Hi,
SS Version: 8.0.5 with 1 Leaf and 1 Master Aggregator
I am using NFS File system to take backups of DB and store it in NFS.
While taking backup of test DB , getting below error:
Logs:
[memsql@rnd-2 singlestoredb-toolbox-1.14.4-e432b273e6]$ sudo exportfs -v
/nfs/general <>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
/home <>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
[memsql@rnd-2 singlestoredb-toolbox-1.14.4-e432b273e6]$ vim /etc/exports
[memsql@rnd-2 singlestoredb-toolbox-1.14.4-e432b273e6]$ ll /nfs
total 0
drwxr-xr-x. 3 memsql memsql 138 Jan 23 10:20 general
[memsql@rnd-2 singlestoredb-toolbox-1.14.4-e432b273e6]$ sudo chmod 777 -R /nfs
[memsql@rnd-2 singlestoredb-toolbox-1.14.4-e432b273e6]$ ./sdb-admin create-backup -r file:///nfs/general test
Toolbox will perform the following actions:
· backup "test" to file:///nfs/general/2023-01-23T103008-04688-12324301643728956639-test
Would you like to continue? [y/N]: y
✘ failed to backup "test" to file:///nfs/general/2023-01-23T103008-04688-12324301643728956639-test:
error running memsqlctl: error running command: `/home/memsql/memsql/singlestoredb-server-8.0.5-249cb54305/memsqlctl --json --yes --config /home/memsql/memsql/memsqlctl.hcl query --memsql-id 284B9D52BBECE935EF0CAA3109684E3017D70904 --sql BACKUP DATABASE `test` TO "/nfs/general/2023-01-23T103008-04688-12324301643728956639-test" --row-timeout 24h0m0s --strict-mode`: exit status 1:
stdout: (empty)
stderr: could not process query results: Error 1789: Leaf Error (<....:3311>): Backup operation could not be completed because path '/nfs/general/2023-01-23T103008-04688-12324301643728956639-test/test_0.backup' on node '<..>:3311' is inaccessible or points to a file that already exists. Error returned from target:Failed to open file: /nfs/general/2023-01-23T103008-04688-12324301643728956639-test/test_0.backup (13:Permission denied)
failed to create 1 backup(s) out of 1
In SS GUI Events sections, it’s saying backup completed.
Snap:
NFS back point:
df -h
<master_aggregator_ip>:/nfs/general 80G 48G 33G 59% /nfs/general
NFS is installed in Master Aggregator VM.
Backup Logs Stored in NFS::
[memsql@rnd-1 nfs]$ cd general/
[memsql@rnd-1 general]$ ls
2023-01-23T110240-463f9-12324301643728956639-test 2023-01-23T110507-4e7fb-12324301643728956639-test file
[memsql@rnd-1 general]$ cd 2023-01-23T110240-463f9-12324301643728956639-test/
[memsql@rnd-1 2023-01-23T110240-463f9-12324301643728956639-test]$ ls
BACKUP_INCOMPLETE test.backup test.backup_columns0.tar test.backup.manifest
[memsql@rnd-1 2023-01-23T110240-463f9-12324301643728956639-test]$ pwd
/nfs/general/2023-01-23T110240-463f9-12324301643728956639-test
[memsql@rnd-1 2023-01-23T110240-463f9-12324301643728956639-test]$ sudo cat BACKUP_INCOMPLETE
{"Backup_Id":"1002","Cluster_Name":"memsql_cluster","Cluster_Id":"12324301643728956639","Database_Name":"test","Start_Timestamp":"2023-01-23 11:02:44","End_Timestamp":"2023-01-23 11:02:45","Num_Partitions":"-1","Backup_Path":"/nfs/general/2023-01-23T110240-463f9-12324301643728956639-test/test.backup","Checksum":"","Status":"Failure","Size":"0","Incr_Backup_Id":"-1","Type":"FULL","Server_Version":"8.0.5","Memory_Usage":"0","Implementation":"Lockfree","Lockfree_Timestamp":"0"}[memsql@rnd-1 2023-01-23T110240-463f9-12324301643728956639-test]$
I have mounted /nfs in leaf1 node:
When I tried to take local back as well as backup in nfs mounted drive, the backup is running.I run SS GUI commands as root user. While running commands inside vm, we are getting error.
Snap:
When I tried to restore DB xyz from NFS Mounted drive, I am getting below errorr:
SS Commands:
DROP DATABASE xyz;
RESTORE DATABASE xyz FROM "/nfs/general";
Output:
RESTORE DATABASE xyz FROM "/nfs/general"
ERROR 1792 ER_BACKUP_IO_ERROR: Failed to stat file: /nfs/general/Arcion_test_0.backup (13:Permission denied)
When I tried to restore DB from Local backup, Restoration of DB is successful.
RESTORE DATABASE xyz FROM "/home/memsql/backup"
OUTPUT:
