Even it looks a very simple task, I have learned a few lessons when configuring the vCSA backup.
Lesson #1: vCSA backup location is <host_name>/<folder_name>
If using FTP protocol, the backup location is not just the FTP server host name or IP address; it MUST include the folder name. There is a “/” between the host name and folder name.
Otherwise, the error message is “FTP location is invalid”.
Lesson #2: vCSA backup supports the FTP virtual host name if entering the username correctly - <ftp virtual hostname>|<ftp username>
See my Lesson #2 in “Setting Up IIS 8 FTP Server Lessons Learned” about the FTP virtual host name login. There is a “|” between the hostname and username.
Otherwise, the error message is “Access to the remote server is denied. Check your credentials and permissions”.
Lesson #3: Use curl to troubleshoot vCSA backup error
After entering the correct settings, vCSA backup wizard validates the settings and starts the backup. The backup fails with “BackupManager encountered an exception. Please check logs for details”, but it does not provide much details or the location of the log file.
After some digging, I found the backup log file in /var/log/vmware/applmgmt/backup.log. In the log file, there is a curl error “Connection time-out”.
This gives me a hint that vCSA backup uses curl to transfer the backup file from vCSA to the FTP location. Recently I am also learning curl to transfer file, so I’m a little familiar with curl. (I will publish what I learn from curl in a future post).
From vCSA console, enter “curl -u <ftp user>:<password> -l <ftp server>”. It should list the file and directory on the FTP server. But I got the timeout error. I also tried running curl on a Windows computer, and got the timeout error too. This leads to me think the problem is on the FTP server. Finally the fix is to restart the FTP service (see Lesson #1 on “Setting Up IIS 8 FTP Server Lessons Learned”).
I am not sure why the wizard was able to successfully validate the FTP server setting when the FTP server connection is blocked by the Windows Firewall. When troubleshooting the Windows Firewall, I thought I could use the FTP command to connect to the FTP site, but using curl would fail. I’m not 100% sure about this, since I can’t replicate the issue again. After restarting the Microsoft FTP service, everything is working okay.
Anyway, using curl is the best tool to troubleshoot the vCSA backup failure.
Lesson #4: vCSA backup location must be an empty folder
After successfully running a backup, I try running the backup one more time with the same setting. I got the following error. (PS. In the screenshot below, I removed the virtual hostname on the FTP site, so I can just use the username).
Thank you for taking the time to write this up. The '/' really got me! I appreciate the help.
ReplyDeleteHad invalid URL error and then I realized the windows firewall was set to 'on'. Then the transfer went through fine for a couple of files and then the wizard was hung on "backup job in progress" while on the FTP server there was no transfer happening. At this point I don't know if my backup is succeeded or not.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteI think the job did finish but it actually tried to make the files READ ONLY and failed. I guess its safe to say the backup is fine
ReplyDeletetail backup.log
2017-10-17 15:09:46,848 [ConfigFilesBackup:PID-21419] INFO: Successfully finished configuration files backup.
2017-10-17 15:09:47,455 [MainProcess:PID-21394] INFO: PNID ******.com' is resolved with addresses: ['10.1.1.30']
2017-10-17 15:09:47,455 [MainProcess:PID-21394] INFO: The location provided: 10.1.1.85:2121/
2017-10-17 15:09:47,617 [MainProcess:PID-21394] ERROR: Failed to make the files readonly at ftp://10.1.1.85:2121; Err- Failed to execute command '['curl', '--fail', '-u', u'****:****', '--connect-timeout', '10', '--silent', '--show-error', '-Q', u'SITE chmod 444 backup-metadata.json', u'ftp://*****/']' at 'ftp://*****/'; ret=21
stdOut:
stdErr: curl: (21) QUOT command failed with 504
2017-10-17 15:09:47,618 [MainProcess:PID-21394] INFO: Finished backup at UTC: 2017-10-17 15:09:47.617927. Duration: 82 seconds.
2017-10-17 15:09:47,625 [MainProcess:PID-21394] INFO: Backup job finished successfully.
I'm receiving the same 'failed to make the files readonly' error and an apparently successful backup. Ever find a fix for this? Have you tried restoring your vcsa from one of these backups?
ReplyDelete[MainProcess:PID-53110] ERROR: BackupManager encountered an exception: Failed to get VC build number; applmgmt version get api failed. E rr: Failure instance has no attribute '__getitem__'
ReplyDeleteEddie'S Blog: Configuring Vcsa 6.5 Backup Lessons Learned >>>>> Download Now
ReplyDelete>>>>> Download Full
Eddie'S Blog: Configuring Vcsa 6.5 Backup Lessons Learned >>>>> Download LINK
>>>>> Download Now
Eddie'S Blog: Configuring Vcsa 6.5 Backup Lessons Learned >>>>> Download Full
>>>>> Download LINK cK