After performing a Veeam Instant Recovery, I was able to bring the customer virtual machine back online in a few short minutes. We had the customer kick the tires to make sure everything was copacetic. Once I got the green light, I started to storage vMotion the VM to a Production datastore. Almost immediately it fails with “Cannot Complete File Creation Operation”.

If it doesn’t work the first time, try again right? No dice. Let’s go look through the logs to see if we can find anything useful. Some interesting entries in /var/log/hostd.log.

2019-05-16T16:24:37.085Z warning hostd[2106341] [Originator@6876 sub=Libs opID=jugkzf5y-905dtzc-h5:702272091-77-99ad 
user=vpxuser:local\administrator] Hostlog_Flush: Failed to open hostlog
 /vmfs/volumes/c8je4592-2kb46b78/vm-www01.local/vm-www01.local-
06fe58fa.hlog for writing: Permission denied

2019-05-16T16:24:37.085Z warning hostd[2106341] [Originator@6876 
sub=Vcsvc.OCM opID=jugk3-auto-5dtzc-h5:702272091-77-99ad 
user=vpxuser:local\administrator] PersistToDisk: failed to persist 
entry /vmfs/volumes/c8je4592-2kb46b78/vm-www01.local/vm-www01.local-
06fe58fa.hlog to disk, operation state may not be correctly reflected.

2019-05-16T16:24:37.085Z info hostd[2106341] [Originator@6876 
sub=Default opID=jugkz3-auto-5dtzc-h5:702272091-77-99ad 
user=vpxuser:local\administrator] AdapterServer caught exception: 
N3Vim5Fault16CannotCreateFile9ExceptionE(Fault cause: 
vim.fault.CannotCreateFile

2019-05-16T16:24:37.090Z info hostd[2106341] [Originator@6876 
sub=Solo.Vmomi opID=jugkzauto-5dtzc-h5:702272091-77-99ad 
user=vpxuser:local\administrator] Activation 
[N5Vmomi10ActivationE:0x000000ccc7228e50] : Invoke done [createEntry] 
on [vim.host.OperationCleanupManager:ha-op-cleanup-mgr]

2019-05-16T16:24:37.090Z verbose hostd[2106341] [Originator@6876 
sub=Solo.Vmomi opID=jug3-auto-5dtzc-h5:702272091-77-99ad 
user=vpxuser:local\administrator] Arg entry:

--> (vim.host.OperationCleanupManager.OperationEntry) {

-->    hlogFile = "/vmfs/volumes/c8je4592-2kb46b78/vm-www01.local/vm-
www01.local-06fe58fa.hlog",
-->    opId = 71210656394680,
-->    opState = "running",
-->    opActivity = "nfc",

migrate.hostlog = "vm-www01.local-06fe58fa.hlog"?

Line 1 immediately stands out to me with Permission Denied to the .hlog file. According to VMware:

The .hlog file is a is a log file that is used by vCenter Server to keep track of virtual machine files that must be removed after a certain operation completes.

My first thought is we haven’t performed a storage migration (yet), so do we need this file? Nah, let’s delete it. I proceed to delete the .hlog file from the virtual machine directory, and retry the migration operation. Same failure: “Cannot Complete File Creation”. Despite the file no longer existing, the VM is still looking for it.

Ah ha! Let’s try this again. I unregister the VM from inventory, and manually edit the .vmx file to remove the line that starts with “migrate.hostlog”. Re-register the VM and retry the migration operation. Success!

Once the migration operation succeeds, I’m able to complete the migration of the VM from the temporary NFS mount to the Production datastore.