When managing and scaling a VMware Cloud Foundation (VCF) environment, adding nodes is a routine operation. However, this process can encounter unexpected roadblocks. One such issue is a node addition failure caused by a backup failure lock. This blog post outlines the cause, troubleshooting steps, and a resolution to this problem.


Troubleshooting Steps
1. Verify Backup Status
Log into SDDC Manager and navigate to the Backup and Restore section.
Check the status of recent backups. Identify if any backup tasks have failed or are stuck in progress.
Check Logs for Lock Details
Access the SDDC Manager logs via SSH or the Log Bundle
Search for relevant log entries related to backups:
/var/log/vmware/vcf/operationmanager/operationmanager.log
Resolution :
Access the SDDC Manager logs via SSH.
connect to the database by running the command "psql --host=localhost -U postgres -d platform -c "select * from lock" "
psql --host=localhost -U postgres -d platform -c "select * from lock"
psql --host=localhost -U postgres -d platform -c "select * from lock"
id | creation_time | modification_time | error | locking_contex
t | resource_id | resource_type | status | resource_name | reentrant_key | resource_lock_type
--------------------------------------+---------------+-------------------+-------+-----------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------+-------------+---------------+--------+---------------+---------------+--------------------
6e290456-9125-4008-8dc0-deda19e68a2b | 1735660103488 | 1735660103488 | | {"serviceIdentifier":"448cae80-f83c-4119-ad92-9fa3efa0b63c","description":"SDDC Manager DEPLOYMENT l
ock to perform Backup \u0026 Restore operation.","pollingInterval":0,"expirationTime":0} | | deployment | ACTIVE | LEGACY | |
(1 row)
run the command to release the lock , followed by the id from step 2 command.
psql --host=localhost -U postgres -d platform -c "delete from lock where id='6e290456-9125-4008-8dc0-deda19e68a2b'"
Example output :
psql --host=localhost -U postgres -d platform -c "delete from lock where id='6e290456-9125-4008-8dc0-deda19e68a2b'"
DELETE 1
re-run the command to verify if the locked released or not.
psql --host=localhost -U postgres -d platform -c "select * from lock"
id | creation_time | modification_time | error | locking_context | resource_id | resource_type | status | resource_name | reentrant_key | resource_lock_type
----+---------------+-------------------+-------+-----------------+-------------+---------------+--------+---------------+---------------+--------------------
(0 rows)
Comments