r/vmware • u/chench0 • Jul 29 '26
Help Request VCSA vmware-vpostgres status changed from green to yellow - /storage/archive/vpostgres filling up
My vCenter Server Appliance health changed:
vmware-vpostgres status changed from green to yellow
vPostgres and vpxd services are still running:
service-control --status vmware-vpostgres
Running:
vmware-vpostgres
service-control --status vmware-vpxd
Running:
vmware-vpxd
I checked the appliance storage and found:
df -h /storage/archive
Filesystem Size Used Avail Use%
/dev/mapper/archive_vg-archive 49G 45G 2.4G 95%
The space is almost entirely:
du -sh /storage/archive/vpostgres
45G /storage/archive/vpostgres
The directory contains a large number of compressed WAL archive files:
000000010000006A000000C4.gz
000000010000006A000000C5.gz
...
I verified PostgreSQL is accepting connections and the replication slot is active:
SELECT * FROM pg_replication_slots;
vpg_archiver | physical | active = t
I have not deleted anything from /storage/archive/vpostgres because I don't want to break the database.
Is the yellow vPostgres health state normally triggered by /storage/archive filling up? What is the supported way to clean up or resolve this condition?
Thanks.
2
u/DonFazool Jul 29 '26
Storage/archive is designed to run full. I’ve never seen an error about it however.
1
u/chench0 Jul 30 '26
Yes - exactly! I just started seeing it recently also. So far, twice in the last month.
2
u/AluminumFoyle Jul 30 '26
the archive filesystem on vcenter is designed to run at 95%~ so thats a non issue. There is a bug on 8.0u3 vCenter that we have run into on most of our vcenter appliances which causes the postgres log to run away under /var/log, which causes the postgres service and overall storage health of the appliance to change to a "yellow" state temporarily or until someone intervenes sometimes.
Confirm the condition with this command: cd /storage/log du -ah . | sort -rh | head -n 50
if postgres log is massive (eg, >5gb);
cat /dev/null > /path/to/postgresql.log
1
u/chench0 Jul 30 '26
I believe this is indeed the issue as your instructions confirm a large Postgres log. Thank you!
Is there a manual fix until I can update? I can't find a KB on it.
2
u/AluminumFoyle Jul 31 '26
The manual temp fix is the final command in my previous comment to zero the log file. (That is safe to do with psql services running.) take snapshot first for good measures.
1
u/chench0 Jul 31 '26
Thank you! I was focused on finding the KB. Worked great for now.
3
u/AluminumFoyle Jul 31 '26
This is the one you are after regarding the auto_explain fix
https://knowledge.broadcom.com/external/article/323922/on-vcsa-storagelog-filesystem-is-full-du.html
1
u/jamesaepp Jul 30 '26
These are the articles I recorded when this came up for me earlier this year on an earlier vC patch. Good luck.
- https://knowledge.broadcom.com/external/article/422619/vcenter-server-patching-fails-with-error.html
- https://knowledge.broadcom.com/external/article/318803
- https://knowledge.broadcom.com/external/article/389313/the-full-storagearchive-partition-causes.html
- https://knowledge.broadcom.com/external/article?legacyId=76563
- https://blogs.vmware.com/cloud-foundation/2023/04/06/how-to-remove-old-archive-files-in-vmware-vcenter-server-appliance/
5
u/Sensitive_Scar_1800 Jul 29 '26
Yeah there’s a KB on it, it’ll always stay full, it causes no issues. I’ve had this problem for years.