Guest Agent service fails to start on Windows Server 2019 VM after update from 20250611.00.0@1 to 20251009.01.0@1

#### Problem you have encountered: On multiple Windows Server 2019 VMs running in Google Compute Engine, the Google Compute Engine Windows Guest Agent services fail to start after reboot, causing internal TCP load balancer health checks to fail (backends report UNHEALTHY) and services to be unreachable until the agent is manually started. On a test server, the "Google Compute Engine Agent" was are not running post-boot, while other Google services are running. This behavior started around Oct 21 and appears correlated with a recent guest agent update. Manual start of the “Google Compute Engine Agent” service restores LB health checks to passing and service reachability. This started around October 21, 1:00 AM US Eastern time, after an update was performed for the following packages, as noted below: * `google-compute-engine-metadata-scripts.x86_64` - 20250414.00.0@1 --> 20251009.01.0@1 * `google-compute-engine-windows.x86_64` - 20250611.00.0@1 --> 20251009.01.0@1 ``` INFO : 2025/10/21 01:16:04.280307 googet_update.go:122: Update for package google-compute-engine-metadata-scripts.x86_64, 20250414.00.0@1 installed and 20251009.01.0@1 available from https://packages.cloud.google.com/yuck/repos/google-compute-engine-stable. INFO : 2025/10/21 01:16:04.280721 googet_update.go:126: google-osconfig-agent.x86_64 - latest version installed INFO : 2025/10/21 01:16:04.280721 googet_update.go:126: google-compute-engine-driver-balloon.x86_64 - latest version installed INFO : 2025/10/21 01:16:04.281273 googet_update.go:126: google-compute-engine-vss.x86_64 - latest version installed INFO : 2025/10/21 01:16:04.281339 googet_update.go:126: google-compute-engine-driver-netkvm.x86_64 - latest version installed INFO : 2025/10/21 01:16:04.281339 googet_update.go:126: google-compute-engine-diagnostics.x86_64 - latest version installed INFO : 2025/10/21 01:16:04.281339 googet_update.go:126: certgen.x86_64 - latest version installed INFO : 2025/10/21 01:16:04.281339 googet_update.go:126: googet.x86_64 - latest version installed INFO : 2025/10/21 01:16:04.281339 googet_update.go:126: google-compute-engine-powershell.noarch - latest version installed INFO : 2025/10/21 01:16:04.281339 googet_update.go:122: Update for package google-compute-engine-windows.x86_64, 20250611.00.0@1 installed and 20251009.01.0@1 available from https://packages.cloud.google.com/yuck/repos/google-compute-engine-stable. INFO : 2025/10/21 01:16:04.281339 googet_update.go:126: google-compute-engine-driver-gga.x86_64 - latest version installed ``` ### What you expected to happen: - The Google Compute Engine Windows Guest Agent and Manager services should start automatically at boot - Internal load balancer health checks should remain healthy after reboot without manual intervention. ### Steps to reproduce: I have been unable to reproduce this in a test environment with the same OS, performing the same updates, but this is occurring on multiple machines in the original environment. ### Other information (workarounds you have tried, documentation consulted, etc): #### General setup 1) On an affected Windows VM (example: east1-test-01), the VM is rebooted during the regular maintenance window (~1:30 AM). 2) After the VM reports online, check Services: - “Google Compute Engine Agent” is not running, but the Google Compute Engine Agent Manager is running and sending debug logs to GCE 3) Observe internal TCP load balancer backend health: - Backends show UNHEALTHY - internal nlb endpoint not reachable via expected port(s) 4) Start the “Google Compute Engine Agent” service manually: - Backend health returns to HEALTHY - Connectivity via internal nlb is restored #### Attempt to fix After attempting package verification/reinstall via googet for google-compute-engine-metadata-scripts and google-compute-engine-windows, the packages reinstalled but this did not change the behavior. #### Current status - Immediate workaround: Manually start “Google Compute Engine Agent” post-boot to restore LB health and connectivity. - Verified LB configuration and health checks; removing/re-adding instances; confirmed health returns once agent runs. - Reviewed Google guidance for internal LB troubleshooting and guest environment/agent behavior; verified that lack of agent activity correlates with missing internal nlb listener/route on host prior to agent start. - Enabled verbose logging in C:\Program Files\Google\Compute Engine\instance_configs.cfg under [Core] (cloud_logging_enabled=true, log_level=4, log_verbosity=4) and confirmed debug logs stream into Cloud Logging when manager service is running. #### Diagnostics - Checked service triggers with: `sc qtriggerinfo GCEAgent` and `sc qtriggerinfo GCEAgentManager` (both set to start on first IP availability). - Checked Scheduled Task at startup - this exists and appears to run successfully based on the history - Collected Event Viewer entries showing GuestAgentCorePlugin install/launch failures after upgrade. - We do see "GCEGuestAgentManager: Stop plugin \"GuestAgentCorePlugin_20251009.01\" finished with error: failed to find process with pid 576: OpenProcess: The parameter is incorrect."; This only occurred between the initial update and the next reboot and is not a recurring error now. - Collected googet update logs, attached as `googet.log` - Enabled debug logs; relevant logs from after the last reboot are attached as CSV. ## Scope This specifically started occurring only after the update from 20250611.00.0@1 to 20251009.01.0@1 as documented by the Googet installer log, and has affected multiple machines in different environments in the project. This indicates some change in the agent that was made. [downloaded-logs-20251030-102656_snip.csv](https://github.com/user-attachments/files/23269817/downloaded-logs-20251030-102656_snip.csv) [googet.log](https://github.com/user-attachments/files/23269819/googet.log)