[FEA]: Add option for custom BMC NTP servers in expected-machines.json

[pmilano-nvidia]

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

Medium

Please provide a clear description of problem this feature solves

BMC clocks are wildly off from UTC (by months) on boot, because they are using localhost and 127.0.0.1 as primary and secondary NTP servers. This blocks Carbide from running many commands against BMC because the time is out of sync.

Feature Description

From an automation engineer perspective, I want to be able to configure NTP servers for BMCs in expected-machines.json so that the BMC clock is automatically correct, preventing the need for manual intervention and subsequent deletion, re-exploring with bare-metal-manager.

Describe your ideal solution

Adding a JSON field for "timeserver_ips" that accepts an array of two IPs.

{
  "expected_machines": [
    {
      "bmc_mac_address": "<MAC>",
      "bmc_username": "<user>",
      "bmc_password": "<pass>",
      "chassis_serial_number": "<serial>",
      "timeserver_ips" : [ "1.2.3.4", "5.6.7.8" ]
    }
  ]
}

Describe any alternatives you have considered

Currently, I manually update the time service in the BMC Console (or via curl). Then, I delete the existing "explore" for the BMC. Then, it automatically re-explores and ingestion can continue.

Additional context

No response

Code of Conduct

• I agree to follow NVIDIA Bare Metal Manager's Code of Conduct
• I have searched the open feature requests and have found no duplicates for this feature request

[ajf]

The NTP server is configured via the kea dhcp config. Dhcp4.hooks-libraries.parameters.carbide-nameservers.

So we can maybe get it from there somehow or it needs to replace it. Is there any way to confirm BMC behavior that the DHCP option is being sent but not honored?

If anything I don't really see a reason for this to be configured per host, but should be a per-site configuration.

[ajf]

If the BMCs do not honor the NTP server setting in DHCP, then yeah, we should put this in the site-level config file (not per machine); and use that source for DHCP and setting the BMC NTP server via API.

[benhuntley]

+1 on the feature, we hit something related today as well.

All 58 DPU BMCs in our environment had NTP enabled with empty NTPServers. When the preingestion time sync check failed on a DPU, execute_power_off_and_bmc_reset also failed but the DPU silently stays stuck in TimeSyncReset with no error in the site explorer report.

It would help to log an error and set a visible failure state when the BMC reset doesn't succeed on a DPU, so operators know manual intervention is needed.

[ajf]

@nick-vetro PTAL

[daluz]

Agree with @ajf, this should be a site configuration (perhaps use site-default as a path anchor for those, similar to the credentials).