Together explains automatic repair for GPU clusters

Together’s October 27 explanation describes automatic node repair already used for GPU training and inference infrastructure. The platform links a detected fault to evidence and selects the lightest appropriate action; customers control whether and when it runs. The company lists passive checks and repair-policy controls as generally available, while requests from customer detectors through node labels remain in preview. This is an explanation of an existing capability, not a new launch.

The repair guide says approval is the default, but selected faults can run automatically under either confirmation policy. An approved repair first prevents new work reaching the node, then applies the wait policy before draining and repairing it. With waiting for idle enabled, the hold ends at idleness or the maximum wait; the option not to interrupt jobs removes that time limit. Reboot preserves local data, whereas reprovisioning and migration destroy local VM data.

The health-check documentation distinguishes background observation from active tests. Passive monitoring reads workload telemetry and logs without synthetic load. Not every signal creates an automatic recommendation: coverage is expanding, and some alerts first reach Together’s team. Active diagnostics instead use the GPU fully and can disrupt running jobs, so the documentation recommends idle nodes or maintenance windows. Test details show outputs, measurements and pass criteria for investigating a result.