Together AI adds canary rollouts to Dedicated Model Inference

Together AI has added canary rollouts to Dedicated Model Inference, so the model behind a live endpoint can be upgraded without downtime. Traffic moves from the current deployment to a new checkpoint in gated steps, defaulting to 5%, 25%, 50%, and 100%.

Health checks run before any traffic shifts. After each step, metric gates compare the new model’s p95 latency and error rate with the previous deployment. If a gate trips, the rollout pauses at the canary share until it is resumed, promoted to 100%, or rolled back.

Canary, blue-green, and rolling strategies are available now through the tg CLI, REST API, and Python SDK. Together AI explains how to start a rollout in Canary rollouts: upgrade models in production without downtime.