The one YAML field that decides whether your TPU job runs or hangs

A multi-host model on Google's TPU hardware has exactly one load-bearing line of Ray Serve configuration: the topology field. Declare a 4x4 slice shape, 16 chips arranged four by four, and the tensor-parallel workers (the model shards that split one model across multiple chips) all land on the same ICI mesh, a high-speed chip-to-chip interconnect that lets them talk fast. Omit that field, and the deployment can sit in DEPLOYING indefinitely while TPU-hours accrue against a job that will never start.

TPU chips come in fixed groups called slices, with host VMs (the virtual machines driving each chip group) sharing that ICI interconnect. A multi-host model, one too large for a single host's memory, must land on a whole slice or its workers cannot reach each other and the job hangs. Google's Part 2 tutorial walks through how Ray Serve, Ray Data, and Ray Train all follow the same pattern: declare a topology, let Ray Core reserve the slice at once. Only the declared resource differs.

The failure mode is silent because of what Ray Serve does when the topology field is missing. Serve's TPU backend falls back to per-chip bundles, and on a multi-host model those bundles can scatter across two slices with no ICI between them. The deployment never errors; it just sits there, billable, until someone notices.

For teams scaling a single-host model up to a multi-host TPU slice, the survival note is short: set the topology field. It is the one line of config that decides whether tensor-parallel workers share a mesh or whether the deployment burns TPU-hours forever.