AIBRIEF
Vulkan is a cross vendor compute API whose drivers ship on every supported machine; CUDA is NVIDIA only. Most of the 10x speedup is GPU offload; the real value is driver ubiquity.
PostSlate, a video editing app, runs face detection and face embedding directly on users' machines. To support NVIDIA, AMD, Intel, and Apple Silicon from one codebase, the team picked ncnn's Vulkan backend and rejected CUDA.
CUDA is NVIDIA's proprietary GPU compute stack and only runs on NVIDIA hardware, which would force separate runtimes per vendor. Vulkan is a cross-vendor graphics and compute API whose drivers ship on Windows, Linux, macOS (via MoltenVK), and Android. One model format, one runtime, every supported GPU.
The numbers, measured by PostSlate on an RTX 4070 in half-precision (fp16), are sharp: ArcFace R50 face embedding went from 30 ms on ONNX CPU to 3 ms on ncnn Vulkan; SCRFD face detection from 25 ms to 2.5 ms. Most of that gain is generic GPU offload, not Vulkan specifically. The genuine Vulkan value is the drivers already on the user's machine, a fact no benchmark shows but every release depends on.
The pattern is portable. When shipping ML to consumer hardware, the operative question is not which framework benchmarks fastest but which compute layer already ships on the device. Independent work, including an arXiv survey of on-device deep learning libraries, tracks the same problem. Vulkan is one documented answer; the principle generalizes.