Setting up a machine to train or serve models has always carried the same tedious part: wrestling with GPU drivers, CUDA or ROCm versions, and dependencies that break on the first apt upgrade. Canonical has published a post about Ubuntu 26.04 LTS explaining how they worked on exactly that, so it stops eating hours of your time.
The idea running through the piece is tokens per watt: the amount of useful AI work you produce for each watt of energy you spend. It measures efficiency at the hardware level, but Canonical is clear that the maths doesn’t end there. How long it takes you to get something running, and the productivity of whoever sets it up, count just as much. An expensive GPU sitting idle for three days because the drivers don’t line up produces very few tokens per watt.
CUDA and ROCm in a single line
The most practical new detail is the install of the GPU compute frameworks. Both NVIDIA CUDA and AMD ROCm can now be installed with a single apt install command. Before, this was a multi-step process where dependency and compatibility problems showed up often. If you’ve ever built an inference environment from scratch, you know the drill: repositories to add by hand, versions that have to match each other, and error messages that tell you next to nothing.
Bringing that down to one apt install isn’t magic, but it saves hours or even days of setup per machine. And when the time comes to move a framework up a version, the upgrade path is cleaner than it used to be.
High-performance networking with DOCA-OFED
The other block is networking. Ubuntu 26.04 LTS integrates the NVIDIA DOCA-OFED stack so it installs straight off. The usual sore points here were kernel drift, drivers that stopped being compatible, and CI pipelines that broke on upgrade. For large-scale AI work, where several nodes have to talk to each other at full speed, having the networking layer fit cleanly with the kernel is the difference between an upgrade being routine and being a weekend on call.
What Ubuntu is trying to be here
It’s worth reading one line from Canonical to get the approach. Jon Seager, VP of Engineering, puts it plainly: “Ubuntu is not becoming an AI product.” The commitment, he says, is for Ubuntu to be an enabler for AI. The point isn’t to add its own layers, but to let the optimization work done with silicon vendors and the simplified toolkit install get you to the thing you actually want to do sooner.
That work with silicon spans NVIDIA, AMD, Intel, Arm, Qualcomm, and RISC-V. The goal is to hand the maximum compute to the AI workload and leave the operating system underneath as light as possible.
Who this is for
If your work involves bringing up GPU nodes, this matters to you directly. For everyone else running Ubuntu on the desktop or a generic server, the change won’t show up in daily use, but it’s part of where Canonical is pushing the LTS release. If you haven’t looked at it yet, we went through everything Ubuntu 26.04 LTS brings, from the kernel to long-term support.
Source
Based on the original post published by Canonical: Beyond tokens per watt – using Ubuntu 26.04 LTS for AI, on the official ubuntu.com blog.