CUDA Containers for Edge AI & Robotics

Modular container build system that provides the latest AI/ML packages for NVIDIA Jetson 🚀🤖


ML	`pytorch` `tensorflow` `jax` `onnxruntime` `deepstream` `holoscan` `CTranslate2` `JupyterLab`
LLM	`SGLang` `vLLM` `MLC` `AWQ` `transformers` `text-generation-webui` `ollama` `llama.cpp` `llama-factory` `exllama` `AutoGPTQ` `FlashAttention` `DeepSpeed` `bitsandbytes` `xformers`
VLM	`llava` `llama-vision` `VILA` `LITA` `NanoLLM` `ShapeLLM` `Prismatic` `xtuner` `gemma_vlm`
VIT	`NanoOWL` `NanoSAM` `Segment Anything (SAM)` `Track Anything (TAM)` `clip_trt`
RAG	`llama-index` `langchain` `jetson-copilot` `NanoDB` `FAISS` `RAFT`
L4T	`l4t-pytorch` `l4t-tensorflow` `l4t-ml` `l4t-diffusion` `l4t-text-generation`
CUDA	`cupy` `cuda-python` `pycuda` `cv-cuda` `opencv:cuda` `numba`
Robotics	`ROS` `LeRobot` `OpenVLA` `3D Diffusion Policy` `Crossformer` `MimicGen` `OpenDroneMap` `ZED` `openpi`
Simulation	`Isaac Sim` `Genesis` `Habitat Sim` `MimicGen` `MuJoCo` `PhysX` `Protomotions` `RoboGen` `RoboMimic` `RoboSuite` `Sapien`
Graphics	`3D Diffusion Policy` `AI Toolkit` `ComfyUI` `Cosmos` `Diffusers` `Diffusion Policy` `FramePack` `Small Stable Diffusion` `Stable Diffusion` `Stable Diffusion WebUI` `SD.Next` `nerfstudio` `meshlab` `gsplat`
Mamba	`mamba` `mambavision` `cobra` `dimba` `videomambasuite`
KANs	`pykan` `kat`
xLTSM	`xltsm` `mlstm_kernels`
Speech	`whisper` `whisper_trt` `piper` `riva` `audiocraft` `voicecraft` `xtts`
Home/IoT	`homeassistant-core` `wyoming-whisper` `wyoming-openwakeword` `wyoming-piper`
3DPrintObjects	`PartPacker` `Sparc3D`

See the packages directory for the full list, including pre-built container images for JetPack/L4T.

Using the included tools, you can easily combine packages together for building your own containers. Want to run ROS2 with PyTorch and Transformers? No problem - just do the system setup, and build it on your Jetson:


$ jetson-containers build --name=my_container pytorch transformers ros:humble-desktop

There are shortcuts for running containers too - this will pull or build a l4t-pytorch image that's compatible:


$ jetson-containers run $(autotag l4t-pytorch)

^{jetson-containers run launches docker run with some added defaults (like --runtime nvidia, mounted /data cache and devices)}
^{autotag finds a container image that's compatible with your version of JetPack/L4T - either locally, pulled from a registry, or by building it.}

If you look at any package's readme (like l4t-pytorch), it will have detailed instructions for running it.

Changing CUDA Versions

You can rebuild the container stack for different versions of CUDA by setting the CUDA_VERSION variable:


CUDA_VERSION=12.6 jetson-containers build transformers

It will then go off and either pull or build all the dependencies needed, including PyTorch and other packages that would be time-consuming to compile. There is a Pip server that caches the wheels to accelerate builds. You can also request specific versions of cuDNN, TensorRT, Python, and PyTorch with similar environment variables like here.

Documentation

Check out the tutorials at the Jetson Generative AI Lab!

Getting Started

Refer to the System Setup page for tips about setting up your Docker daemon and memory/storage tuning.


# install the container tools
git clone https://github.com/dusty-nv/jetson-containers
bash jetson-containers/install.sh

# automatically pull & run any container
jetson-containers run $(autotag l4t-pytorch)

Or you can manually run a container image of your choice without using the helper scripts above:


sudo docker run --runtime nvidia -it --rm --network=host dustynv/l4t-pytorch:r36.2.0

Looking for the old jetson-containers? See the legacy branch.

Only Tested and supported Jetpack 6.2 (Cuda 12.6) and JetPack 7 (CUDA 13.x).

[!NOTE] Ubuntu 24.04 containers for JetPack 6 and JetPack 7 are now available (with CUDA support)

LSB_RELEASE=24.04 jetson-containers build pytorch:2.8 jetson-containers run dustynv/pytorch:2.8-r36.4-cu128-24.04

ARM SBSA (Server Base System Architecture) is supported for GH200 / GB200. To install CUDA 13.0 SBSA wheels for Python 3.12 / 24.04:

uv pip install torch torchvision torchaudio \ --index-url https://pypi.jetson-ai-lab.io/sbsa/cu129

See the Ubuntu 24.04 section of the docs for details and a list of available containers 🤗 Thanks to all our contributors from Discord and AI community for their support 🤗