Deploy supported Hugging Face models for end-to-end TensorRT inference in just two commands.
Public Preview — TensorRT-Model-Connect is evolving rapidly and is intended for evaluation and feedback. APIs, scope, and direction may change. See Project status.
- 2026-08-23 · Blog — AI-Native by Design: What We Learned Building TensorRT-Model-Connect
- 2026-08-18 · Announcement — TensorRT-Model-Connect enters public preview
python -m tensorrt_model_connect build Qwen/Qwen3-0.6B \
--max-sequence-length 16384 \
--output qwen3-0.6b.bundle
trtmc run ./qwen3-0.6b.bundle \
--runtime-root /opt/trtmc/lib \
--prompt "What is the capital of France? Answer in one word." \
--use-chat-template true \
--enable-thinking false
# Generated text: ParisThe same bundle works from C++:
#include <iostream>
#include <stdexcept>
#include <trtmc/runtime/family_loader.h>
#include <trtmc/task.h>
auto task = trtmc::load_task("./qwen3-0.6b.bundle", "/opt/trtmc/lib");
auto* text = dynamic_cast<trtmc::ITextGeneration*>(task.get());
if (text == nullptr) throw std::runtime_error("unexpected task");
std::cout << text->generate("What is the capital of France? Answer in one word.").text << '\n';
Ready to try TensorRT-Model-Connect? Follow the Quick Start to build and run your first supported model.
We're glad you're here. Star the repository to keep TensorRT-Model-Connect on your radar as we share new model integrations, releases, examples, and community updates. Ideas and feedback are always welcome through the issue chooser.
TensorRT Model Connect is an extensive collection of AI Model reference implementations in C++, on top of NVIDIA TensorRT. Model Connect is powered by an agentic workflow that continuously adds support for upcoming models, drastically reducing integration effort on user side and time until new models become compatible.
- Use TensorRT-Model-Connect to explore models quickly and evaluate broad model coverage.
- For production LLM/VLM deployment on NVIDIA edge platforms where performance is the priority, start directly with TensorRT Edge-LLM.
- Start from a supported Hugging Face or local checkpoint and build TensorRT engines without an intermediate ONNX export step.
- Hand a versioned
.bundleartifact from the Python-first build environment to native C++ task APIs such as text generation, transcription, image and video generation, segmentation, embedding, and forecasting. - Use model-family-owned builders, runtime pipelines, helper kernels, and validation contracts as concrete blueprints for modification and customization.
- Keep native TensorRT and optional TensorRT-RTX execution behind the same task-oriented application boundary.
Read the Architecture Overview for the architecture boundary, intended users, and comparison with other TensorRT integration paths.
TensorRT-Model-Connect is a reference implementation. Users are responsible for trusting the checkpoints, bundles, native libraries, and local environment they provide when building or running models.
Recommended Quick Start: AI-Native
Give an AI coding agent with terminal, Docker, and NVIDIA GPU access this prompt:
/goal Use the current TensorRT-Model-Connect checkout, or clone
https://github.com/NVIDIA/TensorRT-Model-Connect.git if none is provided. Read
AGENTS.md, then follow website/docs/getting-started/source-build.md and
website/docs/getting-started/quick-start.md exactly. Do not modify source,
tests, Dockerfiles, git history, or remote state. Report the selected GPU,
exact commands, bundle path, inference output, and any deviation from the
documentation.
Want to know more? See the Quick Start documentation.
| Goal | Start here |
|---|---|
| Complete the first Qwen inference | Quick Start |
| Select and install an environment | Quick Start |
| Compile the CLI, backends, and model DSOs | Build from Source |
| Find an exact checkpoint or model recipe | Models & Recipes |
| Look up task and runtime contracts | C++ Task API |
| Understand the bundle contract | Bundle Format |
| Understand the source layout | Source Layout |
| Add an isolated model family | Add a Model Family |
| Understand the dependency boundaries | Architecture |
| Contribute to the project | Contributing |
The Supported Models page is the single source of truth for exact checkpoints, Hugging Face architectures, family-owned tasks, precision, quantization, topology, and validation evidence.
Collect the model, environment, command, and log details maintainers need, then use the issue chooser for usage questions, reproducible bugs, feature or model requests, and documentation corrections.
Do not disclose suspected security vulnerabilities in a public issue. Follow SECURITY.md to report them privately to NVIDIA PSIRT.
TensorRT-Model-Connect is an experimental project. Its APIs, scope, and direction may evolve as we learn from users. This preview is intended to inform future decisions and does not establish a specific product roadmap or release timeline.
- Read CONTRIBUTING.md before proposing source or model integration changes.
- TensorRT-Model-Connect is licensed under the terms in LICENSE.