case://dgx-llm · in production

Production AI, On-Premises

How my team runs four AI workloads in production — code review, security code review, grounded RAG chat, and code generation — on self-hosted LLMs (Ollama × NVIDIA DGX), behind one .NET abstraction, with sensitive data never leaving the organization.

RoleTechnical Team Lead · owner
Team5 engineers
Stack.NET · MEAI · Ollama · DGX
Statuslive · in daily use

01The Problem

Hosted LLM APIs are the fastest way to ship AI features — and a non-starter for a large part of enterprise work. Our source code, internal policies, incident history, and operational documents are exactly the inputs that make AI useful for engineering teams, and exactly the data that cannot cross the organization's boundary.

The naive answers both fail. "Just use the API" fails compliance. "Ban AI" fails the team — engineers were already losing hours to manual code review passes, repetitive security checklists, and hunting through documentation. The real requirement was: bring the model to the data, not the data to the model.

02The Architecture

Every consumer — CI pipelines, the chat UI, IDE tooling — talks to one .NET gateway built on Microsoft.Extensions.AI's IChatClient abstraction. The gateway routes each request by data sensitivity: workloads touching internal code or documents go to open models served by Ollama on NVIDIA DGX hardware, fully on-premises; general-purpose requests can route to hosted models. Sensitivity is a per-request decision, not a per-project rewrite.

CI/CD pipelinescode + security review Chat UIgrounded RAG assistant Engineering toolscode generation .NET AI GATEWAY IChatClient (MEAI) auth · rate limits · telemetry structured output · retries route by data sensitivity ORG BOUNDARY — DATA STAYS INSIDE Ollama · NVIDIA DGX open models · quantized · queued Vector store embeddings of code + docs Hosted APIs (Claude / OpenAI) non-sensitive workloads only INGESTION (OFFLINE) Docs + repos → chunks embed on-prem → index sensitive general
fig.01 — one gateway, two worlds: routing by sensitivity makes on-prem vs hosted a configuration decision

03Four Workloads in Production

WL-01 · CI-INTEGRATED

AI Code Review

Every merge request gets a model pass before human review: correctness risks, N+1 queries, missing cancellation tokens, convention drift. Findings post as structured review comments — humans stay the approvers; the model clears the underbrush.

in diff + surrounding contextout typed findings → MR comments
WL-02 · CI-INTEGRATED

Security Code Review

A separate pass with a security-specific prompt and ruleset: injection risks, authZ gaps, secret leakage, unsafe deserialization. Runs on-prem by necessity — the code being audited is exactly what can't be sent to an external API.

in diff + dependency manifestout severity-ranked findings
WL-03 · INTERACTIVE

Grounded RAG Chat

An assistant answering from our own knowledge: internal docs and repos are chunked, embedded on-prem, and retrieved per question. Answers cite their sources; "grounded" means the model is constrained to retrieved context, not free to improvise policy.

in question + top-k retrieved chunksout cited answer
WL-04 · TOOLING

Code Generation

Scaffolding, tests, migrations, and boilerplate generated against our conventions, with repository context retrieved the same way the chat is grounded. Output is a starting point that compiles against our patterns — not generic samples from the internet.

in spec + retrieved repo contextout code, reviewed like any code

04Decisions & Trade-offs

DecisionWhyThe trade-off we accepted
Self-host on DGXData boundary is non-negotiable for code and internal docsWe own ops, upgrades, and capacity planning; models trail frontier quality
MEAI abstractionRouting is config, not rewrites; providers are swappableA thin layer to maintain; occasional provider-specific features need escape hatches
Structured output everywhereCI can't parse prose — findings must be typed and validatedStricter prompts; a retry-with-error loop for schema violations
Humans stay approversModel findings advise, never gate a merge aloneSlower than full automation — deliberately
Eval set before every swapModel/quantization changes are deploys, not experimentsMaintaining golden datasets takes real time

05What It Changed

The unlock wasn't the model — it was treating AI infrastructure like any other production system: one abstraction, routed by data sensitivity, observable, and evaluated before every change.

Building something similar?
I'm happy to compare notes on running LLMs inside the boundary.

Open channel