Ahmed Lotfy Mahfouz Technical Team Lead — .NET Systems × Production AI

The question isn't "hosted or self-hosted" — it's where is the data boundary? Internal policies, incident reports, and customer records can't cross the org perimeter, so those workloads run on open models via Ollama on NVIDIA DGX hardware. General coding help and public-content generation route to hosted models like Claude, which are simply better at those tasks.

The engineering move that makes this cheap is Microsoft.Extensions.AI: every caller talks to IChatClient, so routing is configuration, not a rewrite. Sensitivity is decided per request, not per project. The lesson from running it: treat the local model server as production infrastructure — auth, rate limits, telemetry, and an eval set that runs before any model or quantization swap.

Payment and POS integrations punish big-bang migrations — a bad cutover is measured in failed transactions, not bug tickets. The pattern that works is the strangler fig: put a gateway in front of the ESB, then move one integration at a time to a dedicated service while everything else keeps flowing through the old path.

Two disciplines make it safe: contract tests so the new service provably speaks the same language as the old route, and shadow traffic — mirroring real requests to the new service and diffing responses before it ever takes live load. Cutover becomes a routing change you can reverse in seconds. Slow is smooth; smooth is fast.

The moment an LLM output feeds a system instead of a human, prose is a liability. The rule I enforce: never regex model output. Define a typed contract (a C# record), request schema-bound structured output, deserialize, validate, and on failure retry with the validation error appended — the model fixes its own mistake far more reliably than string surgery does.

This is how AnimeCV turns an arbitrary resume into a stable character card, and how our classification services stay boring and dependable. Treat the model like any untrusted upstream API: strict contract in, strict validation out, and telemetry on every call so drift shows up in dashboards, not incidents.