Modelsthatneverleavethebuilding.
Local LLMs, retrieval over your BSS and runbooks, LoRA adapters on your vocabulary. Then the chat, search, and workflow UIs people actually use — with an audit trail, not a notebook.
Why did order 4821 fall out?
Waiting…
Grounding
- Runbook TKT-19
- Ticket 4412
- Order 4821 log
- Charging policy
Retrieve first, then generate. Citations come from the index — not from words floating on the glass.
What we actually build
Four pieces. Most engagements need more than one.
Local LLMs
Run Llama, Mistral, Qwen and similar models on your own GPUs, on-prem, or in a private cloud. Inference stays inside the operator network — no data leaves the premises.
RAG on your systems
Retrieval-augmented generation over billing runbooks, product catalogs, tickets, and network docs. Answers cite the source so operations teams can trust them.
LoRA & QLoRA fine-tunes
Adapt a base model to your BSS vocabulary, ticket language, and SOPs with parameter-efficient fine-tuning. Smaller adapters, lower GPU cost, same private deployment.
Apps on the model
We do not stop at a notebook. We ship the chat, search, and workflow UIs that sit on the model — connected to your APIs, identity, and audit trail.
Fine-tuning with LoRA and QLoRA
Base models are generic. Your charging events, ticket codes, and product names are not. We train low-rank adapters on your labelled data so the model speaks BSS, then serve the adapter next to the frozen base weights.
- LoRA when you have GPU headroom and want higher-fidelity adapters
- QLoRA when VRAM is tight — 4-bit base, trainable adapters
- Eval set from real tickets and catalog questions before go-live
- Versioned adapters, rollback, and audit of training data
Typical stack
- Serving
- vLLM, Ollama, or TGI on Kubernetes
- Training
- Hugging Face PEFT, LoRA / QLoRA
- Retrieval
- pgvector, Qdrant, hybrid search
- Apps
- Next.js / React UIs on your APIs
- Ops
- GPU metrics in Zabbix / Prometheus
- Identity
- SSO, role-based retrieval filters
Where this lands
Ops copilot
NOC and billing teams query private runbooks instead of waiting on a specialist. Suggested next actions, not guessed ones.
Catalog and offer Q&A
Ground the model in PCAT so care agents and self-care only describe products that actually exist.
Ticket summarization
Fine-tuned on your historic tickets so summaries, RCA drafts, and routing match how you already work.
Air-gapped knowledge
For operators that cannot send prompts to a public API. The stack runs on Kubernetes with GPU and queue health in the same NOC as the rest of the estate.
How an engagement runs
Use-case cut
One workflow, one data source, one success metric.
Data & eval
What we can ingest, what we must never log.
Private deploy
Model, RAG, and GPU on your cluster.
Ship the UI
Care, NOC, or internal tool — with audit.