LOCAL LLM INFRASTRUCTURE: The Enterprise Engineer's Playbook: Deploy Ollama & vLLM in Production: GPU Sizing, Private Data Ingestion, and Cost-Efficient Hardware for Compliant, Offline AI Systems - Softcover

Wei Chen, Ethan

Softcover

ISBN 13: 9798177225845

Publisher: Independently published, 2026

Synopsis

What happens when your organization needs private AI, but sending sensitive data to a hosted API is no longer an option?
Running an LLM locally is easy.
Running one reliably in production is an infrastructure engineering problem.
You need enough GPU memory for the model and its KV cache. You need to choose the right inference stack. You need to handle concurrency, latency, scaling, authentication, private data, RAG, monitoring, upgrades, failures, and cost.
And when the system handles sensitive enterprise information, “it works on my workstation” is nowhere near enough.
Local LLM Infrastructure: The Enterprise Engineer's Playbook shows you how to design, deploy, secure, observe, and operate self-hosted LLM infrastructure with the discipline expected of a production engineering platform.
Inside, you'll build real, working systems, not toy demos:

1. A defensible TCO model comparing hosted APIs vs. self-hosted inference, the exact numbers to bring to leadership
2. The complete VRAM sizing formula (model weights + KV cache + overhead) with worked tables across every model size, so you stop guessing and start buying the right GPU the first time
3. Production-hardened Ollama and vLLM deployments, including multi-GPU tensor parallelism serving 70B-class models
4. A private, compliant RAG ingestion pipeline (PDF, DOCX, Confluence, SharePoint) where document content never leaves your infrastructure boundary
5. A full threat model, authentication/audit layer, and HIPAA / GDPR / SOC 2 control mapping, with the caveats a real compliance conversation requires
6. Observability, CI/CD, reliability engineering, and cost governance for AI infrastructure, the operational discipline most books skip entirely
7. A complete capstone project, "SecureAssist," a full compliant internal AI platform, where every architecture decision is traced back to the exact chapter that justifies it,

Who is this book for?
Platform and infrastructure engineers, ML infrastructure and MLOps practitioners, DevOps and SRE engineers, solutions architects, security engineers, technical leads, and engineering managers responsible for building or evaluating private AI infrastructure.
You do not need prior experience training machine-learning models. However, this is not a beginner programming book. Familiarity with Linux, Docker, Python, REST APIs, and general infrastructure concepts will help you get the most from it.
Not for you if: you're looking for a beginner's first coding book, or you want model training/fine-tuning theory, this is deployment and infrastructure, cover to cover.
If your organization is moving from “Can we run an LLM locally?” to “Can we operate local AI as production infrastructure?” this book is designed for that transition.
Scroll up and grab your copy now!!!

Edition details

• Publisher: Independently published
• Publication date: 2026
• Language: English
• ISBN 13: 9798177225845
• Binding: Paperback
• Number of pages: 348