Enterprise LLM Strategy: Choose Models by Workload, Cost, and Control

The question “Which LLM should our company use?” sounds as if it should end with one model name. It rarely does.

A model that is excellent at repository-scale coding can be wasteful for classifying support tickets. A cheap open-weight model can become expensive when it occupies GPUs at low utilization. A million-token context window can improve one document workflow and quietly multiply the cost and latency of another. A model that passes a public benchmark can still fail on a company’s terminology, permissions, tools, and error tolerance.

The durable enterprise decision is not a winning model. It is a managed model portfolio: the least expensive model that meets an accepted outcome and risk threshold for each workload, with a measured path to a stronger model when it does not.

This guide turns that principle into a selection method, a July 2026 market map, a practical GPU model, a platform architecture, and a governance roadmap. Model availability and prices change quickly, so current facts were checked on July 23, 2026. Treat the named models as a dated snapshot and the decision system as the part to keep.

Start with work, not a leaderboard

The useful unit of model selection is a workload with an owner and an acceptance test.

“Customer service” is too broad. The same service may contain intent classification, policy retrieval, answer drafting, sentiment detection, conversation summarization, refund execution, and quality review. Those tasks have different consequences and need not use the same model.

Before comparing providers, write a workload card:

The last phrase matters. Cost per token is an input metric. Cost per accepted outcome includes retries, human correction, failed tool calls, escalation, and downstream mistakes. A model that costs half as much per token but requires twice as many retries is not cheaper.

What model differences change in practice

Model quality is not one dimension. Differences usually surface in six ways:

1. Judgment under ambiguity. Stronger models make better assumptions and recover from incomplete instructions. This matters in research, planning, and professional work.
2. Tool and agent reliability. The gap widens when a task requires ten correct choices in sequence. A small per-step advantage compounds across a long agent run.
3. Instruction and schema adherence. Smaller models can be excellent at narrow extraction, but may fail more often when policies conflict or schemas become nested.
4. Domain and language coverage. An aggregate English score can hide weak Korean terminology, product names, or specialist reasoning.
5. Multimodal fidelity. Models differ in charts, screenshots, scanned documents, audio, and video even when their text scores look similar.
6. Speed and verbosity. A reasoning model may reach a better answer while producing more hidden or visible tokens and taking longer. For an interactive product, that difference is part of quality.

Parameter count helps estimate deployment cost for an open model. It does not predict all six outcomes. Training data, architecture, active parameters in a mixture-of-experts model, post-training, tool harness, quantization, and inference settings all matter.

Build a current commercial model map from source

A model-and-price table becomes stale faster than the surrounding architecture. Do not copy one from a dated article into a procurement decision. Generate a dated worksheet from each provider’s official model, pricing, lifecycle, and data-governance pages, then attach the source URL and retrieval time to every row.

Record at least:

Use OpenAI’s model catalog, Anthropic’s model overview, Google’s Gemini model catalog, and xAI pricing as primary starting points, then check the selected product’s exact page and terms. Restato’s GPT-5.6 selection guide shows this request-level method for one family.

Managed catalogs such as AWS Bedrock, Azure AI Foundry, and Google Vertex AI can reduce integration and procurement friction and expose several model families. They do not make those models interchangeable. Region coverage, features, version timing, quota, logging, pricing, and contractual terms may differ from a provider’s direct API, so treat each delivery path as a separate evaluated deployment profile.

A default four-lane portfolio

Most companies can begin with four lanes rather than approving dozens of models:

Add specialized models for embeddings, reranking, speech, vision, moderation, and document parsing. A general LLM is often the wrong and expensive substitute for these components.

Open weights are a deployment option, not a synonym for free

“Open model” can mean open source, open weights, source-available, or simply downloadable. These are not equivalent. The exact checkpoint license determines whether a company may modify, redistribute, host for customers, or use the model above a revenue threshold.

Evaluate an exact checkpoint, not a family label. Pin the source repository, revision or digest, tokenizer, chat template, license text, architecture, total and active parameters, native context, supported modalities, quantization recipe, serving image, and safety or use restrictions. A hosted API release does not imply that corresponding weights were published.

Mixture-of-experts models also need two parameter measures: total weights determine storage and much of the loading footprint, while active parameters influence compute per token. Sparse activation can reduce inference compute without making the full checkpoint occupy only the active-parameter memory.

For quick local testing, Ollama, LM Studio, llama.cpp, MLX, or Transformers may be sufficient. For production throughput, common choices include vLLM, SGLang, TensorRT-LLM, Hugging Face Text Generation Inference, and managed inference endpoints. Download provenance, model revision, tokenizer, chat template, quantization recipe, and serving image should all be pinned. “Qwen 35B” is not a reproducible deployment identity.

Commercial API versus open-weight deployment

The answer can be hybrid. A private model may process sensitive high-volume documents, a balanced API may handle ordinary dialogue, and a frontier model may receive a redacted difficult case. That architecture is useful only when routing rules and fallback behavior are explicit.

Model size, precision, context, and GPU memory

For an open model with P parameters, base weight memory is approximately:

weight memory = parameter count × bytes per parameter

BF16 or FP16 ≈ 2 bytes
INT8         ≈ 1 byte
INT4         ≈ 0.5 byte, plus quantization metadata

That gives a first-pass estimate:

These are not purchase recommendations. The runtime needs additional memory for CUDA graphs, workspaces, fragmentation, vision encoders, and—most importantly—KV cache. The KV cache grows with concurrent sequences and context length. A quantized 35B model may fit on a 24 GB card for a short single-user test and fail immediately when production concurrency or a 128K context is enabled.

Do not convert the base-weight table into generic GPU purchase bands. Architecture, quantization kernels, tensor or pipeline parallelism, KV-cache format, context distribution, batching, speculative decoding, and concurrency can move the requirement materially.

The sizing decision requires a load test with the exact checkpoint, quantization, serving engine, prompt distribution, output distribution, hardware, and concurrency. Measure cold load, time to first token, inter-token latency, accepted requests per second, peak memory, queue time, failure rate, and tail latency. Average tokens per second can hide a bad user experience.

Use the sizing process above as a starting estimate, then validate it with the exact checkpoint, quantization, serving engine, and concurrency profile you intend to operate. Compatibility and memory behavior change too quickly to inherit from a retired model-specific guide.

A million-token window is a limit, not a target

Long context is valuable for codebases, legal records, multimedia, and cross-document reasoning. It does not remove the need for retrieval and information design.

Sending everything on every request creates four problems:

1. Input cost and latency rise even when most tokens are irrelevant.
2. KV cache reduces concurrency or requires more GPUs.
3. Important evidence can become harder to locate in a noisy prompt.
4. Data that did not need to cross the model boundary now does.

A better pipeline retrieves by identity and purpose, reranks, compacts stable facts, preserves citations, and sends only evidence needed for the decision. Prefix caching helps repeated context, but only when prompts are designed for stable shared prefixes and the provider’s cache semantics are understood. Some providers also charge higher rates beyond long-input thresholds.

Which leaderboards are worth watching

No public leaderboard measures a company’s accepted outcome. Several are useful for creating a shortlist if their measurement is understood.

Read the methodology and confidence interval before the rank. Compare results obtained with the same harness and budget. Vendor model cards are valuable technical sources, but their benchmark tables are not neutral league tables.

A practical monitoring rhythm is:

• Weekly: provider release, pricing, deprecation, quota, safety, and incident notices; Arena and Artificial Analysis for shortlist changes.
• Monthly: rerun a small shadow evaluation on promising candidates and review latency, cost, failures, and user escalations.
• Quarterly: challenge the approved portfolio, supplier concentration, contracts, GPU utilization, and open-model economics.
• Before every production change: run the full regression set against the exact model, prompt, retrieval, tool, and policy versions.

The internal evaluation set is the asset that makes this rhythm useful. Build it from real, permission-cleared examples; include common work, the difficult tail, policy boundaries, adversarial inputs, Korean and other required languages, long documents, tool errors, and cases where refusal is correct. Keep a holdout set and refresh it from production failures.

Route by risk and evidence, not by brand loyalty

A production router should consider more than prompt length:

request
  │
  ├─ registered workload and data class
  ├─ required modality, tools, region, and context
  ├─ failure consequence and verification strength
  ├─ latency and cost budget
  └─ current provider health and capacity
          │
          ▼
  eligible model set → cheapest model that clears the eval threshold
          │
          ├─ accepted result → record outcome and cost
          └─ low confidence / failed check → repair, escalate, or hand off

Do not let an unconstrained LLM pick any provider and send it arbitrary data. Policy should first produce an eligible set. A learned or model-based router can then choose within that set.

Examples:

Agents strengthen the case for routing. A capable planner can delegate bounded classification or transformation to cheaper sub-models. The model is only one component: permissions, tools, state, verification, and stop conditions determine whether the overall agent is trustworthy. See the enterprise agent platform architecture for the broader runtime.

The enterprise model platform

A company does not need to train a frontier model to own its AI strategy. It should own the control plane that makes models replaceable and outcomes measurable.

applications and agents
          │
 identity + registered use case + cost center
          │
 model gateway and policy enforcement
   ├─ routing and fallback
   ├─ provider and self-hosted adapters
   ├─ prompt/context cache
   ├─ redaction and data-boundary checks
   └─ quota, rate, and spend controls
          │
 commercial APIs ─ managed catalogs ─ private inference
          │
 traces + outcomes + latency + token and GPU cost
          │
 evaluation, registry, release evidence, and chargeback

The minimum platform has nine products:

1. Model catalog and registry. Record exact IDs, owners, provider or checkpoint, license, approved regions and data classes, context, tools, pricing, limits, retirement date, risk tier, and evaluation status.
2. Gateway and adapters. Give applications a stable interface while preserving provider-specific capabilities. A lowest-common-denominator API can erase the features that justified a model.
3. Policy-aware routing. Filter by data, jurisdiction, use, and action risk before optimizing quality, latency, or cost.
4. Evaluation service. Version datasets, graders, human review, thresholds, and results with the full application configuration.
5. Prompt and context management. Version instructions, retrieval, templates, schemas, cache strategy, and source provenance.
6. Observability and FinOps. Join request, outcome, user, workload, model, tokens, cache, tools, latency, retry, and cost. For self-hosting, add queue depth, utilization, memory, energy, and accepted throughput.
7. Release and change controls. Pin versions, run shadow traffic, canary changes, compare failure modes, and support fast rollback.
8. Capacity and resilience. Manage quotas, provisioned throughput, provider incidents, regional capacity, and tested fallbacks. A fallback that returns a different schema or refuses differently is not ready.
9. Evidence and incident records. Preserve enough information to reproduce material decisions without turning sensitive prompts into an uncontrolled log archive.

This platform is where governance becomes executable. Restato’s enterprise AI governance roadmap describes the operating model and decision rights around it.

Cost optimization without lowering the outcome bar

Token discounts are useful, but the largest savings usually come from system design.

For APIs, model the full request:

API cost = uncached input
         + cache writes and reads
         + output and reasoning tokens
         + tool, search, storage, and retrieval charges
         + retries and failed work
         + priority or provisioned capacity

For private inference:

self-hosted TCO = GPU lease or depreciation
                + idle and peak capacity
                + power, network, and storage
                + serving and platform engineering
                + security, upgrades, support, and on-call
                + evaluation and migration work

Optimize in this order:

1. Remove unnecessary model calls. Rules, search, SQL, and existing software should do deterministic work.
2. Shorten and structure context. Retrieve and rerank; do not attach a data lake to every prompt.
3. Use the smallest passing model. Route the hard tail upward instead of making the frontier tier the default.
4. Cap output and reasoning effort. Long answers and excessive thinking can dominate cost and latency.
5. Cache stable prefixes and results. Design for the provider’s cache boundary, privacy rules, and invalidation needs.
6. Batch asynchronous work. OpenAI, Anthropic, Google, and managed platforms offer batch economics for delay-tolerant jobs.
7. Improve acceptance. Better retrieval, tools, schemas, and verification can save more than a cheaper model.
8. Raise GPU utilization carefully. Continuous batching, quantization, prefix caching, speculative decoding, autoscaling, and right-sized context improve serving economics.
9. Negotiate the workload, not just the discount. Capacity, regions, retention, support, deprecation, and incident terms can matter more than headline tokens.

Track at least cost per request, cost per accepted outcome, p50/p95 latency, cache-hit rate, retry and escalation rate, output length, quality by workload, and supplier concentration. A central total bill without workload attribution cannot be optimized responsibly.

When self-hosting crosses over

There is no universal token break-even. Run an experiment using the expected request distribution and realistic utilization. Self-hosting becomes plausible when several conditions align:

• a suitable open checkpoint clears the same acceptance threshold;
• traffic is stable enough to use provisioned GPUs well;
• data control, offline operation, latency, or customization has material value;
• the team can support the service through upgrades and incidents;
• license and supply-chain review are complete;
• a commercial API remains available for overflow or the difficult tail, if permitted.

Do not compare an API’s variable price with a fully utilized theoretical GPU. Compare the API bill with an operated service at actual average and peak utilization.

Governance for a changing model portfolio

Every approved model or checkpoint should have a short enterprise model card:

• exact version, endpoint or artifact hash, provider, hosting path, and owner;
• permitted workloads, users, regions, data classes, and autonomy levels;
• license, usage restrictions, attribution, and redistribution conditions;
• retention, training-use, residency, subprocessors, and deletion terms;
• context, modalities, tools, limits, known failure modes, and safety behavior;
• evaluation results and acceptance thresholds by workload;
• unit price or serving cost, quota, capacity, and support level;
• fallback, rollback, deprecation, review date, and incident contacts.

Procurement should cover model-specific change risk: notice of price and model changes, version pinning, capacity, service levels, security incidents, output ownership, intellectual-property protection, audit rights, export controls, termination, and data export. The temporary 2026 suspension and redeployment of Anthropic’s Fable 5 illustrates that geopolitical and safety constraints can become runtime dependencies. A provider’s technical availability page is part of enterprise architecture.

Changes that require re-evaluation include:

• a new model or moving alias;
• a prompt, reasoning level, temperature, or output limit change;
• a new retrieval source, embedding, reranker, or context policy;
• a quantization, runtime, GPU, or serving-image change;
• new tools, permissions, user groups, languages, or regions;
• a provider safety, retention, license, or retirement change.

The fallback model must be evaluated too. Fail-open is especially dangerous for agents: a cheaper fallback may misunderstand a tool schema or skip a refusal that the primary model enforced.

What public enterprise cases actually teach

Supplier case studies are selected success stories, not controlled comparisons. Their architecture lessons are still useful.

• Morgan Stanley built internal knowledge retrieval and meeting debrief workflows with OpenAI models, but the repeatable asset was its evaluation practice: domain experts scored summarization, retrieval, translation, and real meeting cases, with daily regression testing and human review. The lesson is to scale an eval system, not a prompt demo. OpenAI case study
• GitLab formed a model evaluation team and evaluates new vendors and model versions across different DevSecOps tasks. Its published approach explicitly chooses the right model for each use case rather than assuming one model serves the full software lifecycle. Anthropic case study
• TELUS combined Gemini-based agents with BigQuery, privacy controls, shared customer context, and human agent support across a large contact-center workload. Whatever model is chosen, data integration, privacy processing, operational feedback, and action design create the service. Google Cloud case study
• Forethought used managed serverless inference and multi-model endpoints to reduce serving cost for customer-service ML workloads. The broader lesson is that utilization and hosting mode can outweigh a model’s nominal size. AWS case study

These examples point to the same pattern: successful companies accumulate evaluation data, workflow integration, controls, and operating knowledge. The provider model is important, but it is not the whole product.

A 90-day enterprise plan

Days 0–30: measure before standardizing

• inventory current model APIs, employee tools, embedded SaaS AI, open checkpoints, and GPU endpoints;
• select five to ten representative workloads with owners, baselines, risks, and cost ceilings;
• create the first evaluation set from real examples and production failure modes;
• approve one fast, one default, and one frontier commercial candidate;
• establish temporary rules for sensitive data, model downloads, licenses, and unregistered endpoints;
• capture tokens, latency, cost center, model version, and accepted outcome through one gateway.

Exit when: the company can explain what models it uses, for which work, under which terms, at what quality and cost.

Days 31–60: create the portfolio

• run candidates with the same prompts, tools, context, and scoring method;
• set acceptance thresholds per workload, including abstention and refusal cases;
• introduce policy-aware routing and a measured escalation path;
• test one open model where privacy, predictable volume, or customization could justify it;
• pin versions, implement shadow tests and canaries, and test provider failure;
• establish model cards and a cross-functional approval path for new models.

Exit when: production selection is evidence-based and a model can be replaced without an emergency rewrite.

Days 61–90: optimize and govern change

• report cost per accepted outcome and quality by workload, not only total tokens;
• optimize context, cache, output, batch, and reasoning settings before negotiating capacity;
• load-test private inference using real context and concurrency distributions;
• set provider concentration limits and retirement playbooks;
• connect evaluations, model cards, releases, incidents, and spend to the AI inventory;
• schedule monthly portfolio challenges and quarterly contract and capacity reviews.

Exit when: a new model release triggers a repeatable comparison rather than a company-wide debate.

What the company should own for the long term

Model weights will keep changing. The durable assets are:

• workload definitions and accepted-outcome metrics;
• permission-cleared evaluation and regression datasets;
• domain data, terminology, ontology, retrieval, and source provenance;
• tool and action contracts with deterministic verification;
• routing rules, policy decisions, and fallback behavior;
• model-independent telemetry and cost attribution;
• provider adapters and versioned deployment artifacts;
• fine-tuning, preference, and synthetic-data pipelines with clear provenance;
• incident knowledge and evidence of what fails in the real workflow.

A company may later fine-tune, distill, or train specialized models. It should do so because the measured workload advantage exceeds the lifecycle cost—not because owning weights sounds strategically complete.

The practical end state is intentionally unglamorous: a small approved portfolio, a fast path for ordinary work, an expensive escape hatch for the hard tail, a private lane where it earns its keep, and a platform that can prove why each request went where it did.

That is how an enterprise follows a fast-moving model market without letting the market redesign its architecture every month.