What uptime SLAs measure
API reachability, response latency under load, and infrastructure availability. These are infrastructure metrics. They do not measure output quality, accuracy, or faithfulness to retrieved context.
TL;DR
An AI vendor’s uptime SLA measures whether the API is reachable — not whether its outputs are correct. Nine red-flag clauses signal that the vendor is contractually offloading accuracy risk onto you. Know what to demand before you sign.
Definition
AI Vendor SLA Red Flag — A contract clause that appears standard but structurally transfers the cost of AI output failures (hallucinations, accuracy regressions, model version changes, data exposure) from the vendor to the buyer. Red flags are typically found in limitation-of-liability sections, acceptable-use policies, and model-update notification language.
Key questions answered
- What is the most common red flag in an AI vendor SLA?
- An uptime SLA with no accuracy or output-quality SLO. A vendor can keep the API available 99.9% of the time while the underlying model hallucinates on every fifth query — and the SLA is technically met. Uptime measures infrastructure availability, not output reliability.
- What should an AI vendor contract say about model version changes?
- Advance notice period (minimum 30 days), version-pinning rights, a regression testing window after forced upgrades, and the right to exit without penalty if a version change materially degrades task accuracy.
- How do AI vendors typically handle hallucination liability?
- Most disclaim all liability for output accuracy through ‘as-is’ language. They do not define hallucination, set an acceptable rate, or commit to remediation timelines when accuracy degrades. The buyer owns all downstream consequences of incorrect outputs.
Why the contract matters more than the demo
Every AI vendor demo works. The model is curated, the prompts are chosen, the edge cases are absent. The decision-maker who asks to see the contract instead of the demo is exercising more due diligence than most procurement teams — and that instinct is correct.
AI service agreements are written by legal teams whose job is to limit vendor liability. They are not written to protect the buyer from the failure modes that matter most in production: wrong answers delivered confidently, model behaviour changes without notice, training on your operational data, and disappearing exit rights. These risks do not appear in the demo. They appear in clauses that most buyers sign without reading.
The AI-sceptic in the room who says “I want to read the contract first” is not a blocker. They are doing the risk management the rest of the room skipped.
This piece is written for that person — and for the operator who has to brief them. It maps nine specific contract clauses to the production failure modes they hide, and gives a checklist of what to demand in response.
The uptime SLA trap
The most common SLA in an AI service agreement measures uptime: the percentage of time the API is reachable and returning responses within a latency threshold. 99.9% uptime is approximately 8.7 hours of downtime per year. 99.5% is approximately 43 hours. These numbers sound rigorous.
They measure exactly one thing: whether the API is reachable. They say nothing about whether the answers it returns are correct.
A model that is available 99.9% of the time and hallucinating on 20% of queries has met its SLA while failing at the only thing that matters for most business use cases. The vendor collects the contract value. The buyer owns the downstream consequences — customer-facing errors, compliance exposure, and reputational damage — with no contractual recourse.
What they do not measure
Hallucination rate, factual accuracy, instruction-following reliability, output consistency across model versions, or any dimension of whether the AI is doing what the buyer needs it to do.
What to demand instead
An accuracy SLO tied to a defined evaluation set, a hallucination rate threshold, and a service-credit mechanism that applies when accuracy falls below the agreed baseline — not only when the API goes down.
If a vendor cannot define an accuracy SLO, that is itself diagnostic. It means they have not committed to the reliability of their outputs — only to the reliability of their infrastructure.
Nine contract red flags — ranked by risk
The table below maps nine specific contract clauses to the production risk they create and the counter-clause to demand. Severity ratings reflect how often the risk materialises and how hard it is to recover from once the contract is signed.
| Red Flag | Risk transferred to buyer | Severity | What to demand |
|---|---|---|---|
| Uptime-only SLA, no accuracy SLO | All output quality risk. The vendor can deliver a working API that returns wrong answers indefinitely. | Critical | Accuracy SLO tied to a defined evaluation set, with service credits on breach. Minimum: a stated hallucination rate threshold for the agreed task type. |
| “Best-effort” or “as-is” reliability language | Any reliability guarantee. “Best effort” has no legal definition — it means nothing is guaranteed beyond what the vendor felt like providing. | Critical | Replace with defined SLOs. At minimum, require the vendor to acknowledge a baseline performance level and define what constitutes a material accuracy regression. |
| Unilateral model version changes without notice | Regression risk. The model you evaluated is not the model that will serve production traffic next quarter. Version changes can silently degrade task accuracy. | Critical | 30-day minimum notice for any model version change affecting production. Right to version-pin for 90 days. Right to exit without penalty if a version change produces a material accuracy regression on your evaluation set. |
| Training data use clause on submitted prompts | Data exposure. Your operational prompts, queries, and completions may be used to train future model versions — including those sold to your competitors. | Critical | Explicit prohibition on using customer data for model training or fine-tuning. Data isolation commitment. Deletion timeline upon contract termination (typically 30–90 days). |
| No audit log access or audit rights | Governance and compliance exposure. You cannot demonstrate what your AI system did, when, and based on what inputs. This is a regulatory liability for regulated industries. | High | Retained access to all API request and response logs for the buyer’s data for a period consistent with regulatory requirements (typically 7 years for financial services). Audit rights for security reviews. |
| Unlimited liability exclusion on AI outputs | All downstream business impact. The vendor is not liable for decisions made using their AI outputs regardless of accuracy. This is standard but still transfers risk. | High | Negotiate a proportionate liability cap tied to contract value, particularly for outputs used in regulated decisions. At minimum, require indemnification for IP infringement in AI outputs. |
| Opaque sub-processor routing | Data sovereignty and compliance risk. Your queries may be routed through third-party models or infrastructure in jurisdictions outside your compliance boundary without disclosure. | High | Full sub-processor list with jurisdictions. Prior written consent required for any new sub-processor that processes your data. Data residency guarantees if required by your regulatory framework. |
| Service credits that don’t reflect business impact | Inadequate remediation. A typical service credit is a discount on the next invoice proportional to downtime. It does not compensate for the business cost of incorrect AI outputs delivered during an incident. | Moderate | Service credit mechanism that applies to accuracy SLO breaches, not only uptime. Escalating credits for repeat breaches. Termination rights for sustained accuracy failures above a defined threshold. |
| No exit or data portability clause | Lock-in risk. You cannot migrate your workflows, data, or evaluation infrastructure to a different vendor without the current vendor’s cooperation — which is not guaranteed. | Moderate | Data export in a portable format within 30 days of termination request. No contractual prohibition on building equivalent workflows with competing vendors. Transition assistance period at standard rates. |
Why hallucination liability is the hardest gap to close
Of the nine red flags above, the hallucination liability gap is the one most vendors actively resist closing — because closing it requires them to commit to something they cannot fully control.
Current large language models produce incorrect outputs with a frequency that varies by task type, prompt construction, retrieval context quality, and model version. No vendor can truthfully guarantee zero hallucinations. The standard response is therefore to disclaim all output accuracy liability and leave the buyer to build whatever validation layer they require.
That is not unreasonable as a starting position. The problem is that most buyers accept it as a final position.
⚠
The hallucination SLO negotiation floor
Even if a vendor cannot guarantee zero hallucinations, they can: (a) define a hallucination rate they will not exceed on a specified task type; (b) commit to a defined evaluation protocol for measuring it; (c) specify a remediation timeline when rate exceeds threshold; (d) acknowledge the buyer’s right to run independent evals. Each of these is negotiable. None requires the vendor to promise the impossible.
The practical outcome of accepting no hallucination SLO is that the buyer builds the validation layer themselves — or deploys without one and discovers its absence in production. The hallucination detection layer is a real engineering cost. It should either be contractually required of the vendor, or explicitly scoped as an in-house build at procurement time — not discovered as a gap six months after go-live.
The published benchmark comparison shows that no current frontier model achieves zero hallucination on any standard evaluation set. The buyer who understands this before signing is in a materially better negotiating position than the buyer who discovers it in production.
The model version change problem — and why it is structural
Model version instability is the AI vendor SLA problem that most closely mirrors traditional software versioning — and yet most AI contracts treat it as invisible.
When a SaaS vendor updates their software, the change is disclosed, documented, and in enterprise contracts often subject to a change-management clause. When an AI vendor updates the underlying model, most contracts permit this without notice, without regression testing windows, and without buyer consent. The product the buyer evaluated is not the product they will operate indefinitely.
This creates a structural reliability problem that uptime SLAs cannot capture:
- A workflow calibrated to GPT-4o’s behaviour may perform differently after a silent model update — not because the API broke, but because the model changed.
- Evaluation sets built to validate the initial deployment become stale as the model evolves.
- Operators who discover a regression have no contractual basis for demanding the vendor revert or compensate — unless the contract explicitly covers model version changes.
The counter-clause is straightforward: require the vendor to treat model version changes as change events subject to the same notification and testing windows as infrastructure changes. Thirty days is reasonable for major version changes; seven days for patch updates. A version-pinning option for the contract duration is the gold standard — it exists for enterprise tiers of most major AI APIs and is worth negotiating for explicitly.
The procurement checklist
Use the checklist below when reviewing any AI vendor service agreement. Mark each item before sign-off. Any unanswered item is an open risk that should be either resolved in contract negotiation or accepted as a documented exception by the decision-maker who owns it.
Pre-signature AI vendor SLA checklist
- Does the contract include an accuracy SLO, or only an uptime SLO?
- Does “best effort” or “as-is” language appear in the reliability section? If yes, what replaces it?
- What is the notice period for model version changes? Is version pinning available?
- Does the vendor reserve the right to use submitted prompts or completions for model training? If yes, is an opt-out available?
- Do we have access to full API request/response logs for our data? For how long?
- What is the limitation of liability for AI output errors? Does it cover downstream business impact?
- Who are the sub-processors? In which jurisdictions do they operate?
- What are the service credit triggers — uptime only, or accuracy SLO breach as well?
- Is there a data export right and a transition assistance clause upon termination?
A vendor who refuses to address items 1, 3, and 4 is signalling that they are unwilling to be held to their product’s output quality. That is commercially legitimate — but the buyer should price the risk of building their own validation layer into the total cost of ownership before signing.
What good contract language looks like
The goal is not to make AI vendors sign contracts they cannot honour. It is to close the gap between what the demo implied and what the contract actually commits to. The following clause patterns are achievable in enterprise AI agreements and represent a reasonable baseline:
Accuracy SLO clause
“Vendor commits to a [X]% accuracy rate on the agreed evaluation set, measured monthly. Accuracy is defined as [task-specific definition]. Persistent accuracy below threshold for [Y] consecutive days triggers service credits per Schedule B.”
Model version change clause
“Vendor shall provide not less than 30 days’ written notice prior to any change to the model version serving Customer’s production workloads. Customer may elect to pin to the current version for up to 90 days following notice.”
Data use restriction clause
“Vendor shall not use Customer Data, including prompts, completions, embeddings, or derived data, to train, fine-tune, or evaluate any model, including models not used to serve Customer’s workloads. This restriction survives termination.”
These are not exotic negotiating positions. They are the clauses that make an AI service agreement legible as a production infrastructure commitment rather than a technology trial. Enterprise buyers of cloud infrastructure expect SLOs, change management windows, and data use restrictions as standard. AI service agreements should be evaluated by the same standard.
The trust-collapse risk — and who owns it
Organisations that deploy AI without reading the contract are not just accepting legal risk — they are accepting trust risk. When an AI output causes a customer-facing error, a regulatory filing mistake, or a reputational incident, the first question is always: who knew this could happen, and did they document it?
A signed contract that omits accuracy SLOs, permits unannounced model changes, and disclaims all output liability is a document that shows the operator accepted all of these conditions knowingly. In a post-incident review, that document is evidence.
The AI-sceptic decision-maker who insists on reading the contract is not slowing down AI adoption. They are creating the paper trail that protects the organisation when — not if — a production AI system produces an output that causes harm. That is the procurement discipline that scales.
A contract that omits accuracy guarantees does not mean the vendor’s AI is unreliable. It means the vendor has decided that reliability is your problem, not theirs.
Evaluating an agentic service provider?
RFE Online’s Agentic Services practice covers production governance, contract evaluation, hallucination validation, and the operational layer most AI vendor agreements leave unaddressed. If you’re reading a vendor SLA before a procurement decision, a discovery call is worth 30 minutes of your time.
Explore Agentic Services →Sources and background reading
- RFE Online: Hallucination Detection for Production AI Agents The defensive validation layer that most AI vendor contracts leave to the buyer to build. The engineering cost that should be surfaced at procurement time, not discovered at post-incident review. rfeonline.com.au/insights/applied-intelligence/hallucination-detection-for-production-ai-agents
- RFE Online: Which AI Hallucinates Least? Published Benchmark Comparison SimpleQA, TruthfulQA, Vectara HHEM, and Galileo Hallucination Index ranked side by side. The empirical context for understanding what a vendor can and cannot honestly commit to in an accuracy SLO. rfeonline.com.au/insights/applied-intelligence/ai-model-hallucination-rates-comparison
- RFE Online: AI Code Production Hardening Service The hardening gate that AI-built codebases require before production — covers the gap between what a vendor delivers and what production reliability actually requires. rfeonline.com.au/insights/applied-intelligence/ai-code-production-hardening-service
- RFE Online: Observability ≠ Hardening: Why Agent Analytics Stop Short Why monitoring your AI system is not the same as guaranteeing its reliability — the governance distinction that contract language should reflect. rfeonline.com.au/insights/applied-intelligence/observability-not-hardening-agent-analytics
RFE services for this topic
If this article reflects a real situation in your organisation, these engagements apply.
Fixed scope · Fixed price
Production Trust Audit
Nine SLA red flags tell you what the vendor won’t cover. The Trust Audit shows what your own governance layer needs to cover as a result — authority scope, escalation triggers, and audit trail requirements specific to your agent fleet. Written findings in five business days.
Start the audit →
Full engagement
Agentic Services
Authority models, approval gates, audit trails, and escalation controls installed for your production agent fleet. Discovery call, trust audit, and targeted hardening across two to three weeks.
See the full engagement →Was this insight useful?