AI guardrails

How it works

A guardrail is code that runs before or after the model call. Input checks inspect what a user or a document sends in: personal data that should be masked, requests outside the assistant's job, text that looks like an injection attempt. Output checks inspect what comes back: whether the answer cites a source from the approved set, whether it quotes a price the system never retrieved, and whether the JSON is valid against the schema.

Some checks are plain rules: regular expressions, allowlists, schema validation. Others use a second, smaller model as a classifier. Open-source and managed options include NVIDIA NeMo Guardrails (opens in a new tab), Guardrails AI, Llama Guard and Amazon Bedrock Guardrails, though many production checks are a short function in your own code. For what the checks should cover, the OWASP Top 10 for LLM applications (opens in a new tab) (2025) is a practical list, from prompt injection to improper output handling and excessive agency.

How businesses use it

Guardrails decide what an assistant may say and do. We set them per risk, for example:

• A support assistant may only answer from retrieved help articles and must link the one it used.
• An agent that drafts quotes cannot offer a discount above a set limit.
• Any message that mentions legal action, a formal complaint or a medical symptom goes to a person.
• Customer names and card numbers are masked before any text is logged.

We write these rules with the process owner during the Map stage, then test them against real past examples before launch.

Where it goes wrong

There are two failure modes, and they pull in opposite directions. Loose guardrails let a confident wrong answer reach a customer. Tight ones refuse so often that staff stop using the tool and go back to email.

Guardrails also cannot fix a weak system underneath. If retrieval returns the wrong document, a citation check tells you the answer is unsupported; it cannot find the right one. Track refusal and escalation rates every week with LLM evaluation and LLMOps tooling, and tune thresholds against real traffic.

You need fewer of them when a person reviews every output before it goes anywhere. An internal drafting assistant needs data masking and little else.