Keystone AI Governed RAG for Regulated Industries

Governed RAG for Regulated Industries

Evidence-backed answers. Fail-closed refusal. On your hardware.

Every response scored for factual consistency. Query-time role-based access control. Hash-chained, tamper-evident audit trails. Two live deployments with quantitative evaluation results.

Built for industries where wrong answers have regulatory or safety consequences.

Try the Live Demo View on GitHub

Demo credentials: demo.getkeystone.ai. Log in as operator1 / demo123. Try: "What atmospheric testing is required before entering a confined space?"

The Problem

Safety-critical knowledge is scattered, unfindable, and unauditable

Hundreds of SOPs live across SharePoint folders, shared drives, and filing cabinets. When a regulator asks which version of a confined space entry procedure was in effect on the day of an incident, finding the right document takes hours. When an internal audit asks who accessed a specific safety procedure last Tuesday, the honest answer is often: we don't know.

Generic AI tools make this worse. They answer with confidence whether or not the evidence supports it, send your operational documents to third-party cloud providers, and produce no audit trail of what was accessed or by whom.

Keystone was built to address this.

Architecture

What the system enforces

Keystone enforces nine architectural properties simultaneously. These are not prompt instructions or configuration options. They are structural constraints in the retrieval pipeline, database schema, and API layer.

Evidence-backed answers with citations to source documents
Every answer cites the specific document and section it draws from. Operators see the source material alongside the synthesized response. No black box. No unsupported assertions.
Refuses to answer when evidence is insufficient
When retrieved evidence does not support an answer, the system refuses rather than generating an unsupported response. This is an architectural constraint enforced outside the language model, not a prompt instruction.
Access control enforced at the retrieval layer, not the UI
User roles are checked before retrieval runs. Unauthorized documents are excluded at the database level and never enter the query context. Role-restricted content cannot be surfaced by reformulating a query.
Tamper-evident audit logging
Every query writes an audit record: user identity, role at query time, documents retrieved, permission decision, and the answer generated. Records are hash-chained with an INSERT-only database role. The application cannot modify or delete them after the fact.
Every response scored for factual consistency
A dedicated hallucination detection model independently verifies that every generated answer is supported by the retrieved evidence. Answers that score below threshold are automatically withheld. This creates a double safety net: the system refuses when evidence is insufficient, and independently verifies when evidence is present.
Runs entirely on your hardware. Air-gap capable.
No external API calls for core operation. No data sent to cloud providers. Deployable on customer infrastructure without internet access. Your documents do not leave your network.
Governed learning loop: field feedback improves procedures through auditable review
When operators flag an answer as unhelpful, a review task is automatically created for a custodian. When a custodian publishes an improved procedure version, re-queries return the improved answer. The complete chain (query, feedback, review, publication) is reconstructable from an append-only audit trail.
Document version tracking: point-in-time procedure retrieval for audit scenarios
Every procedure version is tracked with effective dates. The system can answer "which version of this procedure was active on the day of the incident?" The database enforces at-most-one-active-version per document. Superseded versions are retained, never deleted.
Separation of duties: enforced at the system level, not just policy
The person who flags a knowledge gap cannot declare it resolved. The person who creates a procedure version cannot approve it for publication. Both constraints are enforced at the API layer with 403 responses, not documented in a policy that can be bypassed.
How It Works

Your documents. Your hardware. Cited answers and a full audit trail.

01
Load your documents
SOPs, regulations, and safety procedures are ingested and indexed on your hardware. Documents are assigned access control metadata specifying which roles are permitted to retrieve each one. Nothing leaves your network at any stage.
02
Ask in plain language
Users query the corpus the way they would ask a colleague. No Boolean syntax, no folder navigation, no knowledge of where documents are stored.
03
Access is filtered at query time
Before retrieval runs, the system checks the user's role. Documents the user is not permitted to access are excluded at the database level. This happens at every query, not once at login.
04
Get cited answers with an audit trail
Every response includes citations to specific source documents and sections. Every query generates a tamper-evident log record. If the evidence is insufficient to answer, the system says so rather than guessing.
Continuous Improvement

Closes the loop between written procedure and field reality

After every answer, workers can flag what was useful, what was missing, and what was wrong. Those signals feed a governed review process. When a procedure owner validates a change, the next person who asks the same question gets a better answer. Your procedures improve because your people use them.

Live demo at demo.getkeystone.ai (operator1 / demo123). Eval baseline (KDAT-001B, 2026-04-11): P@1=0.75, MRR=0.79. Adversarial ACL: 8/8 blocked, 0 leaks. Fail-closed: 5/6 (83%). Audit chain intact.

Evaluation

Quantitative evaluation baseline

All capability claims are backed by evaluation evidence. The current baseline (KDAT-001B, 2026-04-11) was run against a 53-document Alberta OHS safety corpus with 2,674 indexed chunks.

Retrieval Quality
  • Precision at 1 (P@1): 0.75
  • Mean Reciprocal Rank (MRR): 0.79
  • Corpus: 53 documents, 2,674 chunks
  • Hybrid retrieval: pgvector + full-text search
Security and Safety
  • Adversarial ACL testing: 8/8 blocked, 0 leaks
  • Fail-closed accuracy: 5/6 (83%)
  • Audit chain: intact, immutable
  • INSERT-only DB role enforced
Not Claimed
  • Enterprise HA or disaster recovery
  • Multi-node or distributed deployment
  • OIDC/SAML production identity integration
  • Third-party penetration testing
  • WCAG accessibility compliance
About

Built from operational and infrastructure experience

Arnaldo Sepulveda built Keystone AI after nearly 13 years delivering and supporting enterprise platforms at Genesys for regulated and public sector customers. Those environments required production reliability, security controls, and the ability to prove what happened, when, and who authorized it.

That operational background is why Keystone is built the way it is. Every design decision maps to a documented requirement. Every capability claim maps to a demonstrated proof artifact. No overclaims, no roadmap presented as current capability.

Based in New Brunswick, Canada.

Two deployments: municipal fire department (SOP and medical response documents) and Alberta OHS compliance demo (53 documents, 2,674 indexed chunks). Eval baseline (KDAT-001B, 2026-04-11): P@1=0.75, MRR=0.79. Adversarial ACL: 8/8 blocked, 0 leaks. Fail-closed: 5/6 (83%).
Stack

Built with

Python, FastAPI, PostgreSQL 16 with pgvector, Ollama (nomic-embed-text, qwen2.5:7b-instruct), React/TypeScript/Tailwind, Docker Compose, Caddy, Cloudflare Tunnels. No cloud dependency for core operation.

Source and evaluation evidence: github.com/getkeystone