AI engineering reference systems.
- Systems
- RAG · conversational workflows · evaluation harnesses
- Stack
- Python · FastAPI · PostgreSQL · pgvector · local inference
- Evidence
- Internal evaluations · retained artifacts · public source
- Boundary
- Separately composed implementations, not one integrated runtime
What Keystone is today.
Keystone is a collection of public reference implementations and retained internal evidence for enterprise AI engineering.
The work covers hybrid retrieval, authorization-aware access, conversational-agent workflows, evaluation artifacts, runtime state, audit integrity mechanisms, and local model serving. Each repository is composed independently. Similar concepts across repositories are engineering patterns and research candidates, not evidence of one deployed platform runtime.
The engineering builds on 12+ years of enterprise contact-center, cloud, routing, integration, migration, customer systems, and incident-response work at Genesys. Hands-on LLM-era AI engineering began in late 2024. The two periods inform each other without being the same claim.
Implemented code, served behavior, optional mechanisms, deployment, internal evaluation, retained artifacts, and research conclusions are separate categories on this page. A passing internal run belongs to its evaluated commit and configuration. It is not independent validation or proof of general production suitability.
Mechanisms explored across separate implementations.
The visual map shows independent systems and the evidence or research surfaces around them. It does not depict a common deployed runtime.
Candidate runtime concerns.
Governed Execution currently treats six dimensions as research hypotheses rather than a complete ontology: Identity, Task state, Tempo, Cost, Currency, and Fidelity. Currency asks: does the original justification still legitimately authorize the intended consequence at the point of execution?
Retrieval and evidence
PostgreSQL full-text search, pgvector retrieval, authorization predicates, evidence thresholds, procedural reranking, and local generation appear in specific repositories with different scopes.
Audit integrity
Engage and Counsel use unkeyed SHA-256 hash chains. Gov uses a keyed HMAC-SHA256 value per record with limited signed-field coverage. These mechanisms do not establish semantic correctness or one common audit service.
Coordination and events
Engage contains five specialist agents across four coordination phases, with a real NATS JetStream client behind an optional path. Neither is the default served route, and public evidence does not establish production distributed execution.
Task state and cost
A nine-state lifecycle plus heartbeat, stuck detection, takeover, and rescheduling mechanisms are implemented. The transition rules are unit tested; the store methods for claim, heartbeat, stuck detection, and takeover have no caller or test coverage today. The served path normally exercises only part of that lifecycle. Distributed ownership, leases, compare-and-swap, and fencing guarantees are not claimed. Cost fields exist, but local inference records zero and end-to-end cost accounting is not demonstrated.
Four reference systems, inspected independently.
Each tab separates current mechanisms from limitations and evidence.
Implemented
Gov exposes a FastAPI path using PostgreSQL full-text search and pgvector semantic retrieval, hybrid retrieval, role/domain/jurisdiction constraints, deterministic procedural reranking, evidence thresholding, local generation, HHEM factual-consistency scoring, and per-record keyed HMAC-SHA256 audit integrity. GOV REPO
Limits
HHEM failure degrades open. The HMAC verifier operates per record and does not cover answer text, cited document ids, or the factual-consistency score. Integrity checks are not proof of authorization validity, semantic correctness, or immutable history.
Served default
The API uses a single EngageOrchestrator on its normal served path. Local dispatch currently records zero cost. That served FastAPI path includes automatic HTTP spans and manual GenAI spans for model, prompt and completion tokens, latency, and session identifier, with OTLP/gRPC export. This is a tracing boundary, not stack-wide metrics or complete cost observability. ENGAGE DOCS
Optional or incomplete
The five-specialist, four-phase Coordinator and NATS event path are optional and not the default route. authorize_tool_call is not wired into the served path. MCP support is scaffolded. Task-state mechanisms exist, but not all states are exercised normally. Cost fields do not establish end-to-end cost accounting, cost-based production selection, or complete budget enforcement.
Internal evaluation
The retained Engage lineage moves from 96/100 to 100/100 at commit d199382 against the served single-agent endpoint at that commit. The evaluated path included an empathy gate later moved into the optional Coordinator, so the result should not be read as validation of every current served behavior or of the Coordinator.
Implemented
Classification and client filtering constrain retrieval candidates. Cross-client regression tests cover denial through classification-filtered and unfiltered branches. Counsel also implements an unkeyed SHA-256 hash-chain audit mechanism. COUNSEL REPO
Limits
The published corpus contains only global records with client_id NULL. Cross-client tests are implementation evidence, not production multi-client deployment evidence. Counsel has no dedicated baseline in keystone-ledger, and historical core metrics do not belong to Counsel.
Implemented
Verify reads endpoint profiles and JSONL cases, applies defined deterministic checks, and writes structured results.json and run_metadata.json artifacts with optional content checksums. VERIFY DOCS
$ keystone-verify run \
--profile example/echo_profile.json \
--cases example/sample_cases.jsonl \
--output results/Limits
Verify does not cryptographically seal results or own historical retention. The current CLI should not be retroactively credited with producing historical keystone-core/* artifacts unless lineage records that fact.
Retained evidence, with lineage kept distinct.
keystone-ledger retains selected internal baselines, failures, remediations, commit references, and hashes where recorded. It is evidence for inspection, not immutable proof or independent validation.
d199382, retained from 96/100 to 100/100.Retaining failures beside remediation runs makes negative evidence inspectable and ties results to evaluated configurations. Checksums and Git history can improve reconstruction. They do not provide independent witnessing, semantic verification, or assurance that later code still behaves identically.
What this body of work demonstrates.
Hands-on AI systems engineering across retrieval, APIs, conversational workflows, evaluation, observability, and runtime-control experiments.
Runtime governance is a research direction.
Orchestration determines how work proceeds. Governance determines whether the intended consequence remains justified to proceed.
Governed Execution is the umbrella research program. A future Governed Execution Runtime would be a composed runtime only when real components exist to integrate. That term does not describe today's Keystone implementations. The Governed Execution Lab is not used as public evidence on this page because its repository is not currently publicly accessible.
Runtime Validity, Track A
Runtime Validity is the bounded Track A reference implementation. Current evidence shows that, under a controlled process-local authority change, full revalidation can change the implemented result from PROCEED/MATCH to HOLD/MISMATCH. With revalidation_mode="none", the authority obligation is NOT_EVALUATED.
This does not establish authentic external revocation, production authentication or authorization, durable persistence, independent witness evidence, structural binding between transition and decision evidence, or real external-consequence enforcement. HOLD is an implementation design choice, not a universal research conclusion.
The public repositories do not demonstrate one integrated runtime, universal fail-closed behavior, universal authorization enforcement, production multi-client Counsel operation, deployed NATS infrastructure shared by workloads, complete cost-aware dispatch, or portability across domains. Logs and integrity checks do not prove that an action was justified.
Follow the evidence.
The source, retained artifacts, and architecture documentation are public. Read the repository-specific limitations alongside the mechanisms.