AI Governance

1.1 Structured Ingestion and Validation

Envoy connects directly to source systems of record (CRM, ERP, POS, email, WhatsApp Business API) to prevent errors associated with manual re-entry. Data ingestion pipelines include:

• •Schema validation: Each field is checked for consistency against expected formats (e.g. dates, email addresses).
• •Deduplication: Prevents duplicate leads, invoices, or knowledge base entries.
• •Enrichment: Cross-checks against authoritative sources to fill missing fields where possible.

1.2 Knowledge Capture from SMEs

A key challenge for SMEs is that operational knowledge often exists outside formal documents. Envoy addresses this by:

• •Allowing PDF and website uploads into the knowledge base.
• •Conducting a structured 100-question SOP survey, designed to surface tacit knowledge about workflows, customer handling, and escalation paths.
• •Building a hybrid knowledge repository that combines written SOPs with real-world Q&A derived from past conversations.

1.3 Feedback Loops for Continuous Improvement

• •All AI outputs are subject to user verification. Users can confirm, correct, or override AI suggestions.
• •Corrections are logged and patterns of repeated corrections are flagged for review.
• •SMEs are provided with usage metrics, showing which knowledge base articles are most frequently referenced by Envoy. This enables SMEs to prioritise updates to the most business-critical content.

1.4 Monitoring and Auditing

• •All actions are logged in structured formats (JSON), including metadata such as time, source, and model version.
• •Tools such as Windmill, Signoz, and Sentry are integrated to monitor operational metrics (e.g. latency, error frequency, fallback events).
• •Logs support audit trails for compliance with regulatory standards such as Singapore's PDPA.

2.1 Retrieval-Augmented Generation (RAG)

Envoy employs RAG to constrain model responses to an SME's knowledge base. This delivers three benefits:

• •Prevents hallucination by anchoring outputs in vetted sources.
• •Keeps AI aligned to SME intent by ensuring only company-specific data is referenced.
• •Provides auditability through source attribution.

Each response is accompanied by citations that show which knowledge articles were used. SMEs can directly trace responses to underlying documents.

2.2 Explainability for Non-Technical Users

• •Envoy highlights specific text segments or knowledge articles that influenced its answer.
• •In customer-facing contexts, explanations are simplified into plain business language.
• •Non-technical stakeholders can therefore understand why a suggestion was made without technical jargon.

2.3 Confidence Scores and Justifications

• •Confidence scores are generated when extracting structured data (e.g. lead details, invoice line items).
• •In production use, low-confidence outputs trigger human escalation workflows (see Section 3).
• •In advanced debug mode (for internal developer use), Envoy displays both confidence scores and reasoning traces that show which tokens or passages influenced the model's decision. This ensures transparency for developers, while keeping the customer-facing interface uncluttered.

2.4 Reasoning Models (GPT-5)

• •Envoy employs reasoning-capable models that can explain their decision pathways when queried.
• •This enables SMEs to ask "Why did Envoy recommend this?" and receive a coherent natural-language reasoning trace.

3.1 Human-in-the-Loop Escalation

Envoy is not designed to operate without oversight. In cases where the model produces low-confidence results, the following workflow applies:

• •The AI escalates the conversation to a human operator within Envoy's Inbox.
• •The AI jots down an internal note, which includes:

• — A summary of the customer request.
• — The reasoning path it attempted.
• — Its confidence score and why escalation was triggered.

The human operator reviews both the customer's request and the AI's notes, enabling them to step in quickly with full context.

3.2 Internal Notes and Collaboration

• •Internal notes allow humans to see the AI's reasoning without exposing raw model traces to customers.
• •Notes form a collaborative layer, where AI agents prepare drafts or suggestions, and human staff finalise them.
• •This ensures accountability and prevents the "black box" effect of AI-only decisioning.

3.3 Lab Feature for Testing and Evaluation

• •Envoy includes a Lab mode, where a simulated "AI Customer" generates test cases by sampling past conversations.
• •This allows SMEs to proactively test how their Envoy agent responds to real-world scenarios.
• •Human reviewers validate the correctness, tone, and consistency of answers, providing structured feedback for ongoing improvement.

4.1 Update Frequency

Unlike systems that continuously fine-tune models, Envoy does not train or fine-tune models on customer data.

Updates occur on-demand through:

• •Refreshing the knowledge base (uploading new documents, updating SOP survey responses).
• •Adjusting prompts or escalation thresholds.

This approach ensures data security, avoiding risks of unintended memorisation.

4.2 Governance and Logging

• •Every AI request, response, and action is logged in full, including metadata such as request type, model version, and escalation outcomes.
• •Logs are retained for compliance and auditability.
• •Developers and administrators can review logs for error diagnosis and continuous improvement.