Skip to content
EncryptTechBD
Home Industries Our Work Blog Pricing About Contact Discuss Your Project
বাংলা (BN)
AI & Agentic Workflows 6 min read

Private knowledge assistant architecture: a buying guide

How to specify a private knowledge assistant: retrieval design, permission-aware answers, evaluation, vendor scorecard, pilot and exit terms.

E
EncryptTechBD · 0 comments
Person wearing a virtual reality headset
Photo: Bruce Mars / StockSnap

A private knowledge assistant is usually bought as a model, then judged on an answer. Both are the wrong starting point. The model is the least differentiated part, and a polished answer can hide a retrieval failure, a permission leak or a stale document. The architecture is what determines whether staff can trust the assistant with commercial and customer information.

Begin with the decisions the architecture must protect

Before comparing products, write down what must remain true after go-live:

  • Source of truth: which repositories are authoritative, and which copies are out of scope.
  • Permission model: answers must respect the access the asking employee already has, not the access of the person who uploaded the file.
  • Answer standard: every answer cites the source passage, states when evidence is missing and refuses when the question falls outside the corpus.
  • Freshness: how quickly a changed policy, price or procedure must be reflected in answers.
  • Language: whether Bangla, English or mixed-language questions and documents must work equally well.
  • Deployment: whether content may leave the country, the cloud account or the building.
  • Audit: which questions, retrieved passages, answers and user actions are retained, for how long and for whom.
  • Volume: documents, pages, queries and concurrent users, with peak assumptions stated.

These decisions become mandatory requirements. A vendor that cannot answer with evidence is not a shortlist candidate.

The reference architecture, part by part

Ingestion and parsing

Connectors pull documents from file shares, email, wikis and line-of-business systems. Parsing matters more than buyers expect: scanned Bangla PDFs, tables, headings and version history all affect what the assistant can retrieve. Each chunk should carry metadata for source, owner, department, classification and effective date.

Retrieval

A strong business assistant combines keyword and vector search, then reranks the candidates. Hybrid retrieval is usually more reliable for policy numbers, product codes and names than embeddings alone. The retrieval layer also enforces the permission filter before any text reaches the model.

Generation and citation

The model receives only the approved passages and a strict instruction set. The output should quote or link the passage, distinguish fact from inference and decline when the retrieved evidence is thin. A model upgrade should not silently change refusal behaviour, so the prompt and evaluation set are versioned together.

Control plane

Identity, role mapping, logging, rate limits, cost controls and a content administration console sit here. Stale documents are removed here, and an owner can see which sources are never used.

Evaluation and feedback

A golden question set, graded answers and a review queue turn opinion into evidence. Track retrieval hit rate separately from answer quality; otherwise every failure looks like a model problem.

Four deployment architectures you can buy

The four architectures differ in where the index and model calls live, and who operates them:

Shared SaaS with a private tenant

Fastest to start and easiest to operate. The trade-off is limited control over retrieval logic, data location and model changes. Suitable when the corpus is low-risk and the connectors already match your systems.

Single-tenant platform in your cloud

You run the vendor's software inside your own environment. This improves data control and isolation, but you inherit upgrades, capacity planning and operations.

On-premises or in-VPC custom build

Maximum fit for unusual permissions, local language requirements or industry-specific documents. It also creates a product to own: retrieval tuning, evaluation, patching and cost management continue after launch.

Managed architecture operated for you

The provider operates the assistant and reviews answer quality on a schedule. This suits organisations without an in-house data team, provided the contract defines service levels and quality remedies.

A scorecard built around architecture

Weight the criteria before demos, and require evidence rather than assurances:

  • Permission enforcement point (20%): where the filter runs and how it tracks role changes.
  • Retrieval design (15%): hybrid search, reranking, metadata filters and superseded versions.
  • Ingestion and parsing (10%): connectors, OCR, tables and Bangla text on your files.
  • Model gateway and portability (10%): whether the model can be swapped without rebuilding retrieval.
  • Evaluation harness (10%): a golden set, regression runs and retrieval quality reported separately.
  • Observability and audit (10%): query logs, retrieved passages, latency and cost per answer.
  • Deployment and network isolation (15%): residency, private connectivity, key management and backup of the index.
  • Operating model and exit (10%): who runs reindexing, and how the index and prompts leave with you.

Architecture questions that change the price

  • Which components are metered: ingestion, embedding storage, retrieval, reranking, generation or seats?
  • What does a full reindex cost, and how often is it required?
  • Does the price change if the model gateway stays in our cloud?
  • Are high availability, backup and a second region included or priced separately?
  • How is network egress from our environment charged?
  • What does it cost to run the evaluation harness on every release?
  • What is the notice period and the exit assistance fee?

Security and compliance questions

Ask for written answers on residency, retention, deletion, subprocessors, model training, access reviews, incident notification and audit log export. If customer or employee records are in scope, map the assistant against Bangladesh's data protection obligations and any sector rules that apply. A vendor that defers these answers is a risk, not a bargain.

Design the pilot as an architecture acceptance test

Run one department, one corpus and a fixed question set for 30 to 60 days. Test the architecture, not just the answers: revoke a source credential mid-pilot, change a document that is already indexed, ask a question the user is not permitted to see, and swap the model behind the gateway. Agree the pass mark before the start: retrieval hit rate, citation accuracy, refusal correctness and reviewer time per answer. Review failures weekly, and record the running cost so the business case uses measured effort, not a projection.

Exit terms that protect the knowledge base

  • Export of original documents with metadata and folder structure.
  • Export of chunks, embeddings, prompts, evaluation sets and answer logs in a usable format.
  • A documented deletion process with written confirmation at contract end.
  • Transition assistance for a defined period, priced in the contract.
  • Clear ownership of custom connectors and prompts built during the engagement.

Warning signs in a vendor shortlist

  • Accuracy claims without a test set, a rubric or a named reviewer.
  • Answers without citations, or citations that point to a whole document rather than a passage.
  • Permission handling described as "the assistant only sees what it should".
  • Unlimited usage pricing with no cap or alerting.
  • A pilot that uses cleaned sample documents instead of yours.
  • No answer for what happens when the model provider withdraws a model.

How we support this purchase

We design private assistants with retrieval, permissions and evaluation treated as one system through Business Automation. Engagement scope and pricing are described on the pricing page. For the conceptual background, read how retrieval-augmented knowledge systems work.

Sources

Services that deliver this

Want this implemented properly?

Pricing →

Every engagement starts with a written scope. If a requirement is outside it, we quote it separately before starting.

Responses (0)

No comments yet. Add the first one below.

Leave a response

Your email address will not be published. Required fields are marked *.

Keep reading

More field notes