November 2026
Policies, stores and gemstones, from 6 cleaned-up documents, with the passage behind each answer.
1,307 items filtered by budget, shape and metal, shown as cards built from the catalog.
Dates are checked in code, so an expired offer is never quoted.
Checks the store's hours in code. Nothing is booked until the customer confirms.
Offers a handoff, alerts staff in Slack and switches to live chat.
Finds the right passage
50 test questions, typos included
01
One streaming endpoint, one agent, three kinds of knowledge.
02
A retrieval system repeats whatever its sources say.
03
Not everything belongs in a vector store.
| Source | Read by | Why |
|---|---|---|
| Policy documents | Hybrid search | Asked in many different ways |
| Product catalog | SQL filters | “Under 40 million” is an exact value |
| Promotions | Added to the prompt | They expire and must be quoted word for word |
04
Vector search for paraphrases, BM25 for exact terms, merged by rank.
Chunking: 6 documents into 48 chunks of about 760 characters
text-embedding-3-large, 3,072 dimensions, stored as halfvec in pgvector with an HNSW index.Search: every question, top 5 chunks to the model
PT900, and “bao hanh” still matches “bảo hành”.05
A wrong discount is the costliest mistake, so dates are checked in code.
06
No router. The model reads six tool descriptions and decides what to call.
| Tool | Called when |
|---|---|
search_knowledge | Policies, stores, gemstones, payment, warranty |
search_products | The customer wants to see or compare products |
show_products | It recommends products or the customer compares |
suggest_replies | It asks a narrowing question |
book_appointment | The customer wants to visit a store |
escalate_to_staff | The customer agreed to talk to a person |
07
The model chooses what to show; the server decides what it says.
show_products takes only product codes. The server loads each product, so a card cannot carry a price the model made up.08
The bot offers the handoff and escalates only once the customer agrees.
09
50 questions with typos and casual phrasing, run through the real search.
0.82
right chunk first
0.94
in the top 3
0.98
in the top 5
0.88
mean reciprocal rank
| Choice | Compared with | Result |
|---|---|---|
| Hybrid search | Vector only | Top 5: 1.00 vs 0.96 |
| text-embedding-3-large | bge-m3, 3-small | MRR 0.90 vs 0.86, 0.79 |
| Heading path in chunks | No prefix | Top 5: 0.94 vs 0.88 |
| gpt-5.4-mini (production) | glm-5.3-flash, gpt-5-mini | 3.7s vs 5.7s, 12.3s |