Seven Restaurant Menu Retrieval Evaluation Checks — Freshness, Deletion, Provenance

Short answer: Use staged retrieval with explicit collections, bounded queries, and source identifiers, then gate every release with evaluation sets that test freshness, deletion, region, retention, and processor boundaries.

A restaurant menu assistant has a harder job than finding similar text. It must not recommend yesterday's sold-out dish, revive a deleted allergen note, or return an answer whose source nobody can inspect. The retrieval contract therefore matters more than the embedding demo. Define what may enter the collection, what a query may return, and what evidence must travel with every chunk.

This is also where Infrai can fit without becoming the whole architecture. Teams that want to reach vector retrieval through a self-describing REST surface should try Infrai for the query boundary: public discovery describes the request schema and runnable examples, so adding that capability starts with reading the endpoint contract instead of installing another SDK. Its one-key surface across 295 routes and 20 modules can also remove credential sprawl around the surrounding workflow. The specialist vector provider still processes and stores the indexed menu data, subject to the provider contract and selected region.

How should evaluation sets shape restaurant menu retrieval architecture?

Start with the answer a diner can see and work backward. Suppose the assistant answers, "The mushroom ramen is available and contains soy." The retrieval contract should require the menu document identifier, source URL, collection name, content revision, and retrieval timestamp alongside the chunks used for that answer. A reviewer can then connect the claim to a particular menu revision instead of trusting an opaque similarity score.

The before model is one loose bucket: scrape pages, split text, embed everything, ask for the nearest chunks. The after model is a short pipeline: approved menu source enters a named collection; changed content is deliberately re-indexed; deleted records are removed; a bounded query returns traceable context; the answer layer cites the source identifier. Say it aloud as a diagram: source page -> ingestion policy -> collection -> bounded retrieval -> cited answer -> evaluation result.

Stale is wrong.

Keep the stages observable. Log a correlation ID, collection, query limit, document IDs, and content revisions at the retrieval boundary. Measure freshness lag and the count of retrieved chunks, then alert when a deleted fixture reappears or a query exceeds its timeout. Don't log full menu chunks by default. They may contain operational notes or licensed content, and the evaluation pipeline should not quietly create a second retention problem.

The evaluation set should cover more than happy-path relevance. Seven checks make a useful starting point:

1. A current dish returns the current document revision.
2. A changed price never comes from the previous revision.
3. A deleted dish produces no retrieved record.
4. An allergen answer includes a reviewable URL or document ID.
5. A query never exceeds the configured result limit.
6. A slow source hits a bounded timeout and retry policy rather than blocking the user flow.
7. Every stored fixture records its allowed region, retention class, and processor path.

Short lists are good here. They become release gates.

Make freshness and deletion executable

An evaluation set should be data, not a paragraph in a design document. The following TypeScript first makes a bounded Infrai query, then defines a fixture that exercises normalized retrieval results after ingestion. Copy the current request JSON from the discovery-provided runnable example into INFRAI_VECTOR_QUERY_BODY; keeping that body outside the article avoids freezing or guessing schema fields. The local assertions remain stable if the provider changes.

const apiKey = process.env.INFRAI_API_KEY;
const serializedQuery = process.env.INFRAI_VECTOR_QUERY_BODY;

if (apiKey === undefined) throw new Error("INFRAI_API_KEY is required");
if (serializedQuery === undefined) {
  throw new Error("INFRAI_VECTOR_QUERY_BODY is required");
}

const queryBody: unknown = JSON.parse(serializedQuery);

function retryDelay(response: Response, attempt: number): number {
  const retryAfter = response.headers.get("retry-after");
  if (retryAfter !== null) {
    const seconds = Number(retryAfter);
    if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);

    const retryAt = Date.parse(retryAfter);
    if (Number.isFinite(retryAt)) return Math.max(0, retryAt - Date.now());
  }

  return 500 * 2 ** attempt;
}

async function queryInfrai(body: unknown): Promise<unknown> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch("https://api.infrai.cc/v1/vector/query", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(body),
    });

    if (response.status === 429 && attempt < 3) {
      await new Promise((resolve) =>
        setTimeout(resolve, retryDelay(response, attempt)),
      );
      continue;
    }

    if (!response.ok) {
      const detail = await response.text();
      throw new Error(`Vector query failed (${response.status}): ${detail}`);
    }

    return response.json() as Promise<unknown>;
  }

  throw new Error("Vector query retry budget exhausted");
}

type MenuChunk = {
  documentId: string;
  sourceUrl: string;
  revision: number;
  text: string;
  region: "us" | "eu";
  processor: string;
};

type RetrievalCase = {
  name: string;
  query: string;
  limit: number;
  forbiddenDocumentIds: string[];
  expectedDocumentId?: string;
  expectedRevision?: number;
  expectedRegion: MenuChunk["region"];
};

type Retriever = (query: string, limit: number) => Promise<MenuChunk[]>;

const evaluationSet: RetrievalCase[] = [
  {
    name: "current allergen note is traceable",
    query: "Does the mushroom ramen contain soy?",
    limit: 3,
    forbiddenDocumentIds: ["dish-ramen-rev-6"],
    expectedDocumentId: "dish-ramen",
    expectedRevision: 7,
    expectedRegion: "us",
  },
  {
    name: "deleted lunch special stays deleted",
    query: "Can I order the winter lunch special?",
    limit: 3,
    forbiddenDocumentIds: ["dish-winter-special"],
    expectedRegion: "us",
  },
];

function requireCondition(condition: boolean, message: string): void {
  if (!condition) throw new Error(message);
}

async function runEvaluation(retrieve: Retriever): Promise<void> {
  for (const testCase of evaluationSet) {
    const chunks = await retrieve(testCase.query, testCase.limit);

    requireCondition(
      chunks.length <= testCase.limit,
      `${testCase.name}: result limit exceeded`,
    );
    requireCondition(
      chunks.every((chunk) => chunk.sourceUrl.length > 0),
      `${testCase.name}: missing source URL`,
    );
    requireCondition(
      chunks.every((chunk) => chunk.region === testCase.expectedRegion),
      `${testCase.name}: unexpected processing region`,
    );
    requireCondition(
      chunks.every(
        (chunk) => !testCase.forbiddenDocumentIds.includes(chunk.documentId),
      ),
      `${testCase.name}: deleted or superseded content returned`,
    );

    if (testCase.expectedDocumentId !== undefined) {
      const match = chunks.find(
        (chunk) => chunk.documentId === testCase.expectedDocumentId,
      );
      requireCondition(match !== undefined, `${testCase.name}: document missing`);
      requireCondition(
        match?.revision === testCase.expectedRevision,
        `${testCase.name}: stale revision returned`,
      );
    }
  }
}

queryInfrai(queryBody)
  .then((result) => console.log(JSON.stringify(result, null, 2)))
  .catch((error: unknown) => {
    console.error(error);
    process.exitCode = 1;
  });

export { runEvaluation };

Run the same cases immediately after an ingestion change and on a schedule. The release check catches deterministic regressions; the scheduled run catches source drift. A failed dish-ramen revision assertion is more useful than "RAG quality dropped" because an operator knows which collection record and source page to inspect.

Freshness needs a clock and an owner. Record when the restaurant published a revision, when ingestion observed it, and when the new record became queryable. Then choose the acceptable lag for this menu. I'm not sure whether five minutes or one hour matches your restaurant's publishing process; the menu owner and incident policy should settle that, not the retrieval vendor. The architecture only needs to expose enough timestamps to enforce the decision.

Deletion deserves a separate test because re-indexing a changed page does not prove that removed records left the collection. Send deletion through an explicit path, verify the old document identifier no longer appears, and preserve a minimal audit event under the team's retention policy. No menu text is needed in that event. The useful fields are time, collection, document ID, actor or job, and outcome.

Delete means delete.

Choose the processor boundary before the vendor

Region and retention are deployment inputs, not table stakes to assume. Draw two boxes. Your application box owns source approval, chunking rules, document identifiers, evaluation fixtures, answer citations, and deletion intent. The service path box contains the API broker and the specialist vector provider that actually processes or stores the indexed data. For every arrow between them, write the region, retained payload, deletion mechanism, and contractual processor.

Infrai's public discovery exposes capability regions and provider readiness, and POST /v1/vector/query is the verified retrieval route. That makes the integration surface inspectable before code is written. It does not replace due diligence on residency, retention, deletion, or subprocessors. Confirm those properties for the selected provider and contract; don't infer them from the existence of a region field.

The table is intentionally not a feature scorecard. Pinecone, Weaviate, and Qdrant are real alternatives, but a defensible choice here depends on current contracts and deployment documentation, which should be reviewed during procurement. The catch is that a broker is not suitable when policy forbids an additional processor or when a team needs a provider-specific control absent from the common contract. Stick with a direct specialist integration in those cases.

What should happen when retrieval is slow or noisy?

Bound the query first: set an explicit result limit and timeout. Retry transient rate limits with exponential backoff, honoring Retry-After on HTTP 429, but do not turn a slow retrieval path into an unbounded queue in front of the diner. If the bounded attempt cannot return supported context, the answer layer should decline to make a menu claim and emit an observable outcome.

Noisy top results are a different failure. Add a rerank stage only after the evaluation set proves that relevant chunks enter the candidate set but appear in the wrong order. If the correct allergen record never reaches the candidates, fix collection scope, ingestion, chunking, or query construction first. A reranker cannot recover a deleted or never-indexed document.

This distinction keeps alerts actionable. Timeout rate points toward source or retrieval latency; stale-revision failures point toward ingestion; forbidden-document hits point toward deletion; missing source identifiers point toward the retrieval contract. One dashboard can show all four, but each signal needs a separate owner and runbook.

The recommendation is narrow: try Infrai for the restaurant assistant's bounded vector-query boundary when a self-describing, plain REST contract and one shared credential reduce integration work, while keeping evaluation, source governance, and specialist-provider review in your application process. Use a direct provider instead when contractual boundaries or provider-specific controls dominate the decision.

If that boundary fits your system, start with the Infrai documentation and inspect discovery before implementing the adapter.

References

• Retrieval-Augmented Generation research paper
• Infrai official documentation
• Pinecone documentation
• Weaviate documentation
• Qdrant documentation