ITADN

[echo] Build a pre-seed search-quality corpus

#8203Closedrjpower 创建于 8 天前
experimentagent-generated
R
rjpowercommented
## TL;DR Echo previously retained ordinary query text only in two-day Cloud Run request logs. Permanent search execution and ranked-result persistence is now deployed. Replay the fixed 413-query baseline, grade it with a fresh rubric, and stage concise wiki and core-document additions for review. ## Description The retained request-log slice contains 355 federated executions with 338 unique normalized queries. Echo also has 51 agent-authored feedback queries and a 74-case source-grounded benchmark. One feedback query is absent from the log slice, producing 413 unique federated queries for the initial baseline. Every future search stores its exact query, normalized form, caller, mode, domains, filters, result count, latency, repository commit, service revision, and ranked result snapshot indefinitely. The preserved historical queries are replayed through normal search one at a time. This produces current result snapshots for grading without a backfill endpoint or result-less database rows. The baseline uses a fresh rubric. Luna graders fetch full sources lazily when ranked metadata is insufficient and cache repeated sources by stable identity. Weak results are routed by cause. Frequent workflows belong in OPS.md, AGENTS.md, or skills. Less frequent events belong in Echo wiki notes, which may overlap and expand core guidance. All proposed wiki and core-document changes are staged for review before bulk upload. ## Hypothesis or Goal Intent-clustered, source-supported documentation can move direct answers into the top three results for observed queries and held-out paraphrases without regressing the existing benchmark. ## Status Permanent retention is deployed on Echo revision `echo-api-00024-vtc`. The 413-query baseline replay and grading pass are in progress. ## Links * Logbook: `.agents/logbooks/echo-pre-seed.md` * W&B Report: * Important updates: ## Decision Log * Keep query history indefinitely and omit network identity. * Replay unique historical queries through normal search; do not add a backfill endpoint. * Use a fresh grading rubric; existing feedback was agent-authored. * Fetch and cache full sources lazily during grading. * Stage all content before bulk upload; core-document changes are in scope. ## Conclusion
关闭于 7 天前 2 条评论