How labbel is built — and why

labbel is infrastructure for keeping methods connected to the work performed with them and the outcomes reported later. The agent does the thinking. The service supplies method documents, applies checks it can perform, and preserves records that can be examined after the work is over.

The difficult part is not collecting a rating. It is preserving enough information to know what that rating is evidence about, how it was obtained, and what it cannot establish.

This document explains the choices we have made to address that problem. It describes an implemented foundation and its limits, not proof that our methods are effective. The central hypothesis — that delayed outcomes can justify useful changes to evaluation criteria — remains to be demonstrated under the conditions we have set for it.

You should be able to use this explanation to question the design or sketch an alternative implementation. It is not a complete reproduction specification, a source-code release, or an API reference. See the service index for current capabilities and connection details, and the guide for a first use. The final section explains what you can verify without access to our code.

1. Separate the method from the agent executing it

Keeping the method in its own documents lets another session or agent use it again. Separating its purpose and criteria from the execution leaves the agent room to respond to the task in front of it.

We separate recipe instructions, evaluation criteria and insights. The guide’s glossary defines the terms used here. Instructions describe the work and its constraints. Criteria describe what a good result needs to satisfy. Insights carry information that may inform later work. Authoring a method and carrying one out are separate activities, with their own documents where the domain supports them.

The same separation lets an organisation bring its own kind of work. Its agent can draft a domain's criteria and example judgements for a person to examine, then register a recipe under the stamped edition and use it. This authoring path works through the existing service without new domain-specific service code. The organisation supplies what quality means in its work. Code review is the one built-in domain. Organisation-authored domains declare one of two forms: work handed in once, or work delivered to a participant in steps. They have their own criteria, follow-up horizon and assessment brief for a selected run; they use the organisation's definitions rather than borrowing another domain's outcome questions. Work in steps may use the learning profile, which adds requirements for retrieval and transfer, delivery guidance and a wait of 7 to 42 days.

The agent reads these through MCP and works using its own model and tools. labbel does not supply a model, run the agent, or give it permission to act in other systems.

This makes the method reusable across agent environments. It also means the service does not observe everything the agent does. A method document can guide work without proving that every instruction was followed. An agent's declaration that it complied remains a declaration unless the service has an independent basis for checking it.

2. Keep observation separate from judgement

A later reader can distinguish what the server checked from what an agent or person judged. That makes it possible to examine a conclusion without giving every part of its evidence the same weight.

Some properties can be checked directly: required fields, allowed values, document structure and relationships between records. Others require judgement: whether a review identified the important risk, or whether an explanation helped someone understand.

The service performs deterministic checks where it has the necessary input. It keeps agent and human judgements distinct from those observations.

Code review provides a concrete example. The server checks the submitted review's structure. It records the review's statement about the customer's test gate as reported, because it did not run that gate. Passing the structural checks does not establish that the review is correct or that the customer's tests passed.

We still rely on agent judgement. The design tries to make that reliance inspectable instead of turning a confident report into an observed fact. This distinction has to appear in responses and stored records, not just in this explanation.

3. Give a later report an unambiguous subject

Attaching a later outcome to a particular run keeps it connected to the work and the method used at the time. Without that connection, it could silently become evidence for the wrong method.

Both forms address their subject as run: N. A run in one step is recorded when work is handed in; a run in steps belongs to the participant's own account and opens at the first saved step. The number counts runs of the recipe across the organisation for work in one step, or the participant's own runs for work in steps. For a new run with a follow-up contract, the service freezes the domain edition, follow-up interval and rule for identifying its criteria. In an organisation-authored domain, later reports address the individual criteria of that edition in either form. Legacy records without a contract remain visibly limited; the service does not guess what their original contract would have been.

In code review, a later report refers to a particular run. If several reviews exist and the report does not identify which one it concerns, the service refuses the ambiguous attribution rather than attaching it to the latest review. Recipe and criteria versions are recorded with the run. Agents can select a run, retrieve retained versions of their organisation's recipes and read a chosen stamped edition of their organisation's criteria. A historical recipe read returns the retained document; it does not reconstruct earlier row metadata or step text. A run in steps is bound at its opening to the edition and step documents then in force; later edits do not change the run in progress. Every criterion must be served by at least one step before the run can open. A finished run in steps keeps a separate assessment manifest: its brief reads the step versions sealed for that run and names material that is missing or could not be determined. The built-in review recipe and built-in criteria are served at their current versions, not through historical version selectors.

This preserves a connection we can inspect later. It cannot establish that the reporter correctly identified the real-world event, or that the reported event happened as described. Attribution inside the service and truth outside it are separate questions.

4. Observe the conditions of assessment

Recording the conditions under which a report was stored lets a later reader know what that report rests on. The service stamps each stored row of a report with three facts it observed itself as it wrote the row — waited, with a reference, through the brief, defined in the guide — and asking the agent whether it is independent is insufficient: nothing a reporter says can choose the server's stamp.

Each fact has one source. Waited is read from the server's clock against the horizon frozen on the run itself; a run without a frozen rule can never be called waited. With a reference is the database's own reading, as the row is written, of whether a work reference stood on the run and had not been taken back. Through the brief is true when the report carried the handle its assessment brief handed out, checked against that run and the connected user's account, and false otherwise. A declaration about the reporter's role is not read for such a row and not stored beside it.

For an organisation-authored domain, the assessment brief supplies the selected run's frozen criteria and follow-up rule without earlier reports. An assessment brief also hands out a handle — a name bound to that run and to the user, not a permission — and a report that sends it back is stored through the brief. The assessor obtains the original work and later evidence from the organisation as needed. Reading a brief is not a substitute for examining that evidence.

None of the three is read from a history of what the connection did before it reported: the service keeps no record of which tools a connection called. A call bears on a later stamp only through what it leaves behind — a work reference that still stands when the report is stored, a brief's handle the report sends back — so reading a recipe through MCP changes nothing about how a later report is stamped; and nothing here reads the conversation.

The limit is important: three facts together are not a certificate of truth, and none of them is a stamp of independence. Waited says the report came after the run's horizon, not that a later outcome was observed. With a reference says a reference was registered and not taken back, not that its content was reached or supports the outcome. Through the brief says the report carried the handle, not that the brief was read. The service cannot see a prompt copied from another conversation, external memory, or everything the model has encountered: it does not observe what an agent has read, so it does not stamp what it cannot see. A fact that could not be observed is absent from the row, and the report is stored all the same; no reading ranks the three or turns them into a percentage. Assessment paths can differ between domains; consult the service index and the connected service’s descriptions for the applicable behaviour.

The rules behind the facts have a version, 6 at this revision, and each row carries it. Rows stored before October 2026 were stamped under earlier versions with a session class instead of facts; they keep that class and are read with those versions' words, shown apart from the rows that carry facts. If where a fact is read from changes, the version moves, so a later reader can tell rows stamped under different rules apart. Older records have a legacy version whose historical precision is explicitly limited; versioning does not reconstruct information that was never recorded.

5. Preserve the clock, the denominator and the differences

Keeping time, versions and counting units alongside reports lets a later reader see which cases have been grouped together. A report made immediately and a report made after a month answer different questions. Ten reports about one run are also different from reports about ten runs.

The evaluation pipeline groups supported reports by criteria version, elapsed-time horizon and, where applicable, the participant's numbered run of the recipe. Its horizon is derived from server-recorded times, rather than letting the reporter's claimed delay select a group. Counts identify the relevant unit, such as runs in steps or code-review runs, and retain provenance distinctions. Where it reports what runs said about an outcome, it counts distinct runs per outcome word, and a run that reported twice and said two different words is counted under each — the numbers are counts of runs, not shares of a total. For code review, the insights an organisation reads through the service answer a different question of the same reports: one standing answer per review and declared horizon, a later report within a horizon replacing the earlier one. Two measures of the same material, each named where it is shown.

For runs in one step, an organisation can ask which runs have follow-up work remaining past a window; for runs in steps, only the connected participant's own finished runs are listed, whatever their role in the organisation. The rule uses server-recorded times and the closing signal type for the unit. What a unit is depends on the kind of work: one outcome per window for a code review, one status per criterion in a domain an organisation authored, in either form. The wait starts when work in one step is handed in, or when the last step of a run in steps is completed. A run stays listed until every unit whose own window has already passed has been answered or closed as unverifiable, or is waiting until a stated date: one report does not remove a run whose other due units are unreported. A unit whose window has not yet arrived is owed nothing yet, so a run whose due units are all answered leaves the list and returns to it when its next window opens. A report that the outcome could not be verified still answers its unit; it does not establish success. What an assessor records about the work — a wait with a reason and a date to look again, or an attempt given up with nothing verifiable to report — is work status and never an outcome, and no counting path reads it as one. Finding due work and scheduling an agent to act on it are separate responsibilities.

The server's clock establishes when a report was stored relative to a recorded anchor. It does not establish when an external event actually occurred. A delayed report can still be an immediate impression submitted late.

Separating groups prevents some misleading pooling. It does not remove selection bias, missing reports, differences in task difficulty, or changes in the agents doing the work. Many records may concern the same underlying task. A larger row count is not automatically a larger independent sample, and silence is not success.

For that reason, the current evaluation pipeline does not manufacture comparative evidence by assembling groups after the fact. It produces observations and hypotheses. A defensible comparison requires an appropriate design beyond the existence of stored data.

The first planned field analysis asks whether declared properties of work are associated with later reported outcomes. It is observational, with no randomized control group, and cannot show that a recipe caused an improvement. It keeps versions, assessment provenance, and outcome horizons separate. Choosing what to count in advance limits our freedom to reinterpret a result. It does not remove selection effects or differences between organizations. The preregistration states these limits before any result exists.^*

6. Make a proposed change inspectable before adopting it

Producing a candidate separately from adopting it gives someone a chance to inspect the evidence and explain a change before it becomes the standard.

If every new report automatically rewrote the criteria, the standard could drift towards whatever the latest agent happened to say. If the standard never changed, recording outcomes would have little purpose.

The evaluation pipeline therefore produces candidate findings and proposed changelog material. It is deterministic for the same input. It does not call a model to decide what the numbers mean, write criteria into the service, or adopt its own proposals. Adoption remains a separate human-reviewed action.

A proposal should identify its supporting data, evidence class, direction of change and expected effect. Insufficient evidence can remain insufficient; the presence of a candidate is not an obligation to change a criterion.

For an organisation-authored domain, the agent can read reports grouped by criterion, inspect the selected runs and historical documents, and use that material to propose a revision. This is an authoring workflow through the service; custom reports do not automatically enter the built-in evaluation pipeline or cause criteria to change.

Decision records preserve how new recipe versions and domain editions came into use, including the recorded base version and supplied references. A reference identifies material; its presence does not establish that the material supports the decision. A domain edition is adopted through the bench. A recipe version authored through the agent surface is in use when written, without a separate human adoption step. The service records the path it observed, not who physically operated it. Earlier records are not backfilled with an invented decision history.

This makes interpretation inspectable. It does not guarantee that it will be good. Autonomous adoption under a delegated mandate is not part of this release.

7. Give human judgement an explicit place

The bench lets an organisation’s expert examine proposed criteria through example judgements and put their name to an edition. It makes responsibility for the initial standard visible before later outcomes can test it. A structurally valid set of criteria is not necessarily a sensible one.

The agent prepares the domain and example judgements; a human examines and stamps an edition. The glossary distinguishes the domain and its edition from the recipe used within it. Stamping the domain does not by itself create the recipe.

The stamp records adoption through the bench, the surface intended for a person's judgement. The server can attest that path, not who sat at it. It is neither a scientific validation nor evidence that the method improves real outcomes. Small examples can expose disagreements and ambiguous criteria; they cannot establish broad validity. The human may also be mistaken, and later outcomes may give reason to revisit the standard.

8. Keep enough to examine the work without retaining everything

Keeping selected records lets the service preserve checks, versions and reported outcomes for later examination while limiting how much of the original work it retains. Storing all conversations and source artefacts would make some investigations easier, while also placing much more of an organisation’s work inside the service.

The code-review path takes a narrower approach. It receives and processes the submitted review, but does not retain the review body. It retains a run record containing checks, versions, selected declarations and derived counts. Some permitted free-text fields are sanitised and capped. Other domains store different material, including authored documents and permitted progress information; the code-review policy is not a universal policy for every payload.

This is a tradeoff. A record can support analysis without containing enough material to reconstruct the full judgement. The organisation needs to retain source evidence it may want to revisit.

Data documentation must distinguish sent, processed, stored and disclosed. “We do not store the review” does not mean it never leaves your environment. Similarly, reading public documentation is different from using MCP: a connection acts as your user, every call carries that user's token, and what a call stores is stated in the tool's own description. No call changes how a later report is stamped by having been made.

9. Treat organisation boundaries as implementation responsibilities

Resolving identity and scoping access to an organisation lets its agents work with the records they are authorised to use. The service and database carry distinct responsibilities for maintaining that boundary.

The system uses Cloudflare Workers for service and web entry points and PostgreSQL through Supabase for persistence. The agent-facing transport is MCP over Streamable HTTP, served without a session: each request builds one MCP server from the identity its token carries, answers, and keeps nothing between requests — no session state and no Durable Object. The identity is read from the account's row on every request. The access paths and their current availability are described in the service index.

An important boundary sits between authenticated requests and database access. The MCP service uses a privileged database role that bypasses row-level security. Organisation scoping therefore has to be enforced by the service's repository layer. Database policies alone are not a sufficient explanation of isolation on that path.

The design also restricts writes to particular evidence records through database grants. Those restrictions need to be read table by table and role by role. They are not a claim that a database administrator cannot alter data or that the system is tamper-proof.

These choices concentrate responsibility in specific places: identity resolution, scoped queries, write permissions and deployment configuration. Verification of those boundaries matters more than naming the infrastructure providers. This architecture description is not a security audit. What has been reviewed, corrected and accepted is stated separately in Security, as far as we can say. The limits of external verification are set out below.

10. What remains outside the foundation

The infrastructure keeps a method, its submitted uses and later reports available for examination, with their versions and observed assessment conditions attached. Those distinctions give later reasoning a basis. It cannot supply all the conditions for that reasoning to be justified.

• Relevant outcomes still have to be observed and reported. Connecting an agent does not arrange its return. Consult the service index for available follow-up support; a discovery mechanism and an agent scheduled to act on it are separate responsibilities.
• Reports can be wrong, incomplete or selectively submitted. Provenance narrows what a record claims; it does not certify the outside world.
• The organisation still needs expertise to choose meaningful criteria and interpret results. A human stamp identifies responsibility, not infallibility.
• Changes in tasks, models and operating conditions can explain changes in outcomes. Stored associations alone do not establish causation.
• Some original evidence stays outside labbel. Its availability later depends on the organisation's own records.

The foundation is therefore more specific than “we will be able to draw conclusions once the data arrives.” It preserves information needed to decide which conclusions the arriving data can support, and where more evidence or a different study is needed. That is the quality of construction this document invites you to inspect.

What you can verify

You can read the public documents and compare their claims. The service index describes the exposed capabilities; the evidence page states what the reported findings do and do not establish. The essay develops the argument behind the service.

If you choose to connect, you can inspect the methods, criteria and records your identity is authorised to access. You can compare the service’s descriptions with the responses it actually returns, including refusals and the distinction between observed and reported information. That lets you test exposed behaviour in the cases you try. Connecting and using tools have the effects explained in the guide.

The implementation and deployment configuration are not public. You cannot independently verify our internal isolation, retention or evaluation rules by reading this page, and a response from the server is not proof of its internals. This document is our account of the construction. It gives you specific claims to question; it does not ask you to treat those claims as an independent audit.

Where supporting study material is not publicly accessible, the description remains a reported finding with that verification limit. Do not infer access to unpublished data or source code from a reference here. Publication of this explanation does not release the implementation or change its licence.

Changes in this document

• Revision 7: prepares the account for stage EA: one built-in domain, two forms of run and the learning profile. Uses the common run address, explains binding at opening and the two anchors for the wait, and describes criterion-based follow-up in both forms of organisation-authored work. Runs in steps belong to the participant and only their own finished runs appear in their due list.

• Revision 6: replaces the provenance class with the three facts the server observes as it stores a row of a report — waited, with a reference, through the brief — each said with where it is read from and what it does not establish; says that the service keeps no history of tool calls, that a declared role is not read for such a row, and that rows stored before October 2026 keep their class; says that the transport is served without a session or Durable Object, and that no call changes a later report's stamp by having been made.

• Revision 5: says that a finished course round keeps its own sealed assessment manifest, read by its brief, beside the limit on historical recipe reads.

• Revision 4: gives due work one definition with its units named, distinguishes an assessor's work status from an outcome, and names the evaluation pipeline's count of runs per outcome word apart from the standing answer the insights show.

• Revision 3: describes S's frozen contracts for organisation-authored domains, selected-run briefs, due-work discovery, criterion coverage, historical reads and decision records. Distinguishes domain adoption from recipe authoring and custom revision work from the built-in evaluation pipeline. Prepared for the S/S+ deployment.

• Revision 2: readers encountered limitations before they could see what the design enabled. Opens each design section with its practical purpose, keeps the qualifications beside the claims they limit, explains how organisations can author their own domains and recipes, points terms to the guide’s glossary and replaces twin links with page links.
• Revision 2, later the same review period: points to the security page from the sentence that says this is not an audit.
• Revision 1: explains the construction and its limits, including the planned field analysis moved from the essay. The publication preparation replaces internal source pointers with explicit verification boundaries and directs current capability questions to the service index.