Why an agent's own report isn't enough

An agent can finish a task with every test green and still miss part of the request. It may skip an acceptance rule, or quietly delete a test that got in its way. A green suite tells you the tests that exist pass. It doesn't tell you whether each requirement has one.

How it works

  1. Write the request down as requirements. Each requirement is one promise with acceptance criteria someone can check, for example “A registered user who asks for a reset gets a link that expires after one use.” Agents can draft these with fp propose. A person approves them.
  2. The agent tags the tests that prove each one.
    // fp:reset_link_single_use
    test('a used reset link is rejected', async () => { ... })
  3. CI records what happened. fp ci-report --junit uploads the results. Each requirement shows as passing, failing or missing (no tagged test), or as present when an agent tagged a test that hasn't run in CI yet.
  4. The pull request is gated on evidence. fp check --junit junit.xml --pr fails the build when the branch makes things worse: a tagged test that now fails, or a test that proved a requirement and has disappeared.

Example

You ask an agent to add password reset. It proposes three requirements: request a link, the link expires after an hour, and the link works only once. You approve them. The agent implements the feature and tags tests for the first two. CI runs. The matrix shows two requirements passing and reset_link_single_use missing, and fp check names it as the next step before the work counts as done.

When the request itself changes

If the agent finds a requirement is wrong, it proposes the change instead of editing the test to fit. The requirement goes back to draft, its existing evidence is marked suspect, and a person reviews the old and new wording side by side. Proposing a change does not clear a failing gate.

Limitations

  • The evidence is only as strong as the tests. FeatureParity doesn't judge whether a test is thorough.
  • It needs test results in JUnit XML, which most test runners can produce.
  • Someone has to approve requirements. Unreviewed drafts aren't enforced.

Next: how coding agents use FeatureParity, step by step.