Contribute a fact

One agent, one test, or one environment is enough to make a useful contribution.

Contribute a result

1. Fork/clone https://github.com/LouisDeconinck/harnessfacts, enter the checkout, create a branch, and install dependencies with bun install --frozen-lockfile (Bun 1.3.10+ and Git).
2. Install and authenticate the coding agent you want to test. Read the security model in the repository.
3. Run bun run hf doctor and bun run hf run codex instructions.nested (substitute your agent/test), or bun run hf run codex --all for all twelve tests.
4. Inspect the generated JSON, stdout, stderr, and artifacts. Keep failures and errors; do not rewrite them as passes.
5. Run bun run validate, commit the result JSON and its entire evidence directory to your fork, and open a pull request. Explain the environment, authentication method (not credentials), and any adapter changes. One test on one configuration is useful; no code change is required.

The harness records the exact versions, platform, date, hashes, and community provenance. Never commit secrets. A redaction change requires regenerating the matching evidence hash; disclose it for review.

For Antigravity CLI, install agy and sign in once interactively, then run bun run hf run antigravity-cli --all. The adapter uses native OS keyring authentication with a temporary home; see its setup notes. No source changes are needed to submit observations from another installed version.

Gemini CLI is a separate, runtime-unknown adapter, not another name for Antigravity CLI. Verification may require a supported enterprise/Cloud or paid API setup depending on account eligibility and authentication. This adapter currently forwards only GEMINI_API_KEY, not Cloud/enterprise login state. See Gemini setup and limitations. Do not purchase access or spend paid API credits merely to fill this gap; leave it unknown when suitable access is unavailable. Do not put real agent runs in project CI.

Add an agent or test

Adapters live under adapters/<id>/ and own invocation and native configuration. Tests live under tests/<category>/<name>/ and own deterministic expectations. Neither should duplicate the other’s responsibility. Optional authors: [{ github: your-handle }] metadata appears on generated pages.

Start with the repository’s adding an agent guide or adding a test guide. The launch issue list has concrete opportunities.

Check compatibility in another repository

After v0.1.0 is tagged, the reusable Action checks the dataset bundled with that revision; it does not execute an agent in the calling repository:

- uses: LouisDeconinck/harnessfacts/check@v0.1.0
  with:
    agent: codex
    version: '0.153.4'
    platform: linux
    require: |
      instructions.root
      mcp.stdio

Every requirement uses one exact version and full recorded environment. Only PASS succeeds; missing/inconclusive observations remain UNKNOWN, distinct from FAIL. The optional version selects an exact version, not a range. Selection rules and outputs · Complete workflow · External data-package example.

Check your change

bun run lint
bun run typecheck
bun test
bun run validate
bun run build

Project checks do not execute paid coding agents. Never add an actual agent run to ordinary pull-request CI. Explain semantic test changes and increment the test version. Reviews should consider methodology and evidence, not just a green status.

Community

Be kind, specific, and honest about uncertainty. Attribution belongs to the people who did the work. Contributions are licensed under Apache-2.0. See CONTRIBUTING.md, CODE_OF_CONDUCT.md, and SECURITY.md in the source repository.