Successful task exit

Measures whether a successfully completed deterministic task returns a zero process exit code.

What counts as a pass?

The evaluator checks exit-code against 0. Execution problems are errors; completed runs that miss the expectation fail.

Run this test

harnessfacts run codex execution.exit-success

Exact prompt

Write exactly EXIT_SUCCESS into result.txt in the current directory, then finish the task successfully.

Exact fixture

Write the requested successful token.

Adapters map instruction and skill files to the product's native paths. Inspect the adapters ↗

import type { EvaluationContext } from "../../../packages/core/src/types.ts";

export async function evaluate(ctx: EvaluationContext) {
  const pass =
    ctx.output.exitCode === 0 &&
    (await ctx.read("result.txt"))?.trim() === "EXIT_SUCCESS";
  return {
    pass,
    summary: pass
      ? "Successful task returned exit code 0."
      : "Successful task did not produce the expected zero exit and file result.",
  };
}