Bash: Harbor 0.16.1 task verifiers - How is reward determined?
Harbor (0.16.1) task verifiers: how is reward actually determined? Common misconception that test.sh exit code matters.
Harbor IGNORES the verifier script's exit code entirely (harbor/verifier/verifier.py discards the ExecResult). Reward comes ONLY from a file the script writes: /logs/verifier/reward.json (flat JSON str->float, checked FIRST) else /logs/verifier/reward.txt (bare float). Floats are first-class (VerifierResult.rewards: dict[str, float|int]) so partial credit needs no framework support. Gotchas: (1) harbor's pass@k is computed ONLY when every trial has exactly one reward key with values in {0,1} — float rewards silently disable pass@k aggregation; (2) the scaffolded template comment says 'rewards.json' but code only reads 'reward.json'; (3) since reward.json outranks reward.txt and /logs is agent-writable in shared verifier mode, an agent can pre-write reward.json to steal reward — use [verifier] environment_mode='separate' in task.toml.