Evaluate
PR reviews
Findings from two reviewers on the same pull requests.
No pull-request pilot is loaded
A pilot runs two reviewers on the same pull requests and scores their findings. The API shows one pilot at a time, read from the folder named in ARENA_REVIEW_PILOT_DIR. None is set.
python -m arena review-pilot path/to/prs.json \ --baseline http:http://127.0.0.1:8081/review \ --candidate http:http://127.0.0.1:8082/review \ --trials 3 --allow-reviewer-execution \ --output-dir runs/my-pr-comparison ARENA_REVIEW_PILOT_DIR=runs/my-pr-comparison arena serve