wtf is harbor-index?

Harbor Index is a benchmark used to measure how well AI coding agents perform. It consists of 80 tasks that were narrowed down from a starting pool of more than 6,000 candidate tasks, filtered through repeated test runs, automatic detection of broken tasks, human review, and checks for what the readme calls "reward hacking," meaning attempts by an AI agent to score well without actually solving the task properly. The main purpose of this repository, based on the readme, is to support a public leaderboard where people can submit results from running their own AI agent against the benchmark and compare performance against others. To submit, someone runs the benchmark using a command line tool called harbor, specifying which AI agent and which underlying language model they want to test, along with API keys for several AI providers. Those provider API keys are needed because part of the scoring uses a panel of AI models acting as judges to evaluate a portion of the tasks, rather than a purely automated check. After running the benchmark, results are uploaded publicly, and the person submitting opens a pull request in a separate leaderboard repository, linking to their uploaded results. From there, automated checks validate the submission and maintainers review the underlying trajectories, meaning the recorded steps the AI agent actually took, before merging it in as a new row on the public leaderboard. The project also includes internal maintenance tooling, such as an automated process for cutting new versions and a documented policy for how long automated verification checks are allowed to run relative to the overall evaluation time budget. This repository is aimed at people building or evaluating AI coding agents, not general software developers.