gitwtfhub

wtf is algorithm-tester?

samar-gulati/algorithm-tester — explained in plain English

Analysis updated 2026-05-18

1PythonAudience · developerComplexity · 3/5Setup · easy

TL;DR

A Python framework that rigorously stress-tests trading strategy backtests to catch the many ways they can lie about performance.

Mindmap

mindmap
  root((Algorithm-tester))
    What it does
      Tests trading strategies
      Catches backtest lies
      Rejects weak ideas
    Tech stack
      Python
      pandas
      pytest
    Guardrails
      Look-ahead audit
      Matched-exposure control
      Block bootstrap
      Deflated Sharpe
    Use cases
      Vet a strategy idea
      Study case studies
      Learn backtest pitfalls
    Audience
      Developers
      Quant learners

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

Why would anyone build with this?

REASON 1

Check a trading strategy backtest for look-ahead bias before trusting its results.

REASON 2

Test whether a strategy's apparent edge survives resampling and crisis exclusion.

REASON 3

Learn common backtesting pitfalls through the included worked case studies.

What's in the stack?

Pythonpandaspytest

How it stacks up

samar-gulati/algorithm-tester0xustaz/streamgatea-bissell/unleash-lite
Stars111
LanguagePythonPythonPython
Setup difficultyeasyhardhard
Complexity3/54/54/5
Audiencedeveloperdeveloperresearcher

Figures from each repo's GitHub metadata at analysis time.

How do you spin it up?

Difficulty · easy Time to first run · 30min

Requires Python 3.10 or newer, clone the repo and install requirements or use pip install -e.

Wtf does this do

Algorithm-tester is a small Python framework built to test trading strategies honestly, written by a high school student who noticed that the backtesting tools they were using made bad ideas look good. Rather than just measuring whether a strategy made money in the past, the framework is designed to try hard to prove a strategy wrong before it gets used with real money. Every strategy idea passes through a sequence of checks. First it runs through a backtest engine that only lets the strategy see prices from before the day it is making a decision, so it cannot accidentally peek into the future. Then a look-ahead audit deliberately corrupts future prices and reruns the strategy to see if its results change, which would reveal cheating. A matched-exposure control compares the strategy against a simple twin that holds the same average amount of risk with no clever timing, to check whether any apparent skill is really just holding more or less cash. A resampling test called block bootstrap checks whether the strategy's edge survives being recombined thousands of times in different orders, and a related check asks whether the results depend on just one unusual historical event, like a single financial crisis. Finally, a deflated Sharpe ratio calculation discounts the strategy's score based on how many variations were tried before landing on this one, since testing many versions and reporting only the best one inflates apparent skill. The framework also tests itself. A property fuzzer runs many randomized fake markets and strategies through the engine to confirm basic rules always hold, and a mutation test deliberately plants known bugs into the code to confirm the test suite actually catches them. The repository ships with a generic demo trading strategy, since the author's real private strategy is not included, along with three written case studies describing strategy ideas the framework rejected. Setup requires Python 3.10 or newer, and the project can be installed by cloning the repository or directly from GitHub.

Yoink these prompts

Prompt 1
Help me install this framework and run the demo strategy backtest.
Prompt 2
Explain what the look-ahead audit and matched-exposure control checks are testing for.
Prompt 3
Walk me through writing my own trading strategy function to run through this pipeline.
Prompt 4
Show me how the deflated Sharpe ratio check works and why it matters.

Frequently asked questions

wtf is algorithm-tester?

A Python framework that rigorously stress-tests trading strategy backtests to catch the many ways they can lie about performance.

What language is algorithm-tester written in?

Mainly Python. The stack also includes Python, pandas, pytest.

How hard is algorithm-tester to set up?

Setup difficulty is rated easy, with roughly 30min to a first successful run.

Who is algorithm-tester for?

Mainly developer.

View the repo → Decode another repo

This repo across BitVibe Labs

Don't trust strangers blindly. Verify against the repo.