gitwtfhub

wtf is app-factory?

kamyartaher/app-factory — explained in plain English

Analysis updated 2026-05-18

0PythonAudience · developerComplexity · 2/5LicenseSetup · easy

TL;DR

Shares a runnable verification gate that catches AI agents from falsely claiming success, one reusable piece of a larger iOS app-building pipeline.

Mindmap

mindmap
  root((app-factory))
    What it does
      Verifies agent success claims
      Rejects fabricated passes
      Detects stalled loops
    Tech stack
      Python stdlib only
      No dependencies
    Use cases
      Agent pipeline gating
      Reliability testing
      Loop bounding
    Audience
      Developers
      AI agent builders
    Contents
      Gate script
      Unit tests
      Example reports
      Eval set template

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

Why would anyone build with this?

REASON 1

Drop the verification gate into an agent pipeline to reject fabricated success claims.

REASON 2

Study the seven documented tricks to design your own agent reliability checks.

REASON 3

Use the negative fixtures as a template for testing that your own checks can actually fail.

REASON 4

Follow the five step guide to build a small bounded multi-step agent pipeline.

What's in the stack?

Python

How it stacks up

kamyartaher/app-factory0xallam/my-recipe0xhassaan/nn-from-scratch
Stars00
LanguagePythonPythonPython
Last pushed2022-11-22
MaintenanceDormant
Setup difficultyeasymoderatemoderate
Complexity2/52/54/5
Audiencedevelopergeneraldeveloper

Figures from each repo's GitHub metadata at analysis time.

How do you spin it up?

Difficulty · easy Time to first run · 5min
Use freely for any purpose, including commercial use, as long as you keep the copyright notice.

Wtf does this do

app-factory shares pieces of the system the author uses to turn a one sentence app idea into a shipped iOS app, covering discovery, pricing, design, build, App Store screenshots, and submission. More than 20 apps have gone through the full pipeline, with a dozen live on the App Store. The author is publishing reusable parts of that system one at a time rather than the whole thing, since the full setup is described as too tangled and too specific to their own workflow to be useful to others. The first piece released is a verification gate: a single Python file, standard library only, with no dependencies, that decides whether an AI agent's reported success can actually be trusted. It rejects fabricated passes that lack supporting evidence, checks that cited evidence points to a real, current, non-failing result, and includes 17 built in negative test cases designed to make sure the gate correctly refuses bad input. It also detects when an agent's review loop is stalling instead of making progress, enforces limits on both the number of turns and the spend a loop can use, and tracks whether the checker itself is still working correctly over time. The README documents seven specific techniques behind the gate, each mapped to the exact function in the code that implements it, along with a table connecting common symptoms (like a gate that passes wrong output, or a loop that keeps reporting progress with nothing changing) to their underlying causes and fixes. There is a suggested five step path for building a small version of a similar pipeline yourself, starting with chaining a few bounded steps together and adding one negative test case per check. The repository includes the gate script itself, 23 unit tests, example reports showing valid and invalid cases, an evaluation set template, code snippets from the author's write up, and a roadmap of planned future pieces. It is released under the MIT license, and the author welcomes contributions as long as any new check comes with a negative test case attached.

Yoink these prompts

Prompt 1
Explain how the semantic_check function in app-factory decides whether an agent's evidence is trustworthy.
Prompt 2
Show me how to adapt agent_gate_demo.py's RunRegistry to verify my own agent's tool outputs.
Prompt 3
Help me write negative fixtures for my agent pipeline based on app-factory's NEGATIVE_FIXTURES pattern.
Prompt 4
Walk me through building a three step bounded agent chain using app-factory's loop skeleton in snippets/loop.py.

Frequently asked questions

wtf is app-factory?

Shares a runnable verification gate that catches AI agents from falsely claiming success, one reusable piece of a larger iOS app-building pipeline.

What language is app-factory written in?

Mainly Python. The stack also includes Python.

What license does app-factory use?

Use freely for any purpose, including commercial use, as long as you keep the copyright notice.

How hard is app-factory to set up?

Setup difficulty is rated easy, with roughly 5min to a first successful run.

Who is app-factory for?

Mainly developer.

View the repo → Decode another repo

This repo across BitVibe Labs

Don't trust strangers blindly. Verify against the repo.