wtf is one-layer-deeper?

One Layer Deeper is a machine learning competition run by Core Automation and Tilde Research, along with the code needed to take part in it. The challenge asks participants to design the best possible neural network model and training setup for a fixed problem, while staying within a strict limit on how much memory the model can use and how long it can train on a single Nvidia H100 graphics card, which is a type of chip commonly used to train AI models. Entrants control the model's architecture, how deep it is, which optimizer trains it, how the learning rate changes over time, and the training loss function used to measure mistakes during training. The competition organizers, referred to as the evaluator, control everything else: the training data, the outer training loop, and how the final result is scored. A submission is a single Python file that defines a model builder function and an optimizer builder function, following a fixed contract so the evaluator can plug it into its own training pipeline automatically. There are three difficulty tiers. Easy gives 60 seconds of training time and allows many attempts per day, Medium gives 600 seconds with fewer attempts allowed, and Hard gives a full hour of training time but only one accepted attempt per day, using a private dataset that participants cannot inspect directly. The Easy and Medium tiers are meant for practice, while the public leaderboard only ranks each participant's best successful attempt on the Hard tier. The rules are detailed and strict. A model can hold at most 500 million parameters, roughly the individual numbers that make up what the model has learned, submissions cannot depend on files outside the one submitted file, and attempts to game the scoring system on the Hard tier result in an immediate ban. The project provides a command line tool, installed with a package manager called uv, that handles logging in through GitHub, validating a submission file, submitting it for evaluation, checking its status, and viewing the public leaderboard. A basic example submission using a standard Transformer model and the AdamW optimizer is included as a starting point.