• Senior Engineer's Blog on February 21, 2025: In the early days, new engineers had to use the Antithesis platform to beat an NES game. Finding bugs in production code is similar to beating NES games. NES games have advanced data visualization software, while exploring bugs in distributed systems is more challenging. Different software state spaces have different shapes, like the Unicode converter with a rare off-by-one bug and the 49-day crash in Windows 95.
• Gradius: A 1985 NES side-scrolling shooter where players control a spaceship. Enemies drop power orbs for ship upgrades. Beating the game requires reflex, memorization, and upgrade strategy. The Antithesis platform was used to beat Gradius with a two-component approach: tactics generate new inputs, and strategies judge states. Random bytes and random "deltas" tactics were used to solve the sinking ship problem and reach the first boss. Later, a simple strategy of maximizing depth without dying was used, ignoring powerups and other details. The agent's gameplay was different from a human's due to its limited knowledge.
• Seeing like a fuzzer: Gradius is an autoscroller, similar to the 49-day bug in Windows 95. A simple strategy of maximizing depth without dying was discovered. "Depth" means time since console power-on, and "dying" was defined by tracking memory locations. The agent's simplicity enables it to test complex programs with a minimal understanding. This is called "tree fuzzing" and allows for incremental progress.
• Buy somethin’ will ya!: In 2020 during stealth with a single beta customer, Gradius was being fuzzed. Today, many customers onboard and start finding bugs within 2-3 weeks with a simple strategy. The Antithesis platform is effective even with a basic approach.
• Technical addendum: Pauses were unexpected. Tactics generate sequences of inputs called "rollouts" and are sent to the system under test. Pipelining improves performance, and correlated or structured input sequences help overcome local maxima. Holding down the A button in Super Mario Bros. is an example of an action expressed as a sequence of inputs.