The NetHack Lunar Learning Bug: A Case for its Memorialization

Honorable members of the Legendary Computer Bugs Tribunal, I stand before you today to present a most curious case, a bug of such subtle absurdity that it could only arise from the chaotic intersection of human ingenuity and the cold, unfeeling logic of machines. I speak, of course, about the NetHack Lunar Learning Bug, a contender for a place in your hallowed halls of infamy.

Setting the Stage: Where Dungeons Meet Data

Imagine, if you will, a dungeon so deep and so complex that every time you enter, it’s different. This is NetHack, a roguelike game that’s notorious for its unforgiving difficulty and its reliance on player skill and knowledge. Unlike those newfangled games with their fancy graphics and hand-holding tutorials, NetHack is old school. You die, you start over. No save points, no rewinds. It’s you, your wits, and a whole lotta ASCII characters.

Now, you might be wondering, what does a game like NetHack have to do with machine learning? Well, it turns out that the very things that make NetHack so challenging for humans—procedural generation, permadeath, and a vast, intricate ruleset—make it a fascinating playground for AI. Specifically, a type of machine learning called imitation learning, where an AI learns by observing and mimicking expert behavior, is a natural fit for a game like NetHack. Enter Jens Tuyls, a researcher who created a model that captured the essence of expert NetHack gameplay. His work caught the eye of two other researchers, Bartłomiej Cupiał and Maciej Wołczyk, who decided to take it a step further. They fed Tuyls’ model into a neural network, a type of AI that learns by analyzing vast amounts of data, with the goal of creating an AI capable of not just playing NetHack, but actually improving at it. And to their delight, it worked! The neural network, fueled by Tuyls’ model, started racking up points, navigating the treacherous dungeons of NetHack with increasing proficiency.

The Anomaly: When the Code Went Haywire (or Did It?)

Things were going swimmingly. The AI was learning, getting better with each passing game. Scores were climbing steadily, a testament to the power of machine learning. Then, out of nowhere, disaster struck. The AI’s performance plummeted, like a lead weight tossed down a bottomless pit. It was as if the AI had suddenly forgotten everything it had learned, reduced to a bumbling fool stumbling around in the digital dark. The researchers were baffled. What could have caused such a sudden and dramatic decline? They checked the code, scoured the logs, even went as far as restoring the entire software stack to a point in time before the issue occurred. They tried everything they could think of, but the problem persisted, mocking their every attempt to understand it. The mystery deepened, casting a long shadow over the entire project.

Imagine their frustration. It’s like baking a cake, nailing it every time, and then one day, bam! It’s a soggy mess. You follow the recipe to a T, same ingredients, same oven, but the result is an abomination. That’s what these researchers were dealing with, except instead of flour and sugar, they were wrestling with lines of code and algorithms.

A Moment of Desperation and a Glimmer of Hope

Cupiał, bless his tenacious little heart, was at his wit’s end. He’d pulled all-nighters, consumed enough caffeine to fuel a small rocket, and even resorted to sacrificing rubber duckies to the code gods (hey, desperate times…). He took to the digital airwaves, pouring his frustration into a series of increasingly frantic X (you know, the one that used to be Twitter) posts. Think of it as a digital SOS, a plea for help echoing through the vast expanse of the internet.

And who should answer that call but Jens Tuyls himself, the OG NetHack model maestro. You can almost picture him, leaning back in his chair, digital smoke curling from his ears after reading Cupiał’s pleas. His response? Well, it was about as unexpected as a unicorn riding a unicycle while juggling chainsaws: “Oh yes, it’s prob’ly a full moon today.”

Unmasking the Culprit: Blame It on the Moon

Now, for those unfamiliar with the finer points of NetHack, let me drop some knowledge. See, the game has this quirky little feature where it checks your computer’s system clock. And if it just so happens to be a full moon, well, the game throws you a bone. You get a message announcing the lunar event, and your character gets a temporary stat boost. Think of it as the game’s way of saying, “Hey, go nuts for a bit.”

But here’s where things get really weird. You’d think an easier game would mean higher scores, right? Well, not for our AI friend. It turns out, the full moon was messing with its digital mind. See, the AI had been trained on data from games played without the full moon bonus. So, when the full moon rolled around in the real world, and the game got a little bit easier, the AI didn’t know what to do. It was like handing a calculator a paintbrush and expecting a masterpiece. It just wasn’t equipped to handle it.

And the kicker? Cupiał checked. The drop in the AI’s performance coincided perfectly with a real-world full moon in Kraków, where the research was being conducted. Talk about a face-palm moment. All that debugging, all that head-scratching, and the culprit was a giant, glowing rock in space.

Redefining Success: It’s Not About the Points, It’s About the Journey (and Maybe Avoiding Werewolves)

The NetHack Lunar Learning Bug, as it came to be known, forced a serious rethink of what it means for an AI to be “good” at NetHack. It showed that simply chasing high scores, while impressive, doesn’t necessarily equate to true mastery of the game. It’s like judging a chef solely on how much food they can pile onto a plate, without considering taste or presentation. Sure, it’s impressive in a gluttonous kind of way, but it’s not exactly Michelin-star material.

Take, for example, the AutoAscend neural network. Unlike the score-obsessed AI, AutoAscend is all about progression. It’s not interested in racking up points; it wants to beat the game, to delve into the deepest dungeons and emerge victorious. And you know what? It’s actually quite good at it. But even AutoAscend, with its focus on game progression, stumbles when it comes to replicating the nuanced decision-making of a human player. See, NetHack is a game of emergent complexity. There are so many variables, so many possible interactions, that even the most sophisticated AI struggles to grasp the full depth of the game.