so how is everyone putting spiderman into gta 5 with claude?

These videos of people using claude to somehow get spiderman working inside gta 5 are everywhere and the first thing you want to know is, how the hell is this even happening? Is claude rebuilding gta 5 from scratch? Is it somehow reading the whole game and then writing another version of it?

The answer to that is much more fun.

It's not recreating gta 5 at all. What people are doing is giving the ai tools developers and reverse engineers use all the time, and then letting it reverse the game enough to then alter it.

Gta 5 is obviously an enormous program, and you don't have the source code there and slam it open in a text editor to see a line of code that says player.position = x. What you have is a game that has been compiled down, essentially as a bunch of machine code, inside something you recognize as gta5.exe. If you open this in an application that can do some reverse engineering, it's going to seem largely like a bunch of gibberish compared to code you're familiar with.

There will be instructions that move a value from one place to another in memory, and copy something from one CPU register to another, but without telling you this is the position of the player or this is the function that creates a character.

You've got to figure that out.

That's where this reverse engineering part comes in. A human or an ai can take the help of all tools like ghidra, debuggers, and other analysis tools which can dissect the executable and identify that what different parts of the program is doing. You can look for specific strings in the binary and see which functions reference those strings, you can track functions which call other functions and see what's happening in memory while the game runs.

Before long you start forming an idea of what's going on inside.

You may find out that one part of the structure is a car or that one number changes when your health changes etc… You are basically reverse engineering the final product back to the bits that matter to you.

But there's one huge advantage here that many people overlook. The community already had the benefit of years worth of work into this area. There are thousands of existing gta native functions and a huge amount of documentation for the game.

So claude isn't trying to discover a gta 5 executable from scratch and then get told to understand what's in there.

It can leverage existing knowledge of the game and modding interfaces.

Say you want to make spiderman swing from a building. You don't need to understand all of the physics engine of gta 5, you just need to know how to detect where the player is looking, how to detect a point on a building, how to move a player, and apply some sort of swinging physics. Gta already has a physics engine!

It already handles collisions, character movement, animations, cameras.

You just need to take those existing systems and connect them together in a way that make spiderman swing.

So claude could generate code that identifies when you push a button, throws a ray out from the digital camera into the environment, and determines where it intersects a building. Now you've got the landing point for the web. Once you have both the item's coordinates and the hitting point, you calculate the distance and use that for something to move, and then play an animation and draw the web.

Nothing of that necessitates developing a new game engine.

Gta is already doing most of the hardwork.

Then there's the character sprite, which is a further piece of the puzzle. A mod will have a spiderman model, textures, animation and skeleton. Gta already have means to draw a character in the world and how to run the animations so yet again you are just adding your character into an already existing framework. This explains why the phrase 'claude put spiderman into gta' sounds a lot more impressive than it is.

The more interesting bit to me is what happens when the ai doesn't know how something works. This is where it begins to resemble real reverse engineering. Let's say claude needs to fix something in the game but there isn't an obvious documented function that does that thing, it can begin to look into the executable. It might find a chunk of code that seems appropriate, follow the references to that code and look at memory while the game is running, seeing if it can find things that change when you, say, get into a car.

It's essentially detective work. You take a enormous compiled program in which you don't understand most of the code, then gradually build up an understanding of the small part you are interested in. You don't need to reverse every part of gta 5 in order to write one feature. The aim of spider man web swinging is to enable spiderman web swinging.

And that is where ai gets much more interesting. Claude can generate the code, compile it and actually run the game. If it blows up in flames, it can look at what happened.

If spiderman shows up but sinks into the floor, it can troubleshoot the collision issue.

If the web attaches to the wrong thing, it can adjust the calculation and then try again. Instead of the ai process you usually get where you just ask the chatbot for code and then copy and paste it into your project, you can get an agent that is actually working through the problem and testing its work.

The loop is; research, code, compile, play the game, find out what happened, work out what went wrong and then start again. That sounds easy but it really isn't, it is an enormous change. Much of software engineering involves not designing perfect code the first time around but rather making a hunch, testing that hunch and then changing it when reality beats you up. Giving an ai access to the environment where that happens is invaluable.

Another myth is that when people are talking about claude "reverse engineered gta 5" they imply that claude understood everything about the game. That's not necessarily true. Claudes might understand a handful of the game's code deep enough to do something useful.

It might be able to find a specific object in memory, understand a specific function, or call some of the game's internal code.

That small bit of knowledge might be enough to build something impressive without needing to understand the millions of other things going on in the game.

And I honestly believe that that is what makes these demos so compelling. Because the ai doesn't have to rebuild gta. Gta is there. It has the world, the graphics, the physics, the animation system and everything else.

The ai just needs to understand enough of that machine to start controlling it.

And that's a really important distinction because it changes how you should think about these demos. You aren't watching an ai build a video game. You're watching an ai learn how to interact with a video game that has already been built.

So when you watch someone tell claude to make spiderman work inside gta 5, what you're really seeing is years of existing gta reverse engineering, modding tools, known game functions, new code generated by the ai, assets and then a lot of trial and error all coming together.

Claude isn't downloading gta into its brain and suddenly understanding how rockstar built the entire game. It's much closer to having a very fast programmer sitting next to a running copy of gta, with access to all the tools needed to investigate it.

And that's probably the more interesting future here. The crazy part isn't that ai can write a spiderman mod. We already know ai can write code. The crazy part is that an ai agent can potentially inspect a complicated piece of software, form a theory about how something works, write a small program to test that theory, see what happens and then keep going until it gets the result it wants.

At that point you're not really just asking an ai to write code anymore. You're giving it a complicated machine and asking it to figure out how the machine works well enough to change it.

And gta 5 just happens to be a very good example of what that looks like.