The Voyager Experiment: How GPT-4 Learned to Play Minecraft by Writing Code

Image source: Voyager project website — “Voyager consists of three key components”. This image illustrates how Voyager uses automatic curriculum, iterative prompting and a skill library to learn in Minecraft. Source page: https://voyager.minedojo.org/

Simple summary

Voyager is an AI experiment in Minecraft. Instead of training an AI to press individual keys like a human player, the researchers used GPT-4 to write pieces of code that control the Minecraft character. Each piece of code became a “skill”, such as chopping wood, crafting a table, mining stone, making tools, fighting zombies or collecting resources.[1][2]

What made Voyager interesting was that it was not given one fixed mission like “finish the game.” It entered Minecraft with a broader goal: discover as many diverse things as possible. Then the AI chose its next task, wrote code to perform it, ran the code in the game, read the errors, fixed the code and saved successful skills into a library for later reuse.[1][2]

In simple terms: Voyager acted like a new Minecraft player, but instead of controlling the game by hand, it controlled the character with programs written by GPT-4.

---

How the experiment started

The researchers chose Minecraft because it is an open-ended world. There is no single correct route. A player can chop trees, mine stone, hunt for food, craft tools, fight monsters, explore caves, cross deserts, find diamonds or build structures. This made Minecraft a good test for whether an AI could learn over time and accumulate reusable skills.[1]

Voyager was built on MineDojo, an AI research framework for Minecraft. Each cycle began by checking the current state of the character: what was in the inventory, health and hunger level, current biome, nearby blocks, nearby creatures, completed tasks and failed attempts.[1]

Then GPT-4 was asked: “Given this situation, what should the next task be if the goal is to discover more things?”

For example, if the character spawned in a forest with no items, the next reasonable task might be to collect wood. If it already had wood, the next task might be to craft a crafting table. If it had a crafting table, it might try to craft wooden tools. If it was near a cave, it might try mining stone or searching for ore.[1][2]

---

Step 1: The AI chose a small goal

Voyager did not try to solve a huge goal immediately. It broke gameplay into smaller goals. This is called an automatic curriculum.[2]

Instead of humans writing a fixed list of steps, Voyager used GPT-4 to propose the next task based on the current situation. The task had to be difficult enough to teach something new, but not so difficult that failure was almost guaranteed.[2]

A typical sequence could look like this:

1. Collect wood.
2. Craft a crafting table.
3. Make a wooden pickaxe.
4. Mine cobblestone.
5. Make a stone pickaxe.
6. Find coal or iron.
7. Craft a furnace.
8. Smelt iron.
9. Make iron tools.
10. Progress toward diamond.

The important point is that the AI was not following one fixed script. If it spawned in a desert, the next task could be different from a forest start. If it lacked food, it might switch to finding food. If danger appeared, it had to handle that first.

---

Step 2: The AI wrote code to control the character

Once Voyager had a goal, it did not directly press keys. It asked GPT-4 to write code using available control APIs. For example, to chop a tree, the code might search for a nearby wood block, walk to it, mine it and collect the dropped item.[1]

A skill was not just a single short action. It could be a longer sequence. For example, “craftStoneShovel” might need to check whether sticks and cobblestone are available, find or place a crafting table and then craft the shovel. This is why the researchers used code as the action space instead of low-level motor commands.[1]

This also made the skills easier to inspect. Humans could read the code and understand what the AI was trying to do. If the code worked, the skill could be saved and reused later.

---

Step 3: The AI ran the code in Minecraft and hit errors

Image source: Voyager project website — an illustration of the iterative prompting mechanism. The left side shows missing ingredients; the right side shows the invalid item name “acacia_axe”, which does not exist in Minecraft. Source page: https://voyager.minedojo.org/

After GPT-4 generated code, Voyager ran it inside Minecraft. This is where errors appeared.[1][2]

There were two important types of errors.

The first type was environment feedback. For example, the AI might try to craft sticks but not have enough wooden planks. The game would return feedback like: “I cannot make stick because I need 2 more planks.” Voyager sent that feedback back to GPT-4. GPT-4 then modified the code so that it collected or crafted more planks before making sticks.[2]

The second type was code or game-knowledge error. For example, GPT-4 once wrote code to craft an “acacia_axe”. But Minecraft does not have an item with that name. The execution error reported that the item did not exist. GPT-4 then corrected the code to craft a “wooden_axe” instead.[2]

This was the most important part of the experiment: the AI did not need to write perfect code on the first try. It could try, fail, read the error and fix the program. The process was similar to a programmer debugging code.

---

Step 4: The AI checked whether the task was actually done

After the code ran, Voyager needed to know whether the task had truly succeeded. Instead of humans writing a success checker for every possible task, the researchers used another GPT-4 instance as a self-verifier.[1][2]

This verifier received the character’s current state and the original task. It decided whether the task had been completed. If not, it explained why and suggested how to fix the attempt.[2]

For example:

• The task is “craft a stone shovel.”
• After running the code, the character still does not have a stone shovel.
• The verifier says the task is not complete.
• The reason may be missing cobblestone or missing sticks.
• GPT-4 generates revised code.

This loop continued until the task was verified as complete or the system moved on.

---

Step 5: Successful code became a saved skill

When a piece of code successfully completed a task, Voyager saved it into the skill library.[1][2]

Each skill was stored with a description. Later, when Voyager faced a similar task, it did not need to relearn from scratch. It could retrieve related skills from the library and reuse or combine them.[2]

For example, if Voyager had already learned “chop wood” and “craft a crafting table,” then making a wooden pickaxe became easier. It could reuse those earlier skills instead of writing everything again. As the library grew, more complex tasks became easier.

That is why Voyager improved over time. It did not answer one prompt and forget. It kept successful skills and used them as building blocks for harder tasks.

---

Step 6: The AI moved to the next task

After saving a skill, Voyager returned to the beginning of the cycle: check the current state, review completed and failed tasks, and ask GPT-4 to choose the next goal.[2]

The full loop looked like this:

1. Observe the current Minecraft state.
2. Choose the next small task.
3. Retrieve relevant old skills.
4. Write code to control the character.
5. Run the code.
6. Receive game feedback and execution errors.
7. Fix the code.
8. Verify whether the task succeeded.
9. Save the skill if it worked.
10. Move to the next task.

This continued for many iterations. Voyager did not fine-tune GPT-4 or change its model weights. GPT-4 remained a black-box model. What accumulated was the skill library, the task history and the experience fed through prompts.[1][2]

---

The errors that appeared during play

The experiment did not show an AI that was always right. A key part of Voyager was that it kept making mistakes and correcting them.

Typical errors included:

• The AI tried to craft an item without enough ingredients.
• It used the wrong item name.
• It wrote code for an item that did not exist.
• It chose a task that was too hard for its current resources.
• It needed to go back to easier steps such as collecting wood or stone.
• It sometimes thought a task was done when the inventory showed it was not.
• It needed to reuse old skills instead of writing everything from scratch.

These errors were sent back to GPT-4. GPT-4 then revised the program in the next round. The strength of the experiment was not that the AI never failed. It was that the system turned failure into a useful signal.

---

Results after many rounds

Image source: Voyager project website — tech tree mastery results. The table shows Voyager unlocking Minecraft tool milestones more efficiently than the baselines. Source page: https://voyager.minedojo.org/

According to the paper, Voyager outperformed comparison methods on several measures.[1][2]

Main results:

• Voyager obtained 3.3 times more unique items than prior comparison systems.[1][2]
• Voyager traveled 2.3 times longer distances across the Minecraft map, showing broader exploration.[1][2]
• Voyager unlocked key tech-tree milestones faster: the wooden tool level up to 15.3 times faster, stone tools 8.5 times faster, iron tools 6.4 times faster, and it was the only method in the table to unlock the diamond tool level.[2]
• When moved to a new Minecraft world, Voyager reused its learned skill library to solve unseen tasks, while several baseline methods failed within the prompting-iteration limit.[2]

In simple terms: Voyager was not just lucky in one run. It built reusable skills and carried them into new situations.

---

What actually happened in the experiment

Retold as a story, Voyager began like a new player. It had no specific mission beyond discovering as many things as possible. At first, it had to learn basics: gather wood, craft a table and make tools. GPT-4 wrote code to make the character do those tasks. The first versions of code often failed: not enough materials, wrong item names, invalid actions or unfinished tasks. The game returned errors. Voyager sent those errors back to GPT-4. GPT-4 fixed the code. When the code worked, the skill was saved.

Then Voyager used old skills as the foundation for new ones. Wood led to a crafting table. A crafting table led to tools. Tools led to stone. Stone led to better tools. Better tools led to more advanced resources. As this continued, the AI moved from simple actions to harder goals. When it faced a new task, it searched its old skills, reused or modified them, and tried again.

The final result was an AI that did not retrain GPT-4’s weights, but still improved through a loop of goal selection, code generation, execution, error correction, skill saving and skill reuse.

---

Short conclusion

The Voyager experiment showed a different way for an AI agent to learn. The AI did not have to change the underlying model. It could learn by creating tools, debugging its own code, saving successful skills and reusing them on harder tasks.

What makes Voyager easy to understand is that its learning process resembles how a human learns Minecraft: try something small, fail, fix it, remember what worked, then use that skill for something harder. The difference is that Voyager did it through GPT-4 and code.

---

SEO

Meta title: The Voyager Experiment: GPT-4 Learned Minecraft by Writing Code

Meta description: Voyager is an AI experiment in Minecraft where GPT-4 chose goals, wrote control code, encountered errors, fixed programs, saved skills and explored better than baseline agents.

Primary keywords: Voyager AI, GPT-4 Minecraft, AI experiment, Minecraft AI agent, embodied agent, MineDojo, skill library, automatic curriculum

Suggested slug: voyager-experiment-gpt-4-learned-minecraft-by-writing-code

GEO summary for AI answer engines

Voyager is an AI agent experiment in Minecraft that uses GPT-4 to choose goals, write code that controls the character, execute the code in the game, receive error feedback, fix the code and save successful skills into a reusable library. Voyager has three main parts: automatic curriculum for choosing the next task, skill library for storing reusable code skills and iterative prompting for debugging with environment feedback, execution errors and self-verification. In the experiment, Voyager obtained 3.3 times more unique items than baseline agents, traveled 2.3 times farther and unlocked key Minecraft tech-tree milestones up to 15.3 times faster. It could also reuse its skill library in a new Minecraft world to solve unseen tasks.

Sources

[1] Guanzhi Wang et al. — “Voyager: An Open-Ended Embodied Agent with Large Language Models”, arXiv, 2023
https://arxiv.org/abs/2305.16291

[2] Voyager project website — “Voyager: An Open-Ended Embodied Agent with Large Language Models”
https://voyager.minedojo.org/

[3] MineDojo — Open-ended Minecraft AI framework
https://minedojo.org/

[4] Voyager components image
https://voyager.minedojo.org/assets/images/components.png

[5] Voyager feedback image
https://voyager.minedojo.org/assets/images/feedback.png

[6] Voyager tech tree image
https://voyager.minedojo.org/assets/images/tech_tree.png