Directing the next action, and adding skills to the pile. Image Credits:<\/strong> MineDojo<\/p>\n<\/div>\n

For instance, night falls and those skeletons come out. The agent has a general idea of this, but it asks itself, What would a good player of this game do when there are monsters nearby? Well, GPT-4 says, if you want to explore the world safely, you\u2019ll want to make and equip a sword, then whack the skeleton with it while avoiding getting hit. And that general sense of what to do gets translated to concrete goals: collect stone and wood, build a sword at the crafting table, equip it, and fight a skeleton.<\/p>\n

Once it\u2019s done those things, they\u2019re entered into a general skill library so that later, when the task is \u201cgo deep into a cave to find iron ore,\u201d it doesn\u2019t have to learn to fight again from scratch. It does still use GPT, but it uses the cheaper and faster GPT-3.5, which tells it the skills most relevant to a given situation \u2014 so it doesn\u2019t try to mine the skeleton and fight the ore.<\/p>\n

It\u2019s similar to an agent like Auto-GPT that, when faced with an interface it doesn\u2019t know yet, has to teach itself to navigate it in order to accomplish its goal. But Minecraft is a much deeper environment than it is used to solving for, so a specialty agent like Voyager does far better. It finds more stuff, learns more skills, and explores a much greater area than the other bots.<\/p>\n

Interestingly but perhaps not surprisingly, GPT-4 wipes the floor with GPT-3.5 (i.e., ChatGPT) when it comes to generating useful code. A test replacing the former with the latter had the agent hit a wall early on, perhaps even literally, and fail to improve. It may not be obvious from talking to the two models that one is much smarter, but the truth is you don\u2019t have to be particularly smart to carry on an apparently intelligent conversation (ask me how I know). Coding is much more difficult and GPT-4 was a big update there.<\/p>\n

The point of this research isn\u2019t to obsolete Minecraft players but to find methods by which relatively simple AI models can improve themselves based on their \u201cexperiences,\u201d for lack of a better word. If we\u2019re going to have robots helping us in our homes, hospitals, and offices, they will need to learn and apply those lessons to future actions.<\/p>\n