In order to deeply understand and model people\u2019s preferences, our recommendation models can have tens of trillions of parameters \u2014 orders of magnitude larger than even the biggest language models used today.<\/p>\n<\/blockquote>\n

I pressed Meta to get a little more specific about these theoretical tens-of-trillions models, and that\u2019s just what they are: theoretical. In a clarifying statement, the company said, \u201cWe believe our recommendation models have the potential to reach tens of trillions of parameters.\u201d This phrasing is a bit like saying your burgers \u201ccan\u201d have 16-ounce patties but then admitting they\u2019re still at the quarter-pounder stage. Nevertheless the company clearly states that it aims to \u201censure that these very large models can be trained and deployed efficiently at scale.\u201d<\/p>\n

Would a company build costly infrastructure for software it doesn\u2019t intend to create \u2014 or use? It seems unlikely, but Meta declined to confirm (though nor did they deny) that they are actively pursuing models of this size. The implications are clear, so while we can\u2019t treat this tens-of-trillions scale model as extant, we can<\/em> treat it as genuinely aspirational and likely in the works.<\/p>\n

\u201cUnderstand and model people\u2019s preferences,\u201d by the way, must be understood to mean behavior analysis of users. Your actual preferences could probably be represented by a plaintext list a hundred words long. It can be hard to understand, at a fundamental level, why you would need a model this large and complex to handle recommendations even for a couple billion users.<\/p>\n

The truth is the problem space is indeed huge: There are billions and billions of pieces of content all with attendant metadata, and no doubt all kinds of complex vectors showing that people who follow Patagonia also tend to donate to the World Wildlife Federation, buy increasingly expensive bird feeders and so on. So maybe it isn\u2019t so surprising that a model trained on all this data would be quite large. But \u201corders of magnitude larger\u201d than even the biggest out there, something trained on practically every written work accessible?<\/p>\n

There isn\u2019t a reliable parameter count on GPT-4, and leaders in the AI world have also found that it\u2019s a reductive measure of performance, but ChatGPT is at around 175 billion and GPT-4 is believed to be higher than that but lower than the wild 100 trillion claims. Even if Meta is exaggerating a bit, this is still scary big.<\/p>\n

Think of it: An AI model as large or larger than any yet created\u2026 what goes in one end is every single action you take on Meta\u2019s platforms, what comes out the other is a prediction of what you will do or like next. Kind of creepy, isn\u2019t it?<\/p>\n

Of course they\u2019re not the only ones doing this. TikTok led the charge in algorithmic tracking and recommendation, and has built its social media empire on its addictive feed of \u201crelevant\u201d content meant to keep you scrolling until your eyes hurt. Its competitors are openly envious.<\/p>\n

Meta is clearly aiming to blind advertisers with science, both with the stated ambition to create the biggest model on the block, and with passages like the following:<\/p>\n