File this one under inevitable but hilarious. Mechanical Turk is a service that from its earliest days seemed to invite shenanigans, and indeed researchers show that nearly half of its \u201cturkers\u201d appear to be using AI to do tasks that were specifically intended to be done by humans because AI couldn\u2019t. We\u2019ve closed the loop on this one; great job everybody!<\/p>\n

Amazon\u2019s Mechanical Turk let users divide simple tasks into any number of small subtasks that take only a few seconds to do, and which pay pennies \u2014 but dedicated piecemeal workers would perform thousands and thereby earn a modest but reliable wage. It was, as Jeff Bezos memorably put it back then, \u201cartificial artificial intelligence.\u201d<\/p>\n

These were usually tasks that were then difficult to automate \u2014 like a CAPTCHA, or identifying the sentiment of a sentence, or a simple \u201cdraw a circle around the cat in this image,\u201d things that people could do quickly and reliably. It was used liberally by people labeling relatively complex data and researchers aiming to get human evaluations or decisions at scale.<\/p>\n