Dehaene favored the second experiment, which also involved the comprehensive decoding of brain patterns. Test subjects would be randomly exposed to faces and objects flashed on a screen while they played a distracting Tetris-like video game. Shortly after an image was shown, the game would stop and the subject would be asked whether they saw it. Dehaene preferred this design because it offered a more clear-cut contrast between conscious and unconscious mental states, which he considered essential to getting unambiguous data on the correlates of consciousness.

Because Kahneman was so familiar with adversarial collaborations, he mentored the three project leaders. But he also warned them that, in his experience, adversaries don’t change their minds after seeing the results of their collaborations. Instead, when faced with an inconvenient result, “their IQ leaps 15 points” as they invent ways to accommodate the new, conflicting data, he said.

Mixed Results, With No Losers

The researchers set to work performing the experiments suggested by the workshop team. The GNWT-versus-IIT experiment that Tononi liked best, which tested with different levels of tasks, finished up first. It was carried out in two different labs using fMRI, MEG and intracranial electroencephalography. In all, six theory-neutral labs and 250 test subjects participated.

On the evening of June 23, an excited audience gathered at NYU to learn the outcome of that experiment. Writ large on a giant screen, the results were shown on a chart marked by red and green highlights, as though the researchers were reporting on a steeplechase with three types of hurdles.

The first hurdle checked how well each theory decoded the categories of the objects that the subjects saw in the presented images. Both theories performed well here, but IIT was better at identifying the orientation of objects.

The second hurdle tested the timing of the signals. IIT predicted sustained, synchronous firing in the hot zone for the duration of the conscious state. While the signal was sustained, it did not remain synchronous. GNWT predicted an “ignition” of the workspace followed by a second spike when the stimulus disappeared. Only the initial spike was detected. In the on-screen scoring for the NYU audience, IIT pulled ahead.

The third hurdle concerned overall connectivity across the brain. GNWT scored better than IIT here, largely because some analyses of the results supported GNWT predictions while the signals across the hot zone were not synchronous.

Both theories were challenged by the results. But in the final tally on screen at the event, IIT scored more green highlights than GNWT, and the audience responded as though a victor had been crowned. Melanie Boly of the University of Wisconsin, Madison, a supporter of IIT, was buoyed enough by the outcome to declare onstage: “The results corroborate IIT’s overall claim that posterior cortical areas are sufficient for consciousness, and neither the involvement of [the prefrontal cortex] nor global broadcasting are necessary.”

When Dehaene took to the stage, he did not admit defeat either. “I’ve decided to follow the advice of Dan Kahneman,” he quipped. He professed to be happy because “the most interesting part of this experiment was the task-irrelevant stimuli.” The question was whether they would indicate the ignition of a conscious percept in the frontal brain. “The answer is yes!” he said.

Later, Dehaene suggested to me that the hurdles for IIT were set lower than those for his theory. “There was no real test of the complex mathematical core of [IIT],” he said. And as Block noted in his remarks that night, the finding that there was support for the back-of-the-brain theories does not specifically support IIT.

Notwithstanding the slightly higher number of green marks scored by IIT, the project leaders themselves are adamant that there was no winner. “These results confirm some predictions of IIT and GNWT, while substantially challenging both theories,” they wrote in a paper describing the results posted on the biorxiv.org preprint server.

Just as Kahneman predicted, the adversaries explained away the discrepancies. Boly argued that the failure to detect sustained synchrony in the hot zone “may be due to sampling limitations.” Dehaene suspected that no “off” signal was detected because the subjects allowed their minds to wander. “My claim is that consciousness became decoupled from the stimulus,” he said.

Boly and Dehaene now await the results of the second experiment, involving the Tetris-like game distraction. Those results won’t be available until next year.

The Pace of Progress

So has science been advanced? Not everyone thinks so.

Some researchers, such as Olivia Carter, a psychologist at the University of Melbourne and past president of the ASSC, think the two theories were too far apart for their predictions to be meaningfully compared. “My personal feeling is they are testing totally different things,” she said. “IIT is focusing on phenomenal content, and GNWT is much more interested in working memory and attention.”

That assessment seems apt. Yet it’s also frustrating, given that a dispositive comparison was the stated purpose of the adversarial collaboration in the first place. If it’s a victory for science, it seems like a qualified one.

The Monash University philosopher Jakob Hohwy, who is part of another Templeton-funded adversarial collaboration, sees it differently. “This goes to the philosophy of science,” he said. He points out the field is still divided over such fundamentals as the definition of consciousness, whether it is closer to thinking or feeling, and even whether self-reported results truly confound the data. For Hohwy, this kind of collaborative effort is the way to move forward. “We will find out as we go along in exactly this type of adversarial collaboration,” he said.

Others, like the computational neuroscientist Megan Peters of the University of California, Irvine, bristled at media coverage that reported the results as a two-horse race between GNWT and IIT rather than a field with multiple contenders. Instead of focusing on winners and losers, Peters said, it’s important to see that science advances by learning from each experimental hurdle. (Having attended the proceedings that night, however, I can attest that the event was arranged to resemble a sporting event.)

Still, Peters remains a fan of adversarial collaborations. During the Covid-19 lockdown, she was inspired by the Templeton process to help organize a series of workshops hosted by the Cognitive Computational Neuroscience conference. In these “generative adversarial collaborations,” researchers engaged in robust debate. “Watching the teams chew on stuff was instructive,” she said.

The first adversarial collaboration on consciousness may not have succeeded in winnowing out any theories from the field. But it did force theorists to make more tangible predictions, and it made experimentalists work out new techniques. “The findings of the collaboration remain extremely valuable,” wrote the University of Sussex neuroscientist Anil Seth in a commentary after the June event. “They will push forward the development of both IIT and [GNWT] — and other theories of consciousness, too — by providing new constraints and new explanatory targets.”

For Melloni, the fact that the adversaries have not changed their minds does not detract from the value of the process. “As Kahneman says, people don’t change their mind, yet the way they react to the challenges makes their theory progress or degenerate,” she said. “If the latter, [then] over time the theory ‘dies’ and scientists abandon it.”

Corrections added August 24, 2023:
A missing paragraph describing the adversarial collaborations arising from the workshop, including one for testing HOTs and first order theories, was restored. Also, some details of the descriptions of the experiments that the adversarial collaboration developed for testing GNWT against IIT were clarified.