wp-plugin-hostgator
domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init
action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home4/scienrds/scienceandnerds/wp-includes/functions.php on line 6114ol-scrapes
domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init
action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home4/scienrds/scienceandnerds/wp-includes/functions.php on line 6114Source:https:\/\/techcrunch.com\/2023\/05\/11\/ai2-is-developing-a-large-language-model-optimized-for-science\/<\/a><\/br> PaLM 2<\/a>. GPT-4<\/a>. The list of text-generating AI practically grows by the day.<\/p>\n Most of these models are walled behind APIs, making it impossible for researchers to see exactly what makes them tick. But increasingly, community efforts are yielding open source AI that\u2019s as sophisticated, if not more so, than their commercial counterparts.<\/p>\n The latest of these efforts is the Open Language Model, a large language model set to be released by the nonprofit Allen Institute for AI Research (AI2) sometime in 2024. Open Language Model, or OLMo for short, is being developed in collaboration with AMD and the Large Unified Modern Infrastructure consortium, which provides supercomputing power for training and education, as well as Surge AI and MosaicML (which are providing data and training code).<\/p>\n \u201cThe research and technology communities need access to open language models to advance this science,\u201d Hanna Hajishirzi, the senior director of NLP research at AI2, told TechCrunch in an email interview. \u201cWith OLMo, we are working to close the gap between public and private research capabilities and knowledge by building a competitive language model.\u201d<\/p>\n One might wonder \u2014 including this reporter \u2014 why AI2 felt the need to develop an open language model when there\u2019s already several to choose from (see Bloom<\/a>, Meta\u2019s LLaMA<\/a>, etc.). The way Hajishirzi sees it, while the open source releases to date have been valuable and even boundary-pushing, they\u2019ve missed the mark in various ways.<\/p>\n AI2 sees OLMo as a platform, not just a model \u2014 one that\u2019ll allow the research community to take each component AI2 creates and either use it themselves or seek to improve it. Everything AI2 makes for OLMo will be openly available, Hajishirzi says, including a public demo, training dataset and API, and documented with \u201cvery limited\u201d exceptions under \u201csuitable\u201d licensing.<\/p>\n \u201cWe\u2019re building OLMo to create greater access for the AI research community to work directly on language models,\u201d Hajishirzi said. \u201cWe believe the broad availability of all aspects of OLMo will enable the research community to take what we are creating and work to improve it. Our ultimate goal is to collaboratively build the best open language model in the world.\u201d<\/p>\n OLMo\u2019s other differentiator, according to Noah Smith, senior director of NLP research at AI2, is a focus on enabling the model to better leverage and understand textbooks and academic papers as opposed to, say, code. There\u2019s been other attempts at this, like Meta\u2019s infamous Galactica<\/a> model. But Hajishirzi believes that AI2\u2019s work in academia and the tools it\u2019s developed for research, like Semantic Scholar, will help make OLMo \u201cuniquely suited\u201d for scientific and academic applications.<\/p>\n \u201cWe believe OLMo has the potential to be something really special in the field, especially in a landscape where many are rushing to cash in on interest in generative AI models,\u201d Smith said. \u201cAI2\u2019s unique ability to act as third-party experts gives us an opportunity to work not only with our own world-class expertise but collaborate with the strongest minds in the industry. As a result, we think our rigorous, documented approach will set the stage for building the next generation of safe, effective AI technologies.\u201d<\/p>\n That\u2019s a nice sentiment, to be sure. But what about the thorny ethical and legal issues around training \u2014 and releasing \u2014 generative AI? The debates raging around the rights of content owners (among other affected stakeholders), and countless nagging issues, have yet to be settled in the courts.<\/p>\n To allay concerns, the OLMo team plans to work with AI2\u2019s legal department and to-be-determined outside experts, stopping at \u201ccheckpoints\u201d in the model-building process to reassess privacy and intellectual property rights issues.<\/p>\n \u201cWe hope that through an open and transparent dialogue about the model and its intended use, we can better understand how to mitigate bias, toxicity, and shine a light on outstanding research questions within the community, ultimately resulting in one of the strongest models available,\u201d Smith said. What about the potential for misuse? Models, which are often toxic and biased to begin with, are ripe for bad actors intent on spreading disinformation and generating malicious code.<\/p>\n Hajishirzi said that AI2 will use a combination of licensing, model design and selective access to the underlying components to \u201cmaximize the scientific benefits while reducing the risk of harmful use.\u201d To guide policy, OLMo has an ethics review committee with internal and external advisors (AI2 wouldn\u2019t say who, exactly) that\u2019ll provide feedback throughout the model creation process.<\/p>\n We\u2019ll see to what extent that makes a difference. For now, a lot\u2019s up in the air \u2014 including most of the model\u2019s technical specs. (AI2 did reveal that it\u2019ll have around 70 billion parameters, parameters being the parts of the model learned from historical training data.) Training\u2019s set to begin on LUMI\u2019s supercomputer in Finland \u2014 the fastest supercomputer in Europe, as of January \u2014 in the coming months.<\/p>\n AI2 is inviting collaborators to help contribute to \u2014 and critique \u2014 the model development process. Those interested can contact the OLMo project organizers here<\/a>.\u00a0<\/span><\/p>\n<\/p><\/div>\n <\/br><\/br><\/br><\/p>\n
\nAI2 is developing a large language model optimized for science<\/br>
\n2023-05-11 22:01:45<\/br><\/p>\n
<\/span><\/p>\n