\nMicrosoft doubles down on AI with new Bing features<\/br>
\n2023-05-04 22:23:52<\/br><\/p>\n
Microsoft is embarking<\/span> on the next phase of Bing\u2019s expansion. And \u2014 no surprise \u2014 it heavily revolves around AI.<\/p>\n At a preview event this week in New York City, Microsoft execs including Yusuf Mehdi, the CVP and consumer chief marketing officer, gave members of the press, including this reporter, a look at the range of features heading to Bing over the next few days, weeks and months.<\/p>\n They don\u2019t so much reinvent the wheel as they build on what Microsoft has injected into the Bing experience over the past three months or so. Since launching Bing Chat, its AI-powered chatbot powered by OpenAI\u2019s GPT-4<\/a> and DALL-E 2<\/a> models, Microsoft says that visitors to Bing \u2014 which has grown to exceed 100 million daily active users \u2014 have engaged in over half a billion chats and created more than 200 million images.<\/p>\n Looking ahead, Bing will become more visual, thanks to more image- and graphic-centric answers in Bing Chat. It\u2019ll also become more personalized, with capabilities that\u2019ll allow users to export their Bing Chat histories and draw in content from third-party plugins (more on those later). And it\u2019ll embrace multimodality, at least in the sense that Bing Chat will be able to answer questions within the context of images.<\/p>\n \u201cI think it\u2019s safe to say that we\u2019re underway with the transformation of search,\u201d Mehdi said in prepared remarks. \u201cIn our minds, we think that today will be the start of the next generation of this \u2018search mission.’\u201d<\/p>\n As of today, the new Bing \u2014 the one with Bing Chat \u2014 is now available waitlist-free. Anyone can try it out by signing in with a Microsoft Account.<\/p>\n It\u2019s more or less the experience that launched several months ago. But as alluded to earlier, Bing Chat will soon respond with images \u2014 at least where it makes sense. Answers to questions (e.g. \u201cWhere is Machu Picchu?\u201d) will be accompanied by relevant images if any exist, much like the standard Bing search flow but condensed into a card-like interface.<\/p>\n Answers with visuals, new in Bing Chat. Image Credits:<\/strong> Microsoft<\/p>\n<\/div>\n In a demo at the event, a spokesperson typed the question \u201cDoes the saguaro cactus grow flowers?\u201d and Bing Chat pulled up a paragraph-long response alongside an image of the cactus in question. For me, it evoked the \u201cknowledge panels<\/a>\u201d in Google Search.<\/p>\n Microsoft isn\u2019t saying which categories of content, exactly, might trigger an image. But it does have filtering in place to prevent explicit images from appearing \u2014 or so it claims.<\/p>\n Sarah Bird, the head of responsible AI at Microsoft, told me that Bing Chat benefits from the filtering and moderation already in place with Bing search. Beyond this, Bing Chat uses a combination of \u201ctoxicity classifiers,\u201d or AI models trained to detect potentially harmful prompts, and blacklists to keep the chat relatively clean.<\/p>\n Those measures didn\u2019t prevent Bing Chat from going off the rails when it first rolled out in preview in early February, it\u2019s worth noting. Our coverage<\/a> found the chatbot spouting vaccine misinformation and writing a hateful screed from the perspective of Adolf Hitler. Other reporters got it to make threats, claim multiple identities and even shame them for admonishing it.<\/p>\n In another knock<\/a> against Microsoft, the company just a few months ago laid off the ethics and society team within its larger AI organization. The move left Microsoft without a dedicated team to ensure its AI principles are closely tied to product design.<\/p>\n Bird, though, asserts that meaningful progress has been made and that these sorts of AI issues aren\u2019t solved overnight \u2014 public though Bing Chat may be. Among other measures, a team of human moderators is in place to watch for abuse, she said, such as users attempting to use Bing Chat to generate phishing emails.<\/p>\n But \u2014 as members of the press weren\u2019t given the chance to interact with the latest version of Bing beyond curated demos \u2014 I can\u2019t say to what extent all that\u2019s made a difference. It\u2019ll doubtless become clear once more folks get their hands on it.<\/p>\n One aspect of Bing Chat that is<\/em> improving is the transparency around its responses \u2014 specifically responses of a fact-based nature. Soon, when asked to summarize a document or about the contents a document (e.g. \u201cwhat does this page say about the Brooklyn Bridge?\u201d), whether a 20-page PDF or a Wikipedia article, Bing Chat will include citations indicating from where in the text the information came from. Clicking on them will highlight the corresponding passage.<\/p>\n In another new feature on the visual front, Bing Chat will be able to create charts and graphs when fed the right prompt and data. Previously, asking something like \u201cWhich are the most populous cities in Brazil?\u201d would yield a basic list of results. But in a near-future preview, Bing Chat will present those results visually and in the chart type of a user\u2019s choosing.<\/p>\n This seemingly represents a step for Bing toward a full-blown productivity platform, particularly when paired with the enhanced text-to-image generation capabilities coming down the pipeline.<\/p>\n The Image Creator in Bing Chat. Image Credits:<\/strong> Microsoft<\/p>\n<\/div>\n In the coming weeks, Bing Image Creator<\/a> \u2014 Microsoft\u2019s tool that can generate images from text prompts, powered by DALL-E 2 \u2014 will understand more languages aside from English (over 100 total). As with English, users will be able to refine the images they generate with follow-up prompts (e.g. \u201cMake an image of a bunny rabbit,\u201d followed by \u201cnow make the fur pink\u201d).<\/p>\n Generative art AI has been in the headlines a lot, lately \u2014 and not for the most optimistic of reasons necessarily.<\/p>\n Plaintiffs have brought several lawsuits<\/a> against OpenAI and its rival vendors, alleging that copyrighted data \u2014 mostly art \u2014 was used without their permission to train generative models like DALL-E 2. Generative models \u201clearn\u201d to create art and more by \u201ctraining\u201d on sample images and text, usually scraped indiscriminately from the public web.<\/p>\n I asked Bird about whether Microsoft is exploring ways to compensate creators whose work was swept up in training data, even if the company\u2019s official position is that it\u2019s a matter of fair use<\/a>. Several platforms launching generative AI tools, including Shutterstock, have kick-started creators funds along these lines. Others, like Spawning<\/a>, are creating mechanisms to let artists opt out of AI model training altogether.<\/p>\n Bird implied that these issues will eventually have to be confronted \u2014 and that content creators deserve some form of recompense. But she wasn\u2019t willing to commit to anything concrete this week.<\/p>\n Elsewhere on the image front, Bing Chat is gaining the ability to understand images as well as text. Users will be able to upload images and search the web for related content, for example copying a link to an image of a crocheted octopus and asking Bing Chat the question \u201chow do I make that?\u201d to get step-by-step instructions.<\/p>\n Multimodality powers the new page context function in the Edge app for mobile, as well. Users will be able to ask questions in Bing Chat related to the mobile page they\u2019re viewing.<\/p>\n Microsoft wouldn\u2019t say either way, but it seems likely that these new multimodal abilities stem from GPT-4, which can understand images in addition to text. When OpenAI announced<\/a> GPT-4, it didn\u2019t make the model\u2019s image understanding capabilities available to all customers \u2014 and still hasn\u2019t. I\u2019d wager that Microsoft, though, being a major investor in and close collaborator with OpenAI, has some sort of privileged access.<\/p>\n Any image upload tool can be abused, of course, which is why Microsoft is employing automated filtering and hashing to block illicit uploads, according to Bird. The jury\u2019s out on how well these work, though \u2014 we weren\u2019t given the chance to test image uploads ourselves.<\/p>\n Multimodality and new visual features aren\u2019t all that\u2019s coming to Bing Chat.<\/p>\n Soon, Bing Chat will store users\u2019 chat histories, letting them pick up where they left off and return to previous chats when they wish. It\u2019s an experience akin to the chat history feature OpenAI recently<\/a> brought to ChatGPT, showing a list of chats and the bot\u2019s responses to each of those chats.<\/p>\n The specifics of the chat history feature have yet to be ironed out, like how long chats will be stored, exactly. But users will be able to delete their history at any time regardless, Microsoft says \u2014 addressing<\/a> the criticisms several European Union governments had against ChatGPT.<\/p>\n Exporting and sharing chats from Bing Chat.\u00a0Image Credits:<\/strong>Microsoft<\/p>\n<\/div>\n Bing Chat will also gain export and share functionalities, letting users share conversations on social media or to a Word document. Dena Saunders, a partner GM in Microsoft\u2019s web experiences team, told TechCrunch that a more robust copy-and-paste system is in the works \u2014 but not in preview just yet \u2014 for graphs and images created through Bing Chat.<\/p>\n Perhaps the most transformative addition to Bing Chat, though, is plugins. From partners like OpenTable and Wolfram Alpha, plugins greatly extend what Bing Chat can do, for example helping users book a reservation or create visualizations and get answers to challenging science and math questions.<\/p>\n Like chat history, the not-yet-live plugins functionality is in the very preliminary stages. There\u2019s no plugins marketplace to speak of; plugins can be toggled on or off from the Bing Chat web interface.<\/p>\nOpen, and visual<\/h2>\n
![\"Microsoft](\"https:\/\/i0.wp.com\/techcrunch.com\/wp-content\/uploads\/2023\/05\/Visual_Pop_Still.jpg?resize=1024%2C576&ssl=1\")
<\/p>\nProductivity emergent<\/h2>\n
![\"Microsoft](\"https:\/\/i0.wp.com\/techcrunch.com\/wp-content\/uploads\/2023\/05\/Image_Creator_Still.jpg?resize=1024%2C576&ssl=1\")
<\/p>\nMultimodal search<\/h2>\n
New chat features<\/h2>\n
![\"Microsoft](\"https:\/\/i0.wp.com\/techcrunch.com\/wp-content\/uploads\/2023\/05\/Export_Still.jpg?resize=1024%2C576&ssl=1\")
<\/p>\n
![\"Microsoft](\"https:\/\/i0.wp.com\/techcrunch.com\/wp-content\/uploads\/2023\/05\/Edge_Actions_Still.jpg?resize=1024%2C576&ssl=1\")
<\/p>\n