{"id":27928,"date":"2023-03-31T21:44:51","date_gmt":"2023-03-31T21:44:51","guid":{"rendered":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/"},"modified":"2023-03-31T21:44:57","modified_gmt":"2023-03-31T21:44:57","slug":"twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm","status":"publish","type":"post","link":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/","title":{"rendered":"Twitter reveals some of its source code, including its recommendation algorithm"},"content":{"rendered":"

Source:https:\/\/techcrunch.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/<\/a><\/br>
\nTwitter reveals some of its source code, including its recommendation algorithm<\/br>
\n2023-03-31 21:44:51<\/br><\/p>\n

\n

As repeatedly promised<\/a> by Twitter CEO Elon Musk, Twitter has opened<\/a> a portion of its source code to public inspection, including the algorithm it uses to recommend tweets in users\u2019 timelines.<\/p>\n

On GitHub, Twitter published two<\/a> repositories<\/a> containing code for many parts that make the social network tick, including the mechanism Twitter uses to control the tweets users see on the For You timeline. In a blog post, Twitter characterized the move as a \u201cfirst step to be[ing] more transparent\u201d while at the same time \u201c[preventing] risk\u201d to Twitter itself and people on the platform.<\/p>\n

On a Twitter Spaces session today, Musk clarified:<\/p>\n

\u201cOur initial release of the so-called algorithm is going to be quite embarrassing, and people are going to find a lot of mistakes, but we\u2019re going to fix them very quickly,\u201d Musk said. \u201cEven if you don\u2019t agree with something, at least you\u2019ll know why it\u2019s there, and that you\u2019re not being secretly manipulated \u2026 The analog, here, that we\u2019re aspiring to is the great example of Linux as an open source operating system \u2026 One can, in theory, discover many exploits for Linux. In reality, what happens is the community identifies and fixes those exploits.\u201d<\/span><\/p>\n

On that second point in the blog post about preventing risk, the open source releases don\u2019t include the code that powers Twitter\u2019s ad recommendations or the data used to train Twitter\u2019s recommendation algorithm. Moreover, they include few instructions on how to inspect or actually use the code \u2014 reinforcing the idea that the releases are strictly developer-focused.<\/p>\n

\u201c[We excluded] any code that would compromise user safety and privacy or the ability to protect our platform from bad actors, including undermining our efforts at combating child sexual exploitation and manipulation,\u201d Twitter wrote. It\u2019s a bit of mixed messaging coming only weeks after<\/a> Twitter fired much<\/a> of its ethical AI and trust and safety staff, which was responsible for content moderation among other user security-related tasks. But the company nonetheless insists that it \u201c[took] steps to ensure that user safety and privacy would be protected\u201d with today\u2019s code release.<\/p>\n

\"Twitter\"<\/p>\n

A diagram showing how Twitter\u2019s recommendation pipeline works. Image Credits:<\/strong> Twitter<\/p>\n<\/div>\n

Twitter says it\u2019s working on tools to manage code suggestions from the community and sync changes to its internal repository. Presumably, those will be made available at a future date \u2014 there\u2019s no sign of them at the present.<\/p>\n

\u201cWe\u2019re going to look for suggestions, not just on bugs but also on how the algorithm should work,\u201d Musk said on the Spaces session. \u201cIt\u2019s going to be an evolving process. I wouldn\u2019t expect it to be a nonstop upward movement\u2026 but we\u2019re very open to what would improve the user experience.\u201d<\/p>\n

At first glance, the algorithm is fairly complex \u2014 but not necessarily surprising in any way from a technical standpoint. It\u2019s made up of multiple models, including a model for detecting \u201cnot safe for work\u201d or abusive content, determining the likelihood of a Twitter user interacting with another user and calculating a Twitter user\u2019s \u201creputation.\u201d (It\u2019s unclear what \u201creputation\u201d refers to, exactly; the high-level documentation isn\u2019t clear on that.) Several neural networks are responsible for ranking the tweets and recommending accounts to follow, while a filtering component hides tweets to \u2014 forgive the jargon \u2014 \u201csupport legal compliance, improve product quality, increase user trust, protect revenue through the use of hard-filtering, visible product treatments and coarse-grained downranking.\u201d<\/p>\n

\"Twitter<\/p>\n

Twitter\u2019s For You timeline, powered by a recommendation algorithm.\u00a0Image Credits:<\/strong> Twitter<\/p>\n<\/div>\n

In an engineering blog post<\/a>, Twitter reveals more about the recommendation pipeline, which it claims runs approximately five billion times per day:<\/p>\n

\u201cWe attempt to extract the best 1,500 tweets from a pool of hundreds of millions \u2026 Today, the For You timeline consists of 50% [tweets from people you don\u2019t follow] and 50% [tweets from people you follow] on average, though this may vary from user to user,\u201d Twitter wrote. \u201cRanking [tweets] is achieved with a ~48-million-parameter neural network that is continuously trained on tweet interactions to optimize for positive engagement (e.g. likes, retweets and replies).\u201d<\/p>\n

Twitter users don\u2019t see the full 1,500 tweets, of course. They\u2019re filtered according to content restrictions and other criteria and factors considered by the models, like if tweets have \u201cnegative feedback\u201d and if they\u2019re mainly from the same Twitter user, or from users who\u2019ve been blocked or muted.<\/p>\n

Gizmodo notes<\/a> that one thing that doesn\u2019t appear to have been made public is the list of VIPs that Twitter pushes to users. This week, Platformer reported<\/a> that Twitter has a rotating list of noteworthy users, including YouTuber Mr. Beast and Daily Wire founder Ben Shapiro, that it uses to monitor changes to the recommendation algorithm by increasing the visibility of these \u201cpower users\u201d seemingly at will.<\/p>\n

There\u2019s more evidence that the algorithm may treat tweets differently depending on the source. Researcher Jane Manchun Wong noted that Twitter\u2019s algorithm specifically labels whether the tweet author is Elon Musk and has others labels indicating whether the author is a \u201cpower user\u201d as well as whether they\u2019re a Republican or Democrat.<\/p>\n

During the Spaces session this afternoon, a Twitter engineer said that the labels were used only for metrics. But Musk \u2014 who said he wasn\u2019t aware of the labels prior to today \u2014 said that they shouldn\u2019t be there.<\/p>\n

\u201cIt definitely shouldn\u2019t be dividing people into Republicans and Democrats, that makes no sense,\u201d Musk said.<\/p>\n

The release of the source code comes after several controversies involving tweaks to Twitter\u2019s recommendation algorithm in recent months. According to Platformer<\/a>, in February, Musk called on Twitter\u2019s engineers to reconfigure the algorithm so his tweets would be more widely viewed. (Twitter later walked back this change \u2014 at least somewhat.) In November, Twitter began showing<\/a> users more tweets from people they don\u2019t follow \u2014 a move the platform attempted<\/a> prior to Musk\u2019s acquisition but later reversed after a backlash from users.<\/p>\n<\/p><\/div>\n

<\/br><\/br><\/br><\/p>\n

Science, Tech, Technology<\/br>
\n<\/br>
\nSource:https:\/\/techcrunch.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/<\/a><\/br><\/br><\/p>\n","protected":false},"excerpt":{"rendered":"

Source:https:\/\/techcrunch.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/ Twitter reveals some of its source code, including its recommendation algorithm 2023-03-31 21:44:51 As repeatedly promised by Twitter CEO Elon Musk, Twitter has opened a portion of its source code to public inspection, including the algorithm it uses to recommend tweets in users\u2019 timelines. On GitHub, Twitter published two repositories containing code for many […]<\/p>\n","protected":false},"author":1,"featured_media":27929,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","om_disable_all_campaigns":false,"pagelayer_contact_templates":[],"_pagelayer_content":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[26,17,8],"tags":[],"class_list":["post-27928","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-science","category-tech","category-technology"],"yoast_head":"\nTwitter reveals some of its source code, including its recommendation algorithm - Science and Nerds<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Twitter reveals some of its source code, including its recommendation algorithm - Science and Nerds\" \/>\n<meta property=\"og:description\" content=\"Source:https:\/\/techcrunch.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/ Twitter reveals some of its source code, including its recommendation algorithm 2023-03-31 21:44:51 As repeatedly promised by Twitter CEO Elon Musk, Twitter has opened a portion of its source code to public inspection, including the algorithm it uses to recommend tweets in users\u2019 timelines. On GitHub, Twitter published two repositories containing code for many […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/\" \/>\n<meta property=\"og:site_name\" content=\"Science and Nerds\" \/>\n<meta property=\"article:published_time\" content=\"2023-03-31T21:44:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-03-31T21:44:57+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/scienceandnerds.com\/wp-content\/uploads\/2023\/03\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm_6427545441a1a.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"730\" \/>\n\t<meta property=\"og:image:height\" content=\"392\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/\",\"url\":\"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/\",\"name\":\"Twitter reveals some of its source code, including its recommendation algorithm - Science and Nerds\",\"isPartOf\":{\"@id\":\"https:\/\/scienceandnerds.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/03\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm_6427545441a1a.jpeg?fit=730%2C392&ssl=1\",\"datePublished\":\"2023-03-31T21:44:51+00:00\",\"dateModified\":\"2023-03-31T21:44:57+00:00\",\"author\":{\"@id\":\"https:\/\/scienceandnerds.com\/#\/schema\/person\/ea2991abeb2b9ab04b32790dff28360e\"},\"breadcrumb\":{\"@id\":\"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/#primaryimage\",\"url\":\"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/03\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm_6427545441a1a.jpeg?fit=730%2C392&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/03\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm_6427545441a1a.jpeg?fit=730%2C392&ssl=1\",\"width\":730,\"height\":392},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/scienceandnerds.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Twitter reveals some of its source code, including its recommendation algorithm\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/scienceandnerds.com\/#website\",\"url\":\"https:\/\/scienceandnerds.com\/\",\"name\":\"Science and Nerds\",\"description\":\"My WordPress Blog\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/scienceandnerds.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/scienceandnerds.com\/#\/schema\/person\/ea2991abeb2b9ab04b32790dff28360e\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/scienceandnerds.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/7e6e14fc6691445ef2b2c0a3a6c43882?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/7e6e14fc6691445ef2b2c0a3a6c43882?s=96&d=mm&r=g\",\"caption\":\"admin\"},\"sameAs\":[\"https:\/\/scienceandnerds.com\"],\"url\":\"https:\/\/scienceandnerds.com\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Twitter reveals some of its source code, including its recommendation algorithm - Science and Nerds","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/","og_locale":"en_US","og_type":"article","og_title":"Twitter reveals some of its source code, including its recommendation algorithm - Science and Nerds","og_description":"Source:https:\/\/techcrunch.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/ Twitter reveals some of its source code, including its recommendation algorithm 2023-03-31 21:44:51 As repeatedly promised by Twitter CEO Elon Musk, Twitter has opened a portion of its source code to public inspection, including the algorithm it uses to recommend tweets in users\u2019 timelines. On GitHub, Twitter published two repositories containing code for many […]","og_url":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/","og_site_name":"Science and Nerds","article_published_time":"2023-03-31T21:44:51+00:00","article_modified_time":"2023-03-31T21:44:57+00:00","og_image":[{"width":730,"height":392,"url":"https:\/\/scienceandnerds.com\/wp-content\/uploads\/2023\/03\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm_6427545441a1a.jpeg","type":"image\/jpeg"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/","url":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/","name":"Twitter reveals some of its source code, including its recommendation algorithm - Science and Nerds","isPartOf":{"@id":"https:\/\/scienceandnerds.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/#primaryimage"},"image":{"@id":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/03\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm_6427545441a1a.jpeg?fit=730%2C392&ssl=1","datePublished":"2023-03-31T21:44:51+00:00","dateModified":"2023-03-31T21:44:57+00:00","author":{"@id":"https:\/\/scienceandnerds.com\/#\/schema\/person\/ea2991abeb2b9ab04b32790dff28360e"},"breadcrumb":{"@id":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/#primaryimage","url":"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/03\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm_6427545441a1a.jpeg?fit=730%2C392&ssl=1","contentUrl":"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/03\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm_6427545441a1a.jpeg?fit=730%2C392&ssl=1","width":730,"height":392},{"@type":"BreadcrumbList","@id":"https:\/\/scienceandnerds.com\/2023\/03\/31\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/scienceandnerds.com\/"},{"@type":"ListItem","position":2,"name":"Twitter reveals some of its source code, including its recommendation algorithm"}]},{"@type":"WebSite","@id":"https:\/\/scienceandnerds.com\/#website","url":"https:\/\/scienceandnerds.com\/","name":"Science and Nerds","description":"My WordPress Blog","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/scienceandnerds.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/scienceandnerds.com\/#\/schema\/person\/ea2991abeb2b9ab04b32790dff28360e","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/scienceandnerds.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/7e6e14fc6691445ef2b2c0a3a6c43882?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/7e6e14fc6691445ef2b2c0a3a6c43882?s=96&d=mm&r=g","caption":"admin"},"sameAs":["https:\/\/scienceandnerds.com"],"url":"https:\/\/scienceandnerds.com\/author\/admin\/"}]}},"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/i0.wp.com\/scienceandnerds.com\/wp-content\/uploads\/2023\/03\/twitter-reveals-some-of-its-source-code-including-its-recommendation-algorithm_6427545441a1a.jpeg?fit=730%2C392&ssl=1","_links":{"self":[{"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/posts\/27928"}],"collection":[{"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/comments?post=27928"}],"version-history":[{"count":1,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/posts\/27928\/revisions"}],"predecessor-version":[{"id":27930,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/posts\/27928\/revisions\/27930"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/media\/27929"}],"wp:attachment":[{"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/media?parent=27928"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/categories?post=27928"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/scienceandnerds.com\/wp-json\/wp\/v2\/tags?post=27928"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}