Shopping features on Twitter pose content moderation risks and could be used “in a way that leads to individual or societal harm,” according to an internal memo sent to a team of Twitter employees.
The memo, a portion of which was obtained by The Verge, was sent in early July to a group of Twitter employees across various teams, according to a person with knowledge of it. In the email, some existing and not-yet-released features of Twitter shopping are categorized as high risk, with the email warning that content moderation hasn’t been prioritized in shopping.
First rolled out last summer, Twitter shopping allows brands to list items for sale and pin a handful of products at the top of a merchant’s profile. Unlike similar features on Instagram, users can’t buy the product directly on Twitter — items for sale instead link out to a merchant’s website. An expanded version of the shop module was introduced earlier this year, allowing sellers to list and showcase up to 50 products in their storefront, and was available to all US merchants as of June.
In a section of the memo titled “risk assessment,” several elements of Twitter’s e-commerce tool are categorized as “high.” One high-risk concern is merchant-generated fields like shop names and descriptions, which the memo warns could be used by bad actors in harmful ways.
Twitter’s shop feature allows anyone with a professional account selling items in the US to manually add products for sale to their profile. When selecting items to appear in the expanded Twitter shop, merchants can add a custom shop name and descriptions directly on the Twitter dashboard. It’s in these fields that people working on shopping say there are risks.
The memo paints the picture of a bare-bones process to look for and remove potentially abusive or harmful content on Twitter shopping. The platform doesn’t yet have a policy on what is considered a shop name or description violation, according to the memo, and lacks guidance on how violations should be handled. The memo also notes that Twitter lacks the tools to detect violations in shop names and descriptions and that there is no way for users to report storefronts for content in these fields.
A main selling point of Twitter’s shopping features is shareability, and the company has introduced other updates like reminders that allow customers to get notified and tweet about new releases from brands. Currently, Twitter users have the ability to click into merchant shops, view products, and click out to merchant websites, but sharing the storefronts isn’t yet possible. The internal memo lists the ability to share storefronts as “high” on its risk assessment, saying that if the feature were released it could lead to harmful content being amplified further, increasing the visibility of content that violates Twitter rules.
Twitter has some automated detection mechanisms for individual products listed for sale, according to the memo. But proactive measures to detect violations are “limited,” and the company has minimal staff and tools for further review.
“Shareable Shops therefore increase the likelihood that users may see violative Shops, or violative goods contained in a Shop,” the memo reads. “It may also incentivize bad actors to amplify harmful or violative goods by tweeting and sharing their own Shop.”
In a statement to The Verge, Twitter spokesperson Lauren Alexander confirmed the authenticity of the memo and said it was part of a new feature assessment led by the product trust team. Alexander says the assessments are designed so different teams can provide input to ensure new product releases are safe.
“We’re always working to improve the safety of our service and that is particularly true for the addition of new products and features,” Alexander says. “We have been intentional about taking the time to test our new shopping surfaces so that — before we scale or expand to new markets — we have the opportunity to gather learnings to better inform our approach to health and safety and develop additional infrastructure and policy needs.”
Twitter has been trying to diversify its revenue sources beyond advertising in recent years, experimenting with a paid subscription service, ticketed live broadcasts on Spaces, and paid Super Follows. Though the platform has been rolling out and expanding its commerce tools, it’s lagged behind other social networks like Instagram and TikTok; in his attempted takeover of Twitter, Elon Musk indicated that payments had the potential to be a substantial chunk of the company’s business.
Earlier this week, Twitter’s former head of security Peiter “Mudge” Zatko said the company suffered from negligent security and bot removal issues, among other problems, in a wide-ranging whistleblower report. Shortly after news of the complaint, Reuters reported that the platform would combine a “health” team working on reducing misinformation and other toxic content with a services team that works on spam accounts.