While the finer points of running a social media business can be debated, one fundamental truth is that all social media businesses are run with care. Technology leaders are incentivized to grow their user base so that more people can see more ads for more time. It's just good business.
As the owner of Twitter, Elon Musk likely shared that goal. But he insisted he didn't buy Twitter to make money. This allowed him to focus on other passions. It means stopping rival tech companies from scraping his Twitter data without his permission, even if it means turning a blind eye to ads.
Data scraping was a known issue with Twitter. “Scraping was an open secret in Twitter data access. We knew about it. It was okay,” Yoel Roth wrote on Twitter at Blue Sky Alternatives. His AI company, in particular, was famous for swallowing vast amounts of text to train large language models. In Musk's opinion, these companies were worth a lot of money, but the situation wasn't great.
In November 2022, OpenAI debuted ChatGPT, a chatbot that can generate convincing human text. By January 2023, the app had over 100 million users, making it the fastest growing consumer app in history. Three months later, OpenAI secured another round of funding, closing at a staggering $29 billion valuation, more than the value of Twitter, according to Musk's estimates.
OpenAI has been a troubling topic for Musk. Musk was one of the original founders and a major donor until he resigned in 2018 over disagreements with other founders. After launching ChatGPT, Musk made no secret of the fact that he opposed the guardrails OpenAI put in place for chatbots to prevent them from relaying dangerous or sensitive information. Musk said on December 16, 2022, “The dangers of training AI to wake up – to lie – are deadly.” He was considering starting a competitor.
Near the end of June 2023, Musk launched a two-part attack to thwart data scrapers, first telling Twitter employees to temporarily block “logout views.” This change means that only people with Twitter accounts can view tweets.
The “logout view” on Twitter has had a checkered history. It allows dissidents to view tweets without creating a Twitter account, risking their anonymity, and was rumored to have been involved in the Arab Spring. But it was also an easy access point for anyone wanting to collect Twitter data.
After Twitter made this change, Google was temporarily unable to crawl Twitter and display relevant tweets in search results. This can negatively impact Twitter traffic. “We are aware that our ability to crawl Twitter.com has been limited, impacting our ability to display the site's Tweets and pages in search results,” said Google spokesperson Lara Levin. told The Verge.. “Websites can control whether crawlers can access their content,” one person wrote as engineers discussed possible workarounds on Slack.
Engineers then noticed an “explosion in login requests,” according to an internal Slack message. This indicates that the data scraper simply logged into her Twitter and continued scraping. Musk ordered the changes reversed.
On July 1, 2023, Musk launched the second part of his attack. The user scrolled for just a few minutes and suddenly an error message popped up. I got a message that said, “Sorry, your rate is limited.” “Please wait a moment and try again.”