top of page

Elon Musk Rate Limits Twitter Views and Limits Crawling & Scraping

Twitter, traditionally a bastion of free speech and open communication, has been undergoing significant changes under the leadership of Elon Musk. The latest of these changes is a controversial restriction on the number of tweets users can view daily. While the move has been touted as a measure against AI model training, data scraping, and system manipulation, it raises serious concerns about the platform's role in information discovery, the potential risk of unseen and unchecked nefarious activities, and the general impact on safety-related communications.



The New Limits and Their Revisions


Under Musk's unannounced directive, Twitter has limited the daily posts users can read. Initially set at 6,000 posts for verified users, 600 for unverified users, and 300 for new unverified accounts, these limits have been revised twice. The first adjustment raised the limits to 8,000, 800, and 400 posts. A second amendment increased the daily tweet viewing limit to 10,000 for verified users, 1,000 for unverified users, and 500 for new unverified accounts. Twitter has also "temporarily" pulled the public view of all Twitter profiles.

The changes impacted Tweetdeck, and eroded 99% of Google's visibility into the site. As the negative impacts rippled through the Twitterverse, #riptwitter started to trend.

Musk's Rationale Behind the Changes


Musk has defended these measures as necessary to combat "extreme levels of data scraping" and "system manipulation;" however, the sincerity of bots and AI being the drivers behind the changes is cause for skepticism. For over a decade, crawlers, bots, scrapers, and information sleuths have continuously accessed Twitter's data and search features. There is also plenty of cyber threat intelligence that could aid in blocking the more nefarious actions against the platform.


Skepticism and Controversy


Twitter's former head of trust and safety, Yoel Roth, doubts Musk's explanation, disputing that data scraping could cause such severe performance issues so as to warrant restricting tweet viewing behind a login. Web developer Simon Chang also flagged a potential technical issue, suggesting that a bug might be causing an infinite loop of requests resulting in app-driven DDOS. Twitter continually fires off requests to fetch non-existent content due to Musk's decision to block people from reading Twitter without logging in.

Safety, Security, and Unchecked Nefarious Actions


While these Twitter changes are concerning, they raise another, perhaps more insidious, issue. By limiting the number of tweets users can view, Twitter inadvertently provides a safer environment for malicious actors to operate. The less oversight there is, the more opportunities exist for harmful activities to go undetected.


Twitter has always been an essential tool for information discovery. It has facilitated the unearthing of small-scale schemes and large-scale threats to the general public. With Musk's new restrictions, the user community's ability -- particularly those in the security community -- to discover and counteract nefarious behaviors is now severely limited. Given Twitter's limited size trust and safety team, the user community has been a major weapon for reporting threatening tweets.

Broader Context and Speculations


Twitter recently unveiled API pricing packages. As DigitalStakeout and others announced on June 1, these packages are unaffordable, exacerbating the very problem Musk claims is causing issues.


For example, Twitter's new "Pro" package costs $5,000/month for 1 million Tweets. If scraping & bots "created" Twitter performance issues, it comes from a massive response from the market not willing to pay for Tweets at astronomical pricing levels.


There is a material disconnect between Musk's valuation of Twitter data and the market's willingness to pay such high costs. To add, commentators have suggested that the viewing restrictions might be a strategic move to increase subscriptions to Twitter Blue, the platform's paid service that provides users with verified status for $8 per month.


Mitigating the Impact of Twitter View Restrictions


In the face of Twitter's new restrictions, users and security professionals might feel like their hands are tied. However, there are still ways to stay informed and maintain your digital investigation and threat intelligence capabilities.


Focus on Aggregation from Other Sources


While Twitter has long been recognized as a major hub for real-time news and conversations, it's important to remember that it's not the singular epicenter of all information. Indeed, information is distributed across a myriad of social media platforms, each offering its unique lens into unfolding events and public sentiment. Each platform captures a piece of the larger information puzzle.


Twitter isn't a Panacea of Intelligence


Every publicly available data source is awash with valuable information and insights during significant events, reflecting various perspectives and narratives. The value of this intelligence cannot be understated; it paints a holistic picture of the event in question, revealing trends, patterns, and insights that might be missed if we focus solely on one platform like Twitter. Further, Twitter has lately exhibited a pattern of unpredictable and impactful change, so it becomes increasingly risky to place an unusually high dependence on Twitter data.


Always Have a Multi-Source Intelligence Strategy


In fact, cross-platform threat analysis allows for a more comprehensive understanding of events, trends, and shifts. In essence, while impactful, the changes on Twitter do not spell the end of information discovery. They underline the importance of a diversified, platform-agnostic approach to gathering threat intelligence and staying informed in today's digital age.


Leverage our E-Discovery Option


DigitalStakeout Archiver is a powerful browser extension for Edge, Firefox, and Chrome.

With DigitalStakeout Archiver, DigitalStakeout subscribers can manually archive essential pieces of text from any webpage and archive them in DigitalStakeout Scout. Just highlight the required text, and the Archiver handles the rest. The current features in the browser extension are:

  • Archive Selected Text

  • Archive Page Text

  • Archive Images on Page

  • Take Screenshot of Page (Native)

  • Save a Note for URL

  • Archive to Selected DigitalStakeout Container

By integrating tools like the DigitalStakeout Archiver into your toolbox, you can navigate the impacts of Twitter's new viewing restrictions. Armed with such technology, you can continue to monitor, discover, and respond to threats in the ever-evolving digital landscape.


Purchase Access to the Twitter Firehose


As we stated in our June 1 announcement, if you MUST have a real-time Twitter firehose and Twitter's published prices are acceptable to your organization, we will support you in acquiring the capability and using the Twitter Power Track firehose rule system. DigitalStakeout will maintain support for consuming Power Track streams directly into DigitalStakeout for processing, alerting, and analytics. We will also help you maintain compliance with Twitter's terms of service and data use policies.

Post: Blog2_Post
bottom of page