Our Automated Account Experiment Review¶
Given our instance is very small we were not expecting to limit that many accounts, but were still interested in seeing how it all unfolded and to understand the effects on the federated feed as a whole.
Our criteria was that if an account sent a number of public visible posts within a defined period of time that contained the hallmarks of plucking it straight from RSS feeds or only contained a link, then they were classed as being automated.
This may have included a very small amount of accounts that are in fact user driven, but we will review these on a case by case basis.
We reverted everything back to the original state and restarted on the 4th September as insights in the data collection resulted in some edge cases that would cause some accounts to be incorrectly flagged. We also thought it would be a good time to add a couple of more metrics to include as we had previously had to estimate.
Our Instance¶
Over the week we would have limited a total of 327 accounts of which 226 (69.1%) of them were properly marked as being automated.
Three quarters of those we marked as being automated were correctly marked as such accounts so we were quite pleased with that. That being said, slightly disappointing that there are a number of accounts masquerading is user driven. Would probably need to delve into the data more closely to see any patterns.
Of these 327 accounts that posted publicly in an automated fashion, they represented 22.2% of the posts that was seen by our instance in its feeds. Total posts over the week: 238,445 and the number of posts made by what we perceived were automated accounts: 52,875.
Mastodon.Social¶
We also ran a parallel experiment on mastodon.social as it has higher user figures & federation and would have limited 3,370 accounts of which 2,434 (72.2%) of them were properly marked.
The marked to unmarked ratio was pretty similar on both instances, however when we considered the number of posts that these 3,370 accounts contribute to the federated timeline then we see a slightly different picture.
These accounts represented 40.2% of the posts that are seen by mastodon.social in its feeds (almost double the percentage). Over a third of all posts came from automated accounts (or 2 in every 5 posts). Total posts over the week: 699,527 and the number of posts made by what we perceived were automated accounts: 281,130.
What Can We Take From These Figures¶
After a few days, the rate of accounts we were identifying as being automated has dropped significantly and therefore we decided to end the experiment.
We can see that automated accounts are having a significant impact on instance feeds as they accounted for over 30% of all the posts in the public feeds is ashonshing to us and over 40% if you include those that we considered as automated - That is a lot of unnecessary timeline noise (in our eyes).
If they were forced to post in "Quiet Place" or even "Followers Only", we would see a significant reduction in generally low engagement content and a better user experience.
How To Contact Us¶
If there is something you wish to discuss, address or seek clarification, then please do not hesitate to contact the team by email [email protected] or DM us at @[email protected]
Comments
Comment on this blog post using a Fediverse-compatible Account (Mastodon or alike).