The proliferation of spam content on Facebook, much of it generated by artificial intelligence (AI), has become a significant issue for users of the platform. Some of the spam content includes obviously fake AI-generated images, old posts being reshared by bot accounts, and pages sharing random memes or movie clips. This spam not only annoys users but can also be used for malicious purposes, such as scamming or spreading misinformation. Experts studying online behavior warn that this spam could be used by foreign actors to influence events like elections.
Facebook’s intentional shift towards a “discovery engine” that emphasizes entertainment over news content has contributed to the rise of AI-generated spam. This algorithm encourages the creation and sharing of vapid, misleading content, leading to the proliferation of fake images and text by bad actors and engagement farmers. The availability of AI tools has made it easier for malicious users to quickly generate large volumes of fake content on the platform, which can receive significant engagement from users.
Despite efforts by Facebook parent company Meta to combat spam content, the platform continues to be inundated with AI-generated material. Users have reported coming across uncanny and bizarre content that appears benign at first glance but is later revealed to be AI-generated. Some pages share historical photos with AI-generated profile photos, while others recycle images from real influencers without permission. Meta has taken action against some of these spam pages when users flag them for violating spam policies.
The prevalence of AI-generated content on Facebook is difficult to quantify, but researchers have identified numerous pages posting AI-generated images that collectively receive millions of engagements. Various actors may be behind the spam, with some seeking to make money through bonus payments from Facebook or by selling sham products in comments sections. Additionally, some accounts post misinformation or offensive content intermittently to avoid detection. The market for “aged” Facebook accounts is also thriving, with older accounts being used to evade spam filters and enforce misinformation campaigns.
AI technology has enabled bad actors to produce large amounts of fake content without the need for human resources. Identifying all AI-generated images uploaded to Facebook is a challenging task for the platform due to the rapid advancements in AI technology. Despite efforts by Meta to tag AI-generated content and increase moderation, there are still ways for users to evade detection. The reduced trust and safety staff at Meta, combined with the cat and mouse game between platforms and digital communities, makes it difficult to completely eliminate AI-generated spam.
