The production of misleading content

On social media, much of the content does not undergo detailed verification before publication. Moderation tends to focus on extremely harmful content, such as adult-only material or extremist content. In contrast, ordinary information, opinions and comments often remain in circulation, as the platforms are designed to facilitate the exchange of views and build communities around topics.

The production of misleading content can take an automated form. Multiple accounts can be set up to publish similar messages and replicate the same information, creating the impression that this is a widely accepted fact. To the audience, it appears as though ‘everyone is saying the same thing’, although in reality the message may originate from a single source and is simply being replicated.

In practice, such actions may form part of a struggle for influence, support or the retention of power. At a higher level, this can also take the form of a kind of information warfare, in which various groups or states attempt to influence public opinion. With an overload of information and the rapid circulation of news, it is becoming increasingly difficult to distinguish the truth from messages that merely masquerade as reliable information.

Another important factor is that artificial intelligence accelerates content production. It facilitates the creation of multiple versions of the same message, the writing of comments, and the preparation of descriptions and summaries. This can exacerbate the problem, as a vast amount of material can be produced in a short space of time that appears legitimate, yet still leads audiences to draw the wrong conclusions.

It is also worth remembering that some moderation only comes into play once someone reports a problem. If few people report the content, or if the content is written ‘cleverly’ and does not explicitly breach the terms and conditions, it may circulate for a very long time. Therefore, one should not assume that just because something is on the platform, it has definitely been checked.