If you keep your eye on content moderation as we do, you’ll be aware that the EU’s Digital Services Act (DSA) is on the road to being passed, after the European Commission submitted its proposals for legislation last December.
You’ll also know, of course, that the last year has been a tumultuous time for online content. Between governments trying to communicate accurately about the pandemic, a tumultuous US election cycle, and a number of protest movements moving from social media to the streets, it’s felt like a week hasn’t passed without online content – and how to moderate it – hitting the headlines.
All of which makes the DSA (though at least partly by accident) extremely well-timed. With expectations that it will overhaul the rules and responsibilities for online businesses around user-generated content, EU member states will be keen to ensure that it offers an effective response to what many are coming to see as the dangers of unmanaged online discourse, without hindering the benefits of digitalized society that we’ve all come to rely on.
There’s a lot we still don’t know about the DSA. As it is reviewed and debated by the European Council and the European Parliament, changes might be made to everything from its definition of illegal content to the breadth of companies that are affected by each of its various new obligations. It’s absolutely clear, though, that businesses will be affected by the DSA – and not only the ‘Very Large Platforms’ like Google and Facebook which are expected to be most heavily targeted.
Many people looking at the DSA will instinctively think back to the last time the EU made significant new law around online business with the GDPR. The impact of that regulation is still growing, with larger fines being levied year-on-year, but it’s perhaps more important that internet users’ sense of what companies can or should do with data has been shifted by the GDPR. Likewise, the DSA will alter the terrain for all online businesses, and many industries will have to do some big thinking over the coming years as the act moves towards being agreed upon.
Content moderation, of course, is our expertise here at Besedo, and making improvements to how content is managed will be a big part of how businesses adapt to the DSA. That’s why we decided to help get this conversation started by finding out how businesses are currently thinking about it. Surveying UK-based businesses with operations in the EU across the retail, IT, and media sectors, we wanted to take the temperature of firms that will be at the forefront of the upcoming changes.
We found that, while the act is clearly on everyone’s radar, there is a lot of progress to be made if businesses are to get fully prepared. Nearly two-thirds of our respondents, for example, knew that the DSA is a wide-ranging set of rules which applies beyond social media or big tech. However, a similar proportion stated that they understand what will be defined as ‘illegal content’ under the act – despite the fact that that definition is yet to be finalized.
Encouragingly, we also found that 88% of respondents are confident that they will be ready for the DSA when it comes into force. For most, that will mean changing their approach to moderation: 92% told us that achieving compliance will involve upgrading their moderation systems, their processes, or both.
As the DSA is discussed, debated, and decided, we’ll continue to look at numbers like these and invite companies together to talk about how we can all make the internet a safer, fairer place for all its users. If you’d like to get involved or want insight on what’s coming down the road, our new research report, ‘Are you ready for the Digital Services Act?’, is the perfect place to start.
Over the past 2 decades, we’ve seen a ton of in-house tools created to support content moderators in the important task to keep users safe.
While these tools have mostly been created by very talented developers, it has also often been evident that the team behind them has not been able to put the time and resources into creating optimized moderation solutions. Likely because they’ve made the smart business decision to prioritize developing new features or polish for their core product. The platform their users use.
The problem is that the trade-off comes back to impact end-users at some point. Ineffective moderation processes (caused by unwieldy tools) will ultimately mean users get a worse user experience and substandard service.
Obviously, in-house tools vary wildly in features, capacity, and user-friendliness, but over the years we’ve come across 5 features that we regularly see in-house tools lacking.
March marks the 1-year anniversary of WHO declaring Covid 19 a global pandemic. While vaccines are now being rolled out and a return to normality is inching closer, online trade is still heavily influenced and characterized by a year in and out of lockdown. And so are the content moderation challenges we meet in our day-to-day work with platforms across the globe.
Shortage in graphics cards increases electronic frauds.
Whether for work or entertainment, being homebound has caused people to shop for desktop computers at a level we haven’t seen for a decade. For the past 10 years, mobile-first has been preached by any business advisor worth listening to, but lockdowns have given desktop computers a surprising comeback and increased demands for PC parts.
The increased interest in PCs combined with the late 2020 release of the new console generation and the reduced production caused by pandemic mandated lockdowns has created an unexpected niche for scammers.
We’re currently seeing a worldwide shortage of graphic cards, needed for both consoles and desktop computers and scammers haven’t wasted a second to jump on the opportunity.
In March we’ve seen a significant increase in fraud cases related to graphics cards with gaming capabilities. In some cases, more than 50% of fraud cases we deal with have been related to graphics cards.
Puppy scams are still sky-high.
In March we post-reviewed puppy scams on 6 popular online marketplaces in the UK. We found that almost 50% of live listings showed signs of being fraudulent.
Pet trade has exploded since the beginning of the pandemic and scammers are still trying to take advantage of those looking for new furry family members.
Sleeper accounts awaken.
Our moderators warn that this month they’ve seen an increase in sleeper accounts engaging in Trojan scams. The accounts post a low-risk item, then lays dormant for a while before they start posting high-value items. The method is used to circumvent moderation setups that only moderate the first items posted by new accounts.
High-risk items posted by these accounts are often expensive electronics in high demand, such as cameras or the Nintendo Switch.
April is looking to be an interesting month in terms of content moderation challenges. With many countries tentatively opening up and others concerned about a 3rd wave, we recommend that all marketplace owners keep a close eye on corona-related scams. From masks to fake vaccines and a potential incoming surge of forged corona passports staying alert, up to date, and keeping your moderators educated will be as important as ever.
If you need help reviewing your content moderation setup or are looking for an experienced team to take it off your hands, we’re here to help.
Whether you’re running an online marketplace, classifieds site, or sharing economy platform you’ll want to optimize so you can manage User Generated Content (UGC) quickly and effectively.
Unfortunately, the larger you scale, and the more users you gain, the harder this becomes. There are always going to be users who abuse your trust – whether they’re posting poor-quality photographs or uploading content with malicious or fraudulent intent.
Moderating content is a complex and constantly evolving process that requires a carefully blended package of tech and human expertise to protect your platform’s trust, content quality, and user experience. Creating that takes time, effort, and know-how. Or in other words, you’ll need to invest a lot of money and resources if you want to build your own tools and processes in-house.
No “One Size Fits All” Solution
Depending on the size and nature of your organization, you’re going to need to manage content moderation in your own unique way – and those solutions are going to need scalability so that your business can grow.
If content moderation itself isn’t your core business, setting it up in-house is going to divert resources away from what’s most important to you.
That’s not to say content moderation isn’t important – far from it. In the last couple of years, you’ll have no doubt heard the debate surrounding social media giants like Facebook outsourcing their content moderation.
There have been calls to bring that process in-house, as detractors argue that outsourcing such a central business function devalues its importance. However, it’s also been highlighted that a more effective content moderation solution in a large social media business like Facebook would require double the number of human moderators.
Choosing the Right In-house Approach to Content Moderation
Most content moderation solutions blend human and AI moderators. But what are the actual costs involved in setting it up in-house?
Broadly speaking, you’ll need to account for the annual costs of staffing a moderation team plus the associated software and equipment required. Because content moderation is unique to every site, there are also going to be AI development and production costs. Let’s break it down.
Before deciding on the right content moderation strategy, you need to know the five methods at your disposal to enhance and secure your online business.
- Manual Pre-Moderation: Using a set of guidelines, all user-submitted content is screened by a moderator before it goes live on your site.
- Manual Post-Moderation: Content that goes live to your site and is then checked afterward by a moderator. This is commonly used on community-based platforms.
- Reactive Moderation: Users report or flag inappropriate or offensive content on your site. It should only be used to supplement other forms of moderation.
- Distributed Moderation: Leave your content moderation to the community itself, allowing them to control it using rating and voting systems.
- Automated Moderation: Filters and tools which automatically moderate specific content. They need continuous review by a human moderator to ensure accuracy and efficiency.
For a more in-depth account of these processes, read our blog on 5 Moderation Methods.
Staffing Content Moderation In-house
If you’re investing in content moderation in-house, there are a number of cost factors to consider.
You might have already considered the expenditure needed for salary and periods of downtime. Of course, that’s after you’ve recruited, trained, and onboarded suitably skilled content moderation staff. You’ll also need to factor in admin support, responsibility for filter management, and leadership.
The salaries for a tech team are going to include (but are not exclusively limited to):
- Front-end developer
- Back-end developer
- DevOps and Sysadmin to support developers and the platform running.
- Data scientist (where machine learning is needed)
- QA engineer for testing
- Project/Product manager
If your content moderation team is based remotely, you’ll also need to ensure the right security and communication systems are in place. Continuous training will also be needed – to align your content moderation with policy changes and to address quality control.
As with any new hire, you’ll also need to consider the costs associated with performance management as well as shrinkage and shift management. For example, you might need seasonal flexibility, or your business may be suddenly forced to move your entire operation to a work from home setup. Are you prepared for that and the security issues that goes with it?
Redundancy is never good PR and best avoided. So, what’s the best hiring solution for your content moderation needs?
Investing in Tech for In-house Content Moderation
We’ve considered staffing. So, what about the tech needed for in-house content moderation?
One of the most important – yet undervalued components – of content moderation is the control panel and its maintenance. Often considered a lower priority than user-facing features, a control panel is essential for moderation staff to do their job effectively.
If there’s no dedicated team responsible for maintaining the control panel it usually falls to product and site development teams – who will always have other priorities. As a result, there can sometimes be internal battles for developers’ time.
This leads to upgrades being delayed and moderators reporting that control panels are outdated and do not work properly. This can negatively impact moderation efficiency.
Furthermore, ineffective content moderation can damage retention and your brand. There’s strong evidence to suggest it reduces conversions – stunting growth. You’ll also see an increase in customer support costs: as dissatisfaction results in higher levels of customer contact.
Hosting a control panel in-house carries further additional costs. If the team or person who built-it leaves or is unavailable who will you turn to for support? Not only do you have the design and maintenance to account for, but you’ve also got the added burden of automation and filter oversight to consider.
An effective control panel requires regular updates to increase productivity, accuracy, and compliance. If it’s too cumbersome you may be faced with poor productivity and encounter problems with staff retention. There also needs to be a cohesive way of reporting and integrate with the back-end team.
Reasons to Outsource Content Moderation
Fraudsters are innovative. Content moderation needs to be preemptive as well as responsive. Even with an in-house team in place, there are also ongoing costs to consider.
With any service, there needs to be enough capacity to improve existing services and scope for fast-paced improvements.
However, the quality of moderation you receive depends on who you outsource it to. Like any outsourced function, you need to be sure you’re getting the best value and return on investment.
Businesses tend to operate globally these days, but that still means understanding the culture and market for individual countries you’re operating in. And for that, you may need native speakers or at least people with a native understanding of the culture.
Why? Because there can be significant cultural differences which require sensitivity depending on where you are in the world you are. What’s deemed offensive in one territory may not be so in another. You need local expertise.
In addition, every technological advance brings new challenges to content moderation. As an example, let’s take recent developments in machine learning such as Generative Adversarial Networks (GAN).
Essentially, GAN can create new data from training data. So, it can ‘learn’ from photographs to create new photographs. In terms of human image synthesis, this has resulted in misuse to create deep fakes.
Does your business have the budget and scalability to constantly keep up with new content moderation challenges? If not, then what’s the solution?
Implio: The All-In-One Moderation Tool
Getting content moderation right for your business is a complicated matter. With an off-the-shelf tool like Besedo’s Implio you’ve got instantly up-to-date content moderation expertise and technology right out the box.
Considering switching to an off-the-shelf solution? We’ve created this handy checklist to help you. And if you’re still not convinced about the benefits of outsourcing your content moderation, why not take a look at our white paper to see how outsourcing can help free up resources?
If you have decided to see what Besedo can do for you, find out how to manage a smooth migration with our cheat sheet
Responsible & Transparent Partnerships
At Besedo, we’ve been working with businesses like yours for over 19 years. Since 2002, we’ve offered content moderation insight and foresight to our partners based on trends and the vast pool of data that we handle.
To give you an idea of scale, we review over 570 million pieces of content and block nearly 40 million fraud and scam attempts every year. You’re proud of your product and we’re passionate about giving your end-users the best possible experience in a safe environment.
Rather than outsourcing your customer support, think of us as an extension to your team. With 350 employees from 20 over different nationalities, we’re a global and multi-cultural company who can help with localized expertise.
We provide tailored content moderation strategies and solutions to suit your businesses’ needs. You know your site best, but if you’re seeking a second opinion, we can even consult with you to identify the strongest setup for your needs.
Want to see how we can help? Find out more about working with us and request a demo today.
It’s been a long time since social media was simply a recreational diversion for its users. While the early days of social networks were dominated by the excitement of reconnecting with old school friends and staying in touch with distant relatives, they have continued to grow rapidly, and in the process have become embedded in every aspect of society.
Today, it’s unremarkable to hear a tweet being read out on the news – fifteen years ago when Twitter was founded, having social media form part of current affairs reporting would have been unimaginable. This growth has been so fast that it’s easy to believe that we have hit a ceiling and that these platforms couldn’t take center stage any more strongly than they already have.
Even though we’re just a couple of months in, 2021 is shaping up to be a year which, once again, proves that belief wrong. The fact that the gravitational pull of social media on the rest of the world is continuing to grow has enormous consequences for businesses: not just the platforms themselves, but every business that deals with user-generated content.
Content moderation: a high priority with high stakes
Late last year, Gartner predicted that “30% of large organizations will identify content moderation services for user-generated content as a C-suite priority” by 2024. It’s not hard to guess why it was on their radar. All of the biggest global stories of 2020 were marked, in one way or another, by the influence of social media.
Facing the pandemic, governments across the world needed to communicate vital health information with their citizens and turned to social media as a fast, effective channel – as did conspiracy theorists and fraudsters. Over the summer, Black Lives Matter protests swept America and spread globally, sparked by a viral video and driven by online organizing. Later in the year, the drama of the US Presidential election happened as much on Facebook and Twitter as it did on American doorsteps and the nightly news.
Across these events, and more, businesses have been at pains to communicate the right things in the right ways, always aware that missteps (even the mishandling of interactions with members of the public whose communication they cannot influence) will be publicized and indelible. As Gartner summarises, social media is “besieged by polarizing content, [and] brand advertisers are increasingly concerned about brand safety and reputational risk on these platforms”.
This year, social is driving the agenda
Content moderation is therefore becoming an essential tool for operating (as almost all companies now do) online. However, while the suggestion that it will rise to be a priority for 30% of C-suites over the next three years certainly isn’t modest, it already feels like Gartner was perhaps thinking too small.
We have since seen an attack on the US Capitol which was, in large part, organized by users on Parler; a mini-crisis on Wall Street spontaneously emerging from conversations on Reddit; and, most recently, an argument between Facebook and the Australian government which resulted in a number of official COVID-19 communications pages on the platform being temporarily blocked.
These are not just social media reactions to ongoing external stories – they are events driven by social media, with user-generated content at their heart. The power of social platforms to affect people, businesses and society at large have not peaked yet.
That’s the context that the UK’s Online Safety Bill and the EU’s Digital Services Act are emerging into, promising to apply new rules and give governments greater influence. As we wait for such legislation to come into force, however, there are immediate questions to consider: how should social platforms move forward, and how should businesses mitigate their own risks?
The path forward for content moderation
These are fraught questions. One reason for the reticence of social media giants to speak openly about content moderation may be that, simply, outlining new processes for ensuring user safety could be taken as an admission of past failure. Another is that content moderation is too often seen as being just one small, careless step away from censorship – which is an outcome nobody wants to see. For businesses that rely on social, meanwhile, handling a flood of content across multiple platforms and their own sites can quickly become overwhelming and unmanageable.
For all of these challenges, the best way forward starts with having a more open conversation. Social media companies and other businesses founded on user-generated content, such as dating and marketplaces, have so far tended to be fairly quiet about innovating new content moderation approaches. We can say from experience, however, that in private many such businesses are actively seeking new technology and smarter approaches. As with any common goal, collaboration and shared learning would benefit all partners here.
It’s encouraging to see partnerships like the Global Alliance for Responsible Media sowing the seeds of these conversations, but more is needed. For our part, Besedo believes that the right technology and processes can make censorship-free moderation a reality. This is not just about the technical definition of censorship: it’s about online spaces that feel fair, allowing free speech but not hate speech within clear rules.
We also believe that good moderation will spread the benefits of social media and user-generated content to everyone. Ultimately, this is now a key part of how we buy, learn, work, and live, and everyone from multinationals to small businesses to end-users need it to be safe. Finding new ways to answer the challenges of harmful content is in everyone’s best interests.
Amongst all of this, of course, one thing is certain: in 2021, content moderation will not be missing from anyone’s radar.
Every month we collect insights; from the clients, we work with, through external audits, and from mystery shopping on popular marketplaces across the world. The goal is to understand current global trends; within online marketplace scam, fraud, and other content challenges and to track how it evolves and changes over time.
The information is shared with clients and internally in our operations with our teams. Recurring trends are also used in the training of new content moderation specialists and to build new generic filters for Implio and to support the training of AI models.
Here’s an overview of some of February’s moderation trends:
Courier frauds increased by 107%
In February we saw a concerning increase in “courier frauds” with 107% more compared to normal levels. Courier fraud is a scammer pretending to be interested in buying an item, then asking the seller to register at a fake courier site. Once the victim has registered, they’re asked to share their credit card information. To circumvent moderation, scammers often redirect the conversation of the marketplace and the scam is performed through offline communication platforms like WhatsApp. However, with good moderation processes and awareness of how the fraudsters operate, users can be protected.
New console releases are still a major scam driver.
Together with cell phones, which remains the top targeted category for scammers with 39% of all scams, consoles are still leading the challenge by constituting 24.66% of fraudulent cases. Most scams in these 2 categories are tied to the release of the new iPhone and the launch of PlayStation 5. After rumors started floating around of a new Nintendo Switch release in 2021, we’ve also begun seeing scams related to the popular handheld console.
Marketplaces now a hub for exam cheats.
As lockdowns make physical tests an impossibility, we’ve seen a surge of offers to take tests and exams on behalf of others.
While the offers themselves may be genuine, the practice is unethical and if discovered could lead to students being expelled and a devaluation of the educational system. As such we generally recommend removing listings advertising these sorts of services.
Valentine’s Day scammers tried to be extra cuddly.
During the lead-up to Valentine’s Day, we did an audit of 6 popular, non-client marketplaces and saw a worrying number of scams. In particular, puppy scams were abundant. In one instance 90% of all puppy listings were fraudulent. The issue isn’t only limited to Valentine’s Day either.
Due to pandemic enforced social distancing and recurring lockdowns, there’s been a rise in pet purchases over the past year and scammers are taking advantage. As such it pays to stay vigilant and keep an extra focus on pet-related listings and categories.
With this quick overview of current trends, we hope to provide you with the tools needed to focus your content moderation efforts where they’re most needed. If you would like input specifically for your site, feel free to reach out.
Every year people flock to online marketplaces to look for presents for their significant others. This year’s ‘lockdown’ Valentine’s Day will be no different, in fact, more people than ever are likely to shop online. As such it’s important that marketplaces remain vigilant and have the right processes in place to protect their end-users from fraudulent products and sellers during a spike in sales. Especially this year where there will be an influx of buyers, who aren’t accustomed to online shopping and as such may more easily fall victim to scammers.
To give a snapshot of the risks and how dangerous it is to purchase your Valentine’s gift online this year, we’ve investigated listings of popular items on online marketplaces. After analyzing nearly three thousand listings in the run-up to this year’s Valentine’s Day, we’ve found that:
- 13% of items reviewed showed signs of being fraudulent or dangerous
- Of particular concern were the newly launched PlayStation 5 and offers of puppies, where 30% and 25% of listings across all marketplaces were found to be fraudulent, respectively
- 22% of listings for popular consumer tech, such as the iPhone 12 were also deemed fraudulent
- 1 in 10 Louis Vuitton Perfume listings were found to be either scams or counterfeit goods
- 17% of Creed perfume listings on one popular auction website were found to be fraudulent
The research shows that even after any filtering and user protection measures that these marketplaces have, a significant number of the products for sale are leaving shoppers open to losing their money or receiving fake goods.
When it comes to typical romantic gifts such as perfumes and beauty products, the buyer’s inability to touch and see items whilst online shopping means that it is easier for scammers to get away with selling fake items. 14% of Louis Vuitton Perfume listings were found to be either scams or counterfeit goods. Creed perfume is also a popular target and on one popular auction website we found that up to 17% of Creed perfume listings were fraudulent.
Perhaps most worrying, however, are the listings for puppies. Pets are always a careful purchase, and much more meaningful to the couples that get them for Valentine’s Day than consumer products. Out of the 250 listings for puppy purchases that Besedo reviewed in January and February, 25% were found to be scams.
How to protect your users during Lockdown Valentine’s Day and beyond?
In our experience, Valentine’s Day scams start picking up 2-3 weeks before the 14th of February and taper off on the day.
While many of the items targeted by scammers during the Valentine’s period overlap with goods they’d generally use to prey on unsuspecting victims, there are things you can do to increase security for users under the duration of the event.
- Monitor popular electronics extra vigilantly
- Publish targeted guidelines teaching users how to spot and avoid scams
- Put extra focus on onsite chat messages between users. Scammers may use 1to1 messages to send fake online greeting cards that link to malicious programs or to flirt with lonely and vulnerable users to get personal information or monetary gains.
- Rentals for romantic getaways. Although this year with many countries in lockdown we expect to see less activity here.
On top of scammers, there’s also a risk of rising in services or goods you may not want on your marketplace.
Keep an eye out for:
- A rise in sex toys and adult movies and services
- Detective services aimed at catching significant others in the act of cheating
Whether you want to allow these services on your site depends on your audience, but it’s worth monitoring to maintain control.
To learn more read the article 4 ways to keep your users safe online this valentine’s day.
If you need help improving your content moderation processes, get in touch with our team.
It’s the job of both AI and human content moderators to ensure the material being shared is not illegal or inappropriate: always acting in the best interest of the end-users.
And if you’re getting the content right for your end-users, they’re going to want to return and hopefully bring others with them. But is content moderation actually a form of censorship?
If every piece of content added to a platform is checked and scrutinized – isn’t ‘moderation’ essentially just ‘policing’? Surely, it’s the enemy of free speech?
Well actually, no. Let’s consider the evidence.
Moderating content vs censoring citizens
Content moderation is not a synonym for censorship. In fact, they’re two different concepts.
Back in 2016, we looked at this in-depth in our Is Moderation Censorship? article – which explains the relationship between content moderation and censorship. It also gives some great advice on empowering end-users so that they don’t feel censored.
But is it really that important in the wider scheme of things?
Well, content moderation continues to make headline news due to the actions taken by high-profile social media platforms, like Twitter and Facebook, against specific users – including, but not limited to, the former US President.
There’s a common misconception that the actions taken by these privately-owned platforms constitute censorship. In the US, this can be read as a violation of the First Amendment rights in relation to free speech. However, the key thing to remember here is that the First Amendment protects citizens against government censorship.
That’s not to say privately-owned platforms have an inalienable right to censorship, but it does mean that they’re not obliged to host material deemed unsuitable for their community and end-users.
The content moderation being enacted by these companies is based on their established community standards and typically involves:
- Blocking harmful or hate-related content
- Labeling content correctly
- Removing potentially damaging disinformation
- Demonetizing pages by removing paid ads and content
These actions have invariably impacted individual users because that’s the intent – to mitigate content which breaks the platform’s community standards. In fact, when you think about it, making a community a safe place to communicate actually increases the opportunity for free speech.
“Another way to think about content moderation is to imagine an online platform as a real world community – like a school or church. The question to ask is always: would this way of behaving be acceptable within my community?”
It’s the same with online platforms. Each one has its own community standards. And that’s okay.
Content curators – Still culpable?
Putting it another way, social media platforms are in fact curators of content – as are online marketplaces and classified sites. When you consider the volume of content being created, uploaded, and shared monitoring it is no easy feat. Take, for example, YouTube. As of May 2019, Statista reported that in excess of 500 hours of video were uploaded to YouTube every minute. That’s just over three weeks of content per minute!
These content sharing platforms actually have a lot in common with art galleries and museums. The items and artworks in these public spaces are not created by the museum owners themselves –they’re curated for the viewing public and given contextual information.
That means the museums and galleries share the content but they’re not liable for it.
However, an important point to consider is, if you’re sharing someone else’s content there’s an element of responsibility. As a gallery owner, you’ll want to ensure it doesn’t violate your values as an organization and community. And like online platforms, art curators should have the right to take down material deemed to be objectionable. They’re not saying you can’t see this painting; they’re saying, if you want to see this painting you’ll need to go to a different gallery.
What’s the benefit of content moderation to my business?
To understand the benefits of content moderation, let’s look at the wider context and some of the reasons why online platforms use content moderation to help maintain and generate growth.
Firstly, we need to consider the main reason for employing content moderation. Content moderation exists to protect users from harm. Each website or platform will have its own community of users and its own priorities in terms of community guidelines.
“Where there is an opportunity for the sharing of user-generated content, there is the potential for misuse. To keep returning to a platform or website, users need to feel a sense of trust. They need to feel safe.”
Content moderation can help to build that trust and safety by checking posts and flagging inappropriate content. Our survey of UK and US users showed that even on a good classified listing site, one-third of users still felt some degree of mistrust.
Secondly, ensuring users see the right content at the right time is essential for keeping them on a site. Again, in relation to the content of classified ads, our survey revealed that almost 80% of users would not return to the site where an ad lacking relevant content was posted – nor would they recommend it to others. In effect, this lack of relevant information was the biggest reason for users clicking away from a website. Content moderation can help with this too.
Say you run an online marketplace for second-hand cars, you don’t want it to suddenly be flooded with pictures of cats. In a recent example from the social media site Reddit, the subreddit r/worldpolitics started getting flooded with inappropriate pictures because the community was tired of it being dominated by posts about American politics and that moderators were frequently ignoring posts that were deliberately intended to gain upvotes. Moderating and removing the inappropriate pictures isn’t censorship, it’s directing the conversation back to what the community originally was about.
Thirdly, content moderation can help to mitigate against scams and other illegal content. Our survey also found that 72% of users who saw inappropriate behavior on a site did not return.
A prime example of inappropriate behavior is hate speech. Catching it can be a tricky business due to coded language and imagery. However, our blog about identifying hate speech on dating sites gives three great tips for dealing with it:
Three ways to regulate content
A good way to imagine content moderation is to view it as one of three forms of regulation. This is a model that’s gained a lot of currency recently and it really helps to explain the role of content moderation.
Firstly, let’s start with discretion. In face-to-face interactions, most people will tend to pick up on social cues and social contexts which causes them to self-regulate. For example, not swearing in front of young children. This is personal discretion.
When a user posts or shares content, they’re making a personal choice to do so. Hopefully, for many users discretion will also come into play: will what I’m about to post cause offense or harm to others? Do I want others to feel offended?
Discretion tells you not to do or say certain things in certain contexts. We all get it wrong sometimes, but self-regulation is the first step in content moderation.
Secondly, at the other end of the scale, we have censorship. By definition, censorship is the suppression or prohibition of speech or materials deemed obscene, politically unacceptable, or a threat to security.
Censorship has government-imposed law behind it and carries the message that the censored material is unacceptable in any context because the government and law deem it to be so.
Thirdly, in the middle of both of these, we have content moderation.
“Unlike censorship, content moderation empowers private organizations to establish community guidelines for their sites and demand that users seeking to express their viewpoints are consistent with that particular community’s expectations.”
This might include things like flagging harmful misinformation, eliminating obscenity, removing hate speech, and protecting public safety. Content moderation is discretion at an organizational level – not a personal one.
Content moderation is about saying what you can and can’t do in a particular online social context.
So what can Besedo do to help moderate your content?
- Keep your community on track
- Facilitate the discussion you’ve built your community for (your house, your rules)
- Allow free speech, but not hate speech
- Protect monetization
- Keep the platform within legal frameworks
- Keep a positive, safe, and engaging community
All things considered, content moderation is a safeguard. It upholds the ‘trust contract’ users and site owners enter into. It’s about protecting users, businesses, and maintaining relevance.
The internet’s a big place and there’s room for everyone.
To find out more about what we can do for your online business contact our team today.
If you want to learn more about content moderation, take a look at our handy guide. In the time it takes to read, another 4,000 YouTube videos will have been uploaded!
Self-regulation is never easy. Most of us have, at some point, set ourselves New Year’s resolutions, and we all know how hard it can be to put effective rules on our own behavior and stick to them consistently. Online communities and platforms founded in the ever-evolving digital landscape may also find themselves in a similar predicament: permitted to self-regulate, yet struggling to consistently provide protection for users. Governments have noticed. Different standards and approaches to online user safety during the last two decades has left them scratching their heads, wondering how to protect users without compromising ease of use and innovation.
Yet, with the pandemic giving rise to more consumers using these platforms to shop, date, and connect in a socially distanced world, the opportunity for fraudulent, harmful, and upsetting content has also risen. As a result, the era of self-regulation – and specifically the ability to use degrees of content moderation – is coming to an end. In fact, during the first lockdown in 2020, the UK fraud rate alone had risen by 33%, according to research from Experian.
In response, legislation such as the Online Safety Bill and the Digital Services Act is set to change the way platforms are allowed to approach content moderation. These actions have been prompted by a rapid growth in online communities which has come with a rise in online harassment, misinformation, and fraud. This often affects the most vulnerable users: statistics from the British government published last year, for example, suggest that one in five children aged 10-15 now experience cyberbullying.
Some platforms have argued that they are already doing everything they can to prevent harmful content and that the scope for action is limited. Yet, there are innovative new solutions, expertise, and technology, such as AI which can help platforms ensure such content does not slip through the net of their moderation efforts. There is an opportunity to get on the front foot when tackling these issues and safeguarding their reputations.
And, getting ahead in the content moderation game is important. For example, YouTube only sat up and took notice of the issue when advertisers such as Verizon and Walmart pulled adverts because they were appearing next to videos promoting extremist views. Faced with reputational and revenue damage, YouTube was forced to get serious about preventing harm by disabling some comments sections and protecting kids with a separate, more limited app. While this a cautionary tale, when platforms are focused on different priorities such as improving search, monetization, and user numbers, it can be easy to forget content moderation, leaving it to an afterthought until it’s too late.
The Online Safety Bill: new rules to manage social media chaos
In the UK, the Online Safety Bill will hold big tech responsible on the same scale at which it operates. The legislation will be social media-focused, applying to companies which host user-generated content that can be accessed by British users, or which facilitate interactions between British users. The duties that these companies will have under the Online Safety Bill will likely include:
- Taking action to eliminate illegal content and activity
- Assessing the likelihood of children accessing their services
- Ensuring that mechanisms to report harmful content are available
- Addressing disinformation and misinformation that poses a risk of harm
Companies failing to meet these duties will face hefty fines of up to £18m or 10% of global revenue.
The Digital Safety Act: taking aim at illegal content
While the Online Safety Bill targets harmful social content in the UK, the Digital Services Act will introduce a new set of rules to create a safer digital space across the EU. These will apply more broadly, forcing not just social media networks, but also e-commerce, dating platforms, and, in fact, all providers of online intermediary services to remove illegal content.
The definition of illegal content, however, is yet to be defined: many propose that this will relate not only to harmful content but also content that is fraudulent, which offers counterfeit goods, or even content that seeks to mislead consumers, such as fake reviews. This means that marketplaces may become directly liable if they do not correct the wrongdoings of third-party traders.
How to get ahead of the legislation
Online communities might be worried about how to comply with these regulations, but ultimately it should be seen as an opportunity for them to protect their customers, while also building brand loyalty, trust, and revenue. Finding the right content moderation best practice, processes, and technology, in addition to the right expertise and people, will be the cornerstone to remaining compliant.
Businesses often rely on either turnkey AI solutions or entirely human teams of moderators, but as the rules of operation are strengthened, bespoke solutions that use both AI and human intervention will be needed to achieve the scalability and accuracy that the new legislation demands. In the long term, the development of more rigorous oversight for online business – in the EU, the UK, and elsewhere across the world – will benefit companies as well as users.
In the end, most, if not all, platforms want to enable consumers to use services safely, all the time. Browsing at a toy store in Düsseldorf, purchasing something from Amazon, making a match on a dating app, or connecting on a social network should all come with the same level of protection from harm. When everyone works together, a little bit harder, to make that happen, it turns from a complex challenge into a mutual benefit.
The Christmas season is here and while the festivities kick off online retailers hold their breath and wait to see whether all of the preparations they have diligently made will pay off in revenue and sales during this ‘Golden Quarter.’ Will the website be able to handle extra demand? Will all orders be able to be shipped before Christmas?
Yet, The National Cyber Security Centre (NCSC) has highlighted another pressing concern which can have a lasting impact on revenue. Last week it launched a major awareness campaign called Cyber Aware advising potential customers to be aware of an increase in fraud on online platforms this year. This is because millions of pounds are stolen from customers through fraud every year – including a loss of £13.5m from November 2019 to the end of January 2020 – according to the National Fraud Intelligence Bureau.
Fraud is a major concern for marketplaces who are aware of the trust and reputational damage that such nefarious characters on their platform can create. While consumer awareness and education can help, marketplaces know that only keeping one eye on the ball when it comes to fraud, especially within User Generated Content (UGC), is not enough. Fraudulent activity deserves full attention and careful monitoring. Trying to tackle fraud is not a one-off activity but a dedication to constant, consistent, rigorous, and quality moderation where learnings are continuously applied, for the on-going safety of the community.
With that in mind, our certified moderators investigated nearly three thousand listings of popular items on six popular UK online marketplaces, in order to understand whether marketplaces have content moderation pinned down, or, whether fraudulent activity is still slipping through the net. After conducting the analysis during the month of November, including the busy Black Friday and Cyber Monday shopping weekend, we found that:
· 15% of items reviewed showed signs of being fraudulent or dangerous, this rose to 19% on Black Friday and Cyber Monday
· Pets and popular consumer electronics are particular areas of concern, with 22% of PlayStation 5 listings likely to be scams, rising to more than a third of PS5 listings being flagged over the Black Friday weekend
· 19% of listings on marketplaces for the iPhone 12 were also found to show signs of being scams
· Counterfeit fashion items are also rife on popular UK marketplaces, with 15% of listings found to be counterfeits.
The research demonstrates that, even after any filtering and user protection measures marketplaces have a significant number of the products for sale on them are leaving customers open to having their personal details stolen or receiving counterfeit goods. We know that many large marketplaces have a solution in place already, but are still allowing scams to pass through the net, while smaller marketplaces may not have thought about putting robust content moderation practices and processes in place.
Both situations are potentially dangerous if not tackled. While it is certainly a challenging process to quickly identify and remove problematic listings, it is deeply concerning that we are seeing such a high rates of scams and counterfeiting in this data. Powerful technological approaches, using AI in conjunction with human analysts, can very effectively mitigate against these criminals. Ultimately, it should be the safety of the user placed at the heart of every marketplace’s priorities. It’s a false dichotomy that fail safe content moderation is too expensive a problem to deal with – in the longer term, addressing even small amounts of fraud that is slipping through the net can have a large and positive long term impact on the financial health of the marketplace through increased customer trust, acquisition and retention.
2020 was a year we would not want to repeat from a fraud perspective – we have not yet won the battle against criminals. As we move into 2021, we’ll be hoping to help the industry work towards a zero-scam future, one where we take the learnings and lessons together from 2020 to provide a better, safer community for users and customers, both for their safety, but also for the long term, sustainable and financial health of marketplaces.