Do you have a machine learning (ML) solution developed in-house, or do you consider building one? We have seen initiatives like this fail time and again. It usually ends up with frustrated quality managers in conflict with developers over prioritization and quality definitions.
This blog post shows you why you should focus your development resources on your core product and consider sourcing a custom made moderation solution.
Assessing Perceived Benefits
Stop! Before you read on, ask yourself one question. Why are you even developing a tool in-house? What are you trying to achieve? Do you believe it to be more cost-efficient? More flexible? Better tailored to your needs? If your answer is yes, you are about to be disappointed. Our extensive experience working with and advising clients on moderation has proven that time and time again the in-house benefits fail to deliver against all expectations. Let’s look at those perceived benefits one by one.
If you think that finding competent developers is difficult and expensive, then wait until you start looking for people with ML capabilities. To create a state of the art Artificial Intelligence (AI) moderation engine you’ll need a team of data scientists and analysts. ML has a lot more to do with statistics than coding and guys with this type of competency are hard to find and come at a premium.
Were you thinking of utilizing your current in-house dev-team? Cross-utilization is a nice dream, but in reality you will find that skilled programmers are not necessarily great machine learning model makers. Even if your current team has the capabilities of creating your ML models, you will find that supporting tools like these tend to get down-prioritized when new features and bugs need to be fixed on your core product. The complaint we hear from quality managers is repeatedly that they end up with lackluster tools that don’t get updated often enough.
With a subpar machine learning moderation solution, you risk more false positives, leading to unhappy customers and more unwanted content slipping through upping the user risk.
If you buy a ready-made product that’s specialized for the needs of your industry, you will in-directly leverage all the experience and knowledge that the development team has acquired throughout the years of building this type of tool. You’re basically buying 100% dedication, competency and ongoing training on new algorithms and better models.
Do you believe that an in-house solution can adapt faster when you need to make changes to the models? Well, probably not! A professional solutions provider will be able to customize the machine learning model and accommodate your needs at least as quickly, as you could if you had your own dedicated in-house team. Your solution partner probably has a better understanding of moderation needs and knows how important speed and flexibility are to keep up the quality of your content. They are dedicated to moderation and won’t risk being tied up with other tasks, which allows for super-quick updates and model tweaks when needed.
You know your industry and product like the backside of your hand, we won’t argue with that. But honestly spoken, can you call yourself an expert in automated content moderation? An experienced solutions provider has the knowledge and can draw on experience from working with sites of all sizes and growth stages and with content ranging from marketplace ads to dating profiles. At Besedo, for example, we have dedicated the last 15 years’ to spotting and dealing with new and upcoming moderation challenges.
If we at Besedo were asked to select the one thing that we’re most excited about when it comes to our Machine Learning solution for moderation, it’s the benefit of being part of a bigger moderation ecosystem.
We have combined filter moderation, machine learning and a manual moderation panel in one tool. This means you only ever have to integrate once. There’s no risk of one tool breaking if another is updated and most importantly it allows for a seamless feedback loop from manual moderation to continuously improve the machine learning model.
If you consider building a similar solution in-house, make sure to investigate how long it would it take and if those resources couldn’t be used better improving your core product.
Specialized Machine Learning Moderation Solutions vs. Generic AI Models
Some companies we have spoken to about ML moderation have mentioned that they are trying generic AI solutions to see if they can solve their content challenges. And while we understand the lure, reality is that Machine learning itself isn’t necessarily going to help you. In the end, it must solve the particular challenges your business has. Generic ML solutions are, as the name indicates, generic and almost never live up to the high standards marketplaces need to protect their content.
If we hear that a potential client is testing out generic solutions, we always ask them to get back to us on what results they are getting. So far the unanimous feedback has been that these kind of solutions are not working for them. It is simply too generic.
Let’s take a quick look at some of the issues with generic models.
As the name states a generic solution will be built around a generic dataset. That might work for very basic challenges, but even something as simple as catching obscene language will be hard with a generic model. The definition of what profanity is can vary wildly depending on target group and the purpose of the site.
The fact that it is generic means your accuracy level will be lower and your false positives much higher. The end result is that you will still need to do a lot of manual moderation to keep your content quality high and your users safe.
A specialized moderation solution, on the other hand, will be tailor-made using your data, meaning the AI will adhere to the unique rules and policies you have set up for your site.
A generic solution is static and won’t help you quickly adapt to changes in the world or in user behavior. It will also often have a very specific use and it is unlikely it will be able to solve all your moderation challenges.
A Car vs an Engine
One of the main issues with generic models is that they can rarely be applied without at least some tweaking. In the end, you will likely end up dedicating a lot more resources than you originally anticipated just getting the generic machine learning model to work reasonably.
When you’re looking for a car, you buy a car, not an engine to build a car. In the end, a tailor-made machine learning solution often ends up being more cost-effective, the time to market is much shorter and it has a lower maintenance cost.
So you see? Either way you look at it, if you are serious about moderation (and when you are trying to disrupt the market you should be), then you really should go with a tailor-made machine learning solution created by experts in the field.