Hate speech detection and sentiment analysis in AVALANCHE

From Transformers to Trust

Every second, thousands of posts, articles, and comments appear online. These digital conversations shape public opinion, influence social dynamics, and reflect how societies think, react, and evolve in real time.

But alongside meaningful discussion, harmful and abusive content also emerges.

At the scale of today’s internet, manual moderation is no longer enough. Human reviewers cannot realistically inspect every article, post, or comment. Artificial intelligence becomes essential, not as a replacement for human judgment, but as a tool that helps people understand, prioritize, and respond to online content more effectively.

In the AVALANCHE project, we developed in-house machine learning models that automatically analyze text, detect hate speech, and evaluate sentiment. These models are designed not only to classify content, but also to understand context, operate efficiently in production environments, and explain their decisions transparently.

Building this kind of system requires more than training a model. It requires the right architecture, explainability, scalable engineering, and a clear understanding of where AI can help and where human judgment remains essential.

Keyword-based detection is not enough

Early approaches to harmful content detection often relied on keyword matching. These systems were simple and fast, but they had a major limitation: they could identify words, not meaning.

Language is complex. The same phrase can appear in very different contexts.

For example, offensive language may appear in:

• a genuinely harmful statement,
• a journalist quoting someone else,
• an academic or legal discussion,
• or a post condemning hate speech.

A keyword-based system treats these cases similarly because it only checks whether certain words appear. It does not understand how those words are being used.

AVALANCHE addresses this limitation using transformer-based language models, derived from the BERT architecture. Instead of analyzing words in isolation, these models examine how words relate to one another within a sentence or document.

In other words, the question changes from “Does this word appear?” to “What does this word mean in this context?”

This shift from keyword detection to contextual understanding significantly improves the system’s ability to detect harmful content more accurately.

2 models, 2 perspectives

AVALANCHE integrates two complementary models that analyze content from different angles.

The first model focuses on sentiment analysis. It evaluates the emotional tone of a text and classifies it as positive, neutral, or negative. This helps identify emotionally charged content and provides useful context for analysts.

The second model focuses on hate speech detection. It identifies content that may contain abusive, harmful, or targeted language directed at individuals or groups.

Both models rely on transformer encoders to generate contextual representations of text. These representations allow the system to capture nuance, tone, and semantic relationships between words.

Together, the two models give AVALANCHE a richer understanding of online content. Sentiment analysis helps describe the emotional tone, while hate speech detection focuses on potential harm.

Making AI work in the real world

Accuracy is only one part of the challenge.

In production environments, AI systems must also be fast, reliable, and scalable. Online articles and documents can be long, sometimes containing thousands of words. Processing them efficiently without losing important meaning requires careful engineering.

AVALANCHE addresses this challenge through embedding-based summarization during inference.

Instead of processing every part of a long document in the same way, the system identifies the most informative parts of the text and generates embeddings that capture its overall meaning.

This improves:

• processing speed,
• prediction consistency,
• scalability across large volumes of content,
• and the ability to operate efficiently in real-world pipelines.

These optimizations allow AVALANCHE’s models to work reliably within production infrastructure, where performance matters as much as accuracy.

Confidence Scores: knowing when the model is certain

Machine learning predictions are not always absolute.

Some classifications are clear. Others are more uncertain, especially when the language is subtle, ambiguous, or highly contextual.

For this reason, AVALANCHE provides confidence scores alongside each prediction. These scores indicate how strongly the model supports a particular classification.

This is important because confidence scores help analysts prioritize their attention. A highly confident prediction may require less immediate review, while a low-confidence case may require human inspection.

This approach reinforces the role of AI as a decision-support system. The model does not replace analysts. It helps them focus on the cases where their judgment is most valuable.

Explainable AI: opening the black box

One of the biggest challenges with modern AI is explainability.

Deep learning models can produce accurate predictions, but they often do not clearly explain how they reached them. This lack of transparency can make it difficult for developers, analysts, and users to trust the system.

To address this, AVALANCHE integrates SHAP –  SHapley Additive exPlanations. SHAP helps identify which words or tokens contributed most to a model’s decision. This means analysts can see which parts of the text influenced a classification toward hate speech, negative sentiment, or a safer category.

This transparency has several benefits.

It helps developers understand model behavior and improve performance. It helps analysts verify predictions. Most importantly, it helps build trust in the system.

Explainability turns AI from a black box into a transparent decision-support tool.

The hardest challenge: understanding intent

Despite major advances in natural language processing, understanding human intent remains one of the hardest problems in AI.

Consider two examples:

• An author expresses harmful beliefs.
• An author reports someone else’s harmful statement.

The words may be similar, but the intent is completely different.

Humans often understand this distinction naturally. Machines must infer it from patterns, structure, and context.

Transformer-based models significantly improve contextual understanding, but subtle cases remain challenging. This is especially true for long-form content, where meaning may depend on relationships across multiple sentences or paragraphs.

For AVALANCHE, improving contextual understanding remains an important area of ongoing development.

From research to production

A model that performs well in experimentation is not enough.

To create real value, it must function reliably in production.

AVALANCHE integrates its models into scalable infrastructure using optimized APIs and inference pipelines. This allows the system to analyze content efficiently and deliver results close to real time.

The models are also continuously monitored, evaluated, and improved to maintain performance and reliability over time.

This combination of machine learning and software engineering is what transforms research into a robust, production-ready system.

AI as a partner – not a replacement

Artificial intelligence is often presented as a replacement for human judgment. In practice, its greatest value lies in augmentation.

AVALANCHE’s models help humans process information faster, identify risks earlier, and analyze online content more effectively.

By combining transformer-based architectures, explainable AI, confidence scoring, and optimized deployment pipelines, AVALANCHE provides a powerful platform for understanding digital communication at scale.

Language is complex. Context is subtle. Human communication is deeply nuanced.

AI may never perfectly understand every aspect of language. But systems like AVALANCHE represent an important step forward.

They help transform overwhelming volumes of digital content into actionable insight.

And in a world increasingly shaped by online communication, that capability has never been more important.