AI Toxic Language Detection Project

Artificial Intelligence project 14

AI Toxic Language Detection Project

Build a complete toxic language detection workflow with reproducible Python code, transparent inputs, reviewable output, and responsible-use limits.

Explore AI training in VizagView all project ideas

AI objective

Estimate a toxicity probability while marking uncertain results for review.

Data or knowledge source

Sixteen small English teaching examples balanced between ordinary disagreement and direct insults.

Requirements

  • Python 3.10 or later
  • A terminal or command prompt
  • python -m pip install scikit-learn
  • About 45-60 minutes to build and review

How the system works

Train class-weighted logistic regression on TF-IDF word and phrase features, then apply a decision threshold and a wider review band.

Validation checklist

  • Safe disagreement examples are included
  • Probability remains visible
  • Uncertain cases trigger review
  • No account or user action is automated

Complete Python code

Save the program as ai_toxic_language_detector.py. The code runs locally and requires no paid API key or model download.

"""A small toxicity-classification teaching baseline with human review."""

from __future__ import annotations

from dataclasses import dataclass
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline


SAFE = ["thank you for the explanation", "I disagree with this result", "please review the issue", "this feature needs improvement", "can someone help me", "the answer is incorrect", "I am frustrated by the delay", "let us discuss another approach"]
TOXIC = ["you are an idiot", "shut up you fool", "everyone here is stupid", "I hate you moron", "you are worthless", "what a dumb person", "get lost loser", "only an idiot would say that"]


@dataclass(frozen=True)
class ToxicityResult:
    toxic_probability: float
    label: str
    needs_review: bool


def train_detector() -> Pipeline:
    texts = SAFE + TOXIC
    labels = [0] * len(SAFE) + [1] * len(TOXIC)
    return Pipeline([("tfidf", TfidfVectorizer(ngram_range=(1, 2), sublinear_tf=True)),
                     ("model", LogisticRegression(max_iter=1000, class_weight="balanced", random_state=42))]).fit(texts, labels)


def assess(model: Pipeline, text: str, threshold: float = 0.6) -> ToxicityResult:
    probability = float(model.predict_proba([text])[0, 1])
    return ToxicityResult(probability, "toxic" if probability >= threshold else "not_toxic", 0.4 <= probability < 0.7)


def main() -> None:
    model = train_detector()
    for text in ("Please review the incorrect answer", "You are a stupid fool"):
        print(text, "->", assess(model, text))
    print("Tiny English examples are not suitable for automated moderation or decisions about people.")


if __name__ == "__main__":
    main()

Run the project

  1. Create and activate a virtual environment.
  2. Install dependencies with python -m pip install scikit-learn when packages are required.
  3. Run python ai_toxic_language_detector.py.
  4. Review confidence, fallbacks, sources, or error metrics rather than accepting output automatically.
  5. Test additional normal, edge, unsupported, and adversarial inputs.

Expected output

Toxic probability, teaching label, and needs-review flag.

Accuracy, privacy, and responsible-use limits

The tiny English dataset misses context, dialect, reclaimed language, harassment patterns, quotations, and power dynamics. Do not use it for automated moderation or decisions about people.

Ways to extend the project

Create careful annotation guidance, multilingual and dialect evaluation, context windows, subgroup error review, calibrated thresholds, and appeals.

Continue learning Artificial Intelligence

Try the next project, return to the Softenant project library, or explore the AI training in Vizag for guided NLP, retrieval, evaluation, automation, and responsible AI practice.