AI Language Identification Project

Artificial Intelligence project 15

AI Language Identification Project

Build a complete language identification workflow with reproducible Python code, transparent inputs, reviewable output, and responsible-use limits.

Explore AI training in VizagView all project ideas

AI objective

Identify one of four known languages from character-pattern features.

Data or knowledge source

Thirty-two short teaching sentences across English, Spanish, French, and German.

Requirements

  • Python 3.10 or later
  • A terminal or command prompt
  • python -m pip install scikit-learn
  • About 45-60 minutes to build and review

How the system works

Train character-within-word TF-IDF features from three-to-five-character n-grams and a multiclass logistic-regression classifier.

Validation checklist

  • All four languages have equal sample counts
  • Character features tolerate some unseen words
  • Probability is returned with the label
  • Unknown and mixed-language handling is documented as missing

Complete Python code

Save the program as ai_language_identifier.py. The code runs locally and requires no paid API key or model download.

"""Identify four languages with character n-gram features."""

from __future__ import annotations

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline


SAMPLES = {
    "English": ["good morning how are you", "please send the project details", "this lesson is easy to follow", "where is the railway station", "thank you for your help", "we are learning artificial intelligence", "the weather is pleasant today", "I would like a cup of tea"],
    "Spanish": ["buenos dias como estas", "por favor envia los detalles", "esta leccion es facil de seguir", "donde esta la estacion de tren", "gracias por tu ayuda", "estamos aprendiendo inteligencia artificial", "hace buen tiempo hoy", "quisiera una taza de te"],
    "French": ["bonjour comment allez vous", "envoyez les details du projet", "cette lecon est facile a suivre", "ou est la gare", "merci pour votre aide", "nous apprenons intelligence artificielle", "il fait beau aujourd hui", "je voudrais une tasse de the"],
    "German": ["guten morgen wie geht es dir", "bitte senden sie die projektdetails", "diese lektion ist leicht zu verstehen", "wo ist der bahnhof", "danke fuer ihre hilfe", "wir lernen kuenstliche intelligenz", "das wetter ist heute angenehm", "ich moechte eine tasse tee"],
}


def train_identifier() -> Pipeline:
    texts, labels = [], []
    for language, examples in SAMPLES.items():
        texts.extend(examples)
        labels.extend([language] * len(examples))
    return Pipeline([("characters", TfidfVectorizer(analyzer="char_wb", ngram_range=(3, 5), min_df=1)),
                     ("model", LogisticRegression(max_iter=1500, random_state=42))]).fit(texts, labels)


def identify(model: Pipeline, text: str) -> tuple[str, float]:
    probabilities = model.predict_proba([text])[0]
    index = int(probabilities.argmax())
    return str(model.classes_[index]), float(probabilities[index])


def main() -> None:
    model = train_identifier()
    for text in ("muchas gracias por la ayuda", "please send the lesson", "wo ist die station"):
        print(text, "->", identify(model, text))
    print("Short, mixed-language, transliterated, and unseen languages need an unknown option and review.")


if __name__ == "__main__":
    main()

Run the project

  1. Create and activate a virtual environment.
  2. Install dependencies with python -m pip install scikit-learn when packages are required.
  3. Run python ai_language_identifier.py.
  4. Review confidence, fallbacks, sources, or error metrics rather than accepting output automatically.
  5. Test additional normal, edge, unsupported, and adversarial inputs.

Expected output

Predicted language and confidence for each input string.

Accuracy, privacy, and responsible-use limits

Short, mixed-language, transliterated, misspelled, and unsupported-language text can be misclassified. Production systems need an unknown option and broad native-speaker evaluation.

Ways to extend the project

Add Indian languages, native scripts, an unknown class, minimum length, calibrated confidence, code-switching labels, and external test data.

Continue learning Artificial Intelligence

Try the next project, return to the Softenant project library, or explore the AI training in Vizag for guided NLP, retrieval, evaluation, automation, and responsible AI practice.