Recommendations for Project-2 (AI Engineering Bootcamp)

AI-Eng-project-2-recommendations.md

Project-2 Recommendations

Core steps & things to practice

• EDA (e.g. make sure to check if the dataset is imbalanced)
• Data splitting
  • Option 1: Train/Test split (e.g. 80/20) + Cross-Validation
  • Option 2: Train/Validation/Test (e.g. 70/15/15)
  • In any case, use a stratified split
• Text preprocessing
  • Tokenization
  • Lowercasing (optional)
  • Removing punctuation / special characters (optional)
  • Stopword removal (optional)
  • Stemming / Lemmatization (optional)
• Feature extraction
  • Bag of Words (BoW)
  • TF-IDF
  • n-grams (used with BoW or TF-IDF)
  • Word embeddings (e.g., Word2Vec, GloVe, FastText)
  • RECOMMENDATIONS:
    • Aim to try at least 3 of those options.
    • If you use word embeddings: convert each word to a dense vector, then average word vectors across the document to get one fixed-size vector per text. For a stronger version, use a TF-IDF-weighted average (weight each word's vector by its TF-IDF score before averaging) so common words like "the" contribute less than distinctive ones.
• Model training, evaluation, and hyperparameter tuning
  • Logistic Regression
  • Multinomial Naive Bayes (note: designed for non-negative count/frequency features such as BoW and TF-IDF; not suitable for dense word embeddings)
  • Linear SVM
  • ...
  • RECOMMENDATIONS
    • Set a baseline model (e.g. BoW + Logistic Regression), then try other algorithms.
    • Linear SVM is commonly used for text classification (it may provide good results in combination with TF-IDF)

Bonus

• Transfer Learning (e.g. fine-tuning BERT or DistilBERT)
• LSTM (or Bidirectional LSTML aka Bi-LSTM)
• Deployment