machine-learning-project-structure-recommendations.md

Machine Learning Project Structure: Recommended Practices

This repository structure is a recommended way to organize machine learning projects.

It helps keep notebooks focused on exploration, moves reusable code into Python modules, and makes projects easier to understand, maintain, and reproduce.

Feel free to adapt it to your project's needs —there is no single "correct" structure.

Smaller projects may not need every directory shown here, and different teams may follow different conventions. Start with a simple structure and add folders only when they become useful.

Main directories and files

• data/
  • Directory to store datasets (raw & processed data)
• notebooks/
  • EDA + experiments (e.g., training and comparing different models)
  • Recommendation: notebooks should primarily be used for exploration, experimentation, and visualization. Once code becomes useful in multiple notebooks, move it into src/ and import it instead of copying and pasting.
• src/
  • Reusable Python code that can be imported by notebooks or scripts (for example, functions for data loading, preprocessing, feature engineering, training, evaluation, and prediction)
• models/
  • Saved models
• reports/
  • Final report, figures, metrics, presentations... (another alternative is to put this in the README file)

Possible file structure

titanic-classification/
│
├── data/
│   ├── raw/                # Original downloaded data
│   └── processed/          # Cleaned/transformed data
│
├── notebooks/
│   ├── 01_EDA.ipynb
│   ├── 02_baseline_logistic_regression.ipynb
│   ├── 03_random_forest.ipynb
│   ├── 04_xgboost.ipynb
│   ├── 05_feature_engineering.ipynb
│   ├── 06_hyperparameter_tuning.ipynb
│   └── 07_model_comparison.ipynb
│
├── src/
│   ├── data_loading.py         # Reusable functions to load the data
│   ├── data_preprocessing.py   # Reusable functions to preprocess the data
│   ├── model_training.py       # Reusable functions to train models
│   ├── evaluate.py             # Reusable functions to evaluate models
│   └── predict.py              # Reusable functions to make predictions
│
├── models/
│   ├── baseline_logistic_regression.pkl
│   ├── random_forest.pkl
│   └── xgboost.pkl
│
├── reports/
│
├── requirements.txt
├── README.md
└── .gitignore