How to Fix OSError: [Errno 28] inotify watch limit reached in Python / Streamlit

inotify_watch_limit.md

Here is a guide to understanding and resolving the common OSError: [Errno 28] inotify watch limit reached, often encountered when running Python applications using frameworks like Streamlit, Flask, or FastAPI on Linux-based systems.

---

The Question: What Does inotify watch limit reached Mean?

You've just deployed or updated your Python application, and it suddenly crashes with a cryptic error:

OSError: [Errno 28] inotify watch limit reached

This error appears in your logs, often after a long traceback involving libraries like watchdog or streamlit.watcher. What is the system trying to tell you, and how can you fix it?

The Answer: Understanding the Problem and How to Solve It

What is inotify?

inotify is a Linux kernel subsystem that provides a mechanism for applications to monitor filesystem events. In simple terms, it allows a program to be notified when a file or directory is created, modified, deleted, or accessed.

Frameworks like Streamlit use this feature to power their "auto-reload" magic. By "watching" your source code files, Streamlit instantly knows when you save a change and can automatically rerun your script to update the app.

Why Does the Error Happen?

To prevent abuse and conserve system resources, Linux imposes a limit on the number of files and directories a single user can watch at one time. This limit is defined by a system setting called max_user_watches.

The inotify watch limit reached error occurs when your application tries to watch more files and folders than this system limit allows. This is a common issue in projects that contain:

• A large number of source files.
• Virtual environment directories (venv, .venv).
• Node.js dependencies (node_modules).
• Python cache directories (__pycache__).
• Large datasets or asset folders.

Your application, trying to be helpful, attempts to watch every single item, quickly exhausting the available inotify watches.

---

The Solutions

Here are two effective ways to solve this problem. The best one for you depends on your level of control over the deployment environment.

Solution 1: Tell Your Application What to Ignore (Recommended)

This is the easiest and most portable solution, especially if you are deploying on a shared platform like Streamlit Community Cloud, Heroku, or another service where you don't have administrative (sudo) access.

The idea is to tell your application not to watch directories that don't contain runnable source code.

For Streamlit Applications:

1. In the root of your project, create a directory named .streamlit if it doesn't already exist.

2. Inside this directory, create a file named config.toml.

3. Add the following configuration to your config.toml file:

   [server]
   # A list of folders that should not be watched for changes.
   # This can be used to limit the number of files watched in large projects.
   # Add any folders with a large number of files that don't need to trigger a rerun.
   folderWatchBlacklist = ["venv", ".venv", "__pycache__", "node_modules", "your_large_data_folder"]

This configuration explicitly tells Streamlit to ignore these folders, drastically reducing the number of inotify watches it consumes. You can add any other large, non-essential directories to this list.

Solution 2: Increase the System-Wide inotify Limit (Requires sudo)

If you have administrative (sudo) access to your server, VM, or Docker container, you can directly increase the system's inotify watch limit.

1. Check the Current Limit: See what your current limit is. It's often a low number like 8192.

cat /proc/sys/fs/inotify/max_user_watches

2. Increase the Limit Temporarily (Resets on Reboot): You can set a new, higher limit. 524288 is a common, safe value for development machines.

sudo sysctl fs.inotify.max_user_watches=524288

3. Increase the Limit Permanently: To make the change survive reboots, add the setting to the system's configuration file.

echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

Summary

While increasing the system limit (Solution 2) is a valid fix, it's often better to solve the problem at the application level (Solution 1). By configuring your application to ignore unnecessary directories, you create a more efficient and portable project that is less likely to run into environment-specific limitations.

Recommendation: Always try Solution 1 first. It's the cleanest approach and resolves the issue in most cases without requiring special permissions.