-
-
Save khanhlvg/bbeb5e4ccfca6cbcf18508a44f5964be to your computer and use it in GitHub Desktop.
| ## Sep 2023 update: We've released official support for MediaPipe on Raspberry Pi. | |
| ## It provides many more features to what's available in TFLite Task Library. The guide | |
| ## below has been updated to use MediaPipe instead of TFLite Task Library. | |
| ## Check out this blog post to learn more: | |
| ## https://developers.googleblog.com/2023/08/mediapipe-for-raspberry-pi-and-ios.html | |
| # Show your Raspberry Pi OS version. | |
| cat /etc/os-release | |
| # Update packages on your Raspberry Pi OS. | |
| sudo apt-get update | |
| # Check your Python version. You should have Python 3.8 or later. | |
| python3 --version | |
| # Install virtualenv and upgrade pip. | |
| python3 -m pip install --user --upgrade pip | |
| python3 -m pip install --user virtualenv | |
| # Create a Python virtual environment for the MediaPipe samples (optional but strongly recommended) | |
| python3 -m venv ~/mp | |
| # Run this command whenever you open a new Terminal window/tab to activate the environment. | |
| source ~/mp/bin/activate | |
| # Clone the MediaPipe samples repository with the MediaPipe Raspberry Pi samples. | |
| git clone https://github.com/googlesamples/mediapipe.git | |
| cd mediapipe/examples/object_detection/raspberry_pi | |
| # Install dependencies required by the sample | |
| sh setup.sh | |
| # Run the object detection sample | |
| # **IMPORTANT**: If you SSH to the Pi, make sure that: | |
| # 1. There is a display connected to the Pi. | |
| # 2. Run `export DISPLAY=:0` before proceed to make the object_detection window appear on the display. | |
| python detect.py | |
| #### | |
| # If you see an error running the sample: | |
| # ImportError: libcblas.so.3: cannot open shared object file: No such file or directory | |
| # you can fix it by installing an OpenCV dependency that is missing on your Raspberry Pi. | |
| sudo apt-get install libatlas-base-dev |
@Harbednmez @britgould The error was a bug in the tflite-support 0.4.2. Please use the
tflite-support-nightlyfor now where the issue has been fixed.
hi, i installed tflite-support-nightly, the error keeps coming.
If I uninstall tflite-support 0.4.2 it doesn't work either, since it doesn't appear to be on tflite-support-nightly
Do you have to make any other settings?
This happens when i run detect.py
Hey @kashank, when I try to run "python detec.py", this error pops up: "ERROR: Unable to read from webcam. Please verify your webcam settings." Thanks in advance.
Hey @kashank, when I try to run "python detec.py", this error pops up: "ERROR: Unable to read from webcam. Please verify your webcam settings." Thanks in advance.
Me too!
(tflite) admin@raspberrypi:~/examples/lite/examples/object_detection/raspberry_pi $ python3 detect.py
ERROR: Unable to read from webcam. Please verify your webcam settings.
Worked it straight out from pi4 4gb ! ty @khanhlvg
VNCed to my pi without monitor on local network
(tflite) pi@pi:~/examples/lite/examples/object_detection/raspberry_pi $ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
(tflite) pi@pi:~/examples/lite/examples/object_detection/raspberry_pi $
@cedricbaril I had the same error, because I use the Raspberry Camera Module V3 and cv2 does not support that. As far as I know, only picamera2 supports the V3 camera at the moment. I can barely even program in python, so I'm no expert by any means, but I was able to make it work with the following:
-
create the virtual environment with the additional switch: --system-site-packages
so like:python3 -m venv --system-site-packages ~/tflite
I used picamera2 for the camera and it depends on libcamera to work. This allows the virtual environment to use the systems packages, otherwise libcamera would have to be installed in the virtual environment, which I tried and failed at, so this is much easier. Then activate the environment and continue with the setup. -
Install picamera2 if it's not already installed
Since the environment can use system packages and it doesn't matter if it's installed in the environment or the system itself, I'd install it on the system. (deactivate the environment for the time of the install or open a separate terminal)
pip install picamera2should work for the install, but if not, then check out the https://pypi.org/project/picamera2/ website. -
modify the detect.py file
-
to import picamera2 add:
from picamera2 import Picamera2 -
then at the "# Start capturing video input from the camera" replace:
# Start capturing video input from the camera
cap = cv2.VideoCapture(camera_id)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, width)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, height)
with:
# Start capturing video input from the camera
picam2=Picamera2()
camera_config = picam2.create_still_configuration(main={"size": (640, 480)}, buffer_count=2)
picam2.configure(camera_config)
picam2.start() -
then at "# Continuously capture images from the camera and run inference" I replaced this:
# Continuously capture images from the camera and run inference
while cap.isOpened():
success, image = cap.read()
if not success:
sys.exit( 'ERROR: Unable to read from webcam. Please verify your webcam settings.' )
with this:
# Continuously capture images from the camera and run inference
loop = True
while loop:
image1 = picam2.capture_array("main")
image = cv2.cvtColor(image1, cv2.COLOR_BGR2RGB) -
and at the end of this while loop I replaced this:
# Stop the program if the ESC key is pressed.
if cv2.waitKey(1) == 27:
break
cv2.imshow('object_detector', image)
cap.release()
cv2.destroyAllWindows()
with this:
# Stop the program if the q key is pressed.
cv2.imshow('object_detector', image)
if cv2.waitKey(10) & 0xFF == ord('q'):
loop = False
cv2.destroyAllWindows()
picam2.stop()
break
cv2.destroyAllWindows()
(stopping with the q key is just a preference for me)
This might not be the best solution, but the main idea is that with the V3 camera cv2 doesn't work and picamera2 has to be used. For more on picamera2, here is it's manual: https://datasheets.raspberrypi.com/camera/picamera2-manual.pdf Hope this helps.
I'm getting an Import error , please help
@Rakesh5173 - This should solve your issue:
-
Verify the GLIBCXX version (RaspberryPi Bullseye 64-bit should be GLIBCXX_3.4.28):
strings /lib/aarch64-linux-gnu/libstdc++.so.6 | grep GLIBCXX -
Downgrade tflite-support from v0.4.4 to v0.4.3:
python3 -m pip install --upgrade tflite-support==0.4.3
@cedricbaril @EnziinSystem - you can fix this issue by modifying the /boot/config.txt file as there is an issue with OpenCV/V4L2:
sudo nano /boot/config.txt
start_x=1
gpu_mem=128
Comment out the camera_auto_detect=1 - this is currently a valid flag.
#camera_auto_detect=1
Save the file and reboot
Everything should be up and running.
Reference (Probably not the best place to cite PyTorch.lol): https://pytorch.org/tutorials/intermediate/realtime_rpi.html#video-capture
Thank you @ldiangelis , it worked
I get this error when I try to run detect.py. I have installed all the necessary modules and dependencies. Any help would be greatly appreciated.
File "/home/pi/examples/lite/examples/object_detection/raspberry_pi/utils.py", line 48, in visualize
category = detection.categories[0]
AttributeError: 'Detection' object has no attribute 'categories'
BTW, when I comment out the part with the categories, it works fine, and just draws a bounding box but without any text.
Hi,
I'm getting this error: GLIBCXX_3.4.29 not found when I try to run detect.py (TensorFlow) on RPI4. I have tried to install [sudo apt-get install libatlas-base-dev] but did not work and still having the same error. I have followed same steps in the video and same code but I am stuck at this stage. Can someone help please? Thanks @khanhlvg
I get this error when I try to run detect.py. I have installed all the necessary modules and dependencies. Any help would be greatly appreciated.
File "/home/pi/examples/lite/examples/object_detection/raspberry_pi/utils.py", line 48, in visualize category = detection.categories[0] AttributeError: 'Detection' object has no attribute 'categories'
BTW, when I comment out the part with the categories, it works fine, and just draws a bounding box but without any text.
@charlienicholson3 Hey! Im getting the same error, were you able to solve it?
@agam-11 No, I was not able to
@charlienicholson3 Alright, what did you end up using then? any alternative you are aware of?
@agam-11 @charlienicholson3 @ak34a Sorry for the delay. We've been working hard to release MediaPipe for Raspberry Pi, which has all the features of TFLite Task Library but with many more features such as hand, pose detection etc. I've updated the guide above to point to the MediaPipe samples. TFLite Task Library will eventually be replaced by MediaPipe since the architecture of MediaPipe allows us to release more ML tasks in the future, especially complex tasks that requires chaining multiple models.
You can learn more about MediaPipe for Raspberry Pi in this blog post. Thank you all for the continued support.
https://developers.googleblog.com/2023/08/mediapipe-for-raspberry-pi-and-ios.html
python3 detect.py
Traceback (most recent call last):
File "detect.py", line 21, in
import mediapipe as mp
ModuleNotFoundError: No module named 'mediapipe'
(mp) pi@raspberrypi:~/mediapipe/examples/object_detection/raspberry_pi $ pip install mediapipe
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
ERROR: Could not find a version that satisfies the requirement mediapipe (from versions: none)
ERROR: No matching distribution found for mediapipe
I cant install mediapipe I am using raspberry pi 3b+ with os installed
cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
Hello @khanhlvg a pleasure, I am currently writing to you through this medium, due to the news with mediapipe.
I have seen your mediapipe video Easy on-device Machine Learning with MediaPipe-Google for Developers
I really liked how it contributed to the renewal of machine learning with MediaPipe, something that years ago was not seen as possible due to how difficult it was to install on a Raspberry Pi, and because of how limited the hardware these boards are.
Applying machine learning on raspberry pi improved considerably with the use of Coral USB. How will we continue using Coral USB with mediapipe?
This is the favorite playlist of many, of how machine learning teaches us on a raspberry pi with TensorFlow Lite.
Machine Learning for Raspberry Pi-Tensorflow
I hope we soon have the list of videos with the use of TPUs like the Google Coral USB and MediaPipe.
Thank you very much, a hug.
ImportError: /home/qvinh0301/.local/lib/python3.9/site-packages/cv2/cv2.abi3.so: undefined symbol: __atomic_store_8
I got this error when run the command, Please help me !! thank you
@khanhlvg Hi, I am all new to this so please help, when I tried to run the following:
python3 -m pip install --user --upgrade pip
python3 -m pip install --user virtualenv
I got this error:
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
For more information visit http://rptl.io/venv
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
For more information visit http://rptl.io/venv
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
@dannyhung1 the latest versions of Raspberry Pi OS only allow pip packages to be installed in virtual environments. As such, you have to install the virtual environment package via apt.
You can search for it to see if this has already been installed on your system: apt list --installed | grep python3-venv
If it is not installed run: sudo apt install python3-venv
Once installed, you can create your venv.
Details about the change as well as commands can be found on the Raspberry Pi website (Python on Raspberry Pi section). I am also including a link to the Adafruit tutorial that has a few additional commands that may be useful.
Python on Raspberry Pi: https://www.raspberrypi.com/documentation/computers/os.html#python-on-raspberry-pi
Python Virtual Environment Usage on Raspberry Pi: https://learn.adafruit.com/python-virtual-environment-usage-on-raspberry-pi/overview
@ldiangelis Thank you so much. I was able to create the virtual environment successfully. But I tried to run the program in the virtual environment, I got the following error. I checked my Raspberry Pi camera using the "rpicam-hello" command and it looks ok. Any idea?
(mp) dhung@raspberrypi:/mediapipe/examples/object_detection/raspberry_pi $ python detect.py/mediapipe/examples/object_detection/raspberry_pi $ rpicam-hello
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
[ WARN:[email protected]] global cap_v4l.cpp:1119 tryIoctl VIDEOIO(V4L2:/dev/video0): select() timeout.
ERROR: Unable to read from webcam. Please verify your webcam settings.
(mp) dhung@raspberrypi:
[82:20:14.238467034] [20237] INFO Camera camera_manager.cpp:284 libcamera v0.1.0+118-563cd78e
[82:20:14.294448017] [20240] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[82:20:14.297229511] [20240] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx708@1a to Unicam device /dev/media2 and ISP device /dev/media1
[82:20:14.297329416] [20240] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
Made X/EGL preview window
Mode selection for 2304:1296:12:P
SRGGB10_CSI2P,1536x864/0 - Score: 3400
SRGGB10_CSI2P,2304x1296/0 - Score: 1000
SRGGB10_CSI2P,4608x2592/0 - Score: 1900
Stream configuration adjusted
[82:20:14.788965979] [20237] INFO Camera camera.cpp:1183 configuring streams: (0) 2304x1296-YUV420 (1) 2304x1296-SBGGR10_CSI2P
[82:20:14.789691463] [20240] INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@1/imx708@1a - Selected sensor format: 2304x1296-SBGGR10_1X10 - Selected unicam format: 2304x1296-pBAA
(mp) dhung@raspberrypi:~/mediapipe/examples/object_detection/raspberry_pi $
@dannyhung1 here are a couple of basic methods to try. If the method does not work, make sure to revert any changes before moving to the next step.
- Try running: sudo python detect,py
- Pass the argument in the terminal when calling the script with: python detect.py --device 1
- The default device is 0, but always good to test with another device number
- Replace cap.read() with cap.open() in the detect.py script
- Modify the config file to include start_x=1 and comment out camera_auto_detect=1
- There probably is no additional support for the legacy camera stack so this may not work.
- If this does work it is likely an issue with trying to use the V4L2 driver and the camera module with the script.
If none of these work, there are additional steps that you can try. Always good to start simple!
@ldiangelis Thank you for your advice. I have tried all the steps that you suggested but unfortunately none of them works. Would appreciate further suggestions.
Hi! I've had an error when running python detect.py "No module Cv2 found" is it because I'm running the latest version of the OS (bookworm)? I changed the python version to 3.7.3 tho
@khanhlvg Thanks for the help, detect.py is working fine now. But when i tried to use the model maker in colab notebook, i ran into errors. Any suggestions?
Preparing metadata (setup.py) ... done
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.6/106.6 kB 14.0 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.6/43.6 kB 5.7 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.2/5.2 MB 99.4 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 83.5 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 80.3/80.3 kB 11.8 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/2.8 MB 97.7 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.6/84.6 kB 10.7 MB/s eta 0:00:00
ERROR: Cannot install tflite-model-maker==0.1.2, tflite-model-maker==0.2.0, tflite-model-maker==0.2.1, tflite-model-maker==0.2.2, tflite-model-maker==0.2.3, tflite-model-maker==0.2.4, tflite-model-maker==0.2.5, tflite-model-maker==0.3.3, tflite-model-maker==0.3.4, tflite-model-maker==0.4.0, tflite-model-maker==0.4.1, tflite-model-maker==0.4.2 and tflite-model-maker==0.4.3 because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
@khanhlvg Hi, when im running python detect.py, im getting the error that no module was found with the name of tflite_support.
Can anyone help me out
so if i run the code on the raspberrypi desktop terminal using ssh (VNC) how exactly do i get the screen to display , i had X11 forwarding enabled downloaded the xming whatever but dont understand what exactly to do