crcdng

A temporary note on machine vision with ML Kit and Flutter

1. Firebase ML Kit is deprecated (https://developers.google.com/ml-kit/migration). All 3rd-party published tutorials I could find on September 6, 2020 are referring to the deprecated APIs.

2. As of June 3, 2020 the new products / APIs are ML Kit (https://developers.google.com/ml-kit) and Firebase Machine Learning aka Firebase ML, in beta (https://firebase.google.com/products/ml).

3. Flutter (https://flutter.dev/) features a plugin from the Firebase team named firebase_ml_vision, described as "A Flutter plugin to use the capabilities of Firebase ML, which includes all of Firebase's cloud-based ML features, and ML Kit, a standalone library for on-device ML, which can be used with or without Firebase" (https://pub.dev/packages/firebase_ml_vision). Thus the plugin is supposed to cover both ML Kit (which can be used without Firebase) and Firebase ML. Its name therefore could lead to confusion.

4. As of September 6, 2020, the plugin, however, does no

crcdng

NOTE this tutorial was written Dec 2020 and I don't use Leap on Mac since then. Use at your own risk. It still seems to work for some people, but I cannot support it further or answer questions unless I get a project with Leap / Mac again.

The Leap Motion Controller (https://www.ultraleap.com/product/leap-motion-controller/) has been a favourite input device for user interface designers and hackers alike. It allows to create touchless user interfaces that track hand movement and recognize gestures (not bad in the current situation).

Unfortunately the company who made it has decided to go big and bold on VR and Windows only and stopped supporting development on a Mac. There are still options available, for example getting the Leap Motion data via node.js. Yet Python can be the right choice, especially if you want to feed the data into a machine learning pipeline or use any of the numerous libraries that the Python ecosystem offers.

Getting this to work on macOS requires a few steps (tested on macOS 11

crcdng

prerequisites: git, node.js, yarn, chrome browser as decribed here: https://github.com/hydra-synth/hydra#running-locally. run the commands below in Terminal

note: the parameters reduce the size of the download a bit

git clone --depth 1 --branch "main" https://github.com/hydra-synth/hydra.git

cd hydra

note: the --ignore-engines parameter is necessary because at the time of this writing the dependencies of hydra won't allow node.js > 16

crcdng

#!/bin/sh

# builds and prepares a Flutter/Flame web build for itch.io
# tested on Mac
# added: create output dir if it does not exist
# added: mitigate bug where keyboard keys are working fine locally but aren’t recognized on itch.io. 
# after uploading set the dimensions in the "Embed options" section on itch.io 

# set these parameters before running
PROJECT_NAME='NAME'

crcdng

library;

import 'package:flame/camera.dart';
import 'package:flame/game.dart';
import 'package:flutter/material.dart';

void main() {
  final game = MyGame();
  runApp(GameWidget(game: game));
}

crcdng

library;

import 'package:flame/camera.dart';
import 'package:flame/game.dart';
import 'package:flutter/material.dart';

void main() {
  final game = MyGame();
  runApp(GameWidget(game: game));
}