Jeff Hobbs jeffehobbs

Using Python 3 + Google Cloud Vision API's OCR to extract text from photos and scanned documents

Just a quickie test in Python 3 (using Requests) to see if Google Cloud Vision can be used to effectively OCR a scanned data table and preserve its structure, in the way that products such as ABBYY FineReader can OCR an image and provide Excel-ready output.

The short answer: No. While Cloud Vision provides bounding polygon coordinates in its output, it doesn't provide it at the word or region level, which would be needed to then calculate the data delimiters.

On the other hand, the OCR quality is pretty good, if you just need to identify text anywhere in an image, without regards to its physical coordinates. I've included two examples:

####### 1. A low-resolution photo of road signs

Option 1: Command-line download extension as zip and extract

extension_id=jifpbeccnghkjeaalbbjmodiffmgedin   # change this ID
curl -L -o "$extension_id.zip" "https://clients2.google.com/service/update2/crx?response=redirect&os=mac&arch=x86-64&nacl_arch=x86-64&prod=chromecrx&prodchannel=stable&prodversion=44.0.2403.130&x=id%3D$extension_id%26uc" 
unzip -d "$extension_id-source" "$extension_id.zip"

Thx to crxviewer for the magic download URL.

python for botmakers, on dreamhost shared hosting

On a shared hosting service like dreamhost, how do you get your twitter bot up and running? Problems:

where should I put my script?
you can't install python modules like tweepy (for twitter access) because you don't have root permission
once you get that solved, how do you run your script? cron?

I'm still figuring this stuff out myself, so nothing is clear as it should be. Hope this page will be a resource that will improve over time.

Useful Pandas Snippets

A personal diary of DataFrame munging over the years.

Data Types and Conversion

Convert Series datatype to numeric (will error if column has non-numeric values)
(h/t @makmanalp)

	#!/bin/bash

	# Anh Nguyen <anh.ng8@gmail.com>
	# 2016-04-30
	# MIT License

	# This script takes in same-size images from a folder and make a crossfade video from the images using ffmpeg.
	# Make sure you have ffmpeg installed before running.

	# The output command looks something like the below, but for as many images as you have in the folder.

	from selenium import webdriver
	from selenium.webdriver import DesiredCapabilities

	desired_capabilities = DesiredCapabilities.PHANTOMJS.copy()
	desired_capabilities['phantomjs.page.customHeaders.User-Agent'] = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) ' \
	'AppleWebKit/537.36 (KHTML, like Gecko) ' \
	'Chrome/39.0.2171.95 Safari/537.36'
	driver = webdriver.PhantomJS(desired_capabilities=desired_capabilities)

	#!/usr/bin/env python
	import argparse
	import logging
	try:
	from urllib import splittype
	except ImportError:
	from urllib.parse import splittype

	import boto3

	var GetIpInfo = function(ipAddr) {
	var info = null;
	var infoUrl = "http://ipinfo.io/" + ipAddr;
	$.ajax({
	url: infoUrl,
	type: 'GET',
	dataType: 'json',
	async: false,
	success: function(data) {
	info = data;

	// Returns the value at a given percentile in a sorted numeric array.
	// "Linear interpolation between closest ranks" method
	function percentile(arr, p) {
	if (arr.length === 0) return 0;
	if (typeof p !== 'number') throw new TypeError('p must be a number');
	if (p <= 0) return arr[0];
	if (p >= 1) return arr[arr.length - 1];

	var index = (arr.length - 1) * p,
	lower = Math.floor(index),

	var page = new WebPage();

	//spoof it as opera mini, to get the mobile page working properly
	page.settings.userAgent = "Opera/9.80 (J2ME/MIDP; Opera Mini/6.5.26955/27.1407; U; en) Presto/2.8.119 Version/11.10";

	function doLogin(){
	page.evaluate(function(){
	var frm = document.getElementById("login_form");

	frm.elements["email"].value = "--enter-your-email--";