Raj Singh isc-rsingh
EdmundMartin
/ ScrapeGoogle.py
Last active
August 20, 2020 15:53
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import requests | |
| from bs4 import BeautifulSoup | |
| import time | |
| USER_AGENT = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36'} | |
| def fetch_results(search_term, number_results, language_code): | |
| assert isinstance(search_term, str), 'Search term must be a string' | |
| assert isinstance(number_results, int), 'Number of results must be an integer' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ------------------------------------------------------------------------ | |
| Copyright IBM Corp. 2017 | |
| Licensed under the Apache License, Version 2.0 (the 'License'); | |
| you may not use this file except in compliance with the License. | |
| You may obtain a copy of the License at | |
| http://www.apache.org/licenses/LICENSE-2.0 | |
| Unless required by applicable law or agreed to in writing, software |
isc-rsingh
/ geotest.js
Last active
July 26, 2016 16:26
A simple Node.js script to test Cloudant geo queries for load and/or coverage. `npm install superagent` then `node geotest.js`
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var request = require('superagent'); | |
| // Replace these 3 variables with your own information | |
| // Note: The database must be world readable | |
| const CLOUDANT_ACCOUNT = 'opendata'; | |
| const CLOUDANT_DB = 'pois'; | |
| const GEO_INDEX = '_design/idx/_geo/spatial'; | |
| for (var index = 0; index < 500; index++) { | |
| runQuery(index) |
isc-rsingh
/ updateprops.py
Last active
July 26, 2016 16:26
Loop through all docs in a Cloudant database and update data. This version changes string values to integers but it can easily be adapted for other uses.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import os | |
| from cloudant import cloudant | |
| from cloudant.client import Cloudant | |
| from cloudant.document import Document | |
| CLOUDANT_USERNAME = os.getenv('CLOUDANT_USERNAME', 'myusername') | |
| CLOUDANT_PW = os.getenv('CLOUDANT_PW', 'mypw') | |
| CLOUDANT_DB = 'mydb' | |
| PROPS = ['rec','r','h','m','s','idts','idvc','idn','refts'] |
isc-rsingh
/ cloudant-couchdb-changes.html
Last active
August 29, 2015 14:27
Sets up a live replication link between a Cloudant or CouchDB database and an in-browser PouchDB database, and reports change activity
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!DOCTYPE html> | |
| <html> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"> | |
| <title>Cloudant/CouchDB Changes Listener</title> | |
| <link href="//maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" rel="stylesheet"> | |
| <link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css"> | |
| </head> | |
| <body> |
isc-rsingh
/ shapefile2couchdb.js
Last active
March 24, 2016 19:18
Converts an on-disk Shapefile into GeoJSON and imports the data into a CouchDB-compatible database (e.g. Cloudant) using node.js.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** | |
| * Converts an on-disk Shapefile into GeoJSON and imports the data into a CouchDB-compatible database, | |
| * like Cloudant.com! | |
| * requires Node.js and the 'cradle' and 'shapefile' modules. | |
| * - installation: save this code to a file called index.js | |
| * - at the shell: npm install cradle | |
| * - at the shell: npm install shapefile | |
| * - at the shell, run the program: node index.js | |
| */ | |
| var cradle = require('cradle'); // npm install cradle |
isc-rsingh
/ shapefile2cloudant.js
Last active
August 29, 2015 14:07
Converts an on-disk Shapefile into GeoJSON and imports the data into your Cloudant database using node.js.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** | |
| * Converts an on-disk Shapefile into GeoJSON and imports the data into your Cloudant database | |
| * requires Node.js and the 'cloudant' and 'shapefile' modules. | |
| * - installation: save this code to a file called index.js | |
| * - at the shell: npm install cloudant | |
| * - at the shell: npm install shapefile | |
| * - at the shell, run the program: node index.js | |
| */ | |
| var Cloudant = require('cloudant'); // npm install cloudant | |
| var shapefile = require('shapefile'); // npm install shapefile |