Here is how I copied data from one S3 bucket to another:
aws s3 sync s3://bitly-challenges/hdb_sanitized s3://hughdbrown/data-capstone
Hugh Brown hughdbrown
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy | |
| import scipy.stats as scs | |
| def a_b_test(new_views, new_clicks, old_views, old_clicks, size=10000): | |
| new_site = scs.beta(a=new_clicks + 1, b=new_views + 1).rvs(size=size) | |
| old_site = scs.beta(a=old_clicks + 1, b=old_views + 1).rvs(size=size) | |
| return (new_site > old_site).mean() |
I have a resume, but does it say what I want it to say? Specifically, do machine learning algorithms cluster my resume with the job title I would like them to?
- Linkedin data/resumes for various job titles: developer, devops, data scientist, full stack, etc.
- Create a database of resumes: developer, devops, data scientist, full stack, etc.
- Train a K-means model
Homeaway has data on vacation rentals. The data is not nearly so worked over as AirBNB data. Possibly there is something interesting in there to disover.
- Homeaway API access The main problem with the project is that the Homeaway API is pretty opaque. I can't figure out how to get a data dump. Also, the API requires registration and advance permission.
hughdbrown
/ data-job-recommender.md
Created
August 31, 2015 22:59
Job recommender that bootstraps from list of job postings
So often, job sites give candidates job listings that are far off topic. The job title is often not applicable for the candidate, and less often, the location does not match the cadidate's location.
Can we build a better system for users by applying a recommender system to existing public listings?
- glassdoor.com API
- indeed.com web scraping
I was listening on NPR today and heard that within the UN, there are about a dozen different blocs that vote together on global warming issues:
- Switzerland alone
- Developed countries
- European group
- "77 countries plus China" ... which is actually 134 countries
- Various island nations most affected
hughdbrown
/ how-to-put-your-project-online.md
Last active
August 31, 2015 16:06
Getting your project online
hughdbrown
/ data-chronic-kidney-disease.md
Last active
August 31, 2015 17:35
Chronic kidney disease predictor
- UC Irving 400 rows by 25 attributes