philshem
/ _get_musical_names.py
Created
June 6, 2021 12:43
musical names (yob2020.txt comes from https://www.ssa.gov/oact/babynames/limits.html)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/usr/bin/python3 | |
| musical_notes = list('CDEFGAB') | |
| #musical_notes = list('CDEFGAH') | |
| fp = 'yob2020.txt' | |
| with open(fp, 'r') as f: | |
| lines = f.read().split('\n') | |
| names = [x.split(',')[0].upper() for x in lines if len(x) > 0] |
philshem
/ get.py
Created
May 6, 2021 05:03
s&p index + weights scraping https://www.slickcharts.com/sp500
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import requests | |
| df = pd.read_html(requests.get('https://www.slickcharts.com/sp500',headers={'User-agent': 'Mozilla/5.0'}).text)[0] | |
| print(df) | |
| df.to_csv('sp.csv',index=False) |
philshem
/ publibike_zuerich_elevation.csv
Created
February 19, 2021 19:19
Elevation of Publibike stations around Zürich
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| station_id | latitude | longitude | elevation | elevation_above_lake | |
|---|---|---|---|---|---|
| pb_110 | 47.396401 | 8.4860017 | 399.0173645 | -6.982635498 | |
| pb_149 | 47.392336 | 8.491061 | 399.4473877 | -6.552612305 | |
| pb_286 | 47.3985553 | 8.493822 | 401.0709534 | -4.929046631 | |
| pb_90 | 47.3917964 | 8.5056978 | 402.3087158 | -3.69128418 | |
| pb_395 | 47.3669233 | 8.5420304 | 402.3295898 | -3.670410156 | |
| pb_152 | 47.389823 | 8.49163 | 402.6643982 | -3.335601807 | |
| pb_155 | 47.374711 | 8.543042 | 402.9417419 | -3.058258057 | |
| pb_450 | 47.350578 | 8.560587 | 403.0110779 | -2.988922119 | |
| pb_130 | 47.391366 | 8.488108 | 403.0958862 | -2.90411377 |
philshem
/ Collect Parler Metadata.ipynb
Created
January 12, 2021 07:25
— forked from kylemcdonald/Collect Parler Metadata.ipynb
Collect video URLs and GPS data for Parler videos.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
We can't make this file beautiful and searchable because it's too large.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name,lastName,uri,imageUri,worthChange,age,source,industry,gender,timestamp,headquarters,state,realTimeWorth,date,realTimeRank,realTimePosition,squareImage,title,category,government,position,rank,worth,year,source_url,salary,managementAssets,pay | |
| Abigail Johnson,Johnson,abigail-johnson,abigail-johnson,0.0,58.0,money management,Finance & Investments,F,2020-12-07 08:50:01.435,"Milton, MA",Massachusetts,14974.7,1993-01-01 00:00:00,39.0,39.0,//specials-images.forbesimg.com/imageserve/5f15ddd274642d00070464eb/416x416.jpg?background=000000&cropX1=62&cropX2=1355&cropY1=144&cropY2=1437,,,,,,,1990,http://www.forbes.com/ajax/list/data?year=1990&uri=forbes-400&type=person,,, | |
| Aerin Lauder,Lauder,aerin-lauder,aerin-lauder,0.0,50.0,cosmetics,Fashion & Retail,F,2020-12-07 08:50:01.435,"New York, NY",New York,3215.528,1993-01-01 00:00:00,285.0,285.0,https://specials-images.forbesimg.com/imageserve/5f46b424808a289780083f05/416x416.jpg?background=000000&cropX1=895&cropX2=2304&cropY1=287&cropY2=1696,,,,,,,1990,http://www.forbes. |
philshem
/ deming.csv
Created
November 28, 2020 12:10
deming.csv - https://opendata.stackexchange.com/a/18277/1511
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| x | y | y_ideal | |
|---|---|---|---|
| 10 | 10.9277728455939 | 10 | |
| 10.25 | 10.5933490961181 | 10.25 | |
| 10.5 | 14.0852031368451 | 10.5 | |
| 10.75 | 12.4232922691896 | 10.75 | |
| 11 | 14.7683460256395 | 11 | |
| 11.25 | 9.93532391180117 | 11.25 | |
| 11.5 | 12.4777201750146 | 11.5 | |
| 11.75 | 11.3343548513887 | 11.75 | |
| 12 | 10.2801940515687 | 12 |
philshem
/ at.20200927.csv
Last active
September 27, 2020 18:44
scrape individual state data from https://coronadata.at used for this answer https://opendata.stackexchange.com/a/18094/1511
We can make this file beautiful and searchable if this error is corrected: It looks like row 7 should actually have 11 columns, instead of 10 in line 6.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Datum,Tests,Fälle (Gesamt),Fälle (Aktiv),/ 100.000 Einwohner,Genesene,Verstorbene,Hospitalisiert,Intensivstation,state,url | |
| 27.09.2020 20:00,46.351 +476 (1%),758 +9 (1%),141 0 (0%),48,606 +9 (2%),11 +0 (0%),8 +1 (14%),1 0 (0%),burgenland,https://www.coronatracker.at/burgenland | |
| 26.09.2020 20:00,45.875 +592 (1%),749 +20 (3%),141 +6 (4%),48,597 +14 (2%),11 +0 (0%),7 +4 (133%),1 +1 (100%),burgenland,https://www.coronatracker.at/burgenland | |
| 25.09.2020 20:00,45.283 +908 (2%),729 +8 (1%),135 -5 (-4%),46,583 +13 (2%),11 +0 (0%),3 -4 (-57%),0 0 (0%),burgenland,https://www.coronatracker.at/burgenland | |
| 24.09.2020 20:00,44.375 +528 (1%),721 +4 (1%),140 -1 (-1%),48,570 +5 (1%),11 +0 (0%),7 +3 (75%),0 0 (0%),burgenland,https://www.coronatracker.at/burgenland | |
| 23.09.2020 20:00,43.847 +999 (2%),717 +33 (5%),141 +23 (19%),48,565 +10 (2%),11 +0 (0%),4 -3 (-43%),0 -1 (-100%),burgenland,https://www.coronatracker.at/burgenland | |
| 22.09.2020 20:00,42.848 +303 (1%),684 +14 (2%),118 +2 (2%),40,555 +12 (2%),11 +0 (0%),7 -2 (-22%),1 0 (0%),b |
philshem
/ 53827775.py
Created
September 1, 2020 07:41
refactor stackoverflow answer to read unread emails and also fix for loop https://stackoverflow.com/a/53827775/2327328
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import imaplib | |
| import email | |
| # adapted from: https://stackoverflow.com/a/53827775/2327328 | |
| def read_email_from_gmail(): | |
| mail = imaplib.IMAP4_SSL('imap.gmail.com') | |
| mail.login('MYEMAIL','MYSECRET') | |
| mail.select('inbox') |
philshem
/ 1298632074849587204.py
Last active
August 26, 2020 15:07
https://twitter.com/dancow/status/1298632074849587204 reply to dancow
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pandas as pd | |
| # https://burntsushi.net/stuff/worldcitiespop.csv | |
| df = pd.read_csv('worldcitiespop.csv',low_memory=False) | |
| df = df.query('Country == "us"') | |
| print(len(df)) | |
| # prints 141989 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "1960":316.91, | |
| "1961":317.64, | |
| "1962":318.45, | |
| "1963":318.99, | |
| "1964":319.62, | |
| "1965":320.04, | |
| "1966":321.38, | |
| "1967":322.16, | |
| "1968":323.04, |
NewerOlder