This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import ipinfo | |
| from opencensus.ext.azure.log_exporter import AzureLogHandler | |
| def main(host, port): | |
| print ('Starting honeypot!') | |
| # initialize Azure Monitoring logger object | |
| logger = startLogger() | |
| s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) | |
| s.bind((host, port)) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/usr/bin/env python | |
| """ | |
| Credit: https://gist.github.com/omnidan/1456674 | |
| Copyright (c) 2011, Daniel Bugl | |
| All rights reserved. | |
| Redistribution and use in source and binary forms, with or without | |
| modification, are permitted provided that the following conditions are met: | |
| 1. Redistributions of source code must retain the above copyright | |
| notice, this list of conditions and the following disclaimer. | |
| 2. Redistributions in binary form must reproduce the above copyright |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import sys | |
| import pandas as pd | |
| # Function to retrieve a list of Pandas DataFrames and their sizes in memory | |
| # Call the function: | |
| # pandas_dfs_in_memory(mydir=dir(),parent_vars=globals()) | |
| def pandas_dfs_in_memory(mydir,parent_vars): | |
| # get a list of all the local objects with sizes | |
| objects=[] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import geopandas as gpd | |
| from sqlalchemy import create_engine | |
| # create the sqlalchemy connection engine | |
| db_connection_url = "postgresql://user:password@localhost:5432/gis_db" | |
| con = create_engine(db_connection_url) | |
| # read in the data | |
| gdf = gpd.read_file('shapefile/holc_ad_data.shp') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| CREATE TEMP TABLE fracking AS | |
| SELECT f.name, f.operator, f.status, f.type, co.county | |
| FROM fractracker f | |
| JOIN co_counties co | |
| ON ST_COVEREDBY(f.geom, co.geom); | |
| -- query on TEMP TABLE to find count per county | |
| SELECT county, COUNT(county) well_count | |
| FROM fracking | |
| GROUP BY county |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -- calculate statistics on the totals | |
| SELECT | |
| SUM(f.well_count) n_obs, | |
| COUNT(f.well_count) n_counties, | |
| ROUND(AVG(f.well_count),2) mean, | |
| MIN(f.well_count) min, | |
| MAX(f.well_count) max, | |
| ROUND(PERCENTILE_CONT(0.5) WITHIN GROUP(ORDER BY f.well_count) median, | |
| ROUND(STDDEV(f.well_count),2) as stdev | |
| -- calculate group totals |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| SELECT | |
| PERCENTILE_CONT(0.25) WITHIN GROUP(ORDER BY f.well_count) percentile25, | |
| PERCENTILE_CONT(0.5) WITHIN GROUP(ORDER BY f.well_count) percentile50, | |
| PERCENTILE_CONT(0.75) WITHIN GROUP(ORDER BY f.well_count) percentile75, | |
| PERCENTILE_CONT(1.0) WITHIN GROUP(ORDER BY f.well_count) percentile100 | |
| FROM ( | |
| SELECT COUNT(county) well_count | |
| FROM fracking | |
| GROUP BY county | |
| ) f |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| SELECT f.county, f.well_count, co.geom -- pgAdmin will recognize co.geom | |
| FROM ( -- subquery to get totals by county | |
| SELECT county, COUNT(county) well_count | |
| FROM fracking | |
| GROUP BY county | |
| ) f | |
| LEFT JOIN co_counties co -- all from f, matching from co | |
| ON f.county = co.county |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import geopandas as gpd | |
| # Read in the dataset from geojson file to geopandas dataframe | |
| gdf = gpd.read_file('WFIGS_-_2022_Wildland_Fire_Perimeters_to_Date.geojson') | |
| # print a description of the dataframe | |
| print(gdf.info()) | |
| # Check for NaN records | |
| # Result: geometry does not have any NaN but many other columns do | |
| print(f'Columns with nan records: {gdf.columns[gdf.isna().any()].tolist()}') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import timeit | |
| import sys | |
| import json | |
| import ee | |
| # Authenticate to google earth engine | |
| service_account = '[email protected]' | |
| key_path = 'path_to_your_service_account_key.json' | |
| credentials = ee.ServiceAccountCredentials(service_account, key_path) | |
| ee.Initialize(credentials) |