https://fangohr.github.io/blog/spyder-the-scientific-python-development-environment.html
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests | |
import warnings | |
# Suppress warnings containing the text "Unverified HTTPS request" | |
with warnings.catch_warnings(): | |
warnings.filterwarnings("ignore", message="Unverified HTTPS request") | |
list_of_available_datasets = json.loads( | |
requests.get("https://api.beta.ons.gov.uk/v1/datasets", verify=False).text | |
) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Program for accessing the ONS API (see https://developer.ons.gov.uk). | |
The ONS API can be used to programmatically access certain data that ONS has published. | |
This program provides functions for (1) checking what datasets are available through the | |
ONS API and (2) accessing datasets through the ONS API. To use this program, run the | |
entire script; You will then be able to use the get_list_of_available_datasets() and | |
get_latest_data_from_ons_api() functions. | |
get_list_of_available_datasets() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Check available log names and meta data | |
Get-EventLog -list | |
# Remind yourself when you logged on on a particular day (for clocking hours) | |
# by checking the 'system' log, for days going back as far as 14 days ago | |
# printing only when instanceid is 7001 (user logon notification) | |
Get-EventLog system -after (get-date).AddDays(-14) | where {$_.InstanceId -eq 7001} | |
# Version that shows only the first entry for each day | |
Get-EventLog system -after (get-date).AddDays(-14) | where {$_.InstanceId -eq 7001} | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas as pd | |
# Create a toy dataframe for demonstration purposes | |
my_df = pd.DataFrame({"oa": ["a", "b", "c"]}) # 3 rows | |
""" | |
oa | |
0 a | |
1 b | |
2 c |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#### Issue: np.float not found | |
Explanation: the issue is that np.float is being called for, but np.float is deprecated in newer | |
versions of numpy. | |
Solution(s): | |
(i) use float instead | |
(ii) create np.float = float, (iii) update the package that calls for np.float | |
import numpy as np | |
np.float == float |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas as pd # astype(), info(), memory_usage(), to_numeric() | |
import polars as pl # cast(), estimates_size() | |
import numpy as np | |
import pdcast as pdc # install via pip install pandas-downcast | |
pandas_df = pd.DataFrame( | |
{ | |
"a": np.linspace(1, 10_000_000, 10_000_000), | |
"b": np.linspace(1.12, 10_000_000.12, 10_000_000), | |
"c": np.random.choice([1, 0], 10_000_000), |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests | |
def check_proxy(proxy: str): | |
proxies = { | |
"http": f"http://{proxy}", | |
"https": f"http://{proxy}" | |
} | |
try: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Sources: | |
https://pola-rs.github.io/polars/getting-started/intro | |
https://kevinheavey.github.io/modern-polars/ | |
.../DAP_CATS/intro_to_polars/-/tree/main?ref_type=heads | |
""" | |
import polars as pl | |
from datetime import date, datetime | |
import numpy as np |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- coding: utf-8 -*- | |
""" | |
Generate animated population pyramid gifs for all geographies for years 2011 to 2022. | |
I wrote this program quickly. It has not be quality assured. | |
@author: Matthew Hawkes | |
@date: 03-11-2023 | |
Style: https://black.readthedocs.io | |
""" |