Skip to content

Instantly share code, notes, and snippets.

View david-andrew's full-sized avatar

David-Andrew Samson david-andrew

View GitHub Profile
@david-andrew
david-andrew / gist:83f793bae32d32442ce4ac27040d5ad6
Last active November 17, 2022 22:54
Example results from tf-idf vs BERT vs GPT-3 (Babbage) search
Search: tariff
--------------------------------- text results: ---------------------------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
score: 324.95238095238096
name: TM.TAX.MRCH.IP.ZS;
display name: Share of tariff lines with international peaks, all products (%);
description: Share of tariff lines with international peaks is the share of lines in the tariff schedule with tariff rates that exceed 15 percent. It provides an indication of how selectively tariffs are applied.;
unit: %;
unit description: %;
<<<<<<<<<<<<<<<<<<<<<<<<<<<<
@david-andrew
david-andrew / matches.json
Created November 22, 2022 19:11
World Modelers Ontology Concept Rankings
This file has been truncated, but you can view the full file.
{
"Node(name='wm', examples=())": {
"text": [
[
"name: TM.TAX.MRCH.WM.AR.ZS;\ndisplay name: Tariff rate, applied, weighted mean, all products (%);\ndescription: Weighted mean applied tariff is the average of effectively applied rates weighted by the product import shares corresponding to each partner country. Data are classified using the Harmonized System of trade at the six- or eight-digit level. Tariff line data were matched to Standard International Trade Classification (SITC) revision 3 codes to define commodity groups and import weights. To the extent possible, specific rates have been converted to their ad valorem equivalent rates and have been included in the calculation of weighted mean tariffs. Import weights were calculated using the United Nations Statistics Division's Commodity Trade (Comtrade) database. Effectively applied tariff rates at the six- and eight-digit product level are averaged for products in each commodity group. When the effectively applied
@david-andrew
david-andrew / gist:6bcf3f9ea9f302ca4cecc374b690d4e2
Created January 26, 2023 21:03
raw pdf text extraction results
This file has been truncated, but you can view the full file.
---------------------------------------------------------
data/transition_reports/cb4410en.pdf
(page [0]) Food and Agriculture Organization
(page [0]) of the United Nations
(page [0]) FAOSTAT ANALYTICAL BRIEF 19
(page [0]) Temperature change statistics
(page [0]) 1961–2020
(page [0]) Global, regional and country trends
(page [0]) ISSN 2709-006X [Print] ISSN 2709-0078 [Online]
(page [1]) Temperature change statistics 1961 –2020 - Global, regional and country trends
(chatty) david@blade15:~/dev/askem/data-service$ sudo docker-compose logs
Attaching to data-service-api, data-service_graphdb_1, data-service-rdb, data-service_minio_1
data-service-api | Skipping virtualenv creation, as specified in config file.
data-service-api |
data-service-api | tds is not a package.
data-service-api | Skipping virtualenv creation, as specified in config file.
data-service-api |
data-service-api | tds is not a package.
data-service-api | Skipping virtualenv creation, as specified in config file.
data-service-api |
@david-andrew
david-andrew / gist:8210781c56f772c1a5167c87d1208769
Created September 15, 2023 14:02
Python: How to Forward Type Hints in Decorated Functions
from typing import TypeVar
from typing_extensions import ParamSpec
_R_co = TypeVar("_R_co", covariant=True)
_P = ParamSpec("_P")
# decorated functions will mantain identical typing to their original form
def myDecorator(func: Callable[_P, _R_co]) -> Callable[_P, _R_co]:
def wrapper(*args, **kwargs):
# do wrapper stuff
return func(*args, **kwargs)
@david-andrew
david-andrew / gist:24092d01c3a6a7ea1674ab84a550b98c
Created December 20, 2023 17:48
error output for python installing GDAL
43.19 Building wheel for GDAL (setup.py): started
45.54 Building wheel for GDAL (setup.py): finished with status 'error'
45.60 error: subprocess-exited-with-error
45.60
45.60 × python setup.py bdist_wheel did not run successfully.
45.60 │ exit code: 1
45.60 ╰─> [782 lines of output]
45.60 running bdist_wheel
45.60 running build
45.60 running build_py
@david-andrew
david-andrew / gist:e9fe13a92c85b23c4c8b215ebd5cf270
Created January 3, 2024 20:33
multi-question document assistant demo
>>> how is climate change expected to affect flooding in ethiopia?
|||| Context free query: climate change impact on flooding in Ethiopia
Answer: Climate change is expected to affect flooding in Ethiopia in several ways. Primarily, it will likely lead to an increased frequency and intensity of extreme hydrologic events, causing more pronounced disastrous floods which can negatively impact the economy and society [0][7]. The country is susceptible to floods, and past events have shown considerable loss of life and property [1]. The median temperature increase for Africa is predicted to be 3-4°C by the end of the 21st century, possibly intensifying evapotranspiration, which may negate any benefits from increased rainfall, thus exacerbating drought and flood conditions [1].
Moreover, variability in rainfall is expected to increase due to climate change, resulting in more frequent droughts and floods [9]. These changes threaten the stability and transformation of Ethiopia's agricultural sector, which is heavily
@david-andrew
david-andrew / gist:0a4c2fca21c6c8af290c61f76e9af9c0
Created February 20, 2024 20:56
example output from dojo-auto-annotations
Meta(path=PosixPath('datasets/mock_aqi.csv'), name='Air Quality Index', description='This dataset represents daily air quality observations collected from various monitoring stations across different cities worldwide. Each row corresponds to a single observation with details about the date, time, and location of the observation, along with specific air quality metrics and conditions.')
LLM identified column "year" as a DATE
LLM identified column "month" as a DATE
LLM identified column "day" as a DATE
LLM identified column "time" as a DATE
LLM identified column "lat" as a GEO
LLM identified column "lon" as a GEO
LLM identified column "country" as a GEO
LLM identified column "admin1" as a GEO
LLM identified column "admin2" as a GEO
@david-andrew
david-andrew / gist:8213d202107908112d02565544846bf2
Created February 20, 2024 21:26
LLM annotation output for ACLED dataset
geo=[GeoAnnotation(name='iso', display_name=None, description='The values in the dataset represent the ISO 3166-1 numeric country codes, which are internationally recognized codes assigned to each country and certain territories. In this context, the number 854 corresponds to Burkina Faso. These codes are used for data exchange and to increase clarity and ensure unambiguity when identifying countries on a global scale.', type=<ColumnType.GEO: 'geo'>, geo_type=<GeoType.COUNTRY: 'country'>, primary_geo=None, resolve_to_gadm=None, is_geo_pair=None, coord_format=None, qualifies=None, aliases={}, gadm_level=None), GeoAnnotation(name='region', display_name=None, description='This dataset categorizes events based on their geographic location within the continent of Africa, specifically focusing on the sub-region of Western Africa. This area includes countries along the Atlantic coast, from the Sahara Desert in the north to the Gulf of Guinea in the south.', type=<ColumnType.GEO: 'geo'>, geo_type=<GeoType.COUNTRY: 'c
@david-andrew
david-andrew / gist:0a71ad1e7bab8649d20381f5bf0ddd9b
Created February 21, 2024 13:49
Several LLM Dojo Annotation Examples
Meta(path=PosixPath('datasets/mock_aqi.csv'), name='Air Quality Index', description='This dataset represents daily air quality observations collected from various monitoring stations across different cities worldwide. Each row corresponds to a single observation with details about the date, time, and location of the observation, along with specific air quality metrics and conditions.')
LLM identified column "year" as a DATE
LLM identified column "month" as a DATE
LLM identified column "day" as a DATE
LLM identified column "time" as a DATE
LLM identified column "lat" as a GEO
LLM identified column "lon" as a GEO
LLM identified column "country" as a GEO
LLM identified column "admin1" as a GEO
LLM identified column "admin2" as a GEO