Skip to content

Instantly share code, notes, and snippets.

View tmtmtmtm's full-sized avatar

Tony Bowden tmtmtmtm

View GitHub Profile
#!/bin/env ruby
# Find everyone who has a P39 with a party of the "USR" disambiguation page
# and change it to the "Save Romania Union" item.
require 'csv'
require 'rest-client'
WIKIDATA_SPARQL_URL = 'https://query.wikidata.org/sparql'
@tmtmtmtm
tmtmtmtm / adding-aims-ids.md
Last active September 23, 2018 12:13
Adding Northern Ireland Assembly IDs to Wikdiata

Last week we got a new Wikidata property for "Northern Ireland Assembly ID"

My goal this morning was to add that for all members, current and historic.

Step 1: Get the IDs

The Northern Ireland Assembly has a useful API that can give us all this data, so I knocked up a quick script to get them all: https://github.com/everypolitician-scrapers/ni-assembly-members/blob/master/scraper.rb

I've written hundreds of scrapers like this, and can cobble something like this together from previous versions really quickly. The only part that took any time was splitting out the prefixes and suffixes from people's names, but even that was based on a previous scraper

@tmtmtmtm
tmtmtmtm / scraper.rb
Created November 22, 2018 10:19
Solomon Islands constituency QuickStatements
#!/bin/env ruby
# frozen_string_literal: true
# Produce QuickStatements for creating missing constituency items,
# and adding 'instance of' to existing ones (most of which are missing it)
require 'pry'
require 'scraped'
require 'wikidata_ids_decorator'
@tmtmtmtm
tmtmtmtm / senate18.ps
Created January 16, 2019 16:20
Position Statements for Italian Senators of the 18th Legislature with no Parliamentary Term
# Generated via: http://tinyurl.com/ydetfb96
Q50772223 P39 Q50772223-42D92740-6741-4BD5-91A1-45271EFE8D06 P2937 Q48799610
Q50847261 P39 Q50847261-3D888B95-42D6-4C99-8B56-4087BDB8E1C6 P2937 Q48799610
Q50847271 P39 Q50847271-9778831E-B673-45A6-84BF-246BDDF6F0F4 P2937 Q48799610
Q50875173 P39 Q50875173-AEF7EF37-5399-4F35-8A0B-337970B320B5 P2937 Q48799610
Q50875679 P39 Q50875679-72015D3D-A2F3-473A-83B1-11AA8CCC97F2 P2937 Q48799610
Q50984576 P39 Q50984576-1CC736EB-FB5D-46C4-BCD2-CEC4C743E35B P2937 Q48799610
Q51026222 P39 Q51026222-4091F57E-933A-45FB-9848-8347EB4E56E0 P2937 Q48799610
Q51027227 P39 Q51027227-3394C5B0-7153-4E2A-82FB-8DD75B415F2C P2937 Q48799610
Q51027386 P39 Q51027386-CD5BAD68-096D-4527-A999-099034AC2C99 P2937 Q48799610
@tmtmtmtm
tmtmtmtm / azerbaijan-6.md
Last active June 30, 2020 08:26
Adding Members of the 6th National Assembly of Azerbaijan to Wikidata
@tmtmtmtm
tmtmtmtm / elected_in_Malawi-General-Election.rq
Created July 4, 2020 10:48
Migrate Wikidata 'elected in': '2014 Malawian general election' to the 2014 legislative election instead
# wd sparql elected_in_Malawi-General-Election.rq | jq -r '.[] | "\(.ps) P2715 Q15206416 Q61732445"' | wd update-qualifier --batch --summary "Switch elected in from Malawian general election to the legislative election"
SELECT ?ps ?item ?itemLabel WHERE {
?item p:P39 ?ps .
?ps ps:P39 wd:Q21295990 ; pq:P2715 wd:Q15206416.
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}
@tmtmtmtm
tmtmtmtm / coloured-table.png
Last active July 5, 2020 20:57
Making a Listeria table look nicer
coloured-table.png
@tmtmtmtm
tmtmtmtm / migrate-merged-constituencies.rq
Created July 6, 2020 13:19
Fix 'position held' qualifiers for merged constituencies
# Update 'position held' statements for Members of the Parliament of Ghana
# to point at the correct constituency items, after merging
#
# wd sparql migrate-merged-constituencies.rq | jq -r '.[] | "\(.ps) P768 \(.old) \(.new)"' | wd update-qualifier --batch --summary "Update constituencies qualifiers after merges"
SELECT DISTINCT ?ps ?id ?old ?new WHERE {
?id p:P39 ?ps .
?ps ps:P39 wd:Q61659845 ; pq:P768 ?old .
?old owl:sameAs ?new .
}

A bunch of French municipal elections are showing up on the Recent elections with no Country report. Looking at some of those, it seems there's also a bit of a mismatch of modelling, with some being 'instance of: French municipal elections 2020', and some 'instance of: municipal election'+'part of: French municipal elections 2020'. The latter seems better to me, so I'm going to also migrate all the former to that whilst I'm here.

So, to select those:

SELECT ?item ?itemLabel WHERE {
  ?item wdt:P31 wd:Q60846649 .
  FILTER NOT EXISTS { ?item wdt:P17 [] }
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}
@tmtmtmtm
tmtmtmtm / lithuania-11-members.md
Last active July 13, 2020 10:53
Add Members of the 11th Lithuanian Seimas to Wikidata

Step 1: Add a tracking page

The first step for something like this is always to not only see what Wikidata already knows, but to capture that with a Listeria page, so we can track changes over time. Here that's WikiProject every politician/Lithuania/data/Seimas/11th. That initially has no members, which is sometimes a sign that the data has been entered in different way: e.g. with start/end dates rather than legislative terms. But a check for that approach (https://w.wiki/Wmi) shows no entries either, so we're working from a clean slate, and can continue with the term-based approach already taken for the 12th Seimas.

Step 2: Look for a Wikipedia category

If any of the Wikipedias have a category of "Member of the 11th Seim