Skip to content

Instantly share code, notes, and snippets.

View turicas's full-sized avatar

Álvaro Justen turicas

View GitHub Profile
@turicas
turicas / 2021-02-23-microdados_vacinacao-por-grupo.csv
Created February 23, 2021 15:44
Microdados vacinação por grupo do vacinado
Paciente Grupo Contagem
Trabalhadores de Saúde 3916902
Faixa Etária 1351065
Povos Indígenas 229231
Pessoas de 60 anos ou mais institucionalizadas 180771
118921
Pessoas com Deficiência 16748
Comorbidades 12946
Povos e Comunidades Tradicionais 8795
Forças de Segurança e Salvamento 4204
def abbreviate_number(n, divider=1_000, suffix=None):
"""
>>> abbreviate_number(100)
'100'
>>> abbreviate_number(1_000)
'1.0K'
>>> abbreviate_number(1_000, divider=1_024)
'1000'
>>> abbreviate_number(1_024, divider=1_024)
'1.0K'
@turicas
turicas / README.md
Last active November 2, 2020 11:19
SQL file to extract all Brazilian domains from Chrome history using Chrome's SQLite database

Extract Brazilian Domains from Google Chrome History

Save the chrome-history.sql file and then execute on the terminal:

CHROME_DB=~/.config/google-chrome/Default/History
cat chrome-history.sql | sqlite3 $CHROME_DB

Then access the data in my-chrome-history.csv.

@turicas
turicas / brasil_io.py
Last active February 17, 2025 16:55
Exemplo de código para acessar dados do Brasil.IO
import csv
import gzip
import io
import json
from urllib.parse import urlencode, urljoin
from urllib.request import Request, urlopen
class BrasilIO:
@turicas
turicas / parse-nginx-logs.sh
Created June 14, 2020 11:24
Process all nginx logs for a specific website through goaccess using docker
#!/bin/bash
# Process all nginx logs for a specific website through goaccess using docker
# <https://goaccess.io/>
set -e
cat_site_logs() {
sitename="$1"
log_path="/var/log/nginx/"
@turicas
turicas / README.md
Last active June 10, 2020 09:30
Análise nos dados do auxílio emergencial COVID-19
@turicas
turicas / screenshot.py
Created June 6, 2020 20:34
Python script to take screenshot of a website
#!/usr/bin/env python3
# requires: pip install splinter
import argparse
import datetime
import os
import shutil
import time
from urllib.parse import urlparse
from pathlib import Path
@turicas
turicas / filter_log.py
Last active April 28, 2022 03:00
Parse HTTP access log files, extract unique user agents (unique per file) and export to CSV
import argparse
import csv
import rows
parser = argparse.ArgumentParser()
parser.add_argument("input_filename")
parser.add_argument("output_filename")
args = parser.parse_args()
@turicas
turicas / check_username_conflict.py
Created April 15, 2020 22:13
Check Django username conflict (case senstive)
"""Check conflicts in Django usernames
This script checks every username for a possible conflict, due to Django's default policy
to accept `user` and `User` as different usernames.
"""
from django.contrib.auth import get_user_model
from tqdm import tqdm

Check Errors on CKAN resources

This script connects to a CKAN API and, for all available datasets, check URLs of all resources to identify the ones that are not available.

Installing

pip install scrapy