- Fetch some json/XML file from an external API
- Store the file on S3 (as a backup) with a file name with a i.e. a timestamp. Pass the filename on to next task
- Read the json/xml from S3 and into some table structure (pandas, agate etc) and change field types etc.
- Store the table in a postgres database in a temp table
- Compare the temp table to a "main" table and see if there are changes (some SQL diff). Find out which records have to be added/removed/updated in the "main" table.
- If nothing has changed, abort everything. If it has, pass on which records are new, deleted and updated.
- a) insert new records in main table, alert newsroom on slack of new items.b) delete items in main table not in temp-table. Alert via slack. c) Update records in main table, alert via slack
- The end
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
type: custom:mushroom-template-card | |
primary: >- | |
{% if (states.light | selectattr('state', 'eq', 'on') | list | count |float) > | |
0 %} | |
{{ states.light | selectattr('state', 'eq', 'on') | list | count }} lys på | |
{% else %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#lsusbto find path, lsusb -v to find serial | |
#Tower weather report | |
docker run --rm --name rtlsdr-airband -p 8000:8000 -e RTLSDRAIRBAND_FREQS=125.25 -e RTLSDRAIRBAND_SERIAL=00000001 --device /dev/bus/usb:/dev/bus/usb fredclausen/rtlsdrairband | |
#VHF 16 og sea traffic coord center | |
docker run --rm --name rtlsdr-airband -p 8000:8000 -e RTLSDRAIRBAND_FREQS=156.8,157.025,151.625,156.575 -e LOG_SCAN_ACTIVITY=true -e RTLSDRAIRBAND_SERIAL=00000001 -e RTLSDRAIRBAND_MODE=scan -e RTLSDRAIRBAND_SHOWMETADATA=true -e RTLSDRAIRBAND_NAME=VHF16_Fedje -e RTLSDRAIRBAND_GENRE=VHF --device /dev/bus/usb:/dev/bus/usb fredclausen/rtlsdrairband |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
trafficData(trafficRegistrationPointId: "72220V805744") { | |
trafficRegistrationPoint { | |
id, | |
name, | |
direction { | |
from, | |
to, | |
} | |
}, |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
body { | |
background-color: red; | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WITH ais AS | |
(SELECT * | |
FROM aisdata.bronnbaater_resampled_min_lowspeed), | |
lokaliteter AS | |
(SELECT * | |
FROM fishandfjord.fiskeridir_alle_lokaliteter | |
WHERE loknr = '11763') | |
SELECT truncdate, | |
loknr, | |
lokalitet, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from __future__ import print_function | |
from builtins import range | |
from airflow.operators import PythonOperator, PostgresOperator, DummyOperator | |
from airflow.models import DAG | |
from datetime import datetime, timedelta | |
import time | |
from pprint import pprint | |
seven_days_ago = datetime.combine( |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import agate | |
class Streaks(agate.Computation): | |
""" | |
Computes the streaks of consecutive values in a column. | |
Each streak will be given an increasing | |
integer value so that you can group by this later to | |
find longest consecutive streak. | |
""" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env node | |
'use strict'; | |
var dir = require('node-dir'); | |
var fs = require('fs'); | |
var http = require('http'); | |
var _ = require('lodash'); | |
var CartoDB = require('cartodb'); | |
var pg = require('pg'); | |
NewerOlder