Marcin Bilski bilus

Initialize environment variables

export CLOUD_STORAGE_BUCKET="mbilski-badger"
export DB_PATH="./"
export CDN_BASE_URL="http://35.244.156.33/badger_gz/"
export LOGOS_BASE_DIR="./logos"

Go to the app’s directory:

Create `dbt` service account in GCP

Needs “Bigquery User” and “Bigquery Connection User” roles.

Needs them in both conrad-production-api and conrad-analytics-staging

Install dbt

https://docs.getdbt.com/docs/installation

brew update
brew tap fishtown-analytics/dbt
brew install dbt

radiodns

Broadcast data

Link raw data to remote ids

SELECT n.message.data.remote_content_id, n.message.data.remote_broadcast_id, n.message.data.source, r.message as rawData, n.message as normalizedData
FROM `conrad-production-api.conradEvents.StaticDataRaw` r
LEFT JOIN `conrad-production-api.conradEvents.StaticDataNormalized` n
  ON (n.message.data.update_id = JSON_EXTRACT_SCALAR(r.message, "$.data.update_id"))

1. No next step (how to install Dolt) before or after adding repository

2. Confusing “Dolt builds > Request access here” (is it an extra? do I HAVE to do it?)

3. Tried dolt checkout bilus/test initially, it just hangs there

4. Followed “Setting up a remote”

a. It’s `dolt push …` not `dolt remote`

b. Tried `dolt push` got `error: failed to get remote db` due to auth failure

c. Use dolt sql and did `create table countries (id integer primary key, name varchar(255));`

5. No obvious way to get data in. I want to see if it’s not a toy so I want a reasonably large dataset in.

a. Starting `dolt sql-server` it prints nothing but I tried 3306 nevertheless (have to google it up, am a Postgres guy)

b. Look for a GUI to make life easier, find Sequel Pro, try to use, got “An error occurred while retrieving information. MySQL said: unknown error: expecting “EOF” but got ‘`’ instead” when trying to view the contents of the (empty) countries table.

layout	permalink
docs	/documentation/

Oya

Oya is a command line tool aiming to help you bootstrap and manage deployable projects.

	<?xml version="1.0" encoding="utf-8"?>
	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
	<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
	<head>
	<!-- 2020-03-10 Tue 19:33 -->
	<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
	<meta name="viewport" content="width=device-width, initial-scale=1" />
	<title>stitchingdata</title>
	<meta name="generator" content="Org mode" />