kujenga

Setting up a Helium ETL

This documents explains a quick intro how to setup your own ETL server for the Helium Blockchain. Running an ETL is not an easy task, but hopefully this document will help you get started, and by the end you will have the blockchain up and running.

Server Requirements

• 5TB SSD (NVME prefered) disk

---

Install tools

kujenga

git branch -m master main
git fetch origin
git branch -u origin/main main
git remote set-head origin -a

kujenga

# Add the following 'help' target to your Makefile
# And add help text after each target name starting with '\#\#'
 
help:           ## Show this help.
	@fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\(.*\):.*##\(.*\)/\1:\t\2/'
 
# Everything below is an example
 
target00:       ## This message will show up when typing 'make help'
	@echo does nothing

kujenga

#!/usr/bin/env python3
#
# This script augments the converter tool provided by standard notes to provide
# Markdown instead of HTML as the format for the export, which is useful in
# particular after a conversion from Evernote exports [1]. This allows them to
# be viewed with other editors within the standard notes platform.
#
# The pandoc command like tool must be installed locally in order for this
# script to work.
#

kujenga

Keybase proof

I hereby claim:

• I am kujenga on github.
• I am ataylor (https://keybase.io/ataylor) on keybase.
• I have a public key whose fingerprint is D86B E1CC CA9A D502 8987 B297 42B1 4D00 D191 5CE5

To claim this, I am signing this object:

kujenga

• Install Package Control: https://packagecontrol.io/installation
  • Git
  • GoSublime
  • GitGutter
  • SublimeLinter
  • SublimeLinter-flake8
  • SublimeLinter-rubocop
  • SublimeLinter-contrib-golint
  • SublimeLinter-contrib-govet
• AngularJS

kujenga

#!/usr/bin/env bash
#
# Generate test coverage statistics for Go packages.
#
# Works around the fact that `go test -coverprofile` currently does not work
# with multiple packages, see https://code.google.com/p/go/issues/detail?id=6909
#
# largely based on the below links with a few small changes
# to support classic google appengine
#

kujenga

set :sync_directories, ["public/assets", "public/galleries"]
set :sync_backups, 3

kujenga

# these extensions are intended to make it easy to determine whether or not something exists in the database already
# by either using a specified hash of the interested parameters and their values in the current object
# that can be sent to the database as a query

module ActiveRecordExtension
  # extends the active support concerns in order to enable direct calls on model objects
  extend ActiveSupport::Concern

  # add instance methods here
  def model_match_exists(*attrs)

kujenga

require 'nokogiri'
require 'feedjira'

#url = ARGV[0]
url = "http://25livepub.collegenet.com/calendars/all-campus-events.rss"

feed = Feedjira::Feed.fetch_and_parse url

feed.entries.each { |entry|
  entry_h = {}