jsam’s gists

jsam / gist:37a8fdacac8d197e10771cdc887ff2bb

Last active April 28, 2020 13:01

proposal

	with this PR API usage would be something like this:

	```
	with chdir(project.abs_path):
	with service_callback_communication(user_job):
	import_dataset(...)
	```

	now cleaner version of this could be something like this

jsam / gist:d957f306ed0fe4ff018e902df6a1c8e3

Created March 25, 2020 15:06

123

jsam / gist:895e7544e3edc8597a15944ba1e9f766

Created March 25, 2020 13:22

raw

asdadasd

jsam / DataUpload.scala

Last active January 21, 2020 12:19

The idea of what I'm trying to achieve (sync solution would be acceptable)

	def upload(parts: Vector[Part[F]], blocker: Blocker): F[Response[F]] =
	parts.find(isValidPart) match {
	case Some(file) =>
	val t0 = System.nanoTime()
	val home = sys.env.getOrElse("DP_UPLOAD_DIR", "/tmp/data-parser/upload")
	val filename = file.filename.getOrElse("filename")
	val path = FSUtils.ensurePath(home, filename)

	val writer = fs2.io.file.writeAll(path, blocker)
	val response = file.body

jsam / json

Created October 9, 2019 05:52

live match

{"matchId":5061259433,"radiantScore":34,"direScore":5,"leagueId":0,"averageRank":5807,"radiantLead":28359,"gameTime":1493,"isCompleted":false,"isUpdating":false,"isParsing":true,"radiantTeamId":0,"direTeamId":0,"players":[{"heroId":5,"name":"Oggy","playerSlot":0,"steamId":321610761,"steamAccount":{"id":321610761,"profileUri":"https://steamcommunity.com/profiles/76561198281876489/","timeCreated":1454766491,"cityId":0,"communityVisibleState":3,"name":"Oggy","lastLogOff":1570540806,"avatar":"e9/e97ee8b2f38ebf5a222b5630dee1d1a055bb9c90_full.jpg","primaryClanId":103582791429521408,"soloRank":3949,"partyRank":2077,"isAnonymous":false,"isStratzAnonymous":false,"seasonRank":73,"smurfFlag":0,"smurfCheckDate":"2019-10-07T10:57:00"},"isRadiant":true,"numKills":4,"numDeaths":1,"numAssists":16,"numLastHits":33,"numDenies":2,"goldPerMinute":252,"level":15,"gold":1624,"item0Id":214,"item1Id":102,"item2Id":73,"item3Id":36,"item4Id":42,"playbackData":{"playerUpdatePositionEvents":[{"time":-11,"x":127,"y":127},{"time":1,"x":15

jsam / gist:22af0d76d9456320732728b42611f529

Created October 2, 2019 08:56

Stratz: Live Data

This file has been truncated, but you can view the full file.

	// 20191002103439
	// https://api.stratz.com/api/v1/match/5050619833/live

	{
	"matchId": 5050619833,
	"radiantScore": 9,
	"direScore": 49,
	"leagueId": 0,
	"averageRank": 6219,
	"radiantLead": -48902,

jsam / gist:3329f961d8062d6d917eddbb5dc45cd2

Created September 20, 2019 11:04

	PROJECT_V1_BROKEN = """
	'@context':
	created: http://schema.org/dateCreated
	foaf: http://xmlns.com/foaf/0.1/
	name: notexisting:name
	updated: http://schema.org/dateUpdated
	version: http://schema.org/schemaVersion
	'@type': foaf:Project
	created: 2018-02-22 10:40:14.878009
	name: demo

jsam / test.scala

Created August 24, 2019 20:47

http4s file upload test

	package org.renku.http

	import java.io.File
	import java.net.URL
	import cats.syntax.functor._
	import cats.implicits._
	import cats.effect.{ContextShift, ExitCode, IO, IOApp, Sync}
	import org.http4s.client.dsl.Http4sClientDsl
	import org.http4s.headers._
	import org.http4s.multipart.{Multipart, Part}

jsam / wf2.sh

Created July 19, 2019 10:08

simple workflow

	--- Work/data_repos » renku init test-update
	Initialized empty project in /home/sam/Work/data_repos/test-update
	--- Work/data_repos » cd test-update
	--- data_repos/test-update ‹master› »
	--- data_repos/test-update ‹master› »
	--- data_repos/test-update ‹master› » renku run echo "A" > step1
	--- data_repos/test-update ‹master› » renku run cat step1 > step2
	--- data_repos/test-update ‹master› » renku run cat step2 > step3
	--- data_repos/test-update ‹master› »
	--- data_repos/test-update ‹master› »

jsam / wf1.sh

Created July 19, 2019 10:06

simple workflow test

	--- Work/data_repos » renku init test-update
	Initialized empty project in /home/sam/Work/data_repos/test-update
	--- Work/data_repos » cd test-update
	--- data_repos/test-update ‹master› »
	--- data_repos/test-update ‹master› »
	--- data_repos/test-update ‹master› » renku run echo "A" > step1
	--- data_repos/test-update ‹master› » renku run cat step1 > step2
	--- data_repos/test-update ‹master› » renku run cat step2 > step3
	--- data_repos/test-update ‹master› »
	--- data_repos/test-update ‹master› »

sam jsam