Skip to content

Instantly share code, notes, and snippets.

View xissy's full-sized avatar

Taeho Kim xissy

View GitHub Profile
@bernhardschaefer
bernhardschaefer / spark-submit-streaming-yarn.sh
Last active March 21, 2022 05:04
spark-submit template for running Spark Streaming on YARN (referenced in https://www.inovex.de/blog/247-spark-streaming-on-yarn-in-production/)
#!/bin/bash
# Minimum TODOs on a per job basis:
# 1. define name, application jar path, main class, queue and log4j-yarn.properties path
# 2. remove properties not applicable to your Spark version (Spark 1.x vs. Spark 2.x)
# 3. tweak num_executors, executor_memory (+ overhead), and backpressure settings
# the two most important settings:
num_executors=6
executor_memory=3g

FWIW: I (@rondy) am not the creator of the content shared here, which is an excerpt from Edmond Lau's book. I simply copied and pasted it from another location and saved it as a personal note, before it gained popularity on news.ycombinator.com. Unfortunately, I cannot recall the exact origin of the original source, nor was I able to find the author's name, so I am can't provide the appropriate credits.


Effective Engineer - Notes

What's an Effective Engineer?

@jbkopecky
jbkopecky / seoul256.vim
Created November 10, 2016 13:27
Seoul256 own custom vim Airline theme (vim/autoload/airline/themes/seoul256.vim)
let g:airline#themes#seoul256#palette = {}
function! airline#themes#seoul256#refresh()
let M0 = airline#themes#get_highlight('Special')
let accents_group = airline#themes#get_highlight('Special')
let modified_group = [M0[0], '', M0[2], '', '']
let warning_group = airline#themes#get_highlight2(['airline_warning', 'bg'], ['airline_warning', 'fg'])
let s:N1 = airline#themes#get_highlight2(['Normal', 'bg'], ['StatusLine', 'fg'])
@eferro
eferro / _aws_golang_examples.md
Last active August 21, 2024 05:20
golang aws: examples

AWS Golang SDK examples

@dchowitz
dchowitz / es6-debugging-in-vscode.md
Last active August 30, 2023 06:23
Debugging ES6 in VS Code

Debugging ES6 in VS Code

My current editor of choice for all things related to Javascript and Node is VS Code, which I highly recommend. The other day I needed to hunt down a bug in one of my tests written in ES6, which at time of writing is not fully supported in Node. Shortly after, I found myself down the rabbit hole of debugging in VS Code and realized this isn't as straightforward as I thought initially. This short post summarizes the steps I took to make debugging ES6 in VS Code frictionless.

What doesn't work

My first approach was a launch configuration in launch.json mimicking tape -r babel-register ./path/to/testfile.js with babel configured to create inline sourcemaps in my package.json. The debugging started but breakpoints and stepping through the code in VS Code were a complete mess. Apparently, ad-hoc transpilation via babel-require-hook and inline sourcemaps do not work in VS Code. The same result for attaching (instead of launch) to `babel-node

@ChrisLowe-Takor
ChrisLowe-Takor / ClosureAsRxObservable.swift
Last active November 21, 2021 01:53
Creating an RxSwift Observable from a closure
// Say we have an Alamofire request we want to consume as an Observable..
func fooAsObservable(URL: String, headers: [String: AnyObject?]) -> Observable<Foo> {
return create { observer in
let request = Alamofire(.GET, URL, headers: headers
.responseJSON(completionHandler: { (_, _, result) in
switch result {
@japm48
japm48 / README.md
Created January 17, 2016 11:45
CoreOS: enable UTF8 with a provisional (ugly) workaround
  1. First, you need a Linux distro with a functional UTF-8 locale (i.e. NOT CoreOS).

    Execute my_gen_locale.sh to generate a valid locale-archive by copying a locale from the system. It will use en_US.UTF-8, but can be changed if deemed necessary (the list of locales can be found in glibc sources: https://sourceware.org/git/?p=glibc.git;a=blob;f=localedata/SUPPORTED ).

  2. Now the /usr partition must be edited to install it.

Apparently this is quite discouraged by the devs, but I still don't know how to create a new CoreOS image

Kinesis로 App 로그 다루기

글쓴이: 김정주([email protected])

이 글은 AWS 블로그를 참고하고 추가/보완하여 작성되었습니다.


Kinesis는 아마존 웹서비스(AWS)에서 제공하는 실시간 데이터 처리기이다.

@haje01
haje01 / TensorFlow 시작하기.md
Last active May 3, 2024 07:30
TensorFlow 시작하기

텐서플로우 시작하기

글쓴이: 김정주([email protected])

이 문서는 텐서플로우 공식 페이지 내용을 바탕으로 만들어졌습니다.


소개

텐서플로우(TensorFlow)는 기계 학습과 딥러닝을 위해 구글에서 만든 오픈소스 라이브러리입니다. 데이터 플로우 그래프(Data Flow Graph) 방식을 사용하였습니다.

@cgmartin
cgmartin / logging-middleware.js
Created May 24, 2015 01:43
Morgan JSON log format example
'use strict';
var morgan = require('morgan');
var os = require('os');
morgan.token('conversation-id', function getConversationId(req) {
return req.conversationId;
});
morgan.token('session-id', function getSessionId(req) {
return req.sessionId;