Skip to content

Instantly share code, notes, and snippets.

View Arnold1's full-sized avatar

Arnold1

1 follower · 1 following
View GitHub Profile
@Arnold1
Arnold1 / TweetStreams.scala
Created November 11, 2017 03:21 — forked from samklr/TweetStreams.scala
TweetStream with Spark
import Utils
import org.apache.spark.streaming.StreamingContext._
import org.apache.spark.streaming.{Seconds, StreamingContext}
import StreamingContext._
import org.apache.spark.SparkContext._
import org.apache.spark.streaming.twitter._
import org.apache.spark.SparkConf
import org.apache.spark.{SparkConf, SparkContext}
@Arnold1
Arnold1 / 00-OozieWorkflowHdfsAndEmailActions
Created November 8, 2017 20:23 — forked from airawat/00-OozieWorkflowHdfsAndEmailActions
Oozie workflow application with FS and email actions; Includes sample data, workflow components, commands.
This gist includes components of a simple workflow application that created a directory and moves files within
hdfs to this directory;
Emails are sent out to notify designated users of success/failure of workflow. There is a prepare section,
to allow re-run of the action..the prepare essentially negates the move done by a potential prior run
of the action. Sample data is also included.
The sample application includes:
--------------------------------
1. Oozie actions: hdfs action and email action
2. Oozie workflow controls: start, end, and kill.
@Arnold1
Arnold1 / 00-OozieWorkflowHdfsAndEmailActions
Created November 8, 2017 20:23 — forked from airawat/00-OozieWorkflowHdfsAndEmailActions
Oozie workflow application with FS and email actions; Includes sample data, workflow components, commands.
This gist includes components of a simple workflow application that created a directory and moves files within
hdfs to this directory;
Emails are sent out to notify designated users of success/failure of workflow. There is a prepare section,
to allow re-run of the action..the prepare essentially negates the move done by a potential prior run
of the action. Sample data is also included.
The sample application includes:
--------------------------------
1. Oozie actions: hdfs action and email action
2. Oozie workflow controls: start, end, and kill.
@Arnold1
Arnold1 / cv2ff.cpp
Created November 5, 2017 02:48 — forked from yohhoy/cv2ff.cpp
Convert from OpenCV image and write movie with FFmpeg
/*
* Convert from OpenCV image and write movie with FFmpeg
*
* Copyright (c) 2016 yohhoy
*/
#include <iostream>
#include <vector>
// FFmpeg
extern "C" {
#include <libavformat/avformat.h>
@Arnold1
Arnold1 / main.cpp
Last active October 29, 2017 22:55
screen capture
#include "ScreenCapture.h"
#include <algorithm>
#include <atomic>
#include <chrono>
#include <iostream>
#include <locale>
#include <string>
#include <thread>
#include <vector>
#include <queue>
@Arnold1
Arnold1 / main.cpp
Last active October 29, 2017 19:23
screen capture
#include "ScreenCapture.h"
#include <algorithm>
#include <atomic>
#include <chrono>
#include <iostream>
#include <locale>
#include <string>
#include <thread>
#include <vector>
#include <queue>
@Arnold1
Arnold1 / main.cpp
Last active January 12, 2019 21:40
screen recording
#include "ScreenCapture.h"
#include <algorithm>
#include <atomic>
#include <chrono>
#include <iostream>
#include <locale>
#include <string>
#include <thread>
#include <vector>
#include <queue>
@Arnold1
Arnold1 / gist:289799043291ec27a317c658267587a5
Created September 27, 2017 17:10
apache spark
https://blog.socialcops.com/engineering/machine-learning-for-big-data/
@Arnold1
Arnold1 / gist:7035059c0da71ed5fc061ad1b343b0fd
Created August 14, 2017 21:31
scalding issue 2
$ ./sbt update
[warn] Executing in batch mode.
[warn] For better performance, hit [ENTER] to switch to interactive mode, or
[warn] consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading project definition from /Users/geri/work/scalding/project
[info] Updating {file:/Users/geri/work/scalding/project/}scalding-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[warn] There may be incompatibilities among your library dependencies.
[warn] Here are some of the libraries that were evicted:
@Arnold1
Arnold1 / gist:59cba6c03e065e7f8ed0b2c67a555f59
Created August 14, 2017 21:03
scalding issue
This file has been truncated, but you can view the full file.
$ ./sbt update
[warn] Executing in batch mode.
[warn] For better performance, hit [ENTER] to switch to interactive mode, or
[warn] consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading project definition from /Users/geri/work/scalding/project
[info] Updating {file:/Users/geri/work/scalding/project/}scalding-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[warn] There may be incompatibilities among your library dependencies.
[warn] Here are some of the libraries that were evicted: