Skip to content

Instantly share code, notes, and snippets.

View danish-rehman's full-sized avatar

danish-rehman

View GitHub Profile
@danish-rehman
danish-rehman / README.md
Last active July 12, 2016 19:20
Spark : Standalone setup on development box

Follow this official link

Download - link

Pre-built with hadoop 2.6 Unzip it and keep it in a folder.

Set up environment

@danish-rehman
danish-rehman / spark_debug.md
Last active July 1, 2016 21:28
Spark : Issues and solutions
  1. JSON format as string must contain double quotes around keys and values.
  2. If schema is coming out as corrupt then JSON is un-parsable by Spark standard.
  3. Can not have more then one scContenxt on a pyspark shell.
  4. By specifying a schema, you can speed up your Spark job by cutting down the time Spark uses to infer the schema.
  5. If you have a lot of keys that you don't care about, you can filter for only the keys you need.
  6. Too many keys in your JSON data can trigger an OOM error on your Spark Driver when you infer the schema.
  7. When declaring schema if int is very large do not use IntegerType.
  8. Whenever import file use file:// protocol.
@danish-rehman
danish-rehman / game_of_hanoi.cpp
Last active April 29, 2016 22:07
Tower of Hanoi
/*
* Basic implementation of tower of hanoi
*
* Initial Board
* 1 - -
* 2 - -
* 3 - -
*
* To win the board must be as below
* - - 1
import UIKit
import MediaPlayer
class ViewController: UIViewController {
var moviePlayer:MPMoviePlayerController!
override func viewDidLoad() {
super.viewDidLoad()
mysql> desc gif_user;
+-------------+--------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+-------------------+-----------------------------+
| guid | int(11) | NO | PRI | NULL | auto_increment |
| user_cookie | varchar(500) | NO | MUL | NULL | |
| gif_id | int(11) | NO | | NULL | |
| createdon | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| period | int(11) | YES | | NULL | |
/* Inefficient way */
struct email {
time_t send_date;
int flags;
int length;
char body[EMAIL_BODY_MAX];
};
struct email *email = malloc (sizeof (struct email));
/* This size is fixed and not dependent on the caller */
/* More efficient way
struct email {
time_t send_date;
int flags;
int length;
char body[];
};
/* we have an email of email_length bytes ... */
struct email *email = malloc (sizeof (struct email) + email_length);
@danish-rehman
danish-rehman / tk
Created June 3, 2015 06:52
token_bucket
rate = 5.0; // unit: messages
per = 8.0; // unit: seconds
allowance = rate; // unit: messages
last_check = now(); // floating-point, e.g. usec accuracy. Unit: seconds
when (message_received):
current = now();
time_passed = current - last_check;
last_check = current;
allowance += time_passed * (rate / per);
#include "stdio.h"
#define MAXBUF 10
int main(int argc, char **argv)
{
char buf[MAXBUF]; // enough room for 9 characters and one '\0'
strcpy(buf,"hello "); // use 6 characters
/**
* @brief Prints the string s, starting at the current
* location of the cursor.
*
* If the string is longer than the current line, the
* string should fill up the current line and then
* continue on the next line. If the string exceeds
* available space on the entire console, the screen
* should scroll up one line, and then the string should
* continue on the new line. If '\n', '\r', and '\b' are