Skip to content

Instantly share code, notes, and snippets.

View suensummit's full-sized avatar
🚀
Liftoff

Summit Suen suensummit

🚀
Liftoff
View GitHub Profile
@suensummit
suensummit / PCHome_crawler.py
Created January 10, 2021 00:58
Example PCHome crawler script for AWS Lambda
import numpy as np
import pandas as pd
import requests
import bs4
import lxml.etree as xml
import time
from sqlalchemy import create_engine
import pymysql
import pymssql
@suensummit
suensummit / INSTALLATION_GUIDE.md
Last active November 23, 2020 10:05
A Complete Installation Guide for `bigsnpr`(https://github.com/privefl/bigsnpr) on Azure Windows Server.
  • Install R, Rtools (Required), RStudio (Optional).
  • Install Git for Windows for retrive the latest version of xgboost.
  • Install Visual Studio Community, for c++ development environment.
    • Select Workload: Desktop/Mobile/Linux Development with C++ to obtain C++ CMake tool

Screen Shot 2020-11-23 at 12 18 37 PM

Screen Shot 2020-11-23 at 12 18 52 PM

@suensummit
suensummit / eigenvectors_from_eigenvaluies.R
Created November 16, 2019 15:46
A demo R script of [the paper EIGENVECTORS FROM EIGENVALUES](https://arxiv.org/pdf/1908.03795.pdf), modified from [Yu-Chen Shu](https://www.facebook.com/yuchen.shu/posts/10157876738839228) rewritten in R.
# Generate random matrix with dim = N
N <- 3
A <- matrix(runif(N*N), N, N, byrow=TRUE)
A <- A+t(A)
# Setting up eigenvalues lambda, true eigenvector V and derived eigenvector U
ev <- eigen(A)
(lambda <- ev$values)
(V <- ev$vectors)
U <- matrix(0L, N, N)
@suensummit
suensummit / main.py
Created December 13, 2018 03:17 — forked from khanhnamle1994/main.py
FCN - Full Code
#--------------------------
# USER-SPECIFIED DATA
#--------------------------
# Tune these parameters
num_classes = 2
image_shape = (160, 576)
EPOCHS = 40
BATCH_SIZE = 16
@suensummit
suensummit / kMeans_in_BQ.sql
Created December 5, 2018 10:40 — forked from smrgit/kMeans_in_BQ.sql
kMeans using JavaScript UDFs in BigQuery
CREATE TEMPORARY FUNCTION
-- In this function, we're going to be working on arrays of values.
-- we're also going to define a set of functions 'inside' the kMeans.
-- *heavily borrowing from https://github.com/NathanEpstein/clusters* --
kMeans(x ARRAY<FLOAT64>, -- ESR1 gene expression
y ARRAY<FLOAT64>, -- EGFR gene expression
iterations FLOAT64, -- the number of iterations

Training MNIST using Kubeflow, Minio, and Argo on Azure

This example guides you through the process of taking an example model, modifying it to run better within Kubeflow, and serving the resulting trained model. We will be using Argo to manage the workflow, Minio for saving model training info, Tensorboard to visualize the training, and Kubeflow to deploy the Tensorflow operator and serve the model.

Prerequisites

Before we get started there a few requirements.

Deploy an AKS Cluster

# Load data
# ips <- read.csv("https://lookaside.fbsbx.com/file/ipdatas.csv?token=AWyvZbsOT-DxjUjUySQf4vqWu9VSj7WTKytXNHkow4054oYjLFN_7FW0FPNwDtmUbQXXzJyCm99cQoRV6c-xS-Clpc7ss0lw-C04A41hBtFP3Yy2d4sR_W31BNrLip_XcqyuJbOxhjC9xKBq32jQ8zfxnDO3OdbSdq4-SbOnWnMQSLcjWrx6hfqbKo1a3gVttSZLhKqna7TpJhOm8uKdUvde")
ips <- read.csv("ipdatas.csv")
# 如果已安裝則省略
# install.packages("maps")
# install.packages("ggplot2")
library(ggplot2)
@suensummit
suensummit / moneydj_pdf_parser_example.py
Created March 2, 2017 09:42
Python PDF Parser example code.
import os, csv
from urllib2 import Request, urlopen
from StringIO import StringIO
from PyPDF2 import PdfFileReader
# Get the moneydj urls into a list.
with open('/money_url_list.csv', 'rb') as money_url_list:
reader = csv.reader(money_url_list, delimiter = ',')
moneydj_list = list(reader)
@suensummit
suensummit / meteor.cheatsheet.coffee
Last active August 29, 2015 14:27 — forked from LeCoupa/meteor.cheatsheet.coffee
Meteor Cheatsheet: Meteor Core, Publish and Subscribe, Method, Server Connections, Collections, Session, Accounts, Passwords, Templates, Match, Timers, Deps, EJSON, HTTP, Email, Assets, Command Line. More: http://journal.gentlenode.com/meteor-5-cheatsheet/
# METEOR CORE:
Anywhere: Meteor.isClient
Anywhere: Meteor.isServer
Anywhere: Meteor.startup(func)
Anywhere: Meteor.absoluteUrl([path], [options])
Anywhere: Meteor.settings
Anywhere: Meteor.release

##TUNING##

Configuration

System: set file descriptors to 32K or 64K

vim /etc/security/limit.conf