geofferyzh
geofferyzh
/ PYMK Stage1 Mapper
Created
August 27, 2012 14:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import java.io.IOException; | |
| import org.apache.hadoop.io.LongWritable; | |
| import org.apache.hadoop.io.Text; | |
| import org.apache.hadoop.mapred.MapReduceBase; | |
| import org.apache.hadoop.mapred.Mapper; | |
| import org.apache.hadoop.mapred.OutputCollector; | |
| import org.apache.hadoop.mapred.Reporter; | |
| public class pymk_mapper extends MapReduceBase implements | |
| Mapper<LongWritable, Text, Text, Text> { |
geofferyzh
/ PYMK Stage1 Driver Code
Created
August 27, 2012 14:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import org.apache.hadoop.fs.Path; | |
| import org.apache.hadoop.io.Text; | |
| import org.apache.hadoop.mapred.FileInputFormat; | |
| import org.apache.hadoop.mapred.FileOutputFormat; | |
| import org.apache.hadoop.mapred.JobClient; | |
| import org.apache.hadoop.mapred.JobConf; | |
| import org.apache.hadoop.conf.Configured; | |
| import org.apache.hadoop.util.Tool; | |
| import org.apache.hadoop.util.ToolRunner; |
geofferyzh
/ gist:2777245
Created
May 23, 2012 19:28
RinAction - Nonparametric Test of Group Difference
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ######################################################## | |
| ######################################################## | |
| # ---- Nonparametric tests of group differences ---- # | |
| ######################################################## | |
| ######################################################## | |
| # NOTE: | |
| # When you have more than 2 groups, you can use ANOVA or Nonparametric approaches | |
| # to test group difference. ANOVA assumes data are independently sampled from normal | |
| # populations. If the parametric assumptions are not met, nonparametric method can |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ######################################################## | |
| ######################################################## | |
| # -------------------- T-Test ----------------- # | |
| ######################################################## | |
| ######################################################## | |
| # Group comparisons, assuming continuous outcome variable and normal distribution | |
| ##################### | |
| ## independent t-test |
geofferyzh
/ gist:2777234
Created
May 23, 2012 19:26
RinAction - Basic Statistics - Association & Correlations
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ######################################################## | |
| ######################################################## | |
| # -------- Test of independence & Association -------- # | |
| ######################################################## | |
| ######################################################## | |
| ########################## | |
| # Evidence of Independence | |
| ########################## |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ###################################################### | |
| # converting a table into a flat file via table2flat | |
| ###################################################### | |
| table2flat <- function(mytable) { | |
| df <- as.data.frame(mytable) | |
| rows <- dim(df)[1] | |
| cols <- dim(df)[2] | |
| x <- NULL | |
| for (i in 1:rows) { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #-----------------------------------------------------------------------------# | |
| #-----------------------------------------------------------------------------# | |
| # R in Action - Basic Statistics | |
| # - Frequency Table | |
| #-----------------------------------------------------------------------------# | |
| #-----------------------------------------------------------------------------# | |
| install.packages(c('npmc', 'ggm', 'gmodels', 'vcd', 'Hmisc','pastecs', 'psych', 'doBy', 'reshape')) | |
| ####################################################### |
geofferyzh
/ gist:2776306
Created
May 23, 2012 16:48
RinAction - Basic Statistics & Grouped Statistics
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #-----------------------------------------------------------------------------# | |
| #-----------------------------------------------------------------------------# | |
| # R in Action - Basic Statistics | |
| # - Descriptive Statistics | |
| #-----------------------------------------------------------------------------# | |
| #-----------------------------------------------------------------------------# | |
| install.packages(c('npmc', 'ggm', 'gmodels', 'vcd', 'Hmisc','pastecs', 'psych', 'doBy', 'reshape')) | |
| ############################################## |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ######################################################################### | |
| #-----------------------------------------------------------------------# | |
| # SlopeOne Recommender Implementation in R # | |
| ######################################################################### | |
| ##################### | |
| # Sample Data | |
| ##################### | |
| # sample data 1 |
geofferyzh
/ gist:2494809
Created
April 26, 2012 00:39
Similarity Metrics Calculation in R (LLR,Correlation,Vector,Tanimoto)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ########################################################################## | |
| # -----------------------------------------------------------------------# | |
| # -------------------- Similarity Metrics (CF) --------------------------# | |
| # ---------------------Author: Shaohua Zhang ---------------------------# | |
| ########################################################################## | |
| # sample data 1 | |
| Mov1 <- c(4,4,3,4,2) | |
| Mov2 <- c(NA,2,NA,4,1) |