Skip to content

Instantly share code, notes, and snippets.

@mplourde
Last active May 23, 2016 19:19
Show Gist options
  • Save mplourde/cd2f82738287d15e00ffd2d248c5561b to your computer and use it in GitHub Desktop.
Save mplourde/cd2f82738287d15e00ffd2d248c5561b to your computer and use it in GitHub Desktop.
SystemML diag behavior issue
/*
In this example I create two matrices in the same manner, extract the diagonal, and then turn the nx1
matrix of the diagonal values into a diagonal matrix by calling `diag` again. When the initial matrix contains all 1s,
it works as expected. However, when the initial matrix contains all zeros, the second `diag` does not convert the nx1
matrix of diagonal values to a diagonal matrix. I can work around this behavior, but I thought I should bring it to your attention.
*/
val systemml_alg =
"""
fileOut1 = ""
fileOut2 = ""
test_fun = function(double x) return(matrix[double] out) {
X = matrix(x, 10, 10)
out = diag(diag(X));
}
Out1 = test_fun(0)
Out2 = test_fun(1)
write(Out1, fileOut1)
write(Out2, fileOut2)
"""
import org.apache.sysml.api.MLContext
val ml = new MLContext(sc)
ml.setConfig("scratch", "/mnt/work/scratch")
ml.registerOutput("Out1")
ml.registerOutput("Out2")
val outputs = ml.executeScript(systemml_alg)
val Out1 = outputs.getDF(sqlContext, "Out1")
val Out2 = outputs.getDF(sqlContext, "Out2")
// OUTPUT:
// Out1: org.apache.spark.sql.DataFrame = [ID: double, C1: double]
// Out2: org.apache.spark.sql.DataFrame = [ID: double, C1: double, C2: double, C3: double, C4: double, C5: double, C6: double, C7: double, C8: double, C9: double, C10: double]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment