Created
July 9, 2017 11:46
-
-
Save fpopic/62881238ba1657cc9af61a23ebdbbbfd to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| hdfs dfs -copyFromLocal /home/rovkp/fpopic/movielens-ratings-small.csv.formated.indexed /user/rovkp/fpopic/movielens-ratings-small.csv.formated.indexed | |
| hdfs dfs -copyFromLocal /home/rovkp/fpopic/abcd.csv.indexed /user/rovkp/fpopic/abcd.csv.indexed | |
| spark2-submit --class hr.fer.ztel.thesis.Main --master yarn --deploy-mode cluster --driver-memory 5530m --executor-memory 5530m --conf spark.dynamicAllocation.maxExecutors=28 --conf spark.executor.heartbeatInterval=20s --conf spark.network.timeout=240s --conf spark.locality.wait=6s --conf spark.yarn.maxAppAttempts=1 /home/rovkp/fpopic/spark-recommender-assembly-1.0.jar blocks hdfs:///user/rovkp/fpopic/ movielens-ratings-small.csv.formated.indexed abcd.csv.indexed llr true movielens_llr_true_2048 5 2048 | |
| /user/rovkp/fpopic/movielens-ratings-small.csv.formated.indexed | |
| /user/rovkp/fpopic/movielens-ratings-small.csv.formated.lookup.size | |
| #71 blocks | |
| /user/rovkp/fpopic/movielens_llr_true_2048_2017-07-06T22-23-45-871Z | |
| #72 inner | |
| slika ne moze | |
| #73 outer obicni | |
| stane nakon broadcasta | |
| #74 33min (bez partitionera) | |
| outer-entry | |
| #76 s partitionerom 32min (skoro isto) | |
| #82 s partitionerom i 3 corea i 10% fractionom 30min | |
| agrokor | |
| 2048 18min 13.4 GB | |
| 1024 23min 24.8 GB |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment