Skip to content

Instantly share code, notes, and snippets.

@robinsingh1
Created April 30, 2016 16:31
Show Gist options
  • Save robinsingh1/5547bddc7f498210be0a1855a70321c3 to your computer and use it in GitHub Desktop.
Save robinsingh1/5547bddc7f498210be0a1855a70321c3 to your computer and use it in GitHub Desktop.
COMPLETED
Attempting to fetch counters from logs...
Looking for step log in s3://mrjob-a5c15e170f390339/tmp/logs/j-16XAEQ079TBFL/steps/s-90HW0KUHOI7X...
Parsing step log: s3://mrjob-a5c15e170f390339/tmp/logs/j-16XAEQ079TBFL/steps/s-90HW0KUHOI7X/syslog.gz
Counters: 35
File Input Format Counters
Bytes Read=2415
File Output Format Counters
Bytes Written=15058054
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=3616950
FILE: Number of large read operations=0
FILE: Number of read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=4935
HDFS: Number of bytes written=0
HDFS: Number of large read operations=0
HDFS: Number of read operations=35
HDFS: Number of write operations=0
S3: Number of bytes read=2415
S3: Number of bytes written=15058054
S3: Number of large read operations=0
S3: Number of read operations=0
S3: Number of write operations=0
Job Counters
Data-local map tasks=35
Launched map tasks=35
Total megabyte-seconds taken by all map tasks=21779046016
Total time spent by all map tasks (ms)=13088369
Total time spent by all maps in occupied slots (ms)=680595188
Total time spent by all reduces in occupied slots (ms)=0
Total vcore-seconds taken by all map tasks=13088369
Map-Reduce Framework
CPU time spent (ms)=12973780
Failed Shuffles=0
GC time elapsed (ms)=3793
Input split bytes=4935
Map input records=1
Map output records=43319
Merged Map outputs=0
Physical memory (bytes) snapshot=14318559232
Spilled Records=0
Total committed heap usage (bytes)=34896609280
Virtual memory (bytes) snapshot=78334918656
Removing s3 temp directory s3://mrjob-a5c15e170f390339/tmp/wappalyzer_mrjob.robin.20160429.170456.597284/...
Removing temp directory /var/folders/5c/plv6dbfs2nsfrn36q_mbhmch0000gn/T/wappalyzer_mrjob.robin.20160429.170456.597284...
Removing log files in s3://mrjob-a5c15e170f390339/tmp/logs/j-16XAEQ079TBFL/...
Terminating cluster: j-16XAEQ079TBFL
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment