Last active
February 13, 2019 18:49
-
-
Save vsuthichai/21c2f02b9ee2f791a021cfe2bde40ffa to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [0213 03:07:48 @base.py:274] Start Epoch 1 ... | |
| [0213 04:06:00 @base.py:284] Epoch 1 (global_step 15000) finished, time:58 minutes 11 seconds. | |
| [0213 04:06:00 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 34.37 | |
| [0213 04:06:00 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,491.31 | |
| [0213 04:06:00 @monitor.py:467] Throughput/EpochWallClockDuration: 3491.3 | |
| [0213 04:06:00 @monitor.py:467] Throughput/MeanEpochThroughput: 34.371 | |
| [0213 04:06:00 @base.py:274] Start Epoch 2 ... | |
| [0213 04:55:57 @base.py:284] Epoch 2 (global_step 30000) finished, time:49 minutes 56 seconds. | |
| [0213 04:55:57 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 40.04 | |
| [0213 04:55:57 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 2,996.91 | |
| [0213 04:55:58 @monitor.py:467] Throughput/EpochWallClockDuration: 2996.9 | |
| [0213 04:55:58 @monitor.py:467] Throughput/MeanEpochThroughput: 40.041 | |
| [0213 04:55:58 @base.py:274] Start Epoch 3 ... | |
| [0213 05:46:57 @base.py:284] Epoch 3 (global_step 45000) finished, time:50 minutes 59 seconds. | |
| [0213 05:46:58 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 39.22 | |
| [0213 05:46:58 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,059.72 | |
| [0213 05:46:58 @monitor.py:467] Throughput/EpochWallClockDuration: 3059.7 | |
| [0213 05:46:58 @monitor.py:467] Throughput/MeanEpochThroughput: 39.219 | |
| [0213 05:46:58 @base.py:274] Start Epoch 4 ... | |
| [0213 06:39:15 @base.py:284] Epoch 4 (global_step 60000) finished, time:52 minutes 16 seconds. | |
| [0213 06:39:15 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 38.26 | |
| [0213 06:39:15 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,136.81 | |
| [0213 06:39:16 @monitor.py:467] Throughput/EpochWallClockDuration: 3136.8 | |
| [0213 06:39:16 @monitor.py:467] Throughput/MeanEpochThroughput: 38.255 | |
| [0213 06:39:16 @base.py:274] Start Epoch 5 ... | |
| [0213 07:32:48 @base.py:284] Epoch 5 (global_step 75000) finished, time:53 minutes 32 seconds. | |
| [0213 07:32:49 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 37.36 | |
| [0213 07:32:49 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,212.32 | |
| [0213 07:32:49 @monitor.py:467] Throughput/EpochWallClockDuration: 3212.3 | |
| [0213 07:32:49 @monitor.py:467] Throughput/MeanEpochThroughput: 37.356 | |
| [0213 07:32:49 @base.py:274] Start Epoch 6 ... | |
| [0213 08:27:41 @base.py:284] Epoch 6 (global_step 90000) finished, time:54 minutes 51 seconds. | |
| [0213 08:27:41 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 36.46 | |
| [0213 08:27:41 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,291.45 | |
| [0213 08:27:42 @monitor.py:467] Throughput/EpochWallClockDuration: 3291.4 | |
| [0213 08:27:42 @monitor.py:467] Throughput/MeanEpochThroughput: 36.458 | |
| [0213 08:27:42 @base.py:274] Start Epoch 7 ... | |
| [0213 09:23:58 @base.py:284] Epoch 7 (global_step 105000) finished, time:56 minutes 15 seconds. | |
| [0213 09:23:58 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 35.55 | |
| [0213 09:23:58 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,375.79 | |
| [0213 09:23:59 @monitor.py:467] Throughput/EpochWallClockDuration: 3375.8 | |
| [0213 09:23:59 @monitor.py:467] Throughput/MeanEpochThroughput: 35.547 | |
| [0213 09:23:59 @base.py:274] Start Epoch 8 ... | |
| [0213 10:21:24 @base.py:284] Epoch 8 (global_step 120000) finished, time:57 minutes 24 seconds. | |
| [0213 10:21:24 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 34.84 | |
| [0213 10:21:24 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,444.69 | |
| [0213 10:21:25 @monitor.py:467] Throughput/EpochWallClockDuration: 3444.7 | |
| [0213 10:21:25 @monitor.py:467] Throughput/MeanEpochThroughput: 34.836 | |
| [0213 10:21:25 @base.py:274] Start Epoch 9 ... | |
| [0213 11:20:08 @base.py:284] Epoch 9 (global_step 135000) finished, time:58 minutes 42 seconds. | |
| [0213 11:20:08 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 34.07 | |
| [0213 11:20:08 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,522.38 | |
| [0213 11:20:09 @monitor.py:467] Throughput/EpochWallClockDuration: 3522.4 | |
| [0213 11:20:09 @monitor.py:467] Throughput/MeanEpochThroughput: 34.068 | |
| [0213 11:20:09 @base.py:274] Start Epoch 10 ... | |
| [0213 12:20:06 @base.py:284] Epoch 10 (global_step 150000) finished, time:59 minutes 56 seconds. | |
| [0213 12:20:07 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 33.36 | |
| [0213 12:20:07 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,596.82 | |
| [0213 12:20:08 @monitor.py:467] Throughput/EpochWallClockDuration: 3596.8 | |
| [0213 12:20:08 @monitor.py:467] Throughput/MeanEpochThroughput: 33.363 | |
| [0213 12:20:08 @base.py:274] Start Epoch 11 ... | |
| [0213 13:21:20 @base.py:284] Epoch 11 (global_step 165000) finished, time:1 hour 1 minute 11 seconds. | |
| [0213 13:21:20 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 32.68 | |
| [0213 13:21:20 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,671.61 | |
| [0213 13:21:21 @monitor.py:467] Throughput/EpochWallClockDuration: 3671.6 | |
| [0213 13:21:21 @monitor.py:467] Throughput/MeanEpochThroughput: 32.683 | |
| [0213 13:21:21 @base.py:274] Start Epoch 12 ... | |
| [0213 14:23:37 @base.py:284] Epoch 12 (global_step 180000) finished, time:1 hour 2 minutes 15 seconds. | |
| [0213 14:26:27 @performance.py:71] [ThroughputTracker] Over last epoch, MeanEpochThroughput: 32.12 | |
| [0213 14:26:27 @performance.py:72] [ThroughputTracker] Over last epoch, EpochWallClockDuration: 3,735.80 | |
| [0213 14:26:29 @monitor.py:467] Throughput/EpochWallClockDuration: 3735.8 | |
| [0213 14:26:29 @monitor.py:467] Throughput/MeanEpochThroughput: 32.122 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment