-
-
Save stephanie-wang/482e4881a38383382126d815da44f410 to your computer and use it in GitHub Desktop.
threaded_actors_stress_test output
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:20:37,500 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(23799) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.052433013916016 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:21:07,502 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(23895) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 15.850414276123047 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:21:37,504 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(23993) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 15.887882232666016 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:22:07,505 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(24091) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 15.984310150146484 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:22:37,507 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(24187) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.017303466796875 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:23:07,509 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(24292) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.07272720336914 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.85GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:23:37,510 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(24388) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.07785415649414 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:24:07,512 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(24486) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.079559326171875 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:24:37,513 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(24584) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.04279327392578 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.84GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:25:07,515 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(24680) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.071792602539062 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:25:37,516 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(24780) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.075748443603516 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:26:07,518 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(24876) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.075435638427734 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:26:37,520 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(24974) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.077392578125 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.86GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:27:07,521 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(25072) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.07946014404297 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:27:37,523 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(25168) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.077632904052734 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:28:07,524 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(25268) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.08050537109375 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:28:37,526 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(25364) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.050262451171875 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.85GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:29:07,527 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(25462) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.083011627197266 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:29:37,529 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(25560) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.084308624267578 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:30:07,531 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(25656) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.093425750732422 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:30:37,532 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(25756) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.085391998291016 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:31:07,534 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(25852) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.05449676513672 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.84GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:31:37,535 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(25950) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.09749984741211 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:32:07,537 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(26048) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.082996368408203 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.86GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:32:37,539 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(26144) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.12149429321289 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:33:07,540 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(26247) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.09981918334961 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:33:37,542 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(26345) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.10198211669922 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:34:07,543 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(26441) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.103790283203125 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:34:37,545 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(26541) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.10919189453125 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:35:07,546 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(26637) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.133155822753906 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:35:37,548 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(26735) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.11235809326172 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:36:07,549 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(26833) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.110248565673828 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:36:37,551 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(26929) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.113914489746094 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:37:07,553 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(27027) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.10997772216797 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:37:37,554 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(27125) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.131778717041016 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:38:07,557 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(27221) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.169384002685547 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:38:37,559 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(27321) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.16646957397461 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:39:07,561 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(27417) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.169109344482422 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:39:37,562 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(27515) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.172103881835938 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:40:07,564 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(27613) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.192371368408203 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:40:37,565 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(27709) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.172271728515625 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:41:07,567 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(27807) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.173202514648438 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:41:37,568 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(27905) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.176219940185547 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.89GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:42:07,570 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28001) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.152427673339844 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.87GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:42:37,572 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28099) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.13947296142578 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.85GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:43:07,573 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28202) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.164196014404297 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:43:37,576 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28300) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.18735122680664 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.9GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:44:07,577 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28398) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.16394805908203 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.88GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:44:37,579 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28494) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.18984603881836 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.9GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:45:07,580 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28592) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.196205139160156 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.9GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:45:37,582 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28690) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.195392608642578 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.91GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:46:07,584 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28786) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.205604553222656 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.91GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:46:37,585 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28884) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.197139739990234 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.91GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:47:07,587 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(28982) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.198699951171875 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.91GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:47:37,588 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(29080) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.20287322998047 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.91GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:48:07,590 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(29178) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.199195861816406 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.91GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:48:37,591 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(29274) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.216968536376953 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.91GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:49:07,593 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(29372) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.202835083007812 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.91GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:49:37,595 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(29470) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.205299377441406 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.92GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:50:07,596 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(29566) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.20610809326172 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.92GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:50:37,598 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(29664) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.206871032714844 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.92GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:51:07,600 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(29762) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 15.933712005615234 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.92GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:51:37,601 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(29858) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.0045166015625 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.92GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:52:07,603 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(29958) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.101299285888672 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.92GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:52:37,604 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(30054) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.133262634277344 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.9GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:53:07,606 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(30157) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.131633758544922 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.9GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:53:37,608 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(30255) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.155288696289062 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.92GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:54:07,609 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(30351) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.159778594970703 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.93GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:54:37,611 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(30449) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.158512115478516 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.93GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:55:07,612 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(30547) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.16225814819336 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.93GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1328 0.06GiB ray::MemoryMonitorActor.run() | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9046 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 9012 0.06GiB ray::PiCalculator | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 347 0.05GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/jupyter-notebook --NotebookApp.token=2b61de3a | |
| [2m[33m(raylet)[0m [2022-03-14 17:55:37,614 E 206 206] (raylet) worker_pool.cc:508: Some workers of the worker process(30643) have not registered within the timeout. The process is still alive, probably it's hanging during start. | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Memory usage: 16.15865707397461 / 124.3635368347168 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:Top 10 process memory usage: | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m INFO:root:PID MEM COMMAND | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 148 10.8GiB /home/ray/anaconda3/lib/python3.7/site-packages/ray/core/src/ray/gcs/gcs_server --log_dir=/tmp/ray/s | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 1252 1.24GiB python stress_tests/test_threaded_actors.py --test-runtime 3600 --kill-interval_s 60 | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 173 0.93GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/dash | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 250 0.12GiB /home/ray/anaconda3/bin/python -u /home/ray/anaconda3/lib/python3.7/site-packages/ray/dashboard/agen | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 64 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session web_terminal_server --deploy | |
| [2m[36m(MemoryMonitorActor pid=1328)[0m 384 0.09GiB /home/ray/anaconda3/bin/python /home/ray/anaconda3/bin/anyscale session auth_start |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment