Skip to content

Instantly share code, notes, and snippets.

@brycehuang30

brycehuang30/gist:b514b7248fe73a0e752696310a000abe

Created August 10, 2023 21:40
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save brycehuang30/b514b7248fe73a0e752696310a000abe to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save brycehuang30/b514b7248fe73a0e752696310a000abe to your computer and use it in GitHub Desktop.
Download ZIP
dashboard_agent.log
Raw
gistfile1.txt
2023-08-10 21:20:03,968 INFO agent.py:116 -- Parent pid is 387
2023-08-10 21:20:03,969 INFO agent.py:142 -- Dashboard agent grpc address: 0.0.0.0:6823
2023-08-10 21:20:03,971 INFO utils.py:112 -- Get all modules by type: DashboardAgentModule
2023-08-10 21:20:04,132 INFO utils.py:145 -- Available modules: [<class 'ray.dashboard.modules.event.event_agent.EventAgent'>, <class 'ray.dashboard.modules.healthz.healthz_agent.HealthzAgent'>, <class 'ray.dashboard.modules.job.job_agent.JobAgent'>, <class 'ray.dashboard.modules.log.log_agent.LogAgent'>, <class 'ray.dashboard.modules.log.log_agent.LogAgentV1Grpc'>, <class 'ray.dashboard.modules.reporter.reporter_agent.ReporterAgent'>, <class 'ray.dashboard.modules.runtime_env.runtime_env_agent.RuntimeEnvAgent'>, <class 'ray.dashboard.modules.serve.serve_agent.ServeAgent'>]
2023-08-10 21:20:04,133 INFO agent.py:171 -- Loading DashboardAgentModule: <class 'ray.dashboard.modules.event.event_agent.EventAgent'>
2023-08-10 21:20:04,133 INFO event_agent.py:38 -- Event agent cache buffer size: 10240
2023-08-10 21:20:04,133 INFO agent.py:171 -- Loading DashboardAgentModule: <class 'ray.dashboard.modules.healthz.healthz_agent.HealthzAgent'>
2023-08-10 21:20:04,133 INFO agent.py:171 -- Loading DashboardAgentModule: <class 'ray.dashboard.modules.job.job_agent.JobAgent'>
2023-08-10 21:20:04,133 INFO agent.py:171 -- Loading DashboardAgentModule: <class 'ray.dashboard.modules.log.log_agent.LogAgent'>
2023-08-10 21:20:04,137 INFO agent.py:171 -- Loading DashboardAgentModule: <class 'ray.dashboard.modules.log.log_agent.LogAgentV1Grpc'>
2023-08-10 21:20:04,137 INFO agent.py:171 -- Loading DashboardAgentModule: <class 'ray.dashboard.modules.reporter.reporter_agent.ReporterAgent'>
2023-08-10 21:20:04,137 WARNING utils.py:625 -- Ray currently does not support initializing Raywith fractional cpus. Your num_cpus will be truncated from 7.2 to 7.
2023-08-10 21:20:04,139 INFO agent.py:171 -- Loading DashboardAgentModule: <class 'ray.dashboard.modules.runtime_env.runtime_env_agent.RuntimeEnvAgent'>
2023-08-10 21:20:04,141 INFO agent.py:171 -- Loading DashboardAgentModule: <class 'ray.dashboard.modules.serve.serve_agent.ServeAgent'>
2023-08-10 21:20:04,141 INFO agent.py:176 -- Loaded 8 modules.
2023-08-10 21:20:04,144 INFO http_server_agent.py:74 -- Dashboard agent http address: 0.0.0.0:6822
2023-08-10 21:20:04,144 INFO http_server_agent.py:81 -- <ResourceRoute [GET] <PlainResource /api/local_raylet_healthz> -> <function HealthzAgent.health_check at 0x7fa88962add0>
2023-08-10 21:20:04,144 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <PlainResource /api/local_raylet_healthz> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,144 INFO http_server_agent.py:81 -- <ResourceRoute [POST] <PlainResource /api/job_agent/jobs/> -> <function JobAgent.submit_job at 0x7fa888f370a0>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <PlainResource /api/job_agent/jobs/> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [POST] <DynamicResource /api/job_agent/jobs/{job_or_submission_id}/stop> -> <function JobAgent.stop_job at 0x7fa888f37250>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <DynamicResource /api/job_agent/jobs/{job_or_submission_id}/stop> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [DELETE] <DynamicResource /api/job_agent/jobs/{job_or_submission_id}> -> <function JobAgent.delete_job at 0x7fa888f37400>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <DynamicResource /api/job_agent/jobs/{job_or_submission_id}> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [GET] <DynamicResource /api/job_agent/jobs/{job_or_submission_id}/logs> -> <function JobAgent.get_job_logs at 0x7fa888f375b0>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <DynamicResource /api/job_agent/jobs/{job_or_submission_id}/logs> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [GET] <DynamicResource /api/job_agent/jobs/{job_or_submission_id}/logs/tail> -> <function JobAgent.tail_job_logs at 0x7fa888f37760>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <DynamicResource /api/job_agent/jobs/{job_or_submission_id}/logs/tail> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [GET] <PlainResource /api/ray/version> -> <function ServeAgent.get_version at 0x7fa888b82d40>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <PlainResource /api/ray/version> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [GET] <PlainResource /api/serve/deployments/> -> <function ServeAgent.get_all_deployments at 0x7fa888b82dd0>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <PlainResource /api/serve/deployments/> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [GET] <PlainResource /api/serve/applications/> -> <function ServeAgent.get_serve_instance_details at 0x7fa888b82f80>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <PlainResource /api/serve/applications/> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [GET] <PlainResource /api/serve/deployments/status> -> <function ServeAgent.get_all_deployment_statuses at 0x7fa888b83130>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <PlainResource /api/serve/deployments/status> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [DELETE] <PlainResource /api/serve/deployments/> -> <function ServeAgent.delete_serve_application at 0x7fa888b832e0>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <PlainResource /api/serve/deployments/> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [DELETE] <PlainResource /api/serve/applications/> -> <function ServeAgent.delete_serve_applications at 0x7fa888b83490>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <PlainResource /api/serve/applications/> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [PUT] <PlainResource /api/serve/deployments/> -> <function ServeAgent.put_all_deployments at 0x7fa888b83640>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <PlainResource /api/serve/deployments/> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [PUT] <PlainResource /api/serve/applications/> -> <function ServeAgent.put_all_applications at 0x7fa888b837f0>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <PlainResource /api/serve/applications/> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [GET] <StaticResource /logs -> PosixPath('/tmp/ray/session_2023-08-10_21-20-01_701629_100/logs')> -> <bound method StaticResource._handle of <StaticResource /logs -> PosixPath('/tmp/ray/session_2023-08-10_21-20-01_701629_100/logs')>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:81 -- <ResourceRoute [OPTIONS] <StaticResource /logs -> PosixPath('/tmp/ray/session_2023-08-10_21-20-01_701629_100/logs')> -> <bound method _PreflightHandler._preflight_handler of <aiohttp_cors.cors_config._CorsConfigImpl object at 0x7fa888c23310>>
2023-08-10 21:20:04,145 INFO http_server_agent.py:82 -- Registered 30 routes.
2023-08-10 21:20:04,149 INFO event_agent.py:56 -- Report events to 10.0.32.108:33923
2023-08-10 21:20:04,150 INFO event_utils.py:132 -- Monitor events logs modified after 1691700604.0643125 on /tmp/ray/session_2023-08-10_21-20-01_701629_100/logs/events, the source types are all.
2023-08-10 21:20:08,165 INFO event_utils.py:170 -- Found new event log file: /tmp/ray/session_2023-08-10_21-20-01_701629_100/logs/events/event_AUTOSCALER.log
2023-08-10 21:20:22,276 INFO optional_utils.py:264 -- Connecting to ray with address=10.0.32.108:6379
2023-08-10 21:20:22,290 INFO snapshot_util.py:1045 -- Updated runtime env to {'cgroupv2': {}}
2023-08-10 21:20:22,290 INFO worker.py:1432 -- Connecting to existing Ray cluster at address: 10.0.32.108:6379...
2023-08-10 21:20:22,296 INFO worker.py:1616 -- Connected to Ray cluster. View the dashboard at https://session-cq28n9mv9w9df5lz3lic8dxj37.i.anyscale-test.com 
2023-08-10 21:20:22,311 INFO job_manager.py:891 -- Starting job with submission_id: raysubmit_8QmNX5iAcARJv6xp
2023-08-10 21:20:22,313 INFO job_manager.py:826 -- Head node ID found in GCS; scheduling job driver on head node fc3110ea7d5d73b6276c2f299990bc4772f1a65c333c5e41a385ee49
2023-08-10 21:20:22,332 INFO web_log.py:206 -- 10.0.32.108 [10/Aug/2023:21:20:22 +0000] "POST /api/job_agent/jobs/ HTTP/1.1" 200 245 "-" "Python/3.10 aiohttp/3.8.4"
2023-08-10 21:20:22,440 INFO credentials.py:1051 -- Found credentials from IAM Role: cld_i2w99rzq8b6lbjkke9y94vi5-cluster_node_role
2023-08-10 21:20:24,179 INFO event_utils.py:170 -- Found new event log file: /tmp/ray/session_2023-08-10_21-20-01_701629_100/logs/events/event_JOBS.log
2023-08-10 21:21:25,081 INFO web_log.py:206 -- 10.0.32.108 [10/Aug/2023:21:21:25 +0000] "GET /logs HTTP/1.1" 200 2245 "https://session-cq28n9mv9w9df5lz3lic8dxj37.i.anyscale-test.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36"
2023-08-10 21:21:26,918 INFO web_log.py:206 -- 10.0.32.108 [10/Aug/2023:21:21:26 +0000] "GET /logs/dashboard_agent.log HTTP/1.1" 200 239 "https://session-cq28n9mv9w9df5lz3lic8dxj37.i.anyscale-test.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36"
2023-08-10 21:33:43,632 INFO web_log.py:206 -- 10.0.32.108 [10/Aug/2023:21:33:43 +0000] "GET /logs/dashboard_agent.log HTTP/1.1" 200 239 "https://session-cq28n9mv9w9df5lz3lic8dxj37.i.anyscale-test.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36"
2023-08-10 21:40:22,304 INFO web_log.py:206 -- 10.0.32.108 [10/Aug/2023:21:40:22 +0000] "GET /logs/dashboard_agent.log HTTP/1.1" 200 239 "https://session-cq28n9mv9w9df5lz3lic8dxj37.i.anyscale-test.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment