This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<html> | |
<body> | |
<div style="background-color: #cccccc"> | |
<!-- | |
The <p> element is a block element. White space around it is effectively ignored. | |
Block elements are laid out in the direction of the document, in this case that | |
is top to bottom. | |
--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
~ 17:47:44 | |
❯ alias mp=multipass | |
~ 17:47:47 | |
❯ mp launch -n lxd -c 2 -m 4G -d 40G | |
Launched: lxd | |
~ 21s 17:48:15 | |
❯ mp shell lxd | |
Welcome to Ubuntu 20.04.4 LTS (GNU/Linux 5.4.0-120-generic aarch64) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from boto3 import Session | |
from botocore.credentials import RefreshableCredentials | |
from botocore.session import get_session | |
class AutoRefreshSessionFactory: | |
"""A Factory that creates boto3.Session sessions that auto refresh their credentials. | |
Primarily intended for assuming a role, since assume role credentials expire. | |
""" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# with 0 initial worker pods, the resource demand jumps to 200+ pending | |
# but once a worker pod is connected, the resource demand goes to 10+ pending | |
# see lines 225, 332 which have 200+ pending | |
# then line 1019 which has 10+ pending | |
ray-cluster,default:2022-02-09 15:21:55,711 INFO monitor.py:521 -- Logging raw resource message pulled from GCS. | |
ray-cluster,default:2022-02-09 15:21:55,711 INFO monitor.py:522 -- batch { | |
node_id: "\215\336\277\207<\'\201.[\323\220Ul6\026ba\212\353@\322\202|\212\321\005;0" | |
resources_available { | |
key: "memory" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import ray | |
import time | |
import logging | |
ray.init(address="auto", logging_level=logging.DEBUG) | |
remote_args = { | |
"num_cpus": 1, | |
"num_gpus": None, | |
"max_calls": 0, | |
"max_retries": 3, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import ray | |
import time | |
import logging | |
ray.init(address="auto", logging_level=logging.DEBUG) | |
remote_args = { | |
"num_cpus": 1, | |
"num_gpus": None, | |
"max_calls": 0, | |
"max_retries": 3, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import logging | |
import ray | |
@ray.remote | |
def task(): | |
import time | |
time.sleep(30) | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
======== Autoscaler status: 2022-02-08 09:02:10.177916 ======== | |
Node status | |
--------------------------------------------------------------- | |
Healthy: | |
1 head | |
1 wkr-15cpu30g-ondemand | |
Pending: | |
(no pending nodes) | |
Recent failures: | |
(no failures) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from pprint import pprint | |
import ray | |
ray.init("ray://mycluster.internal:10001") | |
@ray.remote | |
def task(): | |
import time | |
time.sleep(30) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
data "aws_ami" "aws_eks_gpu_optimized" { | |
owners = ["602401143452"] | |
most_recent = true | |
filter { | |
name = "name" | |
values = ["amazon-eks-gpu-node-${var.eks_cluster_version}-v20220112*"] | |
} | |
} |
NewerOlder