Skip to content

Instantly share code, notes, and snippets.

@simon-mo

simon-mo/deploy_pytorch.py

Created September 17, 2020 21:06
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save simon-mo/ba00b0cfcd968aa9b556a7dadfb00cb4 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save simon-mo/ba00b0cfcd968aa9b556a7dadfb00cb4 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
deploy_pytorch.py
import ray
from ray import serve
ray.init(address="auto", ignore_reinit_error=True)
client = serve.connect()
client.create_backend("pytorch_backend", PyTorchBackend)
client.set_traffic("sentiment_endpoint", {"pytorch_backend": 1.0})
Raw
deploy_sklearn.py
import ray
from ray import serve
# Connect to the running Ray Serve instance.
ray.init(address='auto', ignore_reinit_error=True)
client = serve.connect()
# Deploy the model.
client.create_backend("sklearn_backend", SKLearnBackend)
client.create_endpoint("sentiment_endpoint", backend="sklearn_backend", route="/sentiment")
Raw
start_serve.py
import ray
from ray import serve
ray.init(address='auto') # Connect to the running Ray cluster.
serve.start(detached=True) # Start the Ray Serve processes within the Ray cluster.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment