Passing Data with VisiData Remote Control

README.md

Passing Data with VisiData Remote Control

This is a neat discussion thread where a core goal is to send data to an existing VisiData session.

This sample explores one way to do that, combining:

• A remote control plugin
• Ray's shared memory object store

Demo at:

vd_remote_open.py

import socket
from pathlib import Path

import pandas as pd
import ray

# Start Ray and take note of its starting address
ray.init()
ray_address = ray.runtime_context.get_runtime_context().gcs_address

# Store a Pandas DataFrame in Ray's in-memory object store,
# and record a reference that we can unwrap later
df = pd.read_csv("~/code/visidata/sample_data/benchmark.csv")
print("Loaded DataFrame with head:")
print(df.head())
serialized_ref = ray.cloudpickle.dumps(ray.put(df))

# Connect to a VisiData remote control socket. Assumes
# VisiData has been started with:
#
# vd server://moo
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
    print("Opening DataFrame in running VisiData instance")
    s.connect(str(Path("~/.visidata/run/moo").expanduser()))

    # Send a remote control command that:
    # - Connects to a local Ray instance
    # - Fetches a DataFrame from Ray's shared memory object store
    # - Opens a new Pandas sheet with that DataFrame as the source
    s.send(
        f"""
import ray

ray.is_initialized() or ray.init('{ray.runtime_context.get_runtime_context().gcs_address}')
df = ray.get(ray.cloudpickle.loads({serialized_ref}))
vd.push(visidata.PandasSheet('', source=df))
ray.shutdown()
        """.encode(
            "utf8"
        ),
    )
    print(f"Remote command exit code: {s.recv(1024).decode('utf8')}")