Skip to content

Instantly share code, notes, and snippets.

@chathurawidanage

chathurawidanage/cylon_slice.py

Created June 1, 2021 01:25
Show Gist options
  • Save chathurawidanage/fec123da7e0ea1f48b70ed747317f1d8 to your computer and use it in GitHub Desktop.
Save chathurawidanage/fec123da7e0ea1f48b70ed747317f1d8 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
cylon_slice.py
from pycylon import DataFrame, read_csv, CylonEnv
from pycylon.net import MPIConfig
env = CylonEnv(config=MPIConfig())
print("Hello from Worker : ", env.rank)
df1 = read_csv("file1.csv", slice=True)
df1.set_index([0])
df2 = read_csv("file2.csv", slice=True)
df2.set_index([0])
join = df1.merge(df2, left_on=[0], right_on=[0], env=env)
print(join)
env.finalize()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment