Skip to content

Instantly share code, notes, and snippets.

@vadimkantorov
Last active June 12, 2025 11:50
Show Gist options
  • Save vadimkantorov/3dd4073750b3649c7cce4b58b1b80e58 to your computer and use it in GitHub Desktop.
Save vadimkantorov/3dd4073750b3649c7cce4b58b1b80e58 to your computer and use it in GitHub Desktop.
Basic example of using fsspec explaining some quirks on porting from regular Python I/O
import sys, fsspec
with fsspec.open(sys.argv[1], 'rt') as f: # must pass 'rt' explicitly, as in fsspec the default mode is 'rb'
print(f.read()) # msut use context manager as in fsspec the result of fsspec.open(...) does not have method read()
# echo world > hello.txt
# python catfsspec.py hello.txt
# python catfsspec.py file://hello.txt
# python catfsspec.py s3://mybucket/hello.txt
import sys, fsspec
path = sys.argv[1]
listdir = fsspec.filesystem(path.split('://')[0] if '://' in path else 'file').ls # from os import listdir
print(listdir(path))
# python lsfsspec.py .
# python lsfsspec.py file://.
# python lsfsspec.py s3://mybucket/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment