Last active
June 12, 2025 11:50
-
-
Save vadimkantorov/3dd4073750b3649c7cce4b58b1b80e58 to your computer and use it in GitHub Desktop.
Basic example of using fsspec explaining some quirks on porting from regular Python I/O
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import sys, fsspec | |
with fsspec.open(sys.argv[1], 'rt') as f: # must pass 'rt' explicitly, as in fsspec the default mode is 'rb' | |
print(f.read()) # msut use context manager as in fsspec the result of fsspec.open(...) does not have method read() | |
# echo world > hello.txt | |
# python catfsspec.py hello.txt | |
# python catfsspec.py file://hello.txt | |
# python catfsspec.py s3://mybucket/hello.txt |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import sys, fsspec | |
path = sys.argv[1] | |
listdir = fsspec.filesystem(path.split('://')[0] if '://' in path else 'file').ls # from os import listdir | |
print(listdir(path)) | |
# python lsfsspec.py . | |
# python lsfsspec.py file://. | |
# python lsfsspec.py s3://mybucket/ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment