Skip to content

Instantly share code, notes, and snippets.

@betolink

betolink/cloud-optimized-hdf5.ipynb

Created January 30, 2024 20:33
Show Gist options
  • Save betolink/b545c364f80882c113b8cc27b763c729 to your computer and use it in GitHub Desktop.
Save betolink/b545c364f80882c113b8cc27b763c729 to your computer and use it in GitHub Desktop.
Download ZIP
cloud optimized HDF5
Display the source blob
Display the rendered blob
Raw
cloud-optimized-hdf5.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@forrestfwilliams
Copy link

I care! Great job guys!

@betolink
Copy link
Author

betolink commented Jan 30, 2024 •
edited
Loading

Forrest!! yeah haha we do care! next up the cost savings factor of cloud optimized HDF5

@alex-s-gardner
Copy link

someday there will be a parade named in your honor !! Well done

@evetion
Copy link

evetion commented Jan 31, 2024

Very interesting work, that will save many many bytes of bandwith and I/O in the long run.

@betolink
Copy link
Author

Yeah, I'm excited about this, thanks to @ajelenak for his guidance on HDF5 internals. Besides the performance improvement, a good side effect for data providers (with data on AWS) is the co$$$t reduction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment