You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
MinIO AIStor has two independent communication layers, each with an HTTP and RDMA mode. They are controlled separately and can be mixed in any combination.
MinIO AIStor vs Dell AI Data Platform: A Hardware-Level Comparison
In response to Michael Dell's LinkedIn post claiming: "12x faster vector indexing, 3x faster processing with Lightning FS (the fastest parallel file system in the world), feeding GPUs at 150 GB/s per rack."
Let's look at the actual hardware behind both platforms and compare published numbers.
The Elephant in the Room: Storj's S3 Gateway IS MinIO
storj/gateway-st/miniogw implements MinIO's ObjectLayer interface, routing S3 API calls to the Storj network instead of local disks. Storj's entire S3-compatible surface area is a MinIO fork β they are not building an S3 implementation; they are building a storage backend adapter for MinIO.
gateway-mt (the hosted, multi-tenant variant) is the same pattern wrapped in a MultiTenancyLayer. When you connect to Storj's S3 endpoint, you are talking to a MinIO process.
This is not incidental. It means Storj's S3 compatibility ceiling is bounded by whatever version of MinIO they have forked and are maintaining β and their own issue tracker (edge#27: "Replace Minio fork with most recent Apache2 version") shows they are perpetually behind upstream.
PySpark parquet overwrite pattern β tests partition prefix visibility after overwrite on S3/MinIO
PySpark Parquet Overwrite β Partition Prefix Visibility Test
Tests that after Spark overwrites partitioned Parquet files on S3/MinIO,
the date-level partition prefixes remain visible in delimited ListObjectsV2
so that Spark's partition discovery still works correctly.
What the test does
Generates two sample CSV files (batch1.csv, batch2.csv) with the same
schema and same date partitions but different values.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters