This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # ๐๏ธ CLUSTER STORAGE AUDIT (Sequential & Random Suite) | |
| # Usage: ./storage_bench.sh /path/to/mount | |
| DIR=${1:-/data_vast/bench_$(hostname)} | |
| JOBS=16 # Number of simultaneous threads | |
| RUNTIME=10s # Duration of each of the 4 tests | |
| RAMP=5s # Warm-up time to saturate controllers | |
| FILE_SIZE=4G # Data per thread (Total per node = 64GB) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # ๐๏ธ CLUSTER STORAGE AUDIT (Sequential & Random Suite) | |
| # Usage: ./storage_bench.sh /path/to/mount | |
| DIR=${1:-/data_vast/bench_$(hostname)} | |
| JOBS=16 # Number of simultaneous threads | |
| RUNTIME=10s # Duration of each of the 4 tests | |
| RAMP=5s # Warm-up time to saturate controllers | |
| FILE_SIZE=4G # Data per thread (Total per node = 64GB) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # ============================================================================= | |
| # ๐๏ธ NFS STORAGE BENCHMARK โ Audio File I/O Profile | |
| # ============================================================================= | |
| # | |
| # PURPOSE: | |
| # Benchmarks an NFS (or similar networked) filesystem for a distributed ML | |
| # training pipeline that consumes large volumes of WAV/FLAC audio files. | |
| # | |
| # WHAT THIS TESTS: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # ============================================================================= | |
| # ๐๏ธ NFS STORAGE BENCHMARK โ Audio File I/O Profile (PyTorch-Realistic) | |
| # ============================================================================= | |
| # | |
| # PURPOSE: | |
| # Benchmarks an NFS (or similar networked) filesystem for ML training pipelines | |
| # that consume large numbers of small WAV/FLAC audio files using PyTorch-style | |
| # dataloaders. | |
| # |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # ============================================================================= | |
| # ๐๏ธ NFS STORAGE BENCHMARK โ Audio File I/O Profile (PyTorch-Realistic) | |
| # ============================================================================= | |
| # | |
| # PURPOSE: | |
| # Benchmarks an NFS (or similar networked) filesystem for ML training pipelines | |
| # that consume large numbers of small WAV/FLAC audio files using PyTorch-style | |
| # dataloaders. | |
| # |
OlderNewer