This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Benchmark Configuration: | |
| Batch sizes: [1] | |
| Sequence lengths: [32, 64, 128, 256, 512, 1024, 1536, 2048, 4096, 8192, 16384] | |
| Number of heads: [16, 32, 64, 128] | |
| Head dimensions: [64, 128] | |
| Causal: True | |
| Data type: torch.bfloat16 | |
| Warmup iterations: 10 | |
| Benchmark iterations: 100 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/usr/bin/env python3 | |
| """ | |
| Benchmark comparing FlashInfer vs Flash Attention on B200 GPU. | |
| Tests various sequence lengths and head dimensions. | |
| """ | |
| import argparse | |
| import sys | |
| from collections.abc import Sequence | |
| from dataclasses import dataclass |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| > uname -m && cat /etc/*release | |
| x86_64 | |
| DISTRIB_ID=Ubuntu | |
| DISTRIB_RELEASE=18.04 | |
| DISTRIB_CODENAME=bionic | |
| DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS" | |
| NAME="Ubuntu" | |
| VERSION="18.04.5 LTS (Bionic Beaver)" | |
| ID=ubuntu | |
| ID_LIKE=debian |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| > uname -m && cat /etc/*release | |
| x86_64 | |
| DISTRIB_ID=Ubuntu | |
| DISTRIB_RELEASE=18.04 | |
| DISTRIB_CODENAME=bionic | |
| DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS" | |
| NAME="Ubuntu" | |
| VERSION="18.04.5 LTS (Bionic Beaver)" | |
| ID=ubuntu | |
| ID_LIKE=debian |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <html><head> | |
| <meta http-equiv="content-type" content="text/html; charset=UTF-8"> | |
| <script type="text/javascript" src="http://d3js.org/d3.v3.min.js"></script> | |
| <style type="text/css"> | |
| .bar.positive { | |
| fill: steelblue; | |
| } | |
| .bar.negative { | |
| fill: brown; |