Trevor Keller, Ph.D. • NIST MML MSED • May 15, 2018
This document summarizes the side-by-side comparison of dual-socket servers:
2× AMD EPYC 7601 vs. 2× Intel Xeon E5-2697Av4.
The AMD system is a testbed, access to which was provided by KOI Computers. The Intel systems are CTCMS compute nodes, two of which were utilized to enable thread-to-thread comparisons for a parallel HPC workload (phase-field model of solid-state transformation). Serial tasks are not of interest, and were not evaluated.
Disclaimers:
- This experiment was not controlled, and a single data point is insufficient for anything other than a spot check. No conclusions should be drawn.
- Certain commercial entities, equipment, or materials may be identified in this document in order to describe an experimental procedure or concept adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology (NIST), nor is it intended to imply that the entities, materials, or equipment are necessarily the best available for the purpose.
| Intel Xeon E5-2697A | AMD EPYC 7601 | Intel Xeon Gold 6142* | |
|---|---|---|---|
| Release Date | June 2016 | June 2017 | July 2017 |
| Clock (base/boost) | 2.6 GHz / 3.1 GHz | 2.2 GHz / 3.2 GHz | 2.6 GHz / 3.4 GHz |
| Fabrication Process | 14 nm | 14 nm | 14 nm |
| Cores / Threads | 16 / 32 | 32 / 64 | 16 / 32 |
| L3 Cache | 40 MB | 64 MB | 22 MB |
| Vector Instructions | AVX2 | AVX2 | AVX-512 |
| Memory Bandwidth | 4× 17 GB/s | 8× 19 GB/s | 6× 20 GB/s |
| TDP | 145 W | 180 W | 150 W |
| Interconnect | QuickPath | Infinity Fabric | UltraPath |
| PCIe v3 Lanes | 40 | 128 | 48 |
| Max. Scalability | 2-way SMP | 2-way SMP | 4-way SMP |
| MSRP | $2,890 | $4,200 | $2,946 |
* not tested, but included for comparison of similar vintages on paper
Runtimes, in minutes, per million timesteps (source code)
| System | Interconnect | Nodes | MPI Ranks | OMP Threads | Total Threads | Runtime | Speedup |
|---|---|---|---|---|---|---|---|
| Xeon | Infiniband | 2 | 4 | 32 | 128 | 93 | 1* |
| EPYC | Infinity Fabric | 1 | 2 | 64 | 128 | 41 | 2.27 |
* indicates reference run
| System | OMP Threads | Runtime | Speedup |
|---|---|---|---|
| Xeon | 64 | 86 | 1* |
| EPYC | 64 | 71 | 1.21 |
| EPYC | 128 | 44 | 1.95 |
* indicates reference run
The Intel systems showed similar runtimes using one or two nodes, suggesting a bottleneck in node-to-node communication for this application. This result could not be confirmed using the single-node AMD configuration. If true, and if this application is representative of HPC workloads, it is better to configure fewer nodes with more sockets than more nodes with fewer sockets, for constant cluster thread count.
In core-to-core comparisons (64 threads), the AMD system outperformed the current compute nodes by 20%. This could be due to the greater memory bandwidth or other architectural improvements, or it could simply reflect variance between individual processors. The socket-to-socket tests suggest a 2× speedup which, due to the difference in thread counts, suggests parity between individual cores and better performance of the AMD processor for this application due to its higher thread count.
This data/work was created by employees of the National Institute of Standards and Technology (NIST), an agency of the Federal Government. Pursuant to title 17 United States Code Section 105, works of NIST employees are not subject to copyright protection in the United States. This data/work may be subject to foreign copyright.
The data/work is provided by NIST as a public service and is expressly provided “AS IS.” NIST MAKES NO WARRANTY OF ANY KIND, EXPRESS, IMPLIED OR STATUTORY, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT AND DATA ACCURACY. NIST does not warrant or make any representations regarding the use of the data or the results thereof, including but not limited to the correctness, accuracy, reliability or usefulness of the data. NIST SHALL NOT BE LIABLE AND YOU HEREBY RELEASE NIST FROM LIABILITY FOR ANY INDIRECT, CONSEQUENTIAL, SPECIAL, OR INCIDENTAL DAMAGES (INCLUDING DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS INTERRUPTION, LOSS OF BUSINESS INFORMATION, AND THE LIKE), WHETHER ARISING IN TORT, CONTRACT, OR OTHERWISE, ARISING FROM OR RELATING TO THE DATA (OR THE USE OF OR INABILITY TO USE THIS DATA), EVEN IF NIST HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
To the extent that NIST may hold copyright in countries other than the United States, you are hereby granted the non-exclusive irrevocable and unconditional right to print, publish, prepare derivative works and distribute the NIST data, in any medium, or authorize others to do so on your behalf, on a royalty-free basis throughout the world.
You may improve, modify, and create derivative works of the data or any portion of the data, and you may copy and distribute such modifications or works. Modified works should carry a notice stating that you changed the data and should note the date and nature of any such change. Please explicitly acknowledge the National Institute of Standards and Technology as the source of the data. Data citation recommendations are provided at https://www.nist.gov/open/license.
Permission to use this data is contingent upon your acceptance of the terms of this agreement and upon your providing appropriate acknowledgments of NIST’s creation of the data/work.