Quentin Anthony Quentin-Anthony

High-Performance Deep Learning PhD student at OSU NOWLAB

107 followers · 7 following

The Ohio State University
Columbus, OH
https://quentin-anthony.github.io/

View GitHub Profile

Recently created

Least recently created

Recently updated

Least recently updated

Quentin-Anthony / megatron-checks.md

Last active June 7, 2022 05:30

Megatron Spreadsheet

ZeRO Stage	Data-Parallel	MP	PP	MP+PP	MoE	MoE+MP
1	✓	✓	✓	✓	✓	✓
2	✓	✓	N/A	N/A	✓	✓
3	✓	✓	N/A	N/A	N/A	N/A

NewerOlder