Short Name | Full Name | URL |
---|---|---|
CAME | Confidence-guided Adaptive Memory Efficient Optimization (ACL23 outstanding paper award) | https://github.com/huawei-noah/Pretrained-Language-Model/blob/master/CAME/came.py |
LOMO | LOw-Memory Optimization | https://github.com/OpenLMLab/LOMO/blob/main/src/lomo.py |
LORA (Low Rank Adaption): https://github.com/microsoft/LoRA or https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora.py