Fully Sharded Data Parallel

Notes
ai
llm
llm-training

Graph

FSDP (Fully Shared Data Parallel)