Parallelism

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimentation and parallelizatio…