decentralizepy

Node

• The Manager. Optimizations at process level.

Dataset

• Static

Training

• Heterogeneity. How much do I want to work?

Graph

• Static. Who are my neighbours? Topologies.

Mapping

• Naming. The globally unique ids of the processes <-> machine_id, local_rank

Sharing

• Leverage Redundancy. Privacy. Optimizations in model and data sharing.

Communication

• IPC/Network level. Compression. Privacy. Reliability