decentralizepy

Setting up decentralizepy

• Fork the repository.
• Clone and enter your local repository.
• Check if you have python >= 3.8.
• (Optional) Create and activate a virtual environment.

• Update pip::

  pip3 install --upgrade pip pip install --upgrade pip

• Install decentralizepy for development::

  pip3 install --editable .[dev]

Node

• The Manager. Optimizations at process level.

Dataset

• Static

Training

• Heterogeneity. How much do I want to work?

Graph

• Static. Who are my neighbours? Topologies.

Mapping

• Naming. The globally unique ids of the processes <-> machine_id, local_rank

Sharing

• Leverage Redundancy. Privacy. Optimizations in model and data sharing.

Communication

• IPC/Network level. Compression. Privacy. Reliability

Model

• Learning Model