Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
D
decentralizepy
Manage
Activity
Members
Labels
Plan
Issues
0
Issue boards
Milestones
Wiki
Code
Merge requests
0
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Milos Vujasinovic
decentralizepy
Commits
7b367239
Commit
7b367239
authored
3 years ago
by
Rishi Sharma
Browse files
Options
Downloads
Patches
Plain Diff
Add RandomAlpha, Regular Graph
parent
5aaa9722
No related branches found
Branches containing commit
No related tags found
Tags containing commit
No related merge requests found
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
src/decentralizepy/graphs/Regular.py
+32
-0
32 additions, 0 deletions
src/decentralizepy/graphs/Regular.py
src/decentralizepy/sharing/RandomAlpha.py
+79
-0
79 additions, 0 deletions
src/decentralizepy/sharing/RandomAlpha.py
with
111 additions
and
0 deletions
src/decentralizepy/graphs/Regular.py
0 → 100644
+
32
−
0
View file @
7b367239
import
networkx
as
nx
from
decentralizepy.graphs.Graph
import
Graph
class
Regular
(
Graph
):
"""
The class for generating a Regular topology
"""
def
__init__
(
self
,
n_procs
,
degree
,
seed
=
None
):
"""
Constructor. Generates a Ring graph
Parameters
----------
n_procs : int
total number of nodes in the graph
degree : int
Neighbors of each node
"""
super
().
__init__
(
n_procs
)
G
=
nx
.
random_regular_graph
(
degree
,
n_procs
,
seed
)
adj
=
G
.
adjacency
()
for
i
,
l
in
adj
:
self
.
adj_list
[
i
]
=
set
()
# new set
for
k
in
l
:
self
.
adj_list
[
i
].
add
(
k
)
if
not
nx
.
is_connected
(
G
):
self
.
connect_graph
()
This diff is collapsed.
Click to expand it.
src/decentralizepy/sharing/RandomAlpha.py
0 → 100644
+
79
−
0
View file @
7b367239
import
random
from
decentralizepy.sharing.PartialModel
import
PartialModel
class
RandomAlpha
(
PartialModel
):
"""
This class implements the partial model sharing with a random alpha each iteration.
"""
def
__init__
(
self
,
rank
,
machine_id
,
communication
,
mapping
,
graph
,
model
,
dataset
,
log_dir
,
dict_ordered
=
True
,
save_shared
=
False
,
metadata_cap
=
1.0
,
):
"""
Constructor
Parameters
----------
rank : int
Local rank
machine_id : int
Global machine id
communication : decentralizepy.communication.Communication
Communication module used to send and receive messages
mapping : decentralizepy.mappings.Mapping
Mapping (rank, machine_id) -> uid
graph : decentralizepy.graphs.Graph
Graph reprensenting neighbors
model : decentralizepy.models.Model
Model to train
dataset : decentralizepy.datasets.Dataset
Dataset for sharing data. Not implemented yet! TODO
log_dir : str
Location to write shared_params (only writing for 2 procs per machine)
dict_ordered : bool
Specifies if the python dict maintains the order of insertion
save_shared : bool
Specifies if the indices of shared parameters should be logged
metadata_cap : float
Share full model when self.alpha > metadata_cap
"""
super
().
__init__
(
rank
,
machine_id
,
communication
,
mapping
,
graph
,
model
,
dataset
,
log_dir
,
1.0
,
dict_ordered
,
save_shared
,
metadata_cap
,
)
def
step
(
self
):
"""
Perform a sharing step. Implements D-PSGD with alpha randomly chosen.
"""
random
.
seed
(
self
.
mapping
.
get_uid
(
self
.
rank
,
self
.
machine_id
)
+
self
.
communication_round
)
self
.
alpha
=
random
.
randint
(
1
,
7
)
/
10.0
super
().
step
()
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment