You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In Spark like deployments, Driver is a single point of failure but not workers.
Keeping KVSStore with one of the worker makes one of the worker process as single point of failures.
If KVS can be started as stand alone process, the integration into spark like deployments will be easy. Driver can start this KVS Store and pass the KVSStore IP_Port to all workers.
Rabit has the similar architecture, tracker( like KVStore here) starts with Driver. All workers connects to tracker.
The text was updated successfully, but these errors were encountered:
In Spark like deployments, Driver is a single point of failure but not workers.
Keeping KVSStore with one of the worker makes one of the worker process as single point of failures.
If KVS can be started as stand alone process, the integration into spark like deployments will be easy. Driver can start this KVS Store and pass the KVSStore IP_Port to all workers.
Rabit has the similar architecture, tracker( like KVStore here) starts with Driver. All workers connects to tracker.
The text was updated successfully, but these errors were encountered: