1.2. Workload Distribution Scenario¶
This Scenario performs the action of distributing a high computational effort Task in remote nodes, in order to parallelize the task and do not block any other important actions that may require to run in the same device. It uses the MultiService over DDS communication to publish those tasks in an efficient way.
The Task distributed is the training data-set of an AML model. This model is stored in a Main Node and the training data-set is divided in different Jobs, that are sent along with states of the model to Computing Nodes in order to perform this training in parallel, reducing the workload in the Main Node host, that may require to perform other actions at the same time.
1.2.1. Job Data Type¶
The Job Data Type represents a partial data-set and a model state.
Internally, Jobs sent from a Main Node to a Computing Node are treated as byte arrays of arbitrary size.
So far, the interaction with this class could be done from a void*
, a byte array or a string.
From Python API, the only way to interact with it is by str type.
Note
A more specific Data Type will be implemented in future releases for efficiency improvements.
1.2.2. Job Solution Data Type¶
The Solution Data Type represents an Atomization or new model state.
The Solution sent from a Computing Node to a Main Node is treated as a bytes array of arbitrary size.
So far, the interaction with this class could be done from a void*
, a byte array or a string.
From Python API, the only way to interact with it is by str type.
Note
A more specific Data Type will be implemented in future releases for efficiency improvements.