Iteration Cycle abstract discussion

[bharathappali]

Current EM Implementation has support only for one iteration, EM design should be changed to fit in the requirement for iterations. The Finite state machine (FSM) state transfers are currently based on the EMStageRegistry, the state transfer control should be transferred partially to the states in FSM to take the decision to go back a particular state so that we can get an iteration over some particular states.

[bharathappali]

Sample image which explains the states and the boundaries where we require subcycles

[bharathappali]

Apologies for the mistake in the image. The metrics collection ends at gather metrics phase

[bharathappali]

Input for Iteration:

...
"trial_settings" : {
     ...
     "iterations" : 3,
     ...
}
...

Case if iterations are not mentioned

If iterations are not given EM should consider the iterations to be 1 rather than failing the experiment.

Scope of an iteration:

current stages in EM:

INIT
CREATE_CONFIG
DEPLOY_CONFIG
INITIATE_TRAIL_RUN_PHASE
INITIAL_LOAD_CHECK
LOAD_CONSISTENCY_CHECK
INITIATE_METRICS_COLLECTION_PHASE
COLLECT_METRICS
CREATE_RESULT_DATA
SEND_RESULT_DATA
CLEAN_OR_ROLLBACK_DEPLOYMENT
EXIT

An Iteration is from stage DEPLOY_CONFIG to CREATE_RESULT_DATA

[bharathappali]

Process flow in an iteration:

• Create/Restart a pod with the given config
• Perform data collection for each cycle (Warm-Up / Measurement)
• Aggregate the results and calculate the final result
• Save results in the ETD (Experiment Trial Data)

Changes in ETD required:

The current ETD is designed to store the results of one iteration, here we need a new set of abstractions for storing the results on an iteration basis. As ETD holds EMTrialConfig we now need to create a data holder for results which will be a list of EMIterationData

contents of EMIterationData:

• Iteration Number - int
• Total Cycles - int
• warm-up cycles - int
• measurement cycles - int
• ...
• Iteration results - EMIterationResult object

contents of EMIterationResult:

• WarmupResults - Arraylist of EMMetrics object
• MeasurementResults - Arraylist of EMMetrics object
• SummarisedResults - Arraylist of EMMetrics object
• addToWarmUp - function which receives an EMMetrics Object to be added into warmup results
• addToMeasurement - function which receives an EMMetrics Object to be added into measurement results
• resultAddedTimestamp - updated when either of addToWarmUp or addToMeasurement are called
• calculateSummary - function which calculates the summary results from warm-up and measurement results
• summaryCalculatedTimestamp - updated when calculateSummary is called
• isLatestResult - check if the summary results are latest (check time diff between anything add to warmup or measurement lists and last calculateSummary is run)
• toJSON - implemented from ConvertToJSON interface resulting out the content in form of JSON