Extend run_command to batch execution

[manuelgitgomes]

As stated in #890, a new function was created to run a command and output in real-time:

atom/atom_calibration/templates/configure

Lines 27 to 48 in cd5725e

	def run_command(command, tictoc):
	print('\n\n' + Style.BRIGHT + Fore.BLUE + 'Executing command:' +
	Style.RESET_ALL + '\n' + Fore.BLUE + command + Style.RESET_ALL)
	# Start executing command.
	tictoc.tic()
	proc = subprocess.Popen(command, shell=True, universal_newlines=True, stdout=subprocess.PIPE,
	stderr=subprocess.PIPE)
	while True:
	line = proc.stdout.readline()
	print(line)
	if not line: break
	stdout_data, stderr_data = proc.communicate() # wait for command to finish
	toc = str(round(tictoc.tocvalue(), 5))
	if not proc.returncode == 0: # Check stdout_data of command.
	print(Fore.RED + Back.YELLOW + 'Error running command. stderr is:' + Style.RESET_ALL)
	print(stderr_data)
	exit(0)
	print(Fore.BLUE + 'Command executed in ' + toc + ' secs.' + Style.RESET_ALL)

This function will be extended to the batch execution in this issue.

[manuelgitgomes]

Hello @miguelriemoliveira.

You already have a function for this:

atom/atom_core/src/atom_core/system.py

Lines 18 to 31 in cd5725e

	def execute(cmd, blocking=True, verbose=True):
	""" @brief Executes the command in the shell in a blocking or non-blocking manner
	@param cmd a string with teh command to execute
	@return
	"""
	if verbose:
	print("Executing command: " + cmd)
	p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
	if blocking: # if blocking is True:
	for line in p.stdout.readlines():
	if verbose:
	print(line)
	p.wait()

I can fuse yours and mine and use only this execute

[manuelgitgomes]

This was harder than expected.

The function execute from atom_core.system has been reformulated:

atom/atom_core/src/atom_core/system.py

Lines 13 to 86 in cce5f65

	def execute(cmd, blocking=True, verbose=True, save_path=None, save_filename_additions=''):
	""" @brief Executes the command in the shell in a blocking or non-blocking manner
	@param cmd a string with teh command to execute
	@return
	"""
	# Open files to save stdout and stderr if save_path is provided
	stdout_file = None
	stderr_file = None
	if save_path:
	stdout_filename = save_path + '/' + save_filename_additions + 'stdout.txt'
	stderr_filename = save_path + '/' + save_filename_additions + 'stderr.txt'
	stdout_file = open(stdout_filename, 'w')
	stderr_file = open(stderr_filename, 'w')
	# Print the command being executed and measure the execution time if verbose is True
	if verbose:
	print("Executing command: " + cmd)
	tictoc = TicToc()
	tictoc.tic()
	# Execute the command in the shell using subprocess.Popen
	proc = subprocess.Popen(cmd, shell=True, universal_newlines=True, stdout=subprocess.PIPE,
	stderr=subprocess.PIPE)
	stdout_data = ''
	# Read the stdout of the process and print it in real-time
	if blocking:
	while True:
	output = proc.stdout.read(1)
	if output == '' and proc.poll() is not None:
	break
	if output:
	if verbose:
	print(output, end='')
	# sys.stdout.flush()
	stdout_data += output
	# Write the stdout data to the file if stdout_file is provided
	if stdout_file:
	stdout_file.write(removeColorsFromText(stdout_data))
	stdout_file.close()
	# Wait for the command to finish and get the stderr data
	stdout_data, stderr_data = proc.communicate()
	if not blocking:
	# Write the stdout data to the file if stdout_file is provided
	if stdout_file:
	stdout_file.write(removeColorsFromText(stdout_data))
	stdout_file.close()
	# If the return code is not 0, print the stderr data and exit
	if not proc.returncode == 0:
	print(Fore.RED + Back.YELLOW + 'Error running command. stderr is:' + Style.RESET_ALL)
	print(stderr_data)
	if stderr_file:
	stderr_file.write(removeColorsFromText(stderr_data))
	stderr_file.close()
	if stdout_file:
	stdout_file.close()
	exit(0)
	# Print the execution time if verbose is True
	if verbose:
	toc = str(round(tictoc.tocvalue(), 5))
	print(Fore.BLUE + 'Command executed in ' + toc + ' secs.' + Style.RESET_ALL)
	# Close the stdout and stderr files if provided
	if stdout_file:
	stdout_file.close()
	if stderr_file:
	stderr_file.close()

It now prints in "real time" what the subprocess is outputting.
It is also now responsible to write stdout and stderr to a file.
This has been done in a way that writing is independent of blocking and the filename can have extra addons.
Time keeping is also done with the parameter verbose enabled.

Then, this has been added to configure and to batch_execution.
configure became significantly simpler, as most local functions have been delegated to this one.
batch_execution suffered some changed to adapt to this, but it is now fully functional.
To achieve this, the function removeColorsFromText needed to be moved from atom_core.utilities to atom_core.system, to remove circular imports.