Bisect: regenerates ground-truth results unnecessarily

[JohnJacobsonIII]

Bug Report

Describe the problem

1. Bisect regenerates the ground-truth.csv result file (by running all baseline tests) many times throughout a bisect run. This may be seen by calling bisect with VERBOSE defined in the Make environment and looking for calls to ./gtrun --output ground-truth.csv, or by analyzing log events in Event logging #331.

   It should be noted that these unexpected event duplications break the logical program dependencies used in tracing the FLiT process for Event logging #331.

2. Bisect currently has no means to pass Makefile output to an arbitrary stream; all output from makefiles is currently conveyed through stdout during the bisect process, although Python output may be written elsewhere via command line arguments.

Possible Fixes

• Extraneous test runs:
  • Ground truth tests should only be run once, so eliminate any extraneous test calls to gtrun.
  • The cause is undetermined, but dependencies of the gtrun executable need to be thoroughly reviewed between both the base Makefile.in and the bisect Makefile_bisect_binary.in files.
  • Tests should be added to check for extraneous events (particularly compilation, links, test runs, and test comparisons.)
• Make output:
  • This is minor, but makes for inconsistent integration testing at least. A couple immediate thoughts:
    1. Leave as is, and adapt integration testing to access stdout for parsing as necessary.
    2. Propagate stream argument to all bisect functions for consistency

[mikebentley15]

Pull request #337 fixes this issue