Summary of failures

[topr]

Description

When test(s) fail it would be very useful to have the failing tests listed in a form of a summary at the bottom.

Additional information

On a project with many tests, when there is a failure, it is far from convenient finding out which tests and why are failing. One needs to scroll up to see what happened, assuming the console buffer even still keeps the information.

At the moment it's possible to go around the issue with the configuration set like:

testlogger {
    showPassed false
    showSkipped false
    showFailed true
}

However this denies the whole idea of the awesome output which this plugin brings.
It really is awesome, thanks! My teammates and I like it a lot. Not sure can we afford using it on a project with about 5000 tests for that reason though. I takes a lot of scrolling...
Unless there is already a configuration option which allows failure summary but I've missed it, it would be great to add such.

Can't tell as I haven't checked the plugin codebase yet but I'd hope it should not be a great deal of work. I guess, it would be sufficient to display the same output which is printed during a test run but only for the failing tests again after the test run is finished.

[radarsh]

Thanks for the idea and I'm glad you like the plugin. What is wrong with having the configuration you suggested? Is it because you want to have the satisfaction of being able to see passing tests and at the same time want to see a quick summary of all the failures?

[topr]

Thanks for the quick reply. Yes, you're on spot. Seeing tests as they run is satisfying. Plus the plugin highlights the long running tests which otherwise wouldn't be visible. That's a helpful feature though.
In general, installing a plugin to display tests in a nice way just to configure it so tests are not displayed at all, well...

I've scripted quickly like a dozen lines in our project's build.gradle to add some rough summary of failures. The project's lead is already saying to ditch the plugin instead though.

One doesn't see often senior devs cheering at their screen like a kid with a new toy. I must give yours plugin that, after updating with master when the plugin was introduced, they were all like this in our team of five, including the lead :)

[DreierF]

I would also love to have this integrated into the plugin. Our use-case is that we have a lot of modules. We want to see the summary of passed/skipped tests in the CI output because we sometimes had cases where tests ran into deadlocks or only failed when executed after some other test. Therefore the output is very helpful to pinpoint the responsible test.

However as already stated it's cumbersome to find the failed tests in the output. The summary would show up ideally after the build has finished and not directly after a single test task has failed, because the summary will be super-seeded by the next test tasks when running tests with --continue.

Our current workaround:

ext {
	failedTests = []
}

allprojects {
	tasks.withType(Test) {
		afterTest { desc, result ->
			if (result.resultType == TestResult.ResultType.FAILURE) {
				project.failedTests += desc.className + '.' + desc.name
			}
		}
	}
}

gradle.buildFinished {
	def failedTests = project.ext.failedTests
	if (!failedTests.isEmpty()) {
		logger.error("==== FAILED TESTS =============")
		failedTests.forEach {
			logger.error(it)
		}
	}
}