Ability to use custom reporters 💡

[stipsan]

EDIT

Shucks, I hit enter while adding the title, didn't mean to post it just yet!

EDIT (again)

I can see that while the built in reporters can be used multiple times even there does not seem to be a way to add external ones:

dredd/src/options.coffee

Lines 52 to 57 in 54135db

	reporter:
	alias: "r"
	description: """Output additional report format. This option can be used \
	multiple times to add multiple reporters. \
	Options: xunit, nyan, dot, markdown, html, apiary.\n"""
	default: []

dredd/src/configure-reporters.coffee

Lines 36 to 49 in 54aa8a3

	addReporter = (reporter, emitter, stats, tests, path) ->
	switch reporter
	when 'junit'
	xUnitReporter = new XUnitReporter(emitter, stats, tests, path, config.options.details)
	when 'dot'
	dotReporter = new DotReporter(emitter, stats, tests)
	when 'nyan'
	nyanCatReporter = new NyanCatReporter(emitter, stats, tests)
	when 'html'
	htmlReporter = new HtmlReporter(emitter, stats, tests, path, config.options.details)
	when 'markdown'
	mdReporter = new MarkdownReporter(emitter, stats, tests, path, config.options.details)
	when 'apiary'
	apiaryReporter = new ApiaryReporter(emitter, stats, tests, config, runner)

I'm wondering if you would consider allowing it, and if a PR for that would be welcome 😄

One of the things I'd like to do in my custom reporter is to change up this:

dredd/src/reporters/x-unit-reporter.coffee

Lines 73 to 81 in 54aa8a3

	Message:
	#{test.message}
	Request:
	#{prettifyResponse(test.request)}
	Expected:
	#{prettifyResponse(test.expected)}
	Actual:
	#{prettifyResponse(test.actual)}
	"""

We have mile long JSON Schemas and big payloads. The ability to change the output to the following is a huge improvement:

prettyDiff = require 'jest-diff'

      diff = """
      Message:
      #{test.message}
      Diff:
      #{prettyDiff(test.expected, test.actual)}
      Request:
      #{prettifyResponse(test.request)}
      Expected:
      #{prettifyResponse(test.expected)}
      Actual:
      #{prettifyResponse(test.actual)}

This is not the only use case I have where the option to add my own reporter to dredd would be awesome:

• post GitHub PR comments on failures with a markdown formatted diff save a step with drilling down deep on CI logs.
• update a README.md with a list over skipped tests to encourage refactors so that skips are no longer needed.
• adapt junit/xunit output so it's more useful in a CircleCI context. Basically:
  This is not very useful:

But this is extremely helpful (but require changes to the junit output that is more compatible with CircleCI):

btw I think it would be better if this weren't commented out:

dredd/src/configure-reporters.coffee

Lines 50 to 52 in 54aa8a3

	# else
	# Cannot happen, due to 'intersection' usage
	# logger.warn "Invalid reporter #{reporter} selected, ignoring."

IMO it's not user friendly to silently ignore any setting :)

[honzajavorek]

@stipsan These are awesome ideas! In #765, we thought about supporting TAP (see also this). That would allow people to easily extend Dredd with any reporter they like, and it would allow us to completely remove some of the built in reporters.

Do you think that would solve your issues? Is TAP protocol powerful enough to support everything you would need to implement the reporters you mentioned?

[stipsan]

Yes absolutely TAP would be powerful enough! 😄 Great news!

I will close this issue and subscribe to #765 👍

[honzajavorek]

Great! Thanks, this validates the thoughts we had about TAP 🙂