v4.3.0

[4.3.0] => 2021-MAY-24

Bugs

• TESTBOX-299 Bug in XML-escaping in JUnit reporters
• TESTBOX-298 min reporter is making assumptions that url.directory will exist
• TESTBOX-297 code coverage QoQ doesn't account for nulls in sum()
• TESTBOX-287 Simple Reporter doesn't show the Test that fails but the assertion in the origin

Improvements

• TESTBOX-305 Show the type of the actual pass into instance type expectations
• TESTBOX-300 Improvement: shift test summary to the bottom of MinTextReporter
• TESTBOX-295 Update the junit and antjunit reports to NOT include supported locales due to the size of the packet

New Features

• TESTBOX-309 Reworked simple reporter to better visualize fail origins and tag contexts.
• TESTBOX-308 Simple reporter now has editor integrations to open failure and error stacks in your editor
• TESTBOX-307 New text and min text reporters to improve visualizations
• TESTBOX-306 Test failures triggered in beforeAll are counted incorrectly

v4.1.0

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog,
and this project adheres to Semantic Versioning.

---

[4.1.0] => 2020-MAY-27

Fixed

• [TESTBOX-283] - Fix type on test results for bundlestats
• [TESTBOX-286] - DebugBuffer was being removed instead of resetting to empty for getMemento
• [TESTBOX-281] - request.testbox Component ... has no accessible Member with name [$TESTID]

Added

• [TESTBOX-282] - Added cfml engine and version as part of the test results as properties
• [TESTBOX-284] - Update all reporters so they can just build and return the report with no content type or context repsonse resets
• [TESTBOX-285] - make buildReporter public in the testbox core

v3.1.0

What's New With 3.1.0

Major Features

Focused Suites and Specs

This was asked for a long time ago and finally we can oblige. You can now prefix any of the suite and spec methods with the letter f and they will be focused. Meaning, ONLY those suites or specs will be executed.

• fit(), fthen() - Spec Methods
• fdescribe(), fscenario(), fstory(), ffeature(), fgiven(), fwhen() - Suite Methods

This is great if you ONLY want certain things to execute instead of the x prefix which was used to EXCLUDE specs and suites.

{% code-tabs %}
{% code-tabs-item title="tests/FocusedSpecs.cfc" %}

function run( testResults, testBox ){
	// all your suites go here.
	describe( "My First Suite", function(){

		it( "A Spec that should not run", function(){
			fail( 'implement' );
		});

		it( "A Spec that should not run", function(){
			fail( 'implement' );
		});

		fit( "This should execute", function(){
			expect( 1 ).toBeTrue();
		});

		it( "A Spec that should not run", function(){
			fail( 'implement' );
		});

		fit( "This should execute as well", function(){
			expect( 1 ).toBeTrue();
		});

		fdescribe( "All specs here should run", function(){

			it( "should run", function(){

			});
			it( "should run", function(){

			});
			it( "should run", function(){

			});
		} );

		describe( "All specs here should NOT run", function(){

			it( "A Spec that should not run", function(){
				fail( 'implement' );
			});

			it( "A Spec that should not run", function(){
				fail( 'implement' );
			});

			it( "A Spec that should not run", function(){
				fail( 'implement' );
			});

		} );

	});

	fdescribe( "My Focused Suite", function(){

		it( "This should execute", function(){
			expect( 1 ).toBeTrue();
		});

		describe( "All specs here should run", function(){

			it( "should run", function(){

			});
			it( "should run", function(){

			});
			it( "should run", function(){

			});
		} );

		xdescribe( "skipped suites", function(){

			it( "A Spec that should not run", function(){
				fail( 'implement' );
			});

			it( "A Spec that should not run", function(){
				fail( 'implement' );
			});

			it( "A Spec that should not run", function(){
				fail( 'implement' );
			});

		} );

	});

{% endcode-tabs-item %}
{% endcode-tabs %}

Here are the results:

toHaveKey() can now work with key lists

How many times did you want to check if a struct had ALL the keys or NO keys at all. Well, now you can. You can pass a list of keys into the toHaveKey() expectation and we will make sure your structure has ALL the keys in it.

var s = {
	"data" : {},
	"error" : {},
	"name" : {},
	"age" : 0
};

expect( s ).toHaveKey( "error" );
expect( s ).notToHaveKey( "luis" );

// Multiple
expect( s ).toHaveKey( "data,error,name,age" );
expect( function(){
	expect( s ).toHaveKey( "data,error,name,age2" );
}).toThrow();
expect( s ).notToHaveKey( "luis,joe,tom" );
expect( function(){
	expect( s ).toHaveKey( "luis,joe,data" );
}).toThrow();

fail() Improvements

You can now pass in a detail to the fail messages that can be used to track exception details, stacktraces, extended info and much more. Sometimes, this can be a life-saver when dealing with toThrow() expectations:

/**
 * Fail an assertion
 *
 * @message The message to fail with.
 * @detail The detail to fail with.
 */
function fail( message = "", detail = "" ){
	variables.assert.fail( argumentCollection = arguments );
}

Which is exactly what we did. The catching of expected exceptions now pass in the exact stacktrace where they failed instead of the nested exception. Which goes nicely into our next major improvement.

Nested Exceptions

We all hate them and not easy to track. In this release we at least try to add more debugging on the results by introducing the following new keys and output on the reporters:

• failDetail
• failExtendedInfo
• failStacktrace

CFML Engine/Version Detection

The reporters know will output the CFML engine the report is executed on. Also, the raw json/xml reporters will also report back the CFML engine and version.

Release Notes

Bugs

• [TESTBOX-251] - Scripts don't output if runner.cfm has enableCFoutputOnly set to true
• [TESTBOX-252] - JSON,xml,Junit, Ant reporters fails for integration tests with cfhtmlhead and cfheader
• [TESTBOX-257] - Update build process to upload the right version assets

New Features

• [TESTBOX-255] - Added new spec stats to track nested exceptions: failDetail, failExtendedInfo, failStacktrace
• [TESTBOX-256] - Add a detail argument to the fail() method to allow for more in-depth tracking of failures, especially when using the toThrow() expectation
• [TESTBOX-258] - Added CFMLEngine and CFMLEngineVersion to results memento so any consumer can display this information
• [TESTBOX-253] - Display the CFML engine in use when using the simple reporter
• [TESTBOX-198] - Add support to fit, fdescribe, fscenario,fwhen, fgiven, fstory, ffeature, fthen

Improvements

• [TESTBOX-254] - check if fail origin is an array in case internal TestBox borks out in all the reporters
• [TESTBOX-259] - Turn off code coverage when clicking a link in the HTML reporter
• [TESTBOX-260] - toHaveKey() now accepts a list of keys that MUST or MUST NOT exist in the collection

v3.0.0

TestBox 3.0.0 is a major release. It has compatibility changes that you should be aware and lots of good feaures!

Compatibility

The major compatibility issues are the engine support removals:

• Lucee 4.5 Support Dropped
• Adobe ColdFusion 10 Dropped

Updating

It is easy to update, just type update testbox and you are done!

Major Features

The most notable features of this release can be found below.

Code Coverage

This has been fully documented and you can find much more information in the code coverage section.

Static Test Visualizer

The static test visualizer is basically the simple reporter but works in offline mode. This means that it will read a static TestBox results json file and create the report for it in the browser. This is incredibly useful for CI integrations and representing any TestBox results json file visually.

Location

You will find the analyzer under /test-visualizer in the root of the TestBox installation.

/test-analyzer
  + index.html (visualizer)
  + test-results.json (sample test results)

Running It

To run it all you need to do is put alongside of it a test-results.json file and then run the index.html and voila! Test Results Visualized!

Release Notes

Bugs

• [TESTBOX-234] - bddrunner.cfm: now compiles properly on ACF
• [TESTBOX-248] - Skip methods for given/when/then fail without `this` reference

New Features

• [TESTBOX-236] - Add CodeCoverage Reporter to TestBox
• [TESTBOX-239] - Update the UI for the code coverage reporting and code visualizer
• [TESTBOX-243] - Complete UI updates for test reporters
• [TESTBOX-245] - Static Test Visualizer

Improvements

• [TESTBOX-237] - Update usage of htmleditformat to encodeForHTML
• [TESTBOX-242] - Removal of old cfml engines support acf10 and lucee 4.5
• [TESTBOX-244] - streamify the code coverage collection
• [TESTBOX-249] - Add original method name to mocking function so it can help in debugging

v2.8.0

What's New With 2.8.0

TestBox 2.8.0 is a minor release with some great new functionality and tons of fixes. You can find the release notes here and the major updates for this release.

Release Notes

Bugs

• [TESTBOX-224] - Bug on SimpleReporter name not complete
• [TESTBOX-231] - Recurse parameter not honored #71 on html runner
• [TESTBOX-232] - Junit reporter blows up on null values in server scope
• [TESTBOX-233] - Update Expectation.cfc to make toMatchWithCase consistent as it is not working

New Features

• [TESTBOX-225] - Add ability to exclude labels from running using url.excludes

Improvements

• [TESTBOX-230] - Incompatibilities with Adobe on many isValid and isInstanceOf

v2.7.0

What's New With 2.7.0

TestBox 2.7.0 is a minor release with some great new functionality and tons of fixes. You can find the release notes here and the major updates for this release.

Release Notes

Bugs

• [TESTBOX-217] - new version of Lucee has complex values in UDF implementation that breaks the mock generator
• [TESTBOX-222] - Heap issues and stack overflow issues when normalizing ORM entities in mocking arguments

New Features

• [TESTBOX-221] - Complete refactoring of around,before,after each executions to support concurrency

Improvements

• [TESTBOX-219] - some refactoring to use invoke() instead of evaluate()
• [TESTBOX-220] - some speed improvements by not using createuuid anymore