Skip to content

Latest commit

 

History

History
745 lines (575 loc) · 26.1 KB

FAQ.md

File metadata and controls

745 lines (575 loc) · 26.1 KB
Raw

FAQ

Q. Why a name that can neither be pronounced nor spelled?

A. It's a parallel to arts. Trompe-l'œil, which literally means "trick the eye," refers to an art form where the artist creates something that tricks the viewer into thinking they see something other than what is there. Writing mocks for testing has resemblances to creating Trompe-l'œil art, in that you create mocks that "tricks" the test object as if it was interacting with the intended real world. When you use mocks in a test program, you are the Trompe-l'œil artist, tricking the code under test.

Perhaps Illusionist or Puppeteer would have sufficed as names, but they were taken many times over for other projects, and besides, the author has a soft spot for Trompe-l'œil art.

If you really cannot handle the name, you can use the following renaming mechanism. Assume that you'd like the name chimera instead.

Create a file chimera.hpp with the following contents:

#ifndef CHIMERA_HPP
#define CHIMERA_HPP

#include <trompeloeil.hpp>
namespace chimera = trompeloeil;

#endif /* include guard */

Your tests can now #include <chimera.hpp> and use (for example) chimera::expectation and chimera::deathwatched<T>.

Q. Which compilers supports Trompeloeil?

A. Trompeloeil is known to work well with:

Trompeloeil is known to work somewhat with g++ 4.8.4 and 4.8.5, and somewhat less with g++ 4.8.3. g++ 4.8.x only compiles with a C++11 dialect (e.g. -std=c++11). For details, see "G++ 4.8.x limitations".

Q. How do I use Trompeloeil with XXX unit test framework?

A. By default, Trompeloeil reports violations by throwing an exception, explaining the problem in the what() string.

Depending on your test frame work and your runtime environment, this may, or may not, suffice.

Trompeloeil offers support for adaptation to any test frame work. Adaptation examples for some popular unit test frame works are listed in the Cook Book.

Q. Is Trompeloeil thread-safe?

A. Yes, with caveats.

In a unit test you don't want to depend on the scheduler, which is typically out of your control. However, some times it is convenient to use a unit test like environment to exercise a larger aspect of your code. In this setting, using mock objects with different expectations can make sense when statistically searching for synchronization problems.

To enable this, Trompeloeil uses a global recursive_mutex which protects expectations.

Expectations can come and go in different threads, and mock functions can be called in different threads, all protected by the global lock. However, it is essential that the mock object is not deleted while establishing the expectation or calling the mock function, as per normal thread-safety diligence.

Should you need to access the lock in your tests, you can do so with

  auto lock = trompeloeil::get_lock();

lock holds the recursive_mutex until it goes out of scope.

Q. Can a mock function be marked override?

A. Yes, just add override a third parameter to MAKE_MOCKn() or MAKE_CONST_MOCKn()

Example:

class Interface
{
public:
  virtual ~Interface() = default;
  virtual int func1(int) = 0;
};

class Mock : public Interface
{
public:
  MAKE_MOCK1(func1, int(int), override); // overridden
  MAKE_MOCK1(func2, int(int));           // not overridden
};

Q. Why can't I .RETURN() a reference?

A. You can, but the language is a bit peculiar.

For parameters or returned references from function calls, just use .RETURN(value). For local variables you need .LR_RETURN(), and for both global and local variables you either need to use std::ref(value) or std::cref(value) for it, or just enclose the value in an extra parenthesis, like this .LR_RETURN((value))

Example:

class C
{
public:
  MAKE_MOCK1(lookup, std::string&(int));
};

using trompeloeil::_;
using trompeloeil::lt;

TEST(some_test)
{
  C mock_obj;

  std::map<int, std::string> dictionary{ {...} };

  std::string default_string;

  ALLOW_CALL(mock_obj, lookup(_))
    .LR_RETURN(dictionary.at(_1)); // function call

  ALLOW_CALL(mock_obj, lookup(trompeloeil::lt(0)))
    .LR_RETURN((default_string)); // extra parenthesis

  ALLOW_CALL(mock_obj, lookup(0))
    .LR_RETURN(std::ref(default_string));

  test_func(&mock_obj);
}

Above, the expectations on function lookup() is that any call is allowed and will return an lvalue-reference to either a match in dictionary, or to the local variable default_string. The reference is non-const, so test_func() is allowed to change the returned string.

Q. Why can't I change a local variable in .SIDE_EFFECT()?

A. It would almost certainly be very confusing. All local variables referenced in .WITH(), .SIDE_EFFECT(), .RETURN() and .THROW() are captured by value, i.e. each such clause has its own copy of the local variable. If you could change it, it would change the value in that clause only and not in any of the others.

Example:

class C
{
public:
  MAKE_MOCK1(func, void(int));
};

using trompeloeil::_;

TEST(some_test)
{
  C mock_obj;

  unsigned abs_sum = 0;

  ALLOW_CALL(mock_obj, func(trompeloeil::gt(0)))
    .SIDE_EFFECT(abs_sum+= _1); // illegal code!

  ALLOW_CALL(mock_obj, func(trompeloeil::lt(0))
    .SIDE_EFFECT(abs_sum-= _1); // illegal code!

  ALLOW_CALL(mock_obj, func(0));

  test_func(&mock_obj);

The two SIDE_EFFECT() clauses above each have their own copy of the local variable abs_sum. Allowing them to update their own copies would be very confusing, and it would also be difficult to get the value back to the test.

If you need to change the value of a local variable it is better to use the alternative "local reference" forms LR_SIDE_EFFECT(), LR_WITH(), LR_RETURN() or LR_THROW().

Q. Why the "local reference" .LR_*() variants? Why not always capture by reference?

A. It's safer. Lifetime management can be tricky in C++, and even more so when complex functionality is hiding behind hideous macros in a frame work. Experiences from the alpha phase, where this distinction wasn't made, made the problem glaringly obvious. Making the default safe, and providing the option to very visibly use the potentially unsafe, is considerably better, although it makes the test code somewhat visually unpleasant.

Q. Is it possible to allow all calls to all mocked functions for all mock objects?

A. No, it is not. There are two reasons for this, technical and philosophical.

Technical There is a problem with the return value. It is difficult, if at all possible, to come up with a generic return that works for all types. This could be overcome by allowing all calls to all functions with a certain return type, for all objects.

Philosophical While there are no doubt situations where this would be convenient, it could be a very dangerous convenience that opens up for relaxing tests unnecessarily, simply because it's so easy to allow everything, and then when you introduce a bug, you never notice because everything is allowed. If a safe way of allowing all calls is thought of, then this may change, but having a perhaps unnecessarily strict rule that can be relaxed is safer than the alternative.

Q. Why are parameters referenced by position and not by name?

A. If you can figure out a way to refer to parameters by name, please open an issue discussing the idea. If you can provide a pull request, so much the better.

Q. Why the need to provide the number of parameters in MAKE_MOCKn() when all information is in the signature?

A. If you can figure out a way to infer the information necessary to generate a mocked implementation without an explicit parameter count, please open an issue discussing the idea. If you can provide a pull request, so much the better.

Q. Why C++14 and not C++11 or C++03 that is more widely spread?

A. C++03 and older is completely out. The functionality needed for Trompeloeil isn't there. Lambdas and variadic templates are absolutely necessary.

The only thing "needed" that C++11 doesn't provide is generic lambdas. It is perhaps possible that "needed" is too strong a word, that it is in fact possible without them, in which case a back port to C++11 could be made.

And indeed, since this FAQ question was first answered, a back port of a useful subset of Trompeloeil has been completed for use with C++11. For details, see "Backward compatibility with earlier versions of C++".

Q. Why are my parameter values printed as hexadecimal dumps in violation reports?

A. By default Trompeloeil prints parameter values using the stream insertion operators for the type, but if none exists, it presents a hexadecimal dump of the memory occupied by the value.

You can change that either by providing a stream insertion operator for your type, or by providing a custom formatter for it.

Q. Can I mock a C function API?

A. Trompeloeil can mock member functions only. However, there are tricks you can use to mock a function API, provided that it is OK to use a link seam and link your test program with a special test implementation of the API that calls mocks. Here's an example:

/* c_api.h */
#ifdef __cplusplus
extern "C" {
#endif

int func1(const char*);
const char* func2(int);

#ifdef __cplusplus
}
#endif

With the above C-API mocks can be made:

/* mock_c_api.h */
#ifndef MOCK_C_API_H
#define MOCK_C_API_H

#include <c_api.h>
#include <cassert>
#include <string>
#include <trompeloeil.hpp>

struct mock_api
{
  static mock_api*& instance() { static mock_api* obj = nullptr; return obj; }
  mock_api() { assert(instance() == nullptr); instance() = this; }
  ~mock_api() { assert(instance() == this); instance() = nullptr; }
  mock_api(const mock_api&) = delete;
  mock_api& operator=(const mock_api&) = delete;

  MAKE_CONST_MOCK1(func1, int(std::string)); // strings are easier to deal with
  MAKE_CONST_MOCK1(func2, const char*(int));
};

endif /* include guard */

Note that the mock constructor stores a globally available pointer to the instance, and the destructor clears it.

With the mock available the test version the C-API can easily be implemented:

#include "mock_c_api.h"

int func1(const char* str)
{
  auto obj = mock_api::instance();
  assert(obj);
  return obj->func1(str); // creates a std::string
}

const char* func2(int value)
{
  auto obj = mock_api::instance();
  assert(obj);
  return obj->func2(value);
}

Now your tests becomes simple:

#include "mock_c_api.h"
#include "my_obj.h"
TEST("my obj calls func1 with empty string when poked")
{
  mock_api api;
  my_obj tested;
  {
    REQUIRE_CALL(api, func1(""))
      .RETURN(0);
    tested.poke(0);
  }
}

Q. Can I match a value pointed to by a pointer parameter?

A. You can always match with _ and use LR_WITH() or WITH() using whatever logic you like. But by using matchers you can match the value pointed to using unary operator * on the matcher.

See Matching pointers to values in the Cook Book.

Q. Can I negate the effect of a matcher?

A. You can always match with _ and use LR_WITH() or WITH() using whatever logic you like. But by using matchers you can negate the effect of the matcher, allowing what the matcher disallows and vice versa, using operator ! on the matcher.

See Matching the opposite of a matcher in the Cook Book.

Q. Can I check if an expectation is fulfilled?

Yes, if you use NAMED_ALLOW_CALL(...), NAMED_REQUIRE_CALL(...) or NAMED_FORBID_CALL(...), then you can ask is_satisfied() and is_saturated(). Example:

TEST("something")
{
  mock_obj mock;
  auto ptr = NAMED_REQUIRE_CALL(mock, some_func())
               .TIMES(2,4);
  ...
  if (ptr->is_satisfied()) // at least two call have been made
  ...
  if (ptr->is_saturated()) // four calls have been made
}

Likewise you can ask sequence objects if the sequence they describe is_completed().

These are rarely useful in pure unit tests, but it can be useful for mini integration tests, especially when threading is involved.

Q. What does it mean to mix IN_SEQUENCE and TIMES?

A. Using .TIMES() with .IN_SEQUENCE() is confusing at best, and especially when you have a (possibly open) interval for .TIMES().

Trompeloeil always sees sequences as observed from a sequence object, and a sequence object can only move forward in what it allows.

Example:

trompeloeil::sequence seq;
REQUIRE_CALL(mock, foo1)
  .TIMES(AT_LEAST(1))
  .IN_SEQUENCE(seq);
REQUIRE_CALL(mock, foo2)
  .IN_SEQUENCE(seq);
REQUIRE_CALL(mock, foo3)
  .IN_SEQUENCE(seq);

// later...

mock.foo1();
mock.foo2();
mock.foo1(); // boom!
mock.foo3();

In the example above, a sequence violation is reported on the second call to mock.foo1(). It goes like this:

mock.foo1(); this is the first call for the sequence object, so it is allowed. It says AT_LEAST(1), so it may move to the next step, or it may repeat the same call.

mock.foo2(); The current step in the sequence is mock.foo1(), but it is satisfied, so moving on to the next one is allowed. The next one is mock.foo2(), which matches this call, so everything is good.

mock.foo1(); The current step in the sequence is mock.foo2(). Is is satisfied and saturated, so the sequence object must move to the next step. The next step is mock.foo3(), which is a mismatch, so a sequence violation is reported.

Q. How do I use Trompeloeil in a CMake project?

A. To use Trompeloeil in a project that is built with CMake, there are several options to make it accessible to CMake. (The commands below of for Linux, but it works similarly on other platforms.)

You can include Trompeloeil directly or use it from an installation, which can be per-project, per-user, or a system-wide installation.

If you aren't using Trompeloeil from an system-wide installed package, you can clone Trompeloeil in a subdirectory or add it as a Git submodule to your project (here, in ./my_proj/toolkits). For example:

git clone https://github.com/rollbear/trompeloeil.git my_proj/toolkits/trompeloeil

To include Trompeloeil in a project, add the subdirectory that contains Trompeloeil to the project's CMakeLists.txt (or any other CMakeLists.txt that gets processed before the one that defines the test that use Trompeloeil).

add_subdirectory(toolkits/trompeloeil)

After that, tests can be linked with Trompeloeil without needing find_package() first. For example:

add_executable( my_unit_tests
    test1.cpp
    test2.cpp
    test_main.cpp
)

target_link_libraries( my_unit_tests
    my_library_under_test # provided by an add_library() call elsewhere in your project

    # Nothing to link since both of these libs are header-only,
    # but this sets up the include path correctly too
    Catch2::Catch2
    trompeloeil::trompeloeil
)

# Optional: Use CTest to manage your tests
add_test( run_my_unit_tests my_unit_tests ) # May need to call enable_testing() elsewhere also

Please note that Trompeloeil will not be installed along your project in this case (i.e. when Trompeloeil is added as a subdirectory). In most cases, this is the desired behavior. This behavior can be overridden by setting TROMPELOEIL_INSTALL_TARGETS CMake variable to ON. e.g:

cmake -DTROMPELOEIL_INSTALL_TARGETS=ON -B build .
cmake --build build --target install

Adding Trompeloeil subdirectory and including it in a project's CMakeLists.txt is not the only option. Building, and installing it locally somewhere in your project is another option:

mkdir build ; cd build
cmake -G "Unix Makefiles" .. -DCMAKE_INSTALL_PREFIX=../../my_proj/toolkits
cmake --build . --target install

This will create a directory structure inside toolkits that has include/trompeloeil.hpp and the CMake find modules in lib/cmake/trompeloeil. Whether you add the entire Trompeloeil repo to your source control is up to you, but the minimal set of files for proper CMake support in is in the toolkits directory.

Alternatively, you could install it globally on your system by cloning the repo and installing with root privileges:

git clone https://github.com/rollbear/trompeloeil.git
cd trompeloeil
mkdir build ; cd build
cmake -G "Unix Makefiles" ..
sudo cmake --build . --target install

In either case, add a find_package() call in your project's CMakeLists.txt:

find_package( Catch2      REQUIRED HINTS "${toolkitsDir}/Catch2"      ) # Sample unit test framework
find_package( trompeloeil REQUIRED HINTS "${toolkitsDir}/trompeloeil" )

add_executable( my_unit_tests
    test1.cpp
    test2.cpp
    test_main.cpp
)

target_link_libraries( my_unit_tests
    my_library_under_test # provided by an add_library() call elsewhere in your project

    # Nothing to link since both of these libs are header-only,
    # but this sets up the include path correctly too
    Catch2::Catch2
    trompeloeil::trompeloeil
)

# Optional: Use CTest to manage your tests
add_test( run_my_unit_tests my_unit_tests ) # May need to call enable_testing() elsewhere also

This assumes that you have defined a variable called toolkitsDir pointing to my_proj/toolkits/lib/cmake, that you have Catch2 installed similarly, and that you have defined a target called my_library_under_test in other parts of the CMake files. (If you installed the libraries globally on your system, you should be able to drop the hints in the find_package() calls.)

Finally, you can add Trompeloeil to your project and then either (a) use CMake's find_file() to locate the header and add its path to include_directories(); or (b) use add_subdirectory() (one or two argument version) to add its path to your project.

If you want to specify a version of Trompeloeil, drop the 'v' from the version name in find_package. E.g. find_package( trompeloeil 39 EXACT ).

Q. Why are mock objects not move constructible?

A. Because a move is potentially dangerous in non-obvious ways. If a mock object is moved, the actions associated with an expectation (.WITH(), .SIDE_EFFECT(), .RETURN(), .THROW()) and their LR_ versions, are not moved. If they refer to data members stored in a moved mock object, they will refer to dead data. This is an accepted cost in normal C++ code, but since the effect is hidden under the macros, it is better to play safe.

With that said, you can explicitly make mock objects movable, if you want to. See: trompeloeil_movable_mock.

Q. Why can't I mock a function that returns a template?

Like this:

struct M
{
  MAKE_MOCK2(make, std::pair<int,int>(int,int));
};

A. You can, but there is a limitation in the preprocessor, that makes it work poorly with templates. It sees the parameters to the MAKE_MOCK2() macro above as make, std::pair<int, followed by int>(int,int), which of course is nonsense and causes compilation errors.

One easy way around this is to put the signature into parentheses:

struct M
{
  MAKE_MOCK2(make, (std::pair<int,int>(int,int)));
};

Or if you prefer the legacy way, create an alias:

using pair_int_int = std::pair<int,int>;

struct M
{
  MAKE_MOCK2(make, pair_int_int(int,int));
};

These work around the preprocessor parameter problem.

Another way, if you're mocking an interface, is to use trompeloeil::mock_interface<T> and IMPLEMENT_MOCKn. See CookBook for an intro.

Q. Can I mock a noexcept function?

A. Yes, but with a caveat.

The way to mock a noexcept function is to add a noexcept specifier to MAKE_MOCKn or MAKE_CONST_MOCKn.

struct S
{
    MAKE_MOCK1(func, void(int), noexcept);
    //                          ^^^^^^^^  noexcept function
};

The caveat is that the violation handlers, and specifically the default one, reports violations by throwing an exception, which means that any call made in violation of the expectation for a noexcept function leads to program termination. How much information you can gain from such an event depends on the runtime system of your tools.

Q. What does it mean that an expectation is "saturated"?

If you see a violation report like this:

[file/line unavailable]:0: FATAL ERROR: No match for call of func with signature void(int) with.
  param  _1 == -3

Matches saturated call requirement
  object.func(trompeloeil::_) at example.cpp:240

What this means is that there is an expectation for the call, but that expectation is no longer allowed to be called, its maximum call count has been met.

An example:

test_func()
{
    test_mock obj;
    REQUIRE_CALL(obj, func(trompeloeil::_))
      .TIMES(AT_MOST(2));

   exercise(obj); // calls obj.func. OK. Expectation is alive, no prior calls, this one is accepted
   exercise(obj); // calls obj.func. OK. Expectation is alive, one prior call, this one is accepted
   exercise(obj); // calls obj.func. Fail. Expectation is alive, two prior calls, this one saturated
}