Why Stubs in Unit Testing Improve Integration Testing

A major and influential customer of ours, working on a safety-critical project being developed according to the IEC 61508 functional safety standard, contacted us. They requested guidance in optimizing their developers’ productivity by further reducing noise generated from unit testing—noise produced from lack of stubbing.

We learned that the way our customer was performing their unit testing was closer to integration testing. In their process, the units to be tested were not isolated from their dependent components (other files or functions in the project), and the unit test cases were executed against most of the completed applications.

Such an approach is not classic unit testing. It’s commonly referred to as integration level testing. Integration testing is very efficient in demonstrating good test coverage for functional and nonfunctional requirements. It can also provide excellent test coverage if structural code coverage is enabled.

What Are Stubs in Testing?

Unit testing is more about isolating the function, method, or procedure, otherwise referred to as a unit. This isolation is done by stubbing out dependencies and forcing specific paths of execution.

Stubs take the place of the code in the unit that is dependent on code outside the unit. It also provides the developer or tester with the ability to manipulate the stub response or result so that the unit can be exercised in various ways and for various purposes, for example, to ensure that the unit performs reliably, is safe, and, in some cases, is also free from security vulnerabilities.

Why Are Stubs Used?

The best way to explain why stubs are used and the value they bring is by stepping through a use case. Take a look at the following code, for example.

At the beginning of the function, there’s an if statement that tests whether the buffer for samples was successfully allocated. Most of the test cases for this function were implemented without any stubs, as they are focused on the regular control flow, except the test case, which checks the behavior of the function when buffer allocation fails. This test case requires a stub for the allocateSampleBuffer function to simulate the failure.

Once the stub is added, it will be consistently applied to the tested code. A user working on the “allocation failure” test case will have an easy way to install a special callback function into the stub, which will simulate the desired effect: allocation failure or do nothing since by default the stub returns a null pointer, which is expected for the test case. But all other test cases require attention right now because a stub configuration has to be added for them to avoid unwanted changes in the control flow.

Of course, developers can go back and reconfigure their test case to account for the stub, but it means extra time spent analyzing the reason for failure, preparing a dedicated callback function for the stub, and removing the noise in the testing process, which was the customer’s main concern when they contacted us.

Using Stubs in Unit Testing C & C++ Code

Parasoft C/C++test makes it much easier to automatically generate stubs or manually create stubs. The option is available in two places:

For automatically generated stubs: Test Configuration -> Execution -> Symbols (tab)

For user stubs and auto stubs: Stub settings panel of the Stubs View

With the “Insert call to original function” option checked, Parasoft C/C++test changes the default way the stubs are generated. The change is in a stub behavior when no call-back is installed. The stub generated with the new option will act as a proxy and call the original function definition unless the user provides a test case specific callback function that is meant to perform alternative activities.

Stubs generated without the new option, including legacy stubs, will not try to call the original symbol in the default situation. If there’s no test case specific callback function installed, the stub will do nothing and just return a default value, such as a null pointer or zero numerical value.

When to Use Stubs

To make sure the difference is clear, let’s compare the situation with the “Insert call to original function” option enabled and not enabled for the case when the user did not provide a dedicated callback.

Here’s a stub generated for goo function without the “Insert call to original function” option. It works in the following way:

Here’s how it looks for the same stub generated for goo function with the “Insert call to original function” option checked:

As you can see, stubs added with the new option are transparent for the tested code. They simply perform the proxy call to the original definition, unless someone provides a callback that implements the desired alternative action.

Differences Between Stubs, Mocks, Spies, Drivers, & Dummies

There are various test doubles that you can apply to unit test cases when testing software. In real-time embedded testing of C and C++ code, as shown in this blog post, teams use stubs and mocks as test double mechanisms.

For testing cloud and web applications that use Java, C#, VB.NET, or other languages, teams can apply many types of test doubles including stubs, mocks, spies, dummies, and drivers.

• Stubs are small pieces of code that take the place of another component during testing. The benefit of using a stub is that it returns canned results and answers, making the test easier to write. You can run your tests even if the other component isn’t working yet.
• Mocks are a piece of code with programmed behavior that takes the place of another code component during testing. A mock is a smarter stub. Mocks, in a way, are determined at runtime since the code that sets the expectations must run before it does anything.
• Spies, sometimes referred to as partial mocks, record information based on how it was called. For example, an email service that logs the number of messages it was sent.
• Dummies are objects passed around but never used. They help fill in mandatory parameters, like in a constructor, but have no effect on your test.
• Drivers are complex in implementation and used when the software needs to interact with an external system. They’re most commonly used for integration testing.

What Are Stub Limitations & Challenges

In instances where there’s not an original definition available for a stubbed function, what happens? How does the stub behave without a callback that defines the alternative behavior?

The beauty of C/C++test is that it automatically detects this kind of situation. The stub will reconfigure itself during the test harness build time. It won’t call the original definition when no callback is installed and will return a safe default value.

Stubs in Integration Testing: An Example & Wrap Up

C/C++test highly reduces the amount of interference between different team members when working concurrently on the test cases in this kind of semi-integration testing. A stub added by developer A will not change the behavior of the tested code for the test cases added by developer B.

If developer B decides they need to configure an alternative action for the stubbed function for one of the test cases, they can create a test case specific callback function that implements desired alternative logic for the stubbed function and install this callback in the existing stub as a part of test case configuration.