How to Make API Security Testing an Automated Part of the CI Process

Since penetration testing is expensive and can take a long time to run, we must perform API security testing in a way that is scalable and sustainable.

Discussing API security and why we should care is a little bit like talking about eating our vegetables. We all know that eating our vegetables is good for our health, but how many of us actually do it? Application security is a little bit like that. While it’s essential for the health of our applications and our businesses, striving for it isn’t nearly as interesting as building cool new application features. But we only have to look at recent news headlines to understand how important it is.

Traditionally, validating an application or API for security has been done at the end of the development process. This is inherently problematic, though. It’s usually too late in the process for discovered errors to be fixed: it may be too close to the release date to fix the problems, or the team might have moved on to other projects, or the architecture of the application might be inherently insecure.

In addition, services and applications today are released more often than ever, often releasing up to multiple times a day. This fast release cadence makes the traditional approach untenable. In this blog I’ll review a step-by-step API security checklist to make API security testing an automated part of the CI process.

Enter Continuous Integration

To solve this problem, we will turn to a solution that the industry has been using to tackle software quality problems with accelerated release cycles—continuous integration. Continuous integration produces builds whenever new code is checked in and validates the new code by running static analysis and unit tests for each build. If teams are sophisticated, they might even be creating and running automated functional tests using CI (perhaps not for every build, since functional tests typically take a long time to run, but at least at specified intervals like once a day).

We can apply this same solution to automated security testing for our APIs by bringing penetration testing into our CI workflows. This will ensure that we test for security vulnerabilities sooner, and it will give us security regression tests that can catch new problems as soon as they are introduced. But we will need to be smart about it since penetration testing is expensive and can take a long time to run. We must do it in a way that is scalable and sustainable.

Start With Functional Tests

I am assuming that our teams are already writing and running automated functional tests for our APIs. (If we are not doing this, we need to start here and are not ready to consider automating our security testing.) If we are running automated functional tests for our APIs, then as part of our normal development and QA processes, we can identify a subset of those functional tests to use as security tests. We will prepare and run this subset as security tests.

Let me describe how this works using Parasoft SOAtest and its built-in penetration testing support, which is included in the 2021.2 release.

To start, let’s assume we have a SOAtest scenario with 1 setup test that cleans the database, and 3 tests that make 3 different API calls. We want to perform penetration testing for each of the 3 APIs that are being called in the scenario:

We will first prepare the scenario for security by adding a Penetration Testing Tool to each of the tests in the scenario:

We will then execute this scenario using SOAtest. As each test executes, SOAtest will make the API call defined in the test and capture the request and response traffic. The Penetration Testing Tool on each test will pass the traffic data to an embedded instance of the OWASP ZAP penetration testing tool, which will perform penetration testing on the API based on the API parameters it observes in the traffic data, using its own heuristics.

The Penetration Testing Tool will then report any errors found associated with the test that accessed the API. Here’s a sample SOAtest report with all errors organized by CWE and by severity:

SOAtest results can be further reported into DTP, Parasoft’s reporting and analytics dashboard, for additional reporting capabilities. Here’s a representation of how this works:

Repurposing functional tests for use as security tests gives the following benefits:

1. Since we are already writing functional tests, we can reuse work that has already been done, saving time and effort.
2. To execute certain APIs, we might have to do some setup, like prepping the database or calling other APIs. If we start with functional tests that already work, this setup is already done.
3. Typically, a penetration testing tool will report that a certain API call has a vulnerability, but it doesn’t give any context about the use case and/or requirement to which it is connected. Since we are using SOAtest to execute the test cases, the security vulnerabilities are reported in the context of a use case. When scenarios have been associated with requirements, we now can get additional business context about the impact of the security errors to the application.
4. We have a test scenario that we can use to easily reproduce the error or to validate that it has been fixed.

Preparing Functional Tests for Use as Security Tests

There are a few things to consider when repurposing functional tests for use as penetration tests:

1. We should maintain our functional test scenarios separately from our security test scenarios and run them from separate test jobs. The main reason for this is that adding penetration testing to existing functional tests will likely serve to destabilize the functional tests. We need to select which functional test scenarios should be turned into automated security tests, and then make copies of the functional tests that will be maintained as separate security tests.
2. We need to be selective in which tests we choose since penetration testing is expensive; we need to maximize the attack surface of the API that is covered while minimizing the number of tests. We should consider the following:
   • Penetration testing tools analyze request/response traffic to understand which parameters in the request are available to be tested. We need to select functional tests that exercise all the parameters in each API, to ensure that every input to the API gets analyzed.
   • Within each scenario, we need to decide which API calls should be penetration tested. The same API may be referenced from multiple scenarios, and we don’t want to duplicate penetration testing on an API that is being tested in a different scenario. The Penetration Testing Tool should only get added to the appropriate tests for the API(s) to be penetration tested.
   • The number of scenarios needs to be manageable so that the security test run is short enough to run at least once a day.

3. Our functional test scenarios may have setup or teardown sections for initialization or cleanup. These typically don’t need to be penetration tested.
4. If the functional test has any parameterization, we should remove it. Penetration testing tools don’t need multiple sets of values for the same parameters to know what to test and sending different sets of values could just lead to making the test runs go longer due to duplicated testing.
5. API functional tests will usually have assertions that validate the response from the service. When used as security tests, these assertions can fail but will be noisy when reviewing the results, since in this context we only care about the security vulnerabilities that were found. We should remove all assertions. In my previous example, this would mean removing the JSON Assertor from Test 3.
6. Some API calls add data to the database. When using a penetration testing tool against such APIs, the database can get bloated with information due to the number of attacks that the penetration testing tool directs at the API. In some cases, this can cause unexpected side effects. On one of our development teams, we discovered a performance issue in the application when a particular API added lots of data due to the penetration test attacks. The application performance became so bad that it prevented the automated security test run from finishing in a reasonable amount of time. We had to exclude the security tests for that API from our automated run until we had fixed the problem.

Maintaining a Stable Test Environment

We need to consider whether to run our functional and security tests within the same test environment or a different one. Resetting the environment between the functional and security test runs, or using a separate environment, promotes better test stability but is usually not necessary. We can often reuse the same environment, but when we do, we should run the functional tests first and the security tests last, since the security tests can destabilize the environment for the functional tests. When we use different environments, we need to make sure that we configure the original functional test scenarios with variables so that it is easy to point the tests at different endpoints for different environments. SOAtest supports this using environment variables.

Our APIs may also depend on other APIs outside our control. We can consider using service virtualization to isolate our environment so we don’t depend on those external systems. This will help to stabilize our tests while at the same time preventing unintended consequences to the external systems due to our penetration testing efforts.

Conclusion

We can ensure better quality in our APIs by moving security testing into development and QA as part of an automated process. We can leverage our existing API functional tests to create automated security tests, which will allow us to discover and fix security errors earlier in the process. And hopefully, this will help us not become one of the next big headlines in the news.