As the number of cyber attacks continues to rise, effective security analysis is more crucial than ever in protecting software and data. In addition, compliance requirements are becoming more stringent to defend against growing risks and require regular specialized security testing.
It can be challenging to determine the best fit for your needs, considering the variety of available approaches. On the one hand, pentests provide an effective method of simulating real-world threats to find vulnerabilities in running applications. On the other hand, security code reviews offer a preventive measure by enabling an examination of the source code for security flaws that could be exploited in future attacks. But it does not have to be one or the other. A whitebox pentest combines the thoroughness of a code review with the real-world attack simulation of a pentest to achieve an even more comprehensive security assessment.
This blog post will introduce the three approaches of Code Review, Greybox Pentest and Whitebox Pentest, explore their differences and similarities, highlight their benefits, and discuss which method may be best for your security goals.
Security Code Reviews
In a code review usually static code analysis is performed. In contrast to other security assessments that involve testing a running application, a code review is a measure in which an analyst examines source code without executing it. It is therefore the most prevalent form of static application security testing (SAST). To achieve comprehensive test coverage, analysts combine automated tools for static code scanning with dependency analysis and a meticulous manual inspection of the source code for potential vulnerabilities.
One of the key benefits of a code review is the fact that it can be implemented early in the software development lifecycle. Therefore, applications can be tested even before they are fully built or deployed. This proactive approach enables analysts to identify and fix vulnerabilities before they reach production. In addition, analysts can identify vulnerabilities that are often harder to find in a running application, such as business logic flaws, hardcoded secrets, or issues with cryptography. Finally, a code review allows for the precise documentation of findings by giving the vulnerable code-snippet and providing highly specific recommendations. For example, security experts can highlight specific lines of code that are vulnerable or suggest more secure alternatives for functions prone to security risks.
However, security code reviews also come with their own set of challenges. The process can be resource-intensive and expensive, as it requires considerable time and expertise to perform a thorough review. This is especially true for large or complex codebases. Furthermore, because code reviews are a static process, they are not suited to identify issues with runtime behavior or configuration vulnerabilities that may only manifest in a live environment. For example, issues related to server configurations, network security, or runtime interactions cannot be uncovered through a code review alone.
Greybox Pentests
Pentests simulate real-world threats in order to identify vulnerabilities before they can be exploited by actual attackers. In contrast to code reviews, in pentests, analysts dynamically test an application during runtime, following the dynamic application security testing (DAST) methodology. Therefore, pentests are particularly well-suited for finding vulnerabilities connected to runtime behavior, such as problems with session management or issues with the interaction between various components of the application.
In a greybox setting, analysts are provided with partial knowledge of the application. This may include documentation, information about the application's architecture, and access to certain components or log files. Typically, exceptions are configured in upstream protection mechanisms like a WAF or IPS and the pentesters are granted access to accounts to perform authenticated tests. However, the source code is not provided and information about the architecture and other components may be incomplete. This approach effectively simulates an attacker with inside knowledge, for example a former employee or an account compromised by a phishing attack.
One of the biggest advantages of a greybox pentest is its time efficiency. The information provided enables the analysts to achieve a high test coverage within a reasonable timeframe. Furthermore, this testing method is also applicable if you use a third-party application, and therefore cannot access the source code.
While greybox pentests offer a dynamic and quite realistic security assessment, they do have their limitations. Because analysts have only partial access and information, they may not be able to uncover all security flaws. Especially vulnerabilities deeply embedded in the application’s source code are hard or impossible to find this way. Although such vulnerabilities are also more difficult for attackers to find, they remain identifiable and exploitable given enough time and resources. However, a greybox pentest can always be enhanced by providing analysts with further information. At some point, the greybox approach then transitions into a whitebox scenario.
Whitebox Pentests
The whitebox approach combines pentesting and security code review to provide a comprehensive security evaluation. The design of a whitebox pentest can range from a greybox pentest with added code access to a complete combination of pentesting and full code review.
The biggest advantages of the whitebox approach are its increased test depth and efficiency. A whitebox pentest can usually be completed within the same time as the corresponding greybox pentest, while achieving greater test depth. For example, if an endpoint of a web application appears to be vulnerable to SQL injection, a quick look in the corresponding implementation helps to verify the vulnerability. Furthermore, creating a payload that demonstrates the impact is much more time efficient if the routines handling the input in the backend are known to the analyst.
In contrast to a stand-alone code review, analysts also consider the environment in which an application is deployed in a whitebox pentest. While conducting code reviews, we oftentimes receive the feedback that a vulnerability is not exploitable in production due to the architecture of the actual environment the asset is deployed in. Therefore, we recommend using a production-like deployment for the whitebox pentest. Each vulnerability identified in the code can then be tested against the running instance to accurately estimate the likelihood of exploitation and the impact.
The final output of every pentest is the report. It documents the findings of the analysts in a way that helps developers understand the issues, and gives guidelines on how to fix it. The final report, too, can benefit from analysts having access to the source code during analysis: Analysts can enhance their findings with a demonstration of the exploit alongside the vulnerable code-snippet, highlighting the underlying issue in the application’s code.
If a code review is required, e.g., for compliance reasons, the whitebox approach can be designed as a code review enhanced by giving the analysts access to the running application. This enables the code reviewers to use runtime information to quickly confirm or dismiss their assumptions about security flaws in the code. In addition, the results of the code review are complemented with vulnerabilities identified in the live environment, and the documentation of vulnerabilities becomes clearer, for example, through screenshots of their exploitation. The timeframe for such a test is likely to be identical to the timeframe of a standalone code review.
Conclusion
The whitebox approach offers the most comprehensive security coverage available. It is flexible in its design, allowing analysts to focus either on the running application or the source code. Even if testing time is limited, a greybox pentest can be significantly enhanced by giving the analysts access to the source code.
The following table summarizes the three scenarios for quick review:
| Time Efficiency | Test Depth | Absolute Time (Costs) | Ideal Use Case | |
| Code Review | Low The review of the entire codebase can take a long time. | High Thorough inspection of all code paths, logic, input validation, etc. | High Can be very time-consuming, especially with large or complex codebases. | Early-stage development or if the goal is to address code-level issues. |
| Greybox Pentest | Medium Using only dynamic testing is faster than also reviewing the code, but access to the code could make the process more efficient. | Medium Dynamic testing with some internal knowledge. May miss issues that require deep code understanding, such as business logic flaws or misconfigurations hidden in the code. | Low Typically the fastest, since no code review is involved and the testing is focused on runtime vulnerabilities using available internal knowledge. | Fast and cost-effective testing of applications with normal protection needs or if source code is not available. |
| Whitebox Pentest | High Code review and pentest render each other more efficient. | Very High Combines both static and dynamic testing, covering vulnerabilities of both the code and the runtime behavior. Depending on the goal, the focus can be more on dynamically testing the application or statically reviewing the source code. | Depends on the design The time (and cost) can range fluidly from that of a greybox pentest, with time spent on targeted validation using code to verify vulnerabilities, to almost the sum of a standalone code review and pentest. | Applications with high protection needs. Depending on the design suited for both pentest-driven and code-review-driven assessments. |
Are you looking to carry out one of the above security analyses in your company or do you have any questions? Contact us, we will be happy to assist you.
