How do I evaluate the offer and quality of a pentest provider?

15. June 2023

Nowadays, increasing numbers of security standards and norms require companies to conduct regular penetration tests (pentests for short). For example, PCI DSS, DORA regulation, the Supervisory Requirements for IT in Financial Institutions (BAIT) and other requirements of the German Federal Financial Supervisory Authority (BaFin) such as the KAIT. However, selecting the appropriate partner to perform these security analyses regularly poses a challenge to buyers. Pentest providers are a dime a dozen, and many of them make the same promise at first glance: To check IT systems or applications for vulnerabilities and thus contribute to increasing the level of IT security - but at very different price models.

In our article, we present you key criteria that can help you compare providers and make an well-informed decision:

1. Service portfolio

The company's internal IT infrastructure often consists of several components. The pentest provider should therefore be able to cover an in-depth analysis of all relevant areas of your IT infrastructure. These can be, for example:

  • Pentest of internal and external networks and servers
  • Workstations (e.g. laptops)
  • SAP infrastructures
  • Web applications (e.g. online stores)
  • Web services/APIs (e.g. JSON web interfaces)
  • Mobile applications (e.g. iOS and Android apps)
  • Native server/desktop applications (e.g. Windows applications)
  • Cloud infrastructures (e.g., AWS, Azure)
  • Mainframes (mainframes)
  • Single sign-on solutions

For comprehensive support in the context of cyber security, the vendor should be able to support you with red teaming, code reviews, cloud security audits and forensics services in addition to pentests. This way, you can ensure a comprehensive review of all assets from a single source.

2. Definition of the right scope of testing

To ensure that the analysis is optimally tailored to your company, it is important that your scope is individually defined in advance during a consultation. This should take into account the protection needs of your assets, possible risks of compromise, the time available and possible compliance requirements. This is the only way to address the risk directly. Therefore, you should not commission an "off-the-shelf" pentest.

3. Team size

A reliable delivery capability on the part of the pentest provider is especially important when a large number of existing systems and applications need to be analyzed or for time-critical projects. Therefore, when selecting your provider, make sure that they have a team large enough to meet these requirements. Thus, you can ensure that your most critical business processes are always analyzed comprehensively and in a timely manner. 

4. Individuality despite standardization

Standardized procedures, including appropriate quality assurance, should ensure consistently high-quality results - regardless of the security analyst performing the analysis. This approach should be based on internationally recognized standards, best practices and experience, but should also be extendable to meet your individual requirements.

Through the use of professional tools, checks of already known vulnerabilities can be automated. However, make sure that the main part of the penetration test is performed manually by the security analysts, as this is the only way to perform a targeted and comprehensive analysis. You should be wary of completely automated tests. This is usually only a security scan.

Tip: To get a better overview, ask the provider for detailed information about the procedure and tools used.

5. Experience, top expertise and continuous training

The security analysts who perform a penetration test for you should have excellent training, extensive experience with the technologies used, and specific industry knowledge. You can recognize this, among other things, by internationally recognized certifications, such as:

  • Offensive Security Certified Professional (OSCP)
  • Offensive Security Certified Expert (OSCE)
  • SANS SEC660: Advanced Penetration Testing, Exploit Writing, and Ethical Hacking.

If you are looking for a pentest provider for a public company, the security analysts performing the penetration test should also be certified in accordance with theSecurity Clearance Check Act (§ 9 SÜG).

In the field of IT security, it is essential to always be up to date on new vulnerabilities, attack vectors and possible patches. Therefore, the pentest provider should constantly continue to enhance itself beyond its core business and support continuous training of the security analysts. This includes participation in national and international conferences as well as publications of previously unknown, so-called zero-day vulnerabilities.

6. Independence

Make sure that the pentest provider operates as an independent consultant and that its focus is not on selling specific products. Check whether there are already testimonials from other customers. Information on this should be available on the provider's website or upon request.

7. Optimal support during the entire project cycle

The preparation and follow-up of penetration tests can be a complex project. To ensure that the penetration test is performed successfully, it is also necessary to obtain a large amount of relevant information and documents in advance, such as the required access and user accounts. Therefore, make sure that the pentest provider can support you before, during and after the pentest. Especially when analyzing a large number of systems and applications, thorough support during preparation is extremely helpful.

Ideally, the pentest provider can also support you throughout the entire project cycle - from identifying IT assets that require protection, to setting up and operating information security management systems (ISMS), to providing consultation and support in the follow-up of your final pentest reports.

Tip: Ask about the approach to implementing the pentest project as well as interface communication and ask about the vendor's implementation and project management experience.

8. Final pentest report

The final pentest report is a fundamental part of the pentest performance, as it provides you with information about your current IT security level and indicates possible improvements. Thus, you should get a comprehensive result report after the pentest. Here, the identified vulnerabilities should be evaluated according to internationally recognized metrics, such as the Common Vulnerability Scoring System (CVSS), giving you an estimation of the risk that they can be exploited by hackers. In addition, the results report should include detailed recommendations for corrective measures in order to sustainably increase your IT security level and minimize your risks.

Ideally, the pentest provider will give you an additional overview of which attack vectors were tested in connection with which functions and with what results - even if no vulnerability was identified there. Thus, you have more transparency, you can evaluate the quality of the analysis, and you meet compliance requirements for your pentest report.

Tip: Ask for a sample report to verify the above points.

9. Option of proof of concept pentests

If your organization requires a large number of penetration tests per year, it is reasonable to use permanent service providers to perform penetration tests. The above mentioned criterias may help you to convert your longlist into a shortlist. Nevertheless, a real assessment of the quality is only possible to a limited extent. In order to be able to select the suitable service provider from your shortlist, it is helpful to have an insight into the way they work, for example through a "proof of concept pentest" (a kind of project demonstration) of a specific test environment. This allows you to evaluate and compare the quality of the pentests provided by several vendors.


Our experienced security analysts will find vulnerabilities in your IT systems and provide you with in-depth support throughout the entire pentest project. Contact us, we will be happy to support you and advise you on your options.

Also interesting:

DORA Countdown: One Month Left Until the Deadline

DORA Countdown: One Month Left Until the Deadline

DORA, the Digital Operational Resilience Act, will fully apply as of 17 January 2025. We have summarized everything you need to know about the EU regulation, preparation and best practices from our news blog.

Sunset of PCI DSS v4.0 on 31 December 2024: Get Ready!

Sunset of PCI DSS v4.0 on 31 December 2024: Get Ready!

PCI DSS v4.0: In March 2024, version 4.0 of the Payment Card Industry Data Security Standard became mandatory after a two-year transition phase. Just a few months later, version 4.0.1 was released as a minor update of the standard, which will become mandatory on...

Top 3 Vulnerabilities in SSO Pentests

Top 3 Vulnerabilities in SSO Pentests

During their penetration tests (pentests), our security analysts at usd HeroLab repeatedly uncover vulnerabilities that pose significant risks to corporate security. They increasingly encounter the same vulnerabilities. Our blog series "Top 3 Vulnerabilities" presents...

Categories

Categories