Test Suites

Overview

Test Suites module is where you can create custom methodologies, checklists and service catalogues for your customers to pick from when requesting a project; or for you to assign to any new projects you create.

When a Test Suite is assigned to a project, the linked Test Cases will also be assigned to the project - so that the pentester or auditor has a checklist to work from.

A test suite helps:

• Clients understand exactly what was tested on the project

• Developers/Engineers link test cases to vulnerabilities

• Pentesters structure their testing in a methodical, consistent & standardized way

• Organizations create repeatable, standardized & comparable assessments - independent of who was actually performing the assessment

Test cases can provide valuable insight into a penetration test or audit.

Test cases demonstrate:

• What was tested

• How was it tested

• When was it tested

• Who tested it

• What was the outcome

• What is the supporting evidence

See Test Cases for more information on how test cases are used on projects.

My Test Suites

AttackForge comes pre-loaded with dozens of industry methodologies that you can select from, for any given project. The methodologies are gathered from OWASP, MITRE ATT&CK, OSSTMM, NIST and others.

Any new test suites that you create will show in My Test Suites. You can use the actions menu to Edit, Duplicate, Reorder or Delete any of these entries.

You can view the test cases linked to the test suites by clicking on the test suite name.

All test suites are shared and common. This means any entries you create can be used by your peers - pooling together your knowledge to save time & effort.

Creating A New Test Suite

To create a new test suite in the library, click on New button.

Once the form is submitted, the test suite will be immediately available to start assigning test cases.

To add a new test case, click on the test suite, then click on the Test Cases tab, then click on Add Test Case.

You can also search the library for an existing test case, this will pre-fill the fields for you to save you time & effort when you only want to make small changes to an existing test case.

The Code field is used to help with sorting & ordering test cases when displayed in projects & reports.

Code will appear before the details of the test case. For example: WEB-APP-001 Test for X, Y & Z; WEB-APP-002 Test for A, B & C; etc.

You can update or modify the test cases at any time by using the actions menu.

You can also update or modify the test suite at any time by using the page menu.

!IMPORTANT: updates to test cases in your library will apply globally to all projects which are referencing that test case.

!IMPORTANT: any test cases you add or delete on a test suite will not apply retrospectively to existing projects. This is to preserve integrity of what was actually assigned & tested on projects, and avoid situation where a project may be Completed and is now Not Completed as new test cases are assigned.

You can create Custom Fields on your Test Cases from the Administration module.

Importing Test Cases

You can import additional methodologies that AttackForge team has prepared and made available on our GitHub: https://github.com/AttackForge/TestSuites

Red Teaming

• MITRE ATT&CK Enterprise Version 14.1

• MITRE ATT&CK Mobile Version 14.1

• MITRE ATT&CK ICS Version 14.1

• OSSTMM Version 3 - Human Security Testing

• OSSTMM Version 3 - Physical Security Testing

Web Application & API

• OWASP Web Security Testing Guide Version 4.2

• OWASP Application Security Verification Standard (ASVS) Version 4 - Level 1

• OWASP Application Security Verification Standard (ASVS) Version 4 - Level 2

• OWASP Application Security Verification Standard (ASVS) Version 4 - Level 3

Mobile Application

• OWASP Mobile Application Security Testing Guide (MASTG) Version 2 - Level 1

• OWASP Mobile Application Security Testing Guide (MASTG) Version 2 - Level 2

• MITRE ATT&CK Mobile Version 14.1

Network Infrastructure, Hardware and IOT

• OSSTMM Version 3 - Telecommunications Security Testing

• OSSTMM Version 3 - Data Networks Security Testing

• OSSTMM Version 3 - Wireless Security Testing

• MITRE ATT&CK ICS Version 14.1

Cloud Configuration

• CIS Amazon Web Services Foundation v1.2.0

• CIS Microsoft Azure Foundation v1.2.0

• CIS Google Cloud Platform Foundation v1.1.0

• Oracle Cloud Infrastructure

• Kubernetes Infrastructure

Start by clicking on New -> Import Test Cases.

Select a import source:

• AttackForge Community - you can export your test cases in AttackForge Community, and import them into your AttackForge Core/Enterprise.

• JSON - generic JSON import option. Includes a template file to help with preparing your data file for import.

• CSV - generic CSV import option. Includes a template file to help with preparing your data file for import.

Select the test cases you would like to import.

You can make changes to the test cases prior to import.

Receive updates on import progress.

Import Mappings

We recommend setting the following Test Case Custom Fields when importing test cases from the AttackForge built testing methodologies.

MITRE ATT&CK Enterprise, Mobile and ICS

• Key - x_mitre_attack_spec_version

  • Field Type - Input

• Key - x_mitre_permissions_required

  • Field Type - List

• Key - kill_chain_phases

  • Field Type - Table

  • Columns:

    • Key - kill_chain_name

      • Field Type - Input

    • Key - phase_name

      • Field Type - Input

• Key - x_mitre_platforms

  • Field Type - List

• Key - x_mitre_data_sources

  • Field Type - List

• Key - external_references

  • Field Type - Table

  • Columns:

    • Key - source_name

      • Field Type - Input

    • Key - description

      • Field Type - Input

    • Key - url

      • Field Type - Input

    • Key - external_id

      • Field Type - Input

• Key - mitre_domain

  • Field Type - Input

• Key - mitre_tactic

  • Field Type - List

• Key - mitigations

  • Field Type - Table

  • Columns:

    • Key - mitigation

      • Field Type - Input

    • Key - description

      • Field Type - Input

• Key - detections

  • Field Type - Table

  • Columns:

    • Key - data_source

      • Field Type - Input

    • Key - data_component

      • Field Type - Input

    • Key - detects

      • Field Type - Input

• Key - x_mitre_defense_bypassed

  • Field Type - List

Execution Flows

Execution flows can be assigned to each test case.

Execution flows can have many uses such as:

• Documenting steps and procedures guiding a person in how to perform the test case

• Documenting which tools should be used to perform the test case

• Documenting internal processes and procedures required by the test case

• Links to external resources

You can add execution flows to any test case when creating or updating the test case.

Abuse Cases

Abuse cases are project-specific test cases. They are unique test cases which apply to the project. For example, consider a web application pentest for a reverse auction website. Typically the pentest may cover the standard OWASP ASVS test cases, however the customer also requires that business logic tests are performed against the bidding functionality to determine whether it can be cheated or not. Abuse cases can be created to specifically test this functionality and provide higher level of assurance beyond standard test cases.

To create abuse cases on the project, you must be either an Administrator or Project Coordinator.

From the project test cases section, click on Add -> Abuse Case.

Abuse Cases are stored & tracked per project in the Test Suites module under the Project Abuse Cases section.

You can delete Abuse Cases directly from the project.

Archived Test Suites

You can access any archived test suites by clicking on the Archived Test Suites button. Here you can view and restore any test suites if desired.

Any test suites you archive from the library will no longer be available for projects or project requests. However, any historical project using the test suite will not be affected so that integrity of test cases on a project remains in-tact.