Test Cases
Last updated
Last updated
When creating a new project, you can select one or more testing methodologies or checklists to apply to the project - referred to as Test Suites. Each test suite has a collection of test cases which gets assigned to the project.
A test suite helps:
Clients understand exactly what was tested on the project;
Developers/Engineers link test cases to vulnerabilities;
Pentesters structure their testing in a methodical, consistent & standardized way;
Organizations create repeatable, standardized & comparable assessments - independent of whom was actually performing the assessment.
Test cases can provide valuable insight into a penetration test or audit. It shows:
What was tested
When it was tested
Who tested it
What was the status
Supporting external (customer facing) notes
Supporting internal (security team) notes
Supporting evidence
Whether test case passed, failed or remediated
To view the test cases assigned to the project, click on Test Cases from the project menu.
It is the function of pentesters on the project to update the test cases as they work through the assessment. Therefore only users with Edit permissions on the project can update a test case.
Test cases by default are set to Not Tested.
Authorized users can update the status of a test case to any of the following:
Not Tested
Testing In Progress
Tested
Not Applicable
You can create your own sub-status for test cases using custom fields via Administration -> Projects -> Test Cases
You can update a test case by clicking on the status of the test case and selecting an option from the drop-down menu.
You can also update multiple test cases in bulk. Select multiple test cases, then select the desired option.
Each test case can have supporting files which contains the evidence of assessment.
Each test case can have supporting external (customer facing) notes & files which contains the observations from the pentester.
Each test case can also have supporting internal (security team) notes & files which contains the observations and artefacts from the pentester.
Internal notes are only visible to users with Edit access to the project.
For example, if the test case required to perform a scan using a tool - the results of the scan can be uploaded to the internal notes
Execution flows can be assigned to each test case.
Execution flows can have many uses such as:
Document steps and procedures guiding a person in how to perform the test case
Document which tools should be used to perform the test case
Document internal processes and procedures required by the test case
Provides links to external resources
You can tailor your test cases to projects using Custom Fields.
Here's an example of a test case for a Web Application pentest:
However for a Purple Team assessment, the test cases will need to be structured differently to accomodate fields for MITRE ATT&CK mappings, Red Team sections and Blue Team sections.
You can define your Red Team and Blue Team sections and custom fields in Administration -> Projects -> Test Cases
You can use Hide Expressions to configure when those sections and fields should be visible, for example only on Purple Team projects:
You can also configure Custom Field Access Controls to determine who can View or Edit the field.
Administrators and Project Coordinators can assign test cases to any project team members with Edit permissions on the project. This helps to delegate tasks to team members to maximise efficiency during testing, as well as accountability for certain tasks.
You can assign test cases to a user by selecting the test cases and using the actions menu to assign them to a user.
Administrators and Project Coordinators can assign assets to test cases. This helps to delegate tasks to individual assets to increase testing coverage and traceability.
You can assign one or more assets to the test case.
Test cases will automatically show as Passed, Failed or Remediated.
Passed test cases are test cases which have no linked vulnerabilities (no findings)
Failed test cases are test cases which have one or more open linked vulnerabilities (findings discovered and haven't been closed)
Remediated test cases are test cases which have one or more linked vulnerabilities, and all of them are closed (findings discovered and have all been addressed)
You can fail a test case by linking a vulnerability to a test case.
When creating or updating a vulnerability on a project, select the failed test case(s) to link them.
You can add a vulnerability directly from the test cases page, to quickly link the test case to the new vulnerability.
When a vulnerability is linked to a test case, the test case will be automatically marked as failed.
You can click on the Vulnerabilities tab to see all linked vulnerabilities.
If all vulnerabilities linked to a failed test case have been Closed, the test case will be considered Remediated.
As an Admin or Project Coordinator, you have the ability to Lock and Unlock test cases on a project at any given time.
Locking test cases is useful if you need to prevent test cases from being altered or tampered with.
When a test case is locked, it cannot be updated. You cannot add any new notes or evidence either. This provides greater assurance from an auditing perspective.
Locked test cases will not show up on or affect the project status and percentage completion.
Locked test cases will not show up in the reports as reporting is focused on the active test cases.
To lock a Test Case - select the test cases then click on Lock from the actions menu.
To unlock a Test Case - click on Show Locked button, then select test cases and click on Unlock in the actions menu.
As an Administrator or Project Coordinator, you can add additional test suites to a project after the project has been created.
To add new test suites on a project, click on Add -> Test Suites.
By default, the test cases loaded on to the project will be set to Unlocked/Active status.
You can automatically lock the previous test cases if required. This will ensure the previous test cases can’t be tampered with or changed accidentally. It will also reset the project status to Waiting to Start and progress will be set to 0% (based on the new test suites added).
As an Administrator or Project Coordinator, you can delete test cases on a project. This can help if you need to remove test cases which do not need to be actioned on the project.
To delete test cases on a project, select the test cases then click on Delete from the actions menu.
Abuse cases are project-specific test cases. They are unique test cases which apply to the project only, or objective of the assessment.
For example, consider a web application pentest for a reverse auction website. Typically the pentest may cover the standard OWASP ASVS test cases, however the customer also requires that business logic tests are performed against the bidding functionality to determine whether it can be cheated or not. Abuse cases can be created to specifically test this functionality and provide higher level of assurance beyond standard test cases.
To create abuse cases on the project, you must be either an Administrator or Project Coordinator.
Click on Add -> Abuse Case andEnter in the details for the project specific test case.
You can access all abuse cases for each project via the Test Suites module. You can make changes to the abuse cases from here. If you need to delete an abuse case, you can perform this directly from the project (see Deleting Test Cases above).
