EXPORTING TO CSV
Getting Started
This page will help you with exporting your Self-Service RESTful API data to CSV format.
This is useful if you need to analyze the data, create reports, or import the data into other tools that require CSV format.
It is possible to programmatically parse the output from the AttackForge Self-Service RESTful API into CSV format using various tools such as JQ.
JQ is a lightweight and flexible command-line JSON processor.
JQ is supported on multiple operating systems (Linux, OSX, Windows) & is freely available. It is written in C and has no runtime dependencies.
This tutorial will show you how to install JQ and use it to convert AttackForge Self-Service RESTful API data into CSV format.
Installing JQ on Windows
1. Download JQ from here: https://stedolan.github.io/jq/download/. We recommend using JQ v1.6.
2. Go into C:\Users\<username> and create a new folder called Bin.
3. Place the downloaded file into the newly created Bin folder.
4. Rename the file to “jq” (so the full name should be jq.exe).
5. Click on the Windows Start menu, and type Edit the System Environment Variables, and hit Enter.
6. The System Properties window will be displayed. Click on Environment Variables.
7. An Environment Variables window appears and two text boxes are displayed. In the top box, click on Path (third item in the list) and hit Edit.
8. An Edit environment variable windows appears; click on the %USERPROFILE% entry and select Edit Text.
9. An Edit User Variable windows appears. At the end of the %USERPROFILE% string, enter the path to the Bin folder that you created in Step 2. The path should be separated from the %USERPROFILE% string by a semi-colon, and the Bin folder path should contain a semi-colon at the end of it – like following:
10. Click OK to exit the Edit User Variable window. Click OK to exit the Environment Variables window. Click OK to exit System Properties window.
11. Open a command window (Windows Start menu > type CMD > select Command Prompt).
12. Enter this command: echo %PATH%. The new folder that you created in Step 2 should be present at the end of the output text. This indicates that the environment variables have been changed and should now pick up the JQ utility.
13. To check the utility is working correctly, type this command: jq
It should display results similar to below – confirming the tool is installed correctly.
Installing JQ on Linux
1. From command line, type following command: sudo apt-get install jq
2. Other install options are available from https://stedolan.github.io/jq/download/
Creating CSV Using API Data
For this tutorial, we are going to focus on the getVulnerabilities API method. You can however use this approach for any of the Self-Service RESTful API methods.
Open a command window / terminal.
Run the following command (replacing your tenant name & API Key). Make sure you have access to the getVulnerabilities API (see https://support.attackforge.com/attackforge-enterprise/modules/users#managing-access-to-self-service-api)
The command above will produce a CSV titled filtered_vulns.csv in the working directory where you had run the command from.
The CSV will include the following details for each vulnerability:
Created (time/date)
Priority (Critical/High/Medium/Low/Info)
Title (e.g. Blind SQL Injection)
Status (Open/Closed)
Status Updated (time/date)
Affected Asset Name (e.g. attackforge.com)
Retest (Yes/No)
Project Name (e.g. Web App Pentest)
Description
Attack Scenario
Remediation Recommendation
Steps to Reproduce
Id
Changing Fields & Re-Ordering Fields
If you would like to change the existing fields in the CSV, or re-order the fields in the CSV – adjust the bolded section of the command below:
curl -X GET “https://<YOUR-TENANT>.attackforge.com/api/ss/vulnerabilities” -H “Host: <YOUR-TENANT>.attackforge.com” -H “X-SSAPI-KEY: <YOUR-API-KEY>” -H “Content-Type: application/json” -H “Connection: close” | jq -r “(.vulnerabilities[] | [.vulnerability_created, .vulnerability_priority, .vulnerability_title, .vulnerability_status, .vulnerability_status_updated, .vulnerability_affected_asset_name, .vulnerability_retest, .vulnerability_project_name, .vulnerability_description, .vulnerability_attack_scenario, .vulnerability_remediation_recommendation, .vulnerability_steps_to_reproduce, .vulnerability_id]) | @csv” > filtered_vulns.csv
Adding Additional Fields
If you would like to add additional fields to the CSV, you can use any of the fields available in the API (see https://support.attackforge.com/attackforge-enterprise/modules/self-service-api/getvulnerabilities)
For example, the command below will also include Project Groups in the CSV:
curl -X GET “https://<YOUR-TENANT>.attackforge.com/api/ss/vulnerabilities” -H “Host: <YOUR-TENANT>.attackforge.com” -H “X-SSAPI-KEY: <YOUR-API-KEY>” -H “Content-Type: application/json” -H “Connection: close” | jq -r “(.vulnerabilities[] | [.vulnerability_created, .vulnerability_priority, .vulnerability_title, .vulnerability_status, .vulnerability_status_updated, .vulnerability_affected_asset_name, .vulnerability_retest, .vulnerability_project_name, .vulnerability_project_groups, .vulnerability_description, .vulnerability_attack_scenario, .vulnerability_remediation_recommendation, .vulnerability_steps_to_reproduce, .vulnerability_id]) | @csv” > filtered_vulns.csv
Including Column Names
If you would like to include column names in the CSV, you can set the columns using the command below – see highlighted section:
curl -X GET “https://<YOUR-TENANT>.attackforge.com/api/ss/vulnerabilities” -H “Host: <YOUR-TENANT>.attackforge.com” -H “X-SSAPI-KEY: <YOUR-API-KEY>” -H “Content-Type: application/json” -H “Connection: close” | jq -r “[\"Discovered\", \"Priority\", \"Title\", \"Status\", \"Status Last Updated\", \"Affected Asset\", \"Retest\", \"Project\", \"Description\", \"Attack Scenario\", \"Recommendation\", \"Steps To Reproduce\", \"Id\"], (.vulnerabilities[] | [.vulnerability_created, .vulnerability_priority, .vulnerability_title, .vulnerability_status, .vulnerability_status_updated, .vulnerability_affected_asset_name, .vulnerability_retest, .vulnerability_project_name, .vulnerability_description, .vulnerability_attack_scenario, .vulnerability_remediation_recommendation, .vulnerability_steps_to_reproduce, .vulnerability_id]) | @csv” > filtered_vulns.csv
Note the backslashes \" are necessary to escape the quotation marks for each CSV column title. If you do not include the backslashes, you may get a compilation error.
Filtering Vulnerabilities
If you would like to filter the vulnerabilities, you can use any of the existing filters supported in the API (see https://support.attackforge.com/attackforge-enterprise/modules/users#managing-access-to-self-service-api)
For example, the command below will limit the vulnerabilities retrieved from the API to discovered between 01st January 2021 to 30th March 2021:
curl -X GET “https://<YOUR-TENANT>.attackforge.com/api/ss/vulnerabilities?startDate=2021-01-01&endDate=2021-03-30” -H “Host: <YOUR-TENANT>.attackforge.com” -H “X-SSAPI-KEY: <YOUR-API-KEY>” -H “Content-Type: application/json” -H “Connection: close” | jq -r “(.vulnerabilities[] | [.vulnerability_created, .vulnerability_priority, .vulnerability_title, .vulnerability_status, .vulnerability_status_updated, .vulnerability_affected_asset_name, .vulnerability_retest, .vulnerability_project_name, .vulnerability_description, .vulnerability_attack_scenario, .vulnerability_remediation_recommendation, .vulnerability_steps_to_reproduce, .vulnerability_id]) | @csv” > filtered_vulns.csv
Advanced Examples
Example 1 - Get All Closed Vulnerabilities and Reason Why Closed
Vulnerability Id | Created | Priority | Title | Status | Status Last Updated | Affected Asset | Project Name | Project Id | Last Closed Reason | Last Closed On | Last Closed By |
65bc9d9a4534bd000e50db3f | 2024-02-02T07:45:30.179Z | Critical | Out-of-date Version | Closed | 2024-02-02T09:36:41.513Z | https://attackforge.com | Test Project 1 | 65af64ec07bd34000fbac60d | Issue Closed: Risk accepted | 2024-02-02T09:36:41.513Z | Admin Test |
65b56ceff8b879000f887c7e | 2024-01-27T20:51:59.346Z | Critical | Out-of-date Version | Closed | 2024-02-02T05:33:38.928Z | https://portal.attackforge.com | Test Project 1 | 65af64ec07bd34000fbac60d | Issue Closed: Issue has been fixed | 2024-02-02T05:33:38.928Z | Admin Test |
Wrapping up
You can now use the JQ utility for any of the AttackForge Self-Service APIs, to extract any desired information for your reporting purposes.
You can check all the of the available APIs from our support site: https://support.attackforge.com/attackforge-enterprise/modules/self-service-api
Troubleshooting
1. Test jq utility is available
jq
This should result in info/help output in the terminal. This will test to confirm jq utility has been installed & configured successfully on your host.
2. Test cURL works + piping/passing JSON data to jq
curl "https://api.github.com/repos/stedolan/jq/commits?per_page=5" | jq "."
This should output JSON data to the terminal.
3. Test @csv is working
curl "https://api.github.com/repos/stedolan/jq/commits?per_page=5" | jq -r ".[0] | [.url] | @csv"
It should print "https://api.github.com/repos/stedolan/jq/commits/d18b2d078c2383d9472d0a0a226e07009025574f"
in the terminal
4. Try output data into csv file
curl "https://api.github.com/repos/stedolan/jq/commits?per_page=5" | jq -r ".[0] | [.url] | @csv" > output.csv
This should create a file output.csv in the directory you are currently running commands from in command terminal, the contents of the file will be https://api.github.com/repos/stedolan/jq/commits/d18b2d078c2383d9472d0a0a226e07009025574f
5. Store AttackForge getVulnerabilities data in a JSON file
curl -X GET "https://<YOUR-TENANT>.attackforge.com/api/ss/vulnerabilities" -H "Host: <YOUR-TENANT>.attackforge.com" -H "X-SSAPI-KEY: <YOUR-API-KEY>" -H "Content-Type: application/json" -H "Connection: close" > vulns.json
This should create a file vulns.json in the directory you are currently running commands from in terminal, the contents of this file will be vulnerabilities which match the query parameter.
6. Try running JQ on vulns.json
jq -r "(.vulnerabilities[] | [.vulnerability_created, .vulnerability_priority, .vulnerability_title, .vulnerability_status, .vulnerability_status_updated, .vulnerability_affected_asset_name, .vulnerability_retest, .vulnerability_project_name, .vulnerability_description, .vulnerability_attack_scenario, .vulnerability_remediation_recommendation, .vulnerability_steps_to_reproduce, .vulnerability_id]) | @csv" vulns.json
This should output csv data in the terminal.
7. Try exporting vulns.json to csv
jq -r "(.vulnerabilities[] | [.vulnerability_created, .vulnerability_priority, .vulnerability_title, .vulnerability_status, .vulnerability_status_updated, .vulnerability_affected_asset_name, .vulnerability_retest, .vulnerability_project_name, .vulnerability_description, .vulnerability_attack_scenario, .vulnerability_remediation_recommendation, .vulnerability_steps_to_reproduce, .vulnerability_id]) | @csv" vulns.json > filtered_vulns.csv
This should create a file filtered_vulns.csv in the directory you are currently running commands from in terminal, the contents of this file will be vulnerabilities which match the specified fields in the JQ command.
If you get to Step 7 above and it is all working correctly, there should be no restrictions from running the original command (see below):
curl -X GET “https://.attackforge.com/api/ss/vulnerabilities” -H “Host: .attackforge.com” -H “X-SSAPI-KEY: ” -H “Content-Type: application/json” -H “Connection: close” | jq -r “(.vulnerabilities[] | [.vulnerability_created, .vulnerability_priority, .vulnerability_title, .vulnerability_status, .vulnerability_status_updated, .vulnerability_affected_asset_name, .vulnerability_retest, .vulnerability_project_name, .vulnerability_description, .vulnerability_attack_scenario, .vulnerability_remediation_recommendation, .vulnerability_steps_to_reproduce, .vulnerability_id]) | @csv” > filtered_vulns.csv
Last updated
