Revealing the Unknown: Tenable.asm Part 2
Last week, I wrote an integration with Tenable.asm and Tenable.io VM using pytenable and navi. This was to answer what I called a million dollar question:
What IP addresses did Tenable.asm find that I have not scanned with Nessus?
In the short article, I explain how to provide increased visibility to Tenable.io about your external attack surface utilizing Tenable.asm data in an automation script. Let’s take the same approach with Tenable Web app data.
Tenable ASM
As a bit of a review, Tenable.asm is Tenable’s External Attack Surface Management solution acquired from Bit Discovery earlier this year. As you would presume, it maps the internet and discovers connections to internet-facing assets, this of course includes web applications.
The Challenge
While Tenable.asm is integrated into the Tenable.io platform, understanding gaps between sensors is still a bit of a challenge. However, as I said before, with a few lines of code and some creativity we can find the gaps between Tenable.asm and Tenable.io WAS quickly!
For complete visibility, we are looking to answer two important questions:
What Web applications did Tenable.asm find that I have not scanned with Tenable.io WAS?
and
What externally facing Web applications have I scanned that Tenable.asm has yet to detect?
Tenable WAS
Tenable’s Dynamic Application Security Testing(DAST)solution is built into Tenable.io and is a simple, scalable and automated way to enumerate vulnerabilities on Web Applications. Tenable.io DAST also called Tenable.io WAS(web application scanning) provides comprehensive and accurate vulnerability data based primarily on the OWASP Top 10 risks.
Note: Need WAS reporting? Check out the WAS-reporter!
Navi Update
Since Tenable purchased Bit Discovery I have been itching to see how much the products can influence each other and wondered what it would look like. If you follow my articles at all you know I created navi; the Tenable swiss army knife for Tenable.io. So bringing Tenable WAS data into the navi database was an obvious choice.
Retrieving data from raw APIs in mass can be time consuming and error prone, especially for tasks you intend to repeat often. So when I set out to solve this challenge I needed WAS asset_uuids and gathering them from the raw Tenable.io APIs wasn’t going to be scalable for my use-case.
Navi version 7.1.16 has all new WAS endpoints and includes a WAS bulk download capability as you can see in the screenshot above.
To download the last 90 days of web application scans into the local navi database; use the new command below:
navi update was --days 90Keep in mind that this could take a long time depending on my how many apps you are scanning and how frequent. Application data and severity counts are stored in the apps table, while details on findings are stored in the plugins table.
The solution
Now that you have navi 7.1.16 installed you will need to add your Tenable.io API keys and run the above WAS update. Once complete you can confirm you have data in each table by using the navi find query command which enables querying the database directly.
The next step is to get the Authentication to Tenable.asm data configured like below. Note that your ASM API keys are different than your Tenable.io API keys.
import requests
import pprint
import time
from tenable.io import TenableIO
from os import system as sys
# ASM URL
api_url = "https://asm-demo.cloud.tenable.com/api/1.0/inventory?offset=0&limit=10000&sortorder=true&inventory=false"
# ASM API KEY
api_key = '<ASM API KEY>'
# ASM headers for Auth
headers = {"accept": "application/json", "Content-Type": "application/json", "Authorization": api_key}
# Limit ASM data to A records
params = [{"column": "bd.record_type",
"type": "starts with",
"value": "A"}]
# T.io API Keys
access_key = '<ACCESS KEY>'
secret_key = '<SECRET KEY>'
# Authenticate using Pytenable
tio = TenableIO(access_key, secret_key, vendor='Casey Reid', product='Integrate WAS/ASM Assets', build='0.0.1')Now that authentication is out of the way we can pull down all of the A records in ASM and look up the corresponding asset_uuid in the navi database.
def navi_tag_was_assets():
# Authenticate to T.io
sys("navi keys --a {} --s {}".format(access_key, secret_key))
# Update WAS data for our search
sys("navi update was --days 30")
asm_data = requests.request("POST", api_url, headers=headers, json=params)
for asm_asset in asm_data.json()['assets']:
query = "select asset_uuid from apps where target LIKE '%{}%';".format(asm_asset['bd.hostname'])
sys("navi tag --c \"{}\" --v \"{}\" --query \"{}\"".format("ASM", "Externally Found", query))
#
navi_tag_was_assets()The above code loops through each A record and searches the Navi database for the asset_uuid based on the hostname matching the target in WAS. For better visibility you can see the SQL query I am using to grab the asset_uuid below:
“select asset_uuid from apps where target LIKE ‘%{}%’;”.format(asm_asset[‘bd.hostname’])
The next step in the loop is to use navi to tag these assets. Each Web application found will be tagged, “ASM: Externally found”. This will help reduce confusion with internal apps.
Luckily navi has a built in function that will take the query command. Ensure that you are only pulling the asset_uuid in this command; its the only thing required for tagging using this method.
Below are all the tagging options available in navi. I will details these options in an upcoming Tenable API deep dive on granular reporting utilizing advanced tagging.
The Results
Now that each Web application found by ASM is tagged, we can use the explorer view filtering to find those assets that are public but were not found by ASM. Indicating a configuration change to ASM to capture the domains that were missed. This could happen with a new acquisition, for instance.
These assets are public facing assets used for training Web application testing concepts which I have in my Tenable.io WAS lab. In my Tenable.asm lab, you can see I do not have these assets:
As a tenable customer, My next action would be to add the found domains to ASM to get more visibility into the domains missed on the initial configuration.
What about web applications found by ASM but NOT scanned by Tenable.io WAS?
Those assets will be printed to the screen. When navi attempts to tag the assets in question and the query returns zero asset_uuids, navi indicates it with a “0 Assets” message as shown below:
Downloading all Completed Scans for the last 90 days.
This will take some time.
ASM Asset Check: rest.vulnweb.com
Your tag resulted in 0 Assets, therefore the tag wasn't created
ASM Asset Check: *.vulnweb.com
Your tag resulted in 0 Assets, therefore the tag wasn't created
ASM Asset Check: www.vulnweb.com
Your tag resulted in 0 Assets, therefore the tag wasn't created
ASM Asset Check: vulnweb.com
Your tag is being updated
Job UUID : 1e558032da01f494494c5c17c0674552:00d0a38800245560a6677cfc0cafb893114d:1f500cfba60121591488
ASM Asset Check: testphp.vulnweb.com
Your tag is being updated
Job UUID : ede5a18a6b74b1a73d53ff27f70dfb70:00d0d3b4cd6fed8818f2d6cd8c9190617e8c:868dd6aab48ac8f83960
ASM Asset Check: testasp.vulnweb.com
Your tag is being updated
Job UUID : 95c57eb98db80c3a34315e9b47d1ff85:00d0b776026f4a08728ad3c066627db5ca3b:ffbdbfe612c43fd048d5
ASM Asset Check: testaspnet.vulnweb.com
Your tag is being updated
Job UUID : 0d7bd60f18da21de23199fc7f7e8e339:00d05adf5fb0f34132eabf7b215324631d5c:8a0f964f15435b10442c
ASM Asset Check: testhtml5.vulnweb.com
Your tag is being updated
Job UUID : 9664c50158c5fa388c104e293387d7ec:00d00289b8094afc8737d4b5c7b4a7db52c5:d520498e462f6babb558
Process finished with exit code 0The ASM assets which produced “0 Asset” messages in Navi are the “Gaps” in WAS. While I stopped my integration work here, one could easily take it a step further and create WAS scans out of these known URLs by the using navi was scan command or tag these assets as I did in the first article.
While both of these articles came with explicit code working through the problem, I intend to make them each a single command in Navi to ease the deployment in the coming months.
Here is the full code:
https://gist.github.com/packetchaos/580ad8567c96550d277ffda48b18da8c
