Parsing 19506 with Python

If you read my earlier article, “For the love of 19506”, you will recall the myriad of useful data that is packed into the Tenable Informational plugin, 19506. If you haven’t, here is a quick screen shot that summarizes the data in the plugin:

Screenshot from Tenable Plugins page

In the article, I provided a python solution(navi) to analyze the scan duration for all of the IPs/Assets in a particular scan. The goal is to quickly find outliers and other scan problems.

navi scan evaluate --scanid <your scan id> --histid <history ID if desired>

navi, creates a CSV export of the scan in Tenable Vulnerability Management(formally Tenable.io) focusing only on the 19506 plugin, parses the output of the plugin and creates a new CSV for further review, shown below.

Snippet of the CSV output

While the above screenshot doesn’t show all of the fields generated by navi, you can see the value of getting the asset UUID along with the duration for every asset in a particular scan.

Which might get you thinking…

How in the world?

How do I parse the 19506 plugin myself?

Parsing 19506 with Python

Before we begin parsing the plugin and utilizing the data within, we need to export a CSV from one of Tenable’s products. The export only needs to include the 19506 output, which is shown below in the Tenable Vulnerability Management solution.

Note: The code in the article is focused on how the data comes out of Tenable Vulnerability Management(formally Tenable.io) when filtering on the 19506 plugin for a scan export.

Creating an Export by Plugin ID

To simplify getting the data and reducing how much is downloaded I’m using pytenable and filtering the export on the 19506 plugin output.

In the code snippet above you can see the utilization of pytenable to make our export crazy simple; a single line of code:

tio.scans.export(scan_id, ('plugin.id', 'eq', '19506'),
                         format='csv', fobj=fobj, history_id=hist_id)

The code surrounding the export request is to handle the necessary variables for the pytenable export: scan ID, history ID and filename. A snippet of the export is shown below:

Now that we have the export, we need to pull it out of the CSV and begin parsing the data.

Grabbing the Plugin Output

To make things easy and avoid counting the CSV columns I recommend using the DictReader class in the CSV python library.

from csv import DictReader
    with open(filename) as fobj:
        for row in DictReader(fobj):
            plugin_output = row['Plugin Output']
            

With a few lines of code, shown above, I now have the plugin output data as a string. From trial and error I found the best way to accomplish parsing the plugin is a three step process:

1. Split everything in the output by return(“\n”)
2. Then cycle through each record creating a dictionary in the process
3. Split everything by colon(“ : “); avoiding warnings that may happen during a scan.

The process in python is below:

# 1. split the output by return
parsed_output = plugin_output.split("\n")

# 2. put everything but warnings into a dictionary
for info_line in parsed_output:

    try:
        # 3. Split everything by colon to catch warnings
        new_split = info_line.split(" : ")
        plugin_dict[new_split[0]] = new_split[1]

    except:
        # Skip warnings
        pass

Now, we have all of the useable data in a dictionary.

Reconstructing the data in the plugin as a dictionary makes it easy to pull the data we are interested. For instance, getting the scan duration is a simple as:

asset_scan_duration = plugin_dict['Scan duration']

However, it might be more useful to create a new CSV with each data point as a column. This would allow you to quickly see scan differences across everything captured in the plugin.

To accomplish that, we are going to create a second for loop and run through each item in the dictionary creating a new CSV in the process.

with open('{}'.format("Parsed_19506_data.csv"), mode='w', encoding='utf-8', newline="") as csv_file:
    # Pull all of the 'keys' for the CSV headers
    header = list(set(key for headers in asset_dict.values() for key in headers.keys()))

    # Create our writer object
    data_writer = csv.DictWriter(csv_file, fieldnames=header)

    # write the headers first
    data_writer.writeheader()

    # Write all of the plugin data into each column
    for asset in asset_dict.values():
       data_writer.writerow(asset)

The first line in the code above, opens a new file called “Parsed_19506_data.csv”. The second line of code can be quite complicated to understand if you’re not a python developer.

I asked ChatGPT to simplify the code using the prompt below. The code grabs all of the plugin attribute names to they can be used as headers in our CSV.

ChatGPT for the win!

The last few lines in the code example writes each attribute of the 19506 plugin in a row for future analysis. See the results below:

Now we have a CSV representation of the 19506 plugin data! This will allow us to create graphs and dig into assets that are taking longer to scan. In addition, we can reuse this code to evaluate all of our scans.

Here is the full code:

def parse_19506():
    filename = "13-report.csv"
    asset_dict = {}

    import csv
    from csv import DictReader
    with open(filename) as fobj:

        for row in DictReader(fobj):
            plugin_dict = {}
            plugin_output = row['Plugin Output']
            asset_uuid = row['Asset UUID']
            plugin_dict['Asset UUID'] = asset_uuid
            # split the output by return
            parsed_output = plugin_output.split("\n")

            # put everything but warnings into a dictionary
            for info_line in parsed_output:

                try:
                    new_split = info_line.split(" : ")
                    plugin_dict[new_split[0]] = new_split[1]

                except:
                    # Skip warnings
                    pass
            # Reformat the data into a nice dictionary for multiple use-cases
            asset_dict[asset_uuid] = plugin_dict

    with open('{}'.format("Parsed_19506_data.csv"), mode='w', encoding='utf-8', newline="") as csv_file:
        # Pull all of the 'keys' for the CSV headers
        header = list(set(key for headers in asset_dict.values() for key in headers.keys()))

        # Create our writer object
        data_writer = csv.DictWriter(csv_file, fieldnames=header)

        # write the headers first
        data_writer.writeheader()

        # Write all of the plugin data into each column
        for asset in asset_dict.values():
           data_writer.writerow(asset)

I hope this article helped provide a better understanding of how to parse the 19506 plugin for your own needs.

I will leave you with the DALL-E /Banksy representation of the title of this article: