Using sleuth

Many examples are within the source code examples.sh.

Sleuth is a tool designed to allow users to make SQL-like queries on JSON data. It reads a stream of flow objects, processes each one in sequence, and writes out the resulting objects. There are several processing stages, each of which is optional; their order is:

1. Filtering objects (--where)
2. Selecting elements (--select)
3. Splitting into multiple output streams (--split)
4. Computing the distribution (--dist)
5. Computing the sum of particular elements (--sum)

The whole solution is broken into 2 parts. First, sleuth_pkg is a Python package that can be installed using pip. This package provides the underlying system for performing operations on and manipulating JSON data. The second piece is a Python script named sleuth which resides at the top-level of the repository. This is the main entry point for users to interact with Sleuth by issuing command options.

Command options

--select [keys]

• Only show the selected JSON keys, along with their values.
• Exclude any object that does not contain the keys.
• The order of the objects is unchanged.

Example:

$ ./sleuth example.json.gz --select "sa,da,sp,dp,pr"

This will show the 5-tuple (source address, destination address, source port, destination port, protocol) identifier ONLY, for all flow JSON objects.