First, we estimate the Entropy for each nybble in the IPv6 addresses, across the whole dataset. For example, if the last nybble is highly variable, then the corresponding Entropy will be high. Conversely, the Entropy will be zero for nybbles that stay constant across the dataset. Below we plot the normalized value of Entropy for each of the 32 nybbles, along with the 4-bit Aggregate Count Ratio, which was introduced in Plonka and Berger, 2015.
Second, we group adjacent nybbles with similar Entropy to form larger segments, with the expectation that they represent semantically different parts of each address. We label these segments with letters and mark them with dashed lines in the plot below.
Next, we search the segments for the most popular values and ranges of values within them. For that purpose, we use statistical methods for detecting outliers and the DBSCAN machine learning algorithm. We analyze distribution and frequencies of values inside address segments.
Below we present the results, with ranges of values shown as two values in italics (bottom to top). The last column gives the relative frequency across the whole dataset. The /32 prefixes are anonymized.
A: bits 0-32 (hex chars 1- 8)
B: bits 32-36 (hex chars 9- 9)
- 20010db8 100.00%
C: bits 36-64 (hex chars 10-16)
- 0 25.24%
- 2 22.62%
- 1 18.59%
- 4 13.83%
- 5 7.09%
- 6 5.44%
- a 4.30%
- 8 1.53%
- 7 1.37%
D: bits 64-88 (hex chars 17-22)
- 02e275f 0.01%
- * 7000047-906eaad 19.25%
- * b648ddf-d9ffbd8 14.44%
- * 000001e-feff8d8 66.28%
E: bits 88-120 (hex chars 23-30)
- 000000 47.33%
- * 0000d3-ffff4f 52.65%
F: bits 120-128 (hex chars 31-32)
- * 00000004-44ffd1f7 39.11%
- * 44fff3fd-4fffde68 21.68%
- * 4ffff424-52ff928e 2.45%
- * 5400257b-ffffb08c 36.28%
- * 5300704a-53ffde68 0.47%
- 01 47.54%
- * 00-ff 52.46%
Next, we search for statistical dependencies between the segments. For that purpose, we train a Bayesian Network (BN) from data.
Below we show structure of the corresponding BN model. Arrows indicate direct statistical influence. Note that directly connected segments can probabilistically influence each other in both directions (upstream / downstream). Under some conditions, segments without direct connection can still influence each other through other segments: e.g., A can influence C through B if C depends on B and B depends on A (even if there is no direct arrow between A and C).
Learning BN structure from data is in general a challenging optimization problem. Hence, there might be more than one possible BN structure graph for the same dataset.
Finally, below we show an interactive browser that decomposes IPv6 addresses into segments, values, ranges, and their corresponding probabilities. The browser lets for exploring the underlying BN model and see how certain segment values probabilistically influence the other segments.
Try clicking on the colored boxes below. You should see the colors changing, which reflects the fact that some segment values can make the other values more (or less) likely. For instance, in the Sample Report, you may find that clicking on J1 (i.e., the first value in segment J) makes segments C, D, F, H, and I largely predictable (see our paper for more examples).
You may condition the model on many segment values. Clicking on selected values un-selects them. Clicking on the red "Clear" above the color map un-selects them all. Below the browser we show the estimated proportion of the addresses matching your selection (vs. the dataset). If the browser cannot estimate the probabilities in a reasonable time, it asks before trying harder.
Using the BN model, below we generate a few candidate target IPv6 addresses matching the selection above. Note that we anonymize the IPv6 addresses in this report.
As we show in the paper, this technique allowed us to successfully scan IPv6 networks of servers and routers, and to predict the IPv6 network identifiers of active client IPv6 addresses.