Aug 23, 2017
This month Synopsys put out their
State of Fuzzing 2017 report. It's useful data, but the context
of the collection and the metrics used to evaluate failures is very
important. I talked with Chris Clark, Principal Security Engineer
for Strategic Initiatives at Synopsys, to discuss the report.
Key points from the podcast and report include:
- The data comes from the yearly totals of usage by Synopsys
customers. The data is anonymized.
- This means the protocol stacks are likely in
development or QA and being tested. Some percentage of these flaws
are fixed prior to release. The numbers may not be as grim as they
- You need to understand the market and protocol history to make
sense of the numbers.
Two examples on the last point.
- The CAN protocol testing for the automotive sector is almost
entirely of ECU's in a lab environment, not deployed in the
automobile. There are large numbers of ECU's in cars as opposed to
a single or very small number of protocol stacks in a PLC or even
in an entire SCADA or DCS.
- The Modbus TCP numbers look terrible. Modbus TCP is such a
simple protocol that many, if not most, code up their own Modbus
TCP stack. Most large and established ICS vendors have fuzzed and
fixed their Modbus TCP stacks. So the numbers don't likely reflect
the robustness of the deployed Modbus TCP stacks.
Contrast this to DNP3 that has a smaller number of stacks and most
stacks purchased from a single vendor, or OPC UA where the
complexity of the stack encourages buying rather than
- The number of protocol stacks in each protocol (Modbus TCP, OPC
UA, etc.) varies and is based solely on customer use. It could be
one stack or 21 stacks.
- The number of tests and test methodology is entirely determined
by the Synopsys customer. It is not uniform.
- The detection of failure is also not as rigorous as you would
see if the device was monitored for performance of its full
As Chris stated, this should be considered a top level view of
the state of the ICS protocol robustness. The key is to understand
where these numbers come from and not read more into them than the
constraints warrant. And we should appreciate that ICS vendors are
doing this type of testing.
Note: I apologize for the voice quality of this. It was a
combination of a mistake I made in setup and marginal line quality.
It is not difficult to understand, but not pleasant to the ear. I
will do better.