The DARPA VET program program is developing techniques to determine that the software and firmware shipped on commodity IT devices is free of broad classes of back-doors and other hidden malicious functionality. Common examples of commodity IT devices include mobile phones, network routers, printers, and computer workstations. If present, back-doors and other hidden malicious functionality could enable an adversary to use commodity IT devices as tools to accomplish a variety of harmful objectives, including the exfiltration of sensitive data and the sabotage of critical operations. The VET program seeks to develop and demonstrate new tools and techniques to establish that vetting every new device in a timely fashion at scale across large organizations is technically feasible.

The HAVoC project supports the VET goals by developing large, real-world benchmarks for 3rd parties to independently evaluate the effectiveness of their vetting approaches. This cleanly separates the duties of defensive technique development from performance evaluation. This is especially important in security research as otherwise if both duties are performed by the same party, they may unintentionally bias the malice towards the solution or the solution towards malice. These articles are also implemented on full sized designs representative, which enable scalability testing.

HAVoC provides a series of challenges on FPGA firmware, or net lists. Modeling the threat vector of commodity IT devices, no source code is provided, only the flattened netlist with hashed signal names. The type of commodity device and an end-user level guide is provided, which does allow for vetting approaches to be customized to a particular application domain. In some cases, a deployment scenario is also provided. The deployment scenario describes how the IT device is to be utilized, and therefore which features may or may not be in scope of the vetting tests. The Challenges page provides more information on the engagement details.

This material is based on research sponsored by the Defense Advanced Projects Agency (DARPA) and the Space and Naval Warfare Systems Center (SSC) Pacific under agreement number N66001-13-2-4042. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of DARPA or SNWSC.

device

The HAVoC challenges consist of 5 engagements. The reference circuits are representative of router and wi-fi commodity IT devices and are implemented on Xilinx Virtex5. The reference circuits are largely derived from other open source FPGA projects, and were then modified to provide engagement challenges. The engagements become progressively more difficult, ranging from 300 to 140,000 look up tables (LUTs), and may contain artifacts which test both probability of detection and probability of false alarm.

It is highly recommended that these engagements are taken sequentially, such that vet technique developers can learn, adapt, and scale their approaches. The reference engagement is provided such that a technique developer can ensure they are able to correctly parse the netlist before taking a challenge.

If desired, the HAVoC team can act as a virtual moderator to help keep the roles of technique developer and evaluator separate. The challenge can be provided first, and then after a technique developer has indicated they have completed the challenge, the answer keys can be released separately in order to loosely replicate the live engagement process. In order to gain access to the engagements, please email [email protected]

These articles were cleared for release by DARPA on January 18, 2018 under Distribution Statement "A" (Approved for Public Release, Distribution Unlimited).