Topics

• Benchmarking the IoT Trifecta
• SecureMark™-TLS Open Sourced on GitHub
• The Twelve EEMBC Companies Bringing Energy Benchmarking to tinyML
• EEMBC, MLperf, and Google SIG Micro to Collaborate on tinyML Benchmarking
• Tech Talk: Overcoming Obstacles to Benchmarking Wi-Fi
• Member Certifications: Renesas and STMicroelectronics

---

Benchmarking the IoT Trifecta

Three topics dominate any discussion about IoT: security, wireless battery life, and machine learning. Here at EEMBC, we've got the conversation covered with three workgroups dedicated to each of these domains: SecureMark, led by Ruud Derwig of Synopsys; IoTMark-Wi-Fi, led by Chandra Duba of Dialog Semiconductor; and Ultra-low Power Machine Learning, led by Sebastian Ahmed of Silicon Labs. Each of these workgroups is now developing their second generation of benchmarks slated for release within a year. The members of these workgroups democratically develop the behavioral specifications which ultimately become the industry standards for years to come. If you are an OEM/ODM, semiconductor vendor, or product manager who has to make decisions about embedded processors, join EEMBC and have your voice heard. Contact Peter Torelli for information on becoming a member.

SecureMark™-TLS Open Sourced on GitHub

IoT security is challenging, and without the right tools, the costs of implementing modern cryptography on an edge device can be hard to measure against the potential losses for not doing so. SecureMark-TLS was developed by EEMBC to help characterize energy-efficiency of IoT-level crypto to make informed product decisions. Recently, the workgroup voted to release the benchmark on EEMBC's GitHub repository. This repository contains both the benchmark firmware skeleton and an example desktop implementation which allows anyone to quickly get a feel for the benchmark. A corporate license is necessary if you want to publish scores in marketing literature. Submitting scores requires use of the framework GUI, which performs a battery of additional verification checks to make sure the test ran according to the rules.

The Twelve EEMBC Companies Bringing Energy Benchmarking to tinyML

Four months ago, EEMBC shifted its Machine Learning focus from high-performance compute with the MLMark® benchmark, to Ultra-low Power (ULP) constrained endpoints (aka "tinyML"). We’re pleased to announce the development of ULPMark™-ML: a benchmark that will leverage EEMBC's popular ultra-low power measurement framework to measure Machine Learning inference energy-efficiency. The workgroup developing this benchmark is led by Sebastian Ahmed of Silicon Labs, and is currently identifying models suitable to these small devices and defining the run rules by which energy will be measured. Participating in this workgroup are ML experts from the following EEMBC Member Companies: Eta Compute, STMicroelectronics, Synopsys, Cypress, ON Semiconductor, Texas Instruments, Nordic, Arm, Renesas, and Silicon Labs. Adding their experience to the discussion are technical consultants from Altran and Ignitarium. It is not too late to get involved, request membership information here.

EEMBC, MLperf, and Tensorflow SIG Micro to Collaborate on tinyML Benchmarking

For the past few years, EEMBC, MLperf and Google's Tensorflow SIG Micro have been researching and developing the rules and software used for benchmarking neural-net based Machine Learning. Recently, industry focus has shifted in earnest to ML inference at the constrained edge (aka "tinyML") which is an area of overlapping interest to these groups' members. In order to introduce more unified standards, the three organizations are collaborating to divide-and-conquer the long, long list of challenges to benchmarking in this space, with EEMBC bringing its ultra-low power (ULP) research and expertise to the table. Stay tuned for continued announcements as this develops.

Tech Talk: Overcoming Obstacles to Benchmarking Wi-Fi

Anyone familiar with 802.11 knows it can be a complex protocol: parts of the specification are ambiguous, which means implementations vary among vendors. Since repeatability is a fundamental component of all EEMBC benchmarks, these vendor variations create inconsistencies, and the IoTMark-Wi-Fi benchmark must guarantee a constrained, repeatable environment. To accomplish this, we’ve developed custom firmware for our own AP built on the Raspberry Pi platform using the hostapd service. By locking down the Broadcom driver version in the Pi’s image, and by strictly configuring hostapd (and tweaking the source code), the benchmark asserts greater control over the content of the frames and how the AP manages stations. The result is a well-behaved benchmark environment, with control over fragmentation, DTIM, channel hopping, and anything else in the 802.11 link-layer that might introduce surprises.

Member Certifications

Members continue to reinforce the value of EEMBC benchmarks in their datasheets through certifications. Since our last update, several new scores have been certified and published from the following members: