The EEMBC BenchPress

The Original Embedded Microprocessor Benchmark Newsletter
Issue #66 Q1'2022

In This Issue

EEMBC's 25th Anniversary

This May EEMBC celebrates its 25th anniversary. In the two-and-a-half decades since 1997, the embedded landscape has changed significantly. First, there are no longer 50+ major semiconductor manufacturers designing chips: many mergers and acquisitions have sharply reduced the playing field, while others have left the embedded space entirely. This has been a challenge to EEMBC's membership as a non-profit. Second, the rapid expansion of applications for embedded technology has taken EEMBC in directions we never anticipated: we've made benchmarks for smartphones, self-driving cars, machine learning systems, wireless devices using Wi-Fi and BLE, and even web browsers. Looking back, this is quite a long way from the simple 16-bit algorithms we started with in our first test suite. Today we find ourselves in a world where the 1990's dream of ubiquitous computing has become a reality through AI-enabled IoT and the Cloud, with EEMBC in the thick of it. Most importantly, the collective impetus making all of this possible has been our board members, member companies, and many licensees. EEMBC could not exist without them, and we are grateful for their support and direction. I'd also like to thank all of our employees, our current staff for their dedication, and those from the past who built EEMBC's foundation. What will embedded technology look like in 2047? I'll leave predictions to the countless pontificating futurists, but I'm certain EEMBC will be involved! - Peter Torelli, President, EEMBC, March, 2022

EEMBC to Develop the World's First Audio-Processing Benchmark

Earbuds, conferencing systems, voiced-activated navigation and TVs, smart IoT gadgets enabled with Cortana, Alexa, Siri... These products require advanced audio processing to integrate seamlessly into the background but still respond quickly and accurately to voice commands. In this type of deployment, the compute demands on a platform are diverse, and understanding the bottlenecks of such systems requires an equally sophisticated benchmark.

EEMBC is excited to announce the development of a benchmark called AudioMark™, which will characterize a real-world, configurable audio application, and target a wide range of platforms.

Under the hood, you'll find a veritable history of engineering. The typical audio pipeline of today combines technologies that date back to radar and RF broadcasting of the mid-20th century--such as beamforming and direction of arrival--as well as modern filters like acoustic echo cancellation and noise suppression. Keeping in line with recent tech trends, we've added a neural net for audio classification to respond locally or send encoded data to the cloud. AudioMark will exercise different data formats, increase the instruction cache demand, and even allow integration of accelerators such as DSPs or other dedicated audio hardware. However, it will be sufficiently balanced not to allow any one technology to dominate.

The technical lead for this effort is Ashutosh Pandley, from member company Infineon (formerly Cypress). Participating in development are Arm, Renesas, Synopsys, Intel, STMicroelectronics, and Texas Instruments.

A General-Purpose MCU Benchmark for Modern Devices

EEMBC rarely announces the development of two benchmarks simultaneously, but 2022 seems like a good year to change the status quo. In addition to AudioMark, EEMBC has formed a working group to re-visit a general-purpose compute benchmark for modern MCUs that fits somewhere in between CoreMark® and CoreMark-Pro.

We introduced the CoreMark benchmark in June of 2009, and it quickly became the de-facto replacement for Dhrystone on embedded devices. In reality, it's popularity grew as it found use in systems ranging from 8-bit PICs to 64-core Intel servers. Five years later, CoreMark-Pro was released, and focused on High-Performance Computing (HPC) in multiprocessor hosted system, adding larger workloads and dual-precision floating-point. Today there has emerged a middle segment not adequately served by this pair: high performance embedded processors that aren't running on a POSIX based O/S, but that still offer larger caches, fp32 performance, more memory, and improved multitasking for RTOS deployments.

This new benchmark, tentatively dubbed “CoreMark-Pro Version 2” by our clever marketing team, will reflect modern IoT deployments by adding meaningful concurrency, where tasks communicate with each other rather than spinning in loops. Adding asynchronous operations will cause context switches and increased cache fill, all of which will stress the fundamental characteristics of the target's architecture. By designing to common RTOS API calls, EEMBC hopes to minimize the amount of porting effort required to deploy this benchmark, making it even more accessible to a wider variety of platforms.

Participating in development are Arm, Renesas, Synopsys, Intel, STMicroelectronics, and Texas Instruments.

Third-party Crypto Libraries Add New Value to SecureMark®

Many modern MCUs offer native instructions to speed cryptography, and some platforms offer a dedicated hardware crypto engine. The SecureMark benchmark was constructed with a generic API in order to allow developers to seamlessly incorporate hardware accelerators into the benchmark. A consequence of this decision provides an additional benefit: the API also allows the user to swap in a different software library. This offers a neat twist, as we can analyze different software on the same hardware, as opposed to the same software on different hardware, which is our traditional mode. In our score database, we now have several comparisons of crypto libraries on the same MCU.

Oberon, a spinoff of the Swiss Federal Institute of Technology (ETH Zürich) which specializes in cryptography, certified several scores using their ocrypto library: "ocrypto is a modern cryptography library for Cortex-M class devices. It is designed to provide resistance against common side-channel attacks, yet delivers high speed and power efficiency. Its footprint is tiny. It requires neither heap nor static memory, and most functions use less than one KB of stack space. This unique combination yields maximum security with minimum overhead. In an industry first, the ocrypto SecureMarks-TLS scores have been certified by EEMBC and footprint numbers have been disclosed along with the scores."

For more information about Oberon please visit their main site, or read about their performance on SecureMark. Their certifications are also linked later on in this newsletter.

Similarly, EEMBC has worked with wolfSSL to port their popular wolfCrypt library to SecureMark. Formed in 2004, "wolfSSL focuses on providing lightweight and embedded security solutions with an emphasis on speed, size, portability, features, and standards compliance." We received great help from the wolfSSL team, and you can read more on their blog post.

EEMBC is glad to have both companies contribute their scores to our database!

Conferences

Visit EEMBC at the Embedded Technology Convention at the Las Vegas Convention Center, Booth 6019, June 8th and 9th! "The Embedded Tech Convention is at the forefront of educating and showcasing organizations at the heart of the advancements in embedded technology and will offer invaluable opportunities to enhance your exposure and knowledge of the market through engaging keynote content, panel debates and masterclasses."

More info can be found at the Embedded Technology Convention.

Academic Research with EEMBC

In the past few months, several new universities have licensed EEMBC software. Here is some of their research and how EEMBC meets their needs.

National University of Sciences and Technology (NUST)

Dr. Rehan Amed

"We are developing an in-house RISC-V based Embedded Microprocessor SoC. We have implemented the RV32I core which is augmented with M,F,C and B (Z) extensions. We support the peripherals that include, UART, Timer, GPIO, SPI, I2C and PWM in our SoC. The soft core is running like a charm on Intel FPGA development boards (DE-1 SOC, DE-10 Nano, DE2-115 and Arduino 4000 MKR). We are now pushing its chip tapeout using TSMC 65nm GP MPW shuttle. EEMBC benchmarks will help us greatly in benchmarking our core with rest of the crowd. We are already running Coremark (our unnormalized score is 159, with 50MHz the normalized score is 3.18) and now with this access we'll run rest of the benchmarks for evaluation."

Laboratoire de Conception et d'Intégration des Systèmes (LCIS)

Prof. Vincent Beroulle

"LCIS laboratory is a French public research laboratory from Univ. Grenoble Alpes collaborating with CEA-Leti to propose new solutions to mitigate hardware attacks against the Internet of things (IoT) and the Industrial Internet of Things (IIoT). IoT and IIoT find use as home appliances, personal usage, industrial automation. These devices take part in our daily life, handling a good amount of information. They are becoming more frequent targets for cybersecurity malware. These mostly low-cost devices have limited protections to defend against the increasing threats. Many solutions are implemented to try to defend the devices against cybersecurity threats. One of these solutions is the use of Machine Learning. The Machine Learning mechanisms try to understand the behavior of the system under normal operation and under attack. To do so, we provide system information to the algorithm and it tries to learn the complex behaviors as best as possible. One problem that designers of such mechanisms face is the lack of normal behaviors to model. IoT and IIoT find usage in multiple and different applications which the designer is difficult to predict. Further, we need applications that are well studied and used. EEMBC offers many benchmark suites with a variety of different benchmarks. The variety of applications will allow us to model the different behaviors of IoT and IIoT devices. EEMBC suites combine with Machine Learning will allow us to recognize the malware behaviors from normal behaviors."

Certifications

EEMBC offers a certification program that verifies the results of a benchmark. While most of our benchmarks self-check, some Run Rules cannot be explicitly enforced remotely. The certification process is performed at the EEMBC lab, and recreates the scores on the actual platforms. Here we scrutinize the implementation with logic analyzers and power probes to verify correctness, even going so far as to hand-analyze assembly code if necessary. EEMBC certification guarantees that a score is valid.

A large number of scores were certified in the past few months:

onsemi

Member company onsemi added to their RSL certification portfolio with two ULPMark-CoreProfile scores on the RSL15 at 1.8V and 3.0V, scoring 1220 and 1070 marks, respectively, and continues to top the ULPMark-CP leaderboard.

From onsemi:

"RSL15 is an ultra-low power secure Arm® Cortex®-M33 processor-based Bluetooth Low Energy 5.2 wireless MCU designed for connected smart devices in industrial and medical applications. With built-in power management, a wide supply voltage range, flexible GPIO and clocking scheme and an extensive set of peripherals, the RSL15 offers maximum design flexibility for high performance and ultra-low-power applications."

renesas

Renesas certified more CoreMark scores this quarter with the following entries:

The RL78 also submitted a ULPMark-PeripheralProfile certification at 125 @ 3.0V.

From Renesas:

"The RL78/G23 microcontroller group is a new generation of the RL78 family of microcontrollers, with 41μA/MHz CPU operation. The RL78/G23 group has the industry's lowest power consumption with 210nA at stop (4KB SRAM retention), and a snooze mode sequencer which significantly reduces power consumption during intermittent operation. The RL78/G23 group features a wide operating voltage range of 1.6V to 5.5V at up to 32MHz, a broad range of package pin counts from 30 pins to 128 pins, and up to 768KB of flash memory. In addition to enhanced analog and security features, it also incorporates logic and event link controllers (ELCL) and the first capacitive touch sensing unit (CTSU2L) in the RL78 family. A rich development environment is also provided for the RL78/G23, including a fast prototyping board. This board allows users to write and debug programs with an easy-to-use GUI, by simply connecting a USB cable and a smart configurator without any additional tools required. The RL78/G23 is compatible with the existing RL78 family and can be a seamless replacement, making it suitable for a wide range of applications, from home appliance and consumer electronics to industrial equipment.""

oberon

Oberon certified four scores on SecureMark for their ocrypto library. The scores were submitted on STMicroelectronics STM32L476RG and Renesas RA2E2, the later utilizing the RA2's AES engine. This set of scores can be compared at this link.



Thanks for reading!

Peter Torelli, President/CTO, EEMBC

peter.torelli@eembc.org