ULPBench - Beyond the Data Sheets of Ultra-Low-Power MCU

Beyond the Data Sheets of Ultra-Low-Power MCUs:

What Does Energy Efficiency Really Mean?

By Dr. Claus Kühnel and Frank Riemenschneider
Original Article published in Elektronik 16/2015, starting on page 26 on August 11, 2015
http://www.elektroniknet.de/halbleiter/mikrocontroller/artikel/120997

For battery-operated devices, as well as dedicated controllers for the "Internet of Things" (IoT), the energy consumption plays a prominent role. No one appreciates having to recharge their battery-powered devices every evening.

The manufacturers of electronic components, such as microprocessors and microcontrollers, increasingly employ semiconductor technology and circuit design techniques that drastically reduce the power consumption of these components.

The Embedded Microprocessor Benchmark Consortium (EEMBC) was founded in 1997 to develop benchmarks that will help developers understand the performance and energy consumption characteristics of their systems, as well as helping them select the best processor for their applications. The popular performance benchmarks, such as CoreMark, MultiBench, and FPMark, all come from EEMBC.

Recently, EEMBC has developed ULPBench and an accompanying EnergyMonitor to allow users to accurately and systematically analyze and compare the energy consumption of ultra-low power microcontrollers. The EnergyMonitor can measure energy consumption of a device under test running at an operating voltage of 3V and a current consumption of less than or equal to 28 mA.

Depending on whether it is an IoT edge node or other battery-powered application, "Ultra-low power" (ULP) requirements can vary considerably. For example, the lowest power consumption is desired when dealing with very limited power resources (e.g. energy harvesting). On the other hand, security systems spend most the time in the microcontroller’s sleep mode, waking only sporadically for executing a task. Furthermore, ULP can also imply that all tasks are run during a limited time period in order to put the system back to sleep as quickly as possible. Nevertheless, a practical implementation will utilize a combination of the specified modes, as well as other measures in order to meet ULP requirements. An ULP application targeting months or years of operation will present the developer with a number of challenges to effectively optimize the system’s behavior. An increasing number of microcontrollers promise ULP properties in their specifications and data sheets, but these are not always easy to reproduce in the real world.

The EEMBC ULPBench builds on the data sheet claims and provides a method to reliably evaluate and compare the energy efficiency of microcontrollers. The methodology behind ULPBench has been defined in an EEMBC working group in which all the relevant manufacturers are represented, the measurement results presented in this article cannot be denied by the manufacturers without criticizing their own work.

The Description of ULPBench

ULPBench follows a two-part approach. First, there are different software tests for the comprehensive measurement of the microcontroller’s efficiency. These tests are characterized by common stress patterns that are interchangeable between 8-, 16-, and 32-bit microcontrollers. The testing utilizes the microcontroller’s low-power modes, combined with an active period, to analyze the effects of active and low-power operating conditions. Second, there is the so-called EEMBC EnergyMonitor, consisting of the EnergyMonitor board to make the physical energy consumption measurements. The detailed description of ULPBench was previously published in the Electronik in an article entitled "EEMBC benchmark results revealed by low-power microcontrollers" ^[1].

Preparing for the ULPBench Test Measurements

The first steps:

Obtain the EnergyMonitor hardware from EEMBC
Register the EnergyMonitor at the EEMBC website
Accept the license agreement terms, and confirm registration by e-mail received
After completing the registration, download the ULPBench software (ulp_1.1.11.tgz)
Install the software
Connect the USB port of the EnergyMonitor Board (Figure 1) with a USB port on the PC (a dedicated USB hub is preferred). Note: the USB driver must be manually installed because the USB driver is not recognized by the PC. The appropriate drivers are located in the /bin/USB_CDC folder.
Connect the terminals +3 V (VCC) and ground (GND) to the test board (DUT).

Figure 1: EEMBC EnergyMonitor Board
(click for full-size image)

The EnergyMonitor, with its built-in GUI, expects the microcontroller to be running the ULPBench-CoreProfile software. EEMBC supplies the source code of the ULPBench-CoreProfile software (directories: benchmarks, TES workloads, platforms).

If the target (DUT) is one of the standard platforms supported by EEMBC, the build infrastructure can be obtained directly from the Platform directory of the source code. Otherwise, the user must port the benchmark to the appropriate target using the EEMBC-provided template directory. The EEMBC community is grateful for the provision of these ports and would like to see more available in the future.

Once the ULPBench-Core-Profile binary is installed on the target, the EnergyMonitor’s VCC and GND are connected, and the EnergyMonitor GUI is started (bin/Energy Monitor.exe), it’s easy to run and generate the official benchmark result.

ULPBench-EnergyMonitor

By clicking the button "Run ULPBench", the EnergyMonitor supplies the test board with current and measures the energy consumption over the core profile.

When the benchmark is running, the status is depicted in the "Accumulated Energy (uJ)" graph of the EnergyMonitor GUI. At the end of the run, the GUI calculates the EEMBC ULPMark and displays it in the upper right corner. The average energy expenditure for each cycle is displayed In the "History" box of the GUI.

By using the "Submit" option in the File menu, after a successful benchmark run, users can report their results to the EEMBC website database. Subsequently, a Web browser submission page opens, and the user can enter all details of the target device.

The Platforms That Were Analyzed

This section describes the microcontrollers that we analyzed, also including the details of the our experience implementing the ULPBench on each platform.

Texas Instruments MSP430FR5969 LaunchPad Development Kit

The Texas Instruments MSP430FR5969 is a 16-bit microcontroller which includes FRAM (Ferroelectric RAM), known for its ultra-low power, long life, and high write speed. The MSPEXP430FR5969 LaunchPad Development Kit ^[2] is an evaluation module for the MSP430FR5969.

Figure 2: EEMBC ULPBench result for the MSP430FR5969
(click for full-size image)

The document "Getting Started, included with the EEMBC ULPBench, contains a section on the MSP EXP430FR5969 ^[3]. It provides very detailed assistance for users wishing to implement the benchmark. The document recommends using the IAR Embedded Workbench Kickstart (free 8KB version) that can be downloaded from the IAR website under www.ti.com/tool/iar-kickstart (Note: registration with IAR is required). The EEMBC ULPBench result for the MSP430FR5969 shown in Figure 2. A value of 121 is good, but not as high as we expected for an FRAM device. We’ll explain this result in the next section.

Texas Instruments MSP430FG4618/F2013 Experimenter’s Board

The MSP430 family from Texas Instruments represents a wide variety of ultra-low-power 16-bit microcontrollers. These include an assortment of features, of which the vast majority of derivatives are equipped with conventional flash memory, not FRAM. We found it somewhat strange that TI has not released ULPBench values for any of these flash-based controllers on the ULPBench-website of EEMBC.

In this testing exercise, we used the MSP430FG4618 Experimenter's Board ^[4]. The getting-started document ^[5] could be used as guidance only to give a general sense of direction, with a recommendation to use of the free version of the IAR Embedded Workbench Kickstart that can be downloaded from www.ti.com/tool/iar-kickstart (registration with IAR is required).

For testing, you can use the same code from the previous section, but select the alternative microcontroller in the Options section.

Figure 3: EEMBC ULPBench for the MSP430FG4618
(click for full-size image)

The 32-kHz crystal X1 (type LFXT) on the Experimenter Board had to be refitted. Shown in Figure 3 is the result of the EEMBC ULPBench for the MSP430FG4618, but the result of only 31.74 is disastrous. We believe that this low result is because the proprietary 16-bit CPU is so inefficient at processing that the controller was in its active mode significantly longer than almost all competitors in order to process the given workload. This issue is compounded by the fact that the MSP430 is manufactured in a 180 nm process, therefore, it is particularly energy-hungry even in active mode.

MSP432P401R Launchpad

Texas Instruments’ MSP432P401x controller family is the latest addition to the portfolio of efficient Ultralow-power mixed-signal microcontrollers. Instead of using the proprietary 16-bit CPU, the MSP432P401x’s core is now an ARM Cortex-M4 processor ^[6] and includes a variety of configuration options and extensive peripherals.

During our experimentation, we were able to get a ‘Blink’ program to run once, but encountered difficulties trying to get ULPBench to run. We discovered that the cause of this problem lay with the controller’s 32-kHz quartz, and TI assisted in analyzing the error and replacing the board.

Figure 4: EEMBC ULPBench for the MSP432P401R
(click for full-size image)

However, after replacing the board, a similar behavior was observed and the ULPBench launched unreliably. TI attributed this issue to suboptimal settings of quartz-parameters: Setting the Crystal to its lowest drive strength does not work. After we configured it to a slightly higher drive strength, the board starting working as expected. The MSP432P401R’s ULPBench result (after adjusting the settings appropriately), are shown in Figure 4. This new device is about five times more energy efficient than the previously discussed 16-bit derivative.

Microchip PIC24FJ64GA202

The Microchip PIC24FJ64GA202 is a 16-bit microcontroller with an integrated hardware crypto module and the company’s so-called eXtreme Low-Power technology (XLP). This microcontroller family also includes 128 KB Flash, 8 KB RAM and advanced peripherals. The combination of these properties helps target these microcontrollers for low-power embedded security applications.

Figure 5: EEMBC ULPBench for the PIC24FJ64GA202
(click for full-size image)

To assist in the porting of EEMBC’s ULPBench, Microchip provided us with a readily adapted hardware_setup.c file. When we used Microchip’s proprietary compiler to build the code, we obtained a ULPBench result of 40. On the other hand, when Microchip provided us a pre-compiled binary file, we obtained a result of 77.43 (Figure 5).

To sort out this discrepancy, we had a conference call with Microchip staff who clarified various options for obtaining our measurements. First ensure that the controller is not in debug mode. Second, remove the appropriate jumpers (JP9) and connect the EnergyMonitor to J10 (2 and 3); these steps ensure the prevention of parasitic power consumption. Nevertheless, a value of 77.43 doesn’t put the PIC in the top spot, and we attribute this to the microcontroller’s relatively weak CPU computational abilities, requiring the microcontroller to remain longer in its energy-hungry active mode.

STMicro’s STM32L476RG

Figure 6: EEMBC ULPBench for the STM32L476RG
(click for full-size image)

The STMicroelectronics STM32L476xx devices, supporting a clock frequency up to 80MHz, are ultra-low-power microcontrollers based on the ARM Cortex-M4. The Cortex-M4 core has a floating-point unit (FPU) with single precision, although this doesn’t add value for increasing ULPBench efficiency. STMicro provides an STM32 development board called Nucleo, for a cost-effective and flexible way to investigate the STM32L476RG. ST also provides the entire project for the Nucleo-L476, and a patch for the Keil IDE was also delivered. When we attempted to build the ULPBench code, we initially received 30 compiler errors. After receiving compiling, downloading, and debugging tips from ST, we still witnessed relatively high values on the EnergyMonitor. There have been differences concerning IRAM within the file project ulp_stm32l4xx.uvprojx; these have been corrected after changing the boot mode by deactivating BFB2 setting. The EEMBC ULPBench result is 121 for STM32L476RG (Figure 6), and although this is a good value, it’s still not on par with TI’s MSP432, which is also based on the Cortex-M4.

Freescale’s FRDM-KL27Z

The FRDM-KL27Z is an ultra-low-cost development platform for Freescale’s Kinetis L-KL17 and KL27 families based on the ARM Cortex-M0+ ^[7]. The platform includes an integrated debug interface for flash programming and control.

To support our energy analysis project, Freescale provided us with a ULPBench port for the KL27Z for the IAR Workbench. Most of platform’s hardware can be prepared by referring to the FRDM-KL27Z User's Guide. However, Freescale had to clarify that the ULPBench binary must be downloaded to the platform on OpenSDA instead of the additional JLink programmer.

Figure 7: EEMBC ULPBench for the 32-bit KL27Z
(click for full-size image)

Figure 7 shows the ULPBench result for the 32-bit KL27Z at 80.17. This result is only slightly above the 16-bit PIC from Microchip, although the Cortex-M0 + has significantly more computational ability than Microchip's proprietary 16-bit CPU.

Silicon Labs Giant Gecko Starter Kit (EFM32GGSTK3700)

The Giant Gecko Starter Kit (EFM32GGSTK3700) is a platform for evaluation and testing of EFM32GG990F1024. The EFM32GG990F1024 is an ARM Cortex-M3-based microcontroller with a clock frequency up to 48MHz. This microcontroller is part of the product line inherited with the purchase of the Norwegian company, Energy Micro. The ULPBench can be built for the EFM32 using the Simplicity Studio and IAR Embedded Workbench.

Figure 8: EEMBC ULPBench for the EFM32GG990F1024
(click for full-size image)

It’s disappointing to see a ULPBench result of only 59 for EFM32GG990F1024 (Figure 8), especially after Energy Micro long touted its Cortex-M3-based EFM32 as the "most energy friendly microcontrollers in the world." This result can be attributed to the device’s outdated TSMC manufacturing process geometry of 180-nm. Up until a few years ago, ultra-low-power MCUs were not built at mainstream foundries, however, the demand for chips in IoT edge nodes is rapidly changing this situation.

Silicon Labs Zero Gecko Starter Kit (EFM32ZGSTK3200)

The Zero Gecko Starter Kit (EFM32ZG-STK3200) is a platform for evaluation and testing of the EFM32ZG222F32. In addition to its Cortex-M3 MCUs, Silicon Labs offers the EFM32ZG222F32, part of another category of microcontrollers based on an ARM Cortex-M0+. These MCUs have a clock speed up to 24 MHz and are also part of the vast Gecko family.

Figure 9: EEMBC ULPBench for the EFM32ZG222F32
(click for full-size image)

The EEMBC ULPBench result of 115 (Figure 9) is nearly twice as high as the Silicon Labs Cortex-M3-based derivative. It’s also interesting to point out that ULPBench results of the Cortex-M0+ devices from Silicon Labs and Freescale differ by almost 35; possibly because the former is built on the TSMC 180ELL ("Extremely Low Leakage").

Atmel SAML21

The new Atmel Smart Platform is designed specifically for IoT applications in which the battery life is expected to last multiple years. According to the SAM-L21-family data sheet’s specification on power, the devices consume down to 35 uA/MHz in active mode and 200 nA in sleep mode. In addition to low power consumption, these microcontrollers also support Full Speed USB Host & Device, AES, and capacitive touch sensing. For our analysis, we used the Atmel SAM Smart L21 Xplained Pro Evaluation Kit as it is ideal for evaluation and prototyping with the Atmel SAM-ARM Cortex-M0-L21 + -based microcontrollers.

In our analysis, Atmel provided instructions for the modification of the Xplained Pro Evaluation Kit. Specifically, the kit’s modification required the removal of very small SMD components and the severing of conductive traces on the board. These changes were necessary in order to prevent parasitic power consumption (i.e. powering other things on the boards besides the microcontroller itself).

Usually, a user’s program can be downloaded to the board via USB using the Xplained Pro Evaluation Kit Atmel Embedded Debugger (EDBG). However, this is no longer available after the required trace-cutting modifications. Alternatively, you can use an Atmel SAM-ICE or J-Link JTAG emulator, making contact via the 10-pin debug connector.

Figure 10: EEMBC ULPBench for the Atmel SAM L21
(click for full-size image)

When we built the ULPBench code, the compilation in Atmel Studio was quick and without error. When we ran ULPBench, make note that the Brown-Out Detect fuses were disabled. The Atmel SAM L21 got a ULPBench result of 161 (Figure 10) – all the more remarkable, as it was achieved with a Preliminary Device (i.e. not shipping in quantity). For the Rev. B devices, Atmel expects enhanced values. Among all the Cortex-M0+ -based MCUs that we analyzed, the SAM21L is clearly at the top – exceeding the value of Freescale’s KL27 by 100%!

Renesas RL78

Figure 11: EEMBC ULPBench for the RL78
(click for full-size image)

The 16-bit RL78 / G14 microcontroller combines low power consumption (CPU: 66 uA/MHz, standby (STOP): 240 nA) with a computing power of 44 DMIPS (32 MHz). To evaluate this microcontroller, we could have used the QB-R5F104LE-TB Target Board, which includes Renesas’ on-chip debug emulator E1 and programmer. However, Renesas, could not support this project, therefore, we refer to a comparable representation directly from the manufacturer. With a ULPBench result of 84.67 (Figure 11), the RL78 reaches a value on the order of Microchip’s PIC previously described.

The Bottom Line

The EEMBC ULPBench serves a quantitative means of backing up statements from manufacturer on the ULP properties of their microcontrollers. The first version of ULPBench does not include energy consumption of microcontroller peripherals - only the CPU and the memory system are represented. A subsequent version of ULPBench will include support for traditional peripherals such as A/D converter, SPI, and PWM.

In our analysis, we did a hands-on investigation of a variety of microcontrollers, all of which are marketed as ‘ultra-low-power’ MCUs. Furthermore, all the microcontrollers we investigated, were from manufacturers who continue to work in the ULPBench working group of EEMBC and therefore, support this benchmark and its associated measurement methodology.

Throughout our investigation, we used development environments such as IAR Embedded Workbench, Keil MDK 5, Microchip’s MPLAB 2.35, and Atmel’s Studio 6.2, which showed a high degree of complexity. Having some experience with these tools is very advantageous.

Also in our investigation, the manufacturers usually granted us generous support with respect to hardware and toolchain configuration - numerous e-mails and some phone conferences have helped to obtain the results shown here.

Figure 12: The collective EEMBC ULPBench scores
(click for full-size image)

The collective EEMBC ULPBench scores are shown in Figure 12. It is striking that the peak values are derived almost exclusively from controllers with 32-bit ARM CPUs (namely M4 and M0+) - no MCU with proprietary 16-bit CPUs could come close, with the exception of a niche product from TI in the form of a FRAM controller whose memory system power efficiency more than offset the CPU’s weakness.

What’s Coming?

Naturally, you can rightly discuss how representative is ULPBench of real-world ultra-low-power applications. Applications will always exist with different duty cycles (the ratio of active mode to sleep mode) and different workloads, that will bring different results. On the other hand, the manufacturers - to produce a fair comparison – have agreed on the ULPBench format. And even though there are deviations in real applications, this benchmark provides a clue as to which microarchitectures and manufacturing technologies have the energy efficiency lead.

Ultimately, you will have to perform more detailed evaluations to select the appropriate chip for your application, but ULPBench, as well as our investigative work should help you save an enormous amount of time. Hence, we will continue these studies in the future, focusing on the latest products and subsequent versions of EEMBC ULPBench.

Appendix: An example implementation on Atmel SAML21

In this section we will show, by way of example with the SAML21, the necessary ports to run ULPBench. In the ULPBench directory "template", it includes board-specific porting. Specifically, these are board.h and hardware_setup.c files, as shown below:

board.h

#ifndef _BOARD_H_

#define _BOARD_H_

// Get board and chip defines

#endif /* _BOARD_H_ */

hardware_setup.c

/*********************************************************

EEMBC ULPBench Software is a product of EEMBC and is provided under the terms of the ULPBench License that is distributed with the official EEMBC ULPBench Software release. The Software is the proprietary intellectual property of EEMBC and its members and is protected under all applicable laws, including all applicable copyright laws. If you received this EEMBC ULPBench Software without the accompanying ULPBench License, you must discontinue use and download the official release from http://www.eembc.org/benchmark/ulp_sl.php.

**********************************************************/

//=============================================================================

// Platform.c

// Platform-specific declarations

//=============================================================================

// board/chip defines

#include „board.h"

#include „TesTPI.h"

#include „CoreProfile.h"

//=============================================================================

void RTC_Start( void )

{

// $$$ Code to init the RTC.

}

void hardware_setup_part1( void )

{

// $$$ init device phase 1 (device specific startup)

RTC_Start();

}

void hardware_setup_part2( void )

{

// $$$ init device phase 2 (device specific, after TES started)

}

//=============================================================================

/*__interrupt*/ void pltTimer_ISR (void)

{

tesTimerInterrupt();

// $$$ Code to finish Sleep mode.

}

The following settings are required for the Atmel SAML21 port to the EEMBC ULPBench. Detailed knowledge of the respective microcontroller’s registers is necessary to implement the port.

board.h

#ifndef _BOARD_H_

#define _BOARD_H_

// Get board and chip defines

#include <compiler.h>

#include <gclk.h>

#include <port.h>

#endif /* _BOARD_H_ */

hardware_setup.c für SAML21

/*********************************************************

**********************************************************/

//=============================================================================

// Platform.c

// Platform-specific declarations

//=============================================================================

// board/chip defines

#include „board.h"

#include „TesTPI.h"

#include „CoreProfile.h"

//=============================================================================

void RTC_Start( void )

{

// $$$ Code to init the RTC.

//clock for RTC : Xosc32k

OSC32KCTRL->RTCCTRL.bit.RTCSEL = OSC32KCTRL_RTCCTRL_RTCSEL(OSC32KCTRL_RTCCTRL_RTCSEL_XOSC32K_Val);

RTC->MODE0.CTRLA.bit.SWRST=1;

while(RTC->MODE0.SYNCBUSY.bit.SWRST>0);

RTC->MODE0.INTENSET.reg = RTC_MODE0_INTENCLR_CMP0;

RTC->MODE0.COMP[0].reg = 32768;

while(RTC->MODE0.SYNCBUSY.bit.COMP0>0);

//Mode 0

RTC->MODE0.CTRLA.reg = RTC_MODE0_CTRLA_MODE(0)|RTC_MODE0_CTRLA_PRESCALER(RTC_MODE0_CTRLA_PRESCALER_DIV1_Val);

RTC->MODE0.CTRLA.reg = RTC_MODE0_CTRLA_ENABLE|RTC_MODE0_CTRLA_MATCHCLR|RTC_MODE0_CTRLA_MODE(0)|RTC_MODE0_CTRLA_PRESCALER(RTC_MODE0_CTRLA_PRESCALER_DIV1_Val);

while(RTC->MODE0.SYNCBUSY.bit.ENABLE>0);

}

//Initialise BOD33 in sampling mode for RUN and STANDBY

//Threshold is set at approximately

void BOD_init_sampled(void)

{