The Embedded Microprocessor Benchmark Consortium (EEMBC - pronounced embassy) is a non-profit organization that works collaboratively with member companies to develop performance benchmarks that target key applications of embedded systems. EEMBC activities include the EEMBC Technology Center, which delivers technical support to members and licensees, certifies benchmark scores, and oversees the development of benchmark suites.
Publication rules vary between benchmarks, but some members simply choose not to publish results, and not every licensee has the bandwidth to port benchmarks to every platform and publish results. If a processor score is not available on the EEMBC website, you may request that score on line or directly with the corresponding processor vendor. Customer requests for scores on specific processors are a key mechanism by which more scores are published, and EEMBC encourages you to request certified scores for any processor you are considering for a design.
EEMBC offers a certification stamp for members who wish to guarantee that the benchmarks were run according to the specific run rules of the benchmark. The certification process scrutinizes the modified source code, validates that the verification flows (available on some benchmarks) pass, and double checks external hardware interaces with a logic analyzer (if necessary) to verify the integrity of the certified score.
Up until 2012, EEMBC stood for the "EDN Embedded Microprocessor Benchmark Consortium". The name has since been changed to drop the "EDN" (as EDN is no longer in business), but we kept EEMBC because for over 20 years that's what we've been known as: it's our identity. Plus it is easier to Google... And heck, the two E's looks pretty cool.
Once you have become an EEMBC member, as many people in your company as you like may download the EEMBC benchmark suites, but each such individual must be advised of the EEMBC rules and the provisions of the EEMBC Member License Agreement. Moreover, each person accessing the members-only area of the EEMBC Web site to download benchmarks or obtain other information must register individually. You are also responsible for seeing that any employees who leave your company do not take the benchmarks outside of your company.
EEMBC Board of Directors meetings combine both technical and marketing-oriented discussions, and our member representatives include individuals whose roles in their respective companies are purely technical, purely marketing, and everything in between. In general, the most technical discussions in EEMBC take place on the working group level by teleconference and web-meetings outside of Board meetings.
Focus on those parameters that most closely match your application needs. A designer typically uses the benchmark scores as a criterion to pick the best processor for the application. For example, if the application does many table look-ups, then that parameter (in the Automotive/industrial benchmark suite) is most important and the designer will like give the most weight to that benchmark. On the other hand, if the application is a modem, then the benchmark scores in the Telecomm suite become the most important parameters.
Absolutely. Feedback from several processor vendors indicates that they have seen performance differences upwards of 40% simply as a result of using a different compiler. This is why vendors such as Green Hills and IAR are actively involved in the EEMBC process.
Over time, as more processor vendors submit scores from a variety of compilers running on the same processor, this will provide very valuable information. Also, if you held the compiler family constant and varied the processor, this would be a great way to compare a processor's C friendliness.
A compiler vendor's results on the web site are not related to whether or not that vendor is an EEMBC member. The processor vendor member typically makes that decision, and they usually publish scores from the compiler that yields the best scores.
EEMBC-allowed optimizations are decided on a benchmark-by-benchmark basis. There are some fundamental restrictions in place to ensure fair comparison. EEMBC allows optimizations that change the source code to take advantage of libraries or hardware accelerators. These restrictions will allow you to demonstrate an architecture's capability and facilitate practical comparisons. You can also use intrinsics ( i.e. C code modification), or change the algorithm in some cases. For example, if EEMBC has a radix-4 implementation of an FFT, we will allow it to change to radix-2. So it is still an FFT, but on some DSPs that makes a big difference.
1. Log in to EEMBC member section and 'Make a Certification Reservation'.
2. Build and run the benchmark(s) on your platform.
3. Send your platform and tools to EEMBC Technology Center. Depending on the benchmark, you may be required to submit more than one board for verification.
4. Enter your benchmark data in the online disclosure report (in member section), or run the host software that submits the score to the website for you (depends on the benchmark).
The disclosure form contains all the information necessary to recreate the benchmark environment. In addition to all the benchmark scores, it ontains board-level information (memory configuration, etc.), compiler information (settings, libraries, etc.), and processor information (core and bus speeds, cache sizes, on-chip peripherals, etc.) After certification is complete, and the scores are published, the associated disclosure is publicly available alongside the benchmark scores.
FFTs are used more often in Telecomm applications, but they are becoming increasingly important in automotive and industrial applications. Sensors used in engine knock detection, vehicle stability control, occupant safety systems and flow control, all require sophisticated filtering of the incoming signal to determine what is really happening among a lot of noise.
All of the kernels (Bit Allocation, AutoCorrelation, FFT/iFFT, Viterbi, Biquad IIR, Convolutional Encoder) in the suite are intended to be run (for comparison purposes) using 16-bit fixed point math. There are FP versions of some of the kernels which were used to generate the golden data file sets. At this time, we have no plans to extend the "Out of the Box" code to require/use FP math. You may re-code the kernels, using FP for an optimized version of the benchmark if you wish. At this time, we do not have access to any floating point data sets. The data sets which accompany the AutoCorr kernel are in 16-bit (1.15) fixed point format (that is: 1 sign bit, 15 bits of data) to reduce the precision of the data, you can remove (or zero) bits from the LSBs of each number.