Heterogeneous Computing from the First Principle : Part #1

This post is the first in a series where I share with you my experience and knowledge about Heterogeneous Computing (HC), which is the subject of my Master’s degree (“Exploiting High Performance Heterogeneous Computing (HPHC) in Mobile, low power and low-cost Devices”).
We will go through a historical approach up to real world example to understand the processing units from its first generations; from single instruction processor to heterogeneous system processor.

By the end of this series, you will understand HC and write your first application for heterogeneous systems.

Heterogeneous Computing

Heterogeneous Computing simply means executing programs on computer platforms with computer nodes that have different characteristics. This approach takes advantage of multiple processing units such as CPUs, GPUs, DSPs (digital signal processors) and FPGAs (field programmable gate arrays) to improve performance, power efficiency and flexibility. We will talk about each one of this units later don't worry :D.

Now let's break down this definition and start with what we know. Usually, every computer program runs on a CPU (computational processing unit) because this is the main element and the brain of the computer. So where does the word "heterogeneous" come from?

Moore's Law

As computers grow, and customers need more performance for their tasks, we need to develop a new powerful processing units to achieve the user and the industry needs.

In 1965, Gordon E. Moore publish an observation about the market of semiconductors, this observation is known today as Moore's Law.

The law says that the number of transistors will be increasing every year (doubles approximately) in a single chip.
The plot above show's that cramming more transistors on a single-chip increases the frequency of the single core, which will increase the performance. Performance isn't the only metric that will increase, the power consumption also will increase.

By the year of 2005, power consumption enforce that N.transistors ⇒ Performance equation is no longer valid, the number of transistors increases but frequency and performance stay the same. To keep power consumption under control, researchers and manufacturers bring up the multicore system into a single chip and heterogeneous system later on.

In this first post I just want to put you in the context of computer architectures and heterogeneous computings systems, In next post we will talk about different Processing Paradigms and different processor architectures.

Written with ♥️

References

• David Money Harris and Sarah L. Harris. "Digital Design and Computer Architecture". Second Edition 2013.
• Mohamed Zahran. "Heterogeneous Computing: Hardware and Software Perspectives". Association for Computing Machinery, 2019.
• G.E. Moore. "Cramming More Components Onto Integrated Circuits"
• Karl Rupp. "Microprocessor Trend Data". : https://github.com/karlrupp/microprocessor-trend-data.