Theoretical embedded linux requirements

Question

I come from a programmer background using Java, C#, C++, Javascript

I got my self a Raspberry Pi (Model 1 A, the one without ethernet) and played around for a while with it. I used Raspbian and Arch Linux ARM (since it was said it is small and customizable). Unfortunatly I didn't manage to configure them as I want to have them.

I am trying to build a nice looking (embedded) system with the only goal to start (boot) the Raspberry Pi fast and autostart a test application which will be written in C# (Mono), C++ (Qt), Java (Java Runtime) or something in JavaScript/HTML.

Since I was not able to get rid of all the log messages (i got rid of most), the tty login screen, the attempts of connecting to the network (although the Model 1 A does not have ethernet at all) booting was ugly and took long (+1 minute in some cases).

It seems I will have to build a minimum embedded linux but I have a lack in the theory of embedded linux elements and how they fit together.

My question: What are the theoretically required parts of an embedded linux holding either mono, qt, java runtime on a raspberry pi?

So far I know the following parts:

• the hardware (raspberry pi model 1 A) + sd card
• the sd card holds 2 partitions, 1 boot partition (fat32), 1 data partition (ext4)
• a boot loader
• a linux kernel (which can be optimized to the needs of a raspi)

But what then? My research got lost at "use a distro" what I don't want. What are the missing pieces between the kernel and starting an application?

Answer 1

An Embedded Linux system is comprised of many different parts that work together towards the same goal of making things work efficiently.

Ideally, that is not much different from a regular GNU/Linux system, but let's see in detail the building blocks of a generic embedded system.

For the following explanation, I am assuming as architecture ARM. What is written below may differ slightly from implementation to implementation, but is usually a common track for commercial embedded systems.

Blocks of a GNU/Linux Embedded System

• Hardware

SoC

The SoC is where all the processing takes places, it is the main processing unit of the whole system and the only place that has "intelligence". It is in charge of using the other hardware and running your software.

It is made of various and heterogeneous sub-blocks:

1. Core + Caches + MMU - the "real" processor, e.g. ARM Cortex-A9. It's the main thing you will notice when choosing a SoC. May be coadiuvated by e.g. a SIMD coprocessor like NEON.
2. Internal RAM - generally very small. Used in the first phase of the boot sequence.
3. Various "Peripherals" - connected via some interconnect fabric/bus to the Core. These can span from a simple ADC to a 3D Graphics Accelerator. Examples of such IP cores are: USB, PCI-E, SGX, etc.
4. A low power/real time coprocessor - some systems offer one or more coprocessor thought either to help the main Core with real time tasks (e.g. industrial communication buses) or to handle low power states. Its/their architecture might (or not) be a relative of the Core's one.

External RAM

It is used by the SoC to store temporary data after the system has bootstrapped and during the bootstrap itself. It's usually the memory your embedded system uses during regular operation.

Non-Volatile Memory - optional

May or may not be present. In your case it's the SD card you mentioned. In other cases could be a NAND, NOR or SPI Dataflash memory (or any combination of them).

When present, it is often the regular source of data the SoC will read from and usually stores all the SW components needed for the system to work. Could not be necessary/useful in some kind of applications.

External Peripherals

Anything not strictly related to the above. Could be a MAC ID EEPROM, some relays, a webcam or whatever you can possibly imagine.

• Software

  First of all, we introduce what is called the bootchain, which is what happens as soon as you power up your SoC and - someway - tell it to start running. In the following list, the bootchain is the subsequent calls of point 1 to point 4.

Apart from specific/exotic implementations, it is more or less always the same:

1. Boot ROM code - a small (usually masked - aka factory impressed) memory contained in the SoC. The first thing the SoC will do when powered up is to execute the code in it. This code will - generally according to external configuration pins - decide the so-called "boot strategy" or "boot order", which is where (and in what order) to look for additional code to be executed.
   
   The suitable mediums are disparate: USB storage devices, USB hosts, SD cards, NANDs, NORs, SPI dataflashes, Ethernets, UARTs, etc. If none of the above contains something valid, the Boot ROM will usually issue a soft reset of the SoC, and so on.
   
   The code in the medium is not, of course, executed in place: it gets copied into the Internal RAM then executed.

[The following two are contained in what we will call bootloader medium]

2. 1st stage bootloader - it has just been copied by the Boot ROM into the SoC's Internal RAM. Must be tiny enough to fit that memory (usually well under 100kB). It is needed because the Boot ROM isn't big enough and does not know what kind of External RAM the SoC is attached to.
   
   Has the main important function of initializing the External RAM and the SoC's external memory interface, as well as other peripherals that may be of interest (e.g. disable watchdog timers). Once done, it copies the next stage to the External RAM and executes it. Depending on the context, could be called MLO, SPL or else.

3. 2nd stage bootloader - the "main" bootloader. Bigger (could be x10) than the 1st stage one, completes the initializiation of the relevant peripherals (e.g. ethernet, additional storage media, LCD displays).
   
   Allows a much more complicated logic for what to do next and offers - depending on the level of sofistication - high level facilities (filesystem/volume handling, data copy-move-interpretation, LCD output, interactive console, failsafe policies).
   
   Most of the times loads a Linux kernel (and related) into memory from some medium and passes relevant information to it (e.g. if not embedded, for newer kernels the DTB physical address is put in the r2 register - the Kernel then reads the register and retrieves the DTB)

4. Linux Kernel - the core of the operating system. Depending on the hardware platform may or may not be a mainline ("official") version.
   
   Is usually completed by built-in or loadable (from an external source - free or not) modules. Initializes all the hardware needed for the complete system to work according to hardcoded configuration and the DT - enables MMU, orchestrates the whole system and accesses the hardware exlusively. According to the boot arguments (cmdline - usually passed by the previous stage) and/or to compiled options, the Kernel tries to mount a root file system. From the rootfs, it will try to load an init (namely, /sbin/init - where / is the just mounted rootfs).

5. Init and rootfs - init is the first non-Kernel task to be run, and has PID 1. It initalizes literally everything you need to use your system. In production embedded systems, it also starts the main application. In such systems is either BusyBox or a custom crafted application.

More on rootfs and distros

Rootfs contains all of your GNU/Linux systems that is not Kernel (apart from /lib/modules and other bits).
It contains all the applications that manage peripherals like Ethernet, WiFi, or external UMTS modems.

Contains the interactive part of the system, contains the user interface, and everything else you see when you boot a GNU/Linux system - embedded or not.

A "distro" is just a particular collection of userspace (non-Kernel) programs and libraries (usually) verified to work well one with the other, put toghether by a particular group of people. Desktop distros usually also ship with a custom-tailored kernel and a bootloader. Examples are Fedora, Ubuntu, Debian, etc.

In the general sense of the term, nothing stops you from creating your own distro, which is what happens everytime a custom embedded system goes in production: through tools like Yocto or Buildroot (or by hand), in fact, you are able to decide the very particular collection (hence distro, distribution) of softwares fit for the purpose of the system.

To sum up and answer exactly to your question, the missing part you are looking for is init and the process of mounting the rootfs: the Kernel mounts - aka renders available to itself - via its drivers and the passed/builtin parameters - a given volume/partition (the ext4 data partition you mention) to the "/" mount point.

In this volume/partition there is a /sbin/init executable, which the Kernel executes.

This is the "Big Bang" of our GNU/Linux userspace system: the place where everything visible starts. Depending on the configuration scripts (usually located under /etc/init.d) the "application" you mention is either run automatically by init or by the user via a terminal/ssh/whatever that - again - init made you possible to use.