The Yocto framework and MIPS
Although you may not realise it, there’s a good chance that your Wi-Fi router, digital camera and TV have something in common. There’s a high possibility that the software running on them is a heavily customised version of Linux designed for embedded systems. Running a customised version of Linux is ideal if you want to use the OS for a specific task and optimise it for your workflow.
This article will provide a brief look at how designers can use the Yocto open source infrastructure to customise an embedded Linux distribution on a MIPS platform – enabling them to focus on their product, not the complexity of building a customised distribution.
There are many ways to create a custom embedded Linux distribution (distro). The first stage is to obtain the required software libraries and open source packages – these can all be easily located and downloaded from the Internet. After that, everything must be manually cross-compiled and put together by hand. To make this tedious process easier, there are many tools and frameworks available to assist developers.
One of the most widely known and successful of these is provided by the Yocto project and Imagination has been working hard to customise it for use with our MIPS processors. This continues to build on the already comprehensive universe of tools and software available for MIPS.
However, before we detail the work on customising Yocto for MIPS, it makes sense to explain what the Yocto project is.
What is the Yocto Project?
To describe the Yocto Project, I thought we could do no better than to quote from the FAQ on its website:
“The Yocto Project provides an open source, high-quality infrastructure and tools to help developers create their own custom Linux distributions for any hardware architecture and across multiple market segments. The Yocto Project is intended to provide a helpful starting point for developers. The Yocto Project hosts other projects as well, including the Poky build system, Autobuilder automated build and test system, and the Embedded GLIBC (EGLIBC)C library.”
Alternatively, you could watch the video below:
For those familiar with the Buildroot distribution environment, Yocto is another framework that supports more features e.g. more packages and support for package feeds. As a result, it is much more comprehensive.
Introduction and internal structure
Yocto is structured in three primary layers (there are others, but these are outside the scope of this document). The contents and configuration of each layer is defined by metadata called ‘recipes’.
These are the three layers:
– Base layer (Open Embedded Core): This is a collection of core software that is common in embedded Linux distros. One such example is BusyBox; software that provides several stripped-down Unix tools in a single executable file.
– Machine layer: This adds machine/board-specific metadata, such as kernel patches, kernel config, Bootloader etc.
– Software layer: This adds a specific collection of software e.g. Gnome, XFCE, NodeJS etc.
The base layer OE-Core has the recipes needed to create a fully functional embedded Linux distribution and is a core requirement for any distribution based on the Yocto framework. A configuration file can be used to add further layers. Once the layers and recipes are configured, the Bitbake build tool parses these recipes and configuration files to download and compile everything to create the distribution.
What’s the starting point for Yocto?
Yocto is a comprehensive framework that supports the inclusion and configuration of many different tools and systems, and it can be quite daunting to know where to begin. However, the Yocto project provides a reference Linux distribution called Poky that provides an excellent place to start. Poky provides configuration files for a headless system called ‘core-image-minimal’ and a version with a graphical user interface (GUI) called ‘core-image-sato’.
Thanks to these, a product developer only needs to focus on the recipes required to incorporate the software needed for their product; the rest of the infrastructure is provided by Yocto.
For example, I have a software system with a UI written using the QT framework. I can start creating an embedded Linux Distribution using core-image-sato as a reference point. I’ll fetch the QT software layer and add the path to my config file. And I’ll write a custom recipe for my software application describing the dependency on QT. Bitbake will understand and parse the configuration files and create a core-image-sato distribution with QT and my custom software. With a few configuration files and recipes, we have an embedded Linux distribution that supports our QT based software UI.
Where does MIPS fit in?
Historically, MIPS has been included in Yocto as part of the OE-Core layer, and Imagination recently extended this support to add the new MIPS R6 Instruction Set Architecture into OE-Core for the v2.2 Yocto release of October 2016. Imagination also created ‘meta-img’, which is a machine-type layer available on GitHub. This layer adds machine configuration files for various MIPS platforms and QEMU targets including the Creator Ci40, MIPSfpga, SEAD-3 and Rhino Labs SDNA-7130 (UTM-8).
Using Poky and the meta-img layer, one can readily build a QEMU based system, start it running and get to a MIPS shell prompt.
Here’s a walkthrough to get you started:
$ git clone git://git.yoctoproject.org/poky -b morty
$ cd poky
$ git clone https://github.com/MIPS/meta-img.git -b morty
$ source oe-init-build-env
(this will change the current directory to the build one. That is OK)
$ bitbake-layers add-layer ../meta-img
$ MACHINE=qemumips32r2el bitbake core-image-minimal
On a fast machine with a solid internet connection, this process can take around two hours.
After that, you’ll find a kernel and rootfs in tmp/images/deploy/qemumips32r2el.
Yocto also compiles the QEMU emulator and provides an easy script to run it with the correct arguments (‘runqemu’ located in the scripts directory).
Run qemu using
$ ../scripts/runqemu tmp/deploy/images/qemumips32r2el/ nographic
login: root
(use Ctrl-a X to exit QEMU)
Common debug notes:
i) If a missing dependency is reported when building, the error message will usually tell you about a missing utility. Use the package manager for your distribution to install that utility.
If you get an error mentioning locale isn’t UTF-8, try this for a temporary solution.
$ export LC_ALL=en_US.UTF-8
ii) gcc >= 6 was required (on Ubuntu) to build core-image-minimal
Adding custom packages in root filesystem
Now, we’ll add an additional package to customise our system. This could be a group of packages for a particular application or be configured to load the binary blob for proprietary firmware. Here the ‘python’ package is added:
i) First, we will try to find an existing recipe for python on the ‘OpenEmbedded Layer Index’ https://layers.openembedded.org/layerindex/branch/master/recipes/?q=python
ii) We see that a recipe is available in the openembedded-core layer. That particular layer is automatically included in Poky.
iii) All we need to do is add the following to ‘meta-img/conf/layer.conf’
CORE_IMAGE_EXTRA_INSTALL += “python”
This will search for a recipe with the name python in all the configured layers in conf/bblayers.conf.
- iv) Rebuild the QEMU image-minimal
$ cd “to the ‘build’ directory”
$ MACHINE=qemumips32r2el bitbake core-image-minimal
- v) Now when you start QEMU, the python package will have been included and available
$ ../scripts/runqemu tmp/deploy/images/qemumips32r2el/ nographic
login: root
$ python
Python 2.7.12 (default, Feb 7 2017, 15:08:17)
[GCC 6.2.0] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.
>>>
Python has started and run to its prompt, confirming that Python has been correctly included.
Adding custom firmware
The Yocto framework is highly flexible and enables the designer to easily add custom firmware to their root file system as well as using recipes. Let’s look at the Creator Ci40 for an example of such a recipe. The Ci40 Linux kernel driver expects the Wi-Fi firmware to be available in the root file system at the following location. /lib/firmware/img/uccp420wlan/. There are two files MCP_LOADER.ldr and MAC_LOADER.ldr.
The recipe required for putting these files in the file system is quite simple and available at this link on GitHub.
DESCRIPTION = “Closed source binary firmware for the uccp420 wlan on the Ci40”
LICENSE = “CLOSED”
COMPATIBLE_MACHINE = “ci40”
# WiFi firmware version 6.0.4 from https://github.com/CreatorDev/openwrt/blob/target/linux/pistachio/base-files/lib/firmware/img/uccp420wlan/
FILESEXTRAPATHS_prepend := “${THISDIR}/files:”
SRC_URI = “file://MAC_LOADER.ldr;name=MAC_LOADER;”
SRC_URI += “file://MCP_LOADER.ldr;name=MCP_LOADER;”
SRC_URI[MAC_LOADER.md5sum] = “90d45fdabe671ac84c23584022110391”
SRC_URI[MAC_LOADER.sha256sum] = “084c9ec49fd459d2b81926c26e81e427a0de64f30529ad52c9d17cb5dc526248”
SRC_URI[MCP_LOADER.md5sum] = “d3197e5156eb936c94547eda2447798e”
SRC_URI[MCP_LOADER.sha256sum] = “d235846a2a25e23d78a1fb0460561f7811fbeda90c1701f189ec93e8322ce330”
do_install() {
install -d ${D}/lib/firmware/img/uccp420wlan
install -m 0644 ${THISDIR}/files/MAC_LOADER.ldr ${D}/lib/firmware/img/uccp420wlan/MAC_LOADER.ldr
install -m 0644 ${THISDIR}/files/MCP_LOADER.ldr ${D}/lib/firmware/img/uccp420wlan/MCP_LOADER.ldr
}
FILES_${PN} += “/lib/firmware/img/uccp420wlan”
BitBake parses this recipe and the name-value variables defined in them. The SRC_URI variable tells the location of the files. Here, they are present in a subfolder in the same location as the recipe. They can also be referenced using many other means via HTTP, git, URL, etc.
The SRC_URI variable is also populated with the md5 and sha256 checksums of the firmware files. This is particularly useful in the case of firmware files being grabbed from separate locations or the internet. If the firmware at the destination changes, the sha256sum fails and BitBake highlights the reason with an error. This catches the error early on instead of generating a root file system with firmware that might not work due to changes.
do_install is a post-install step hook, which will run the commands needed to copy over the firmware files into the destination rootfs.
Conclusion
Yocto on a MIPS platform provides tools that enable a designer to focus on their product instead of struggling to build their customised distribution. This is just a glimpse of how to use the Yocto framework to customise an embedded Linux distribution. For further details on writing recipes, take a look at the Yocto Reference Manual.