A new way to develop & publish SingleBoardComputing IoT Projects
I want to share a better way to accelerate & automate the development of Single-Board-Computing projects with you. (e.g. the Raspberry pi)
The Raspberry Pi is an optimal platform for hobbyists who want to build prototypes and learn about computers. The way that we share Rasbperry pi projects with others on the Interenet is not optimal.
What are the current problems facing people who want to share their Raspberry Pi Project?
Publishing a full firmware image to share with others is hard for a variety of reasons:
- Firmware images for linux builds are large (For example, The Quake On Lan firmware is 1.38 gigs. https://www.quakeonlan.net)
- Large files have greater hosting providers costs (Note that the Quake on Lan team manages download costs by asking people to pay/register for access to faster downloads https://ufile.io/jvp3et41)
- Projects that go viral have unpredictable and potentially significant hosting costs (See above for large file-distribution coping examples)
- Firmware build process requires perfect execution. Failing to implement perfect, repeatable execution can result in exploitable secrets lingering in the filesystem or History files. It also puts you at risk of difficult to discover configuration mistakes. Unit tests must be written to discover & remove private material from final build image
- Iterative image development is error prone. If a DD operation against a firmware image results in block errors, these can be difficult to correct in successive builds, costing you disk space on the final image. Sharing firmware between developers carries undiscovered mistakes forward into successive builds.
- Distributing large files for incremental updates slows down development. Developers have to wait for the latest firmware image to finish downloading before they can commence making modifications.
- Build procedure documentation goes stale. (Google “Raspberry Pi Wifi hotspot”. Note that the responses reference IP Tables in their configuration, which is no longer included in Raspbian (https://thepi.io/how-to-use-your-raspberry-pi-as-a-wireless-access-point/)
- Human executed installation of software risks mistakes in the final build.
Embedded IoT developers must first come up with a prototype that works. This can involve a lot of trial and error that will leave your environment’s history command filled with both useless or wrong commands as well as the ones that are correct. When you’ve finished making one working prototype, you need to go through the process a second time, while only limiting yourself to the necessary commands for creating the device. Those exact and reduced commands must be documented. Next you have to test your documentation for accuracy. Then you’re finally ready to publish your build process. That decision to share your your project started as a simple act of charity. It becomes a significant investment of time and money if you’re going to do it well. If you decide to skip documenting your steps & instead publish a firmware image of your project, you’re going to pay bandwidth bills for files that usually start at 2 gigs or greater. In that situation, you’re asking users who might have slow internet connections to download very large files.
There are problems for the people who will read your guide as well. Users who follow Raspberry pi project guides spend almost as much time troubleshooting stale documentation as they do troubleshooting their actual device. If someone follows your instructions 2 years after you’ve published, the user may find that linux has moved forward and your commands don’t work. The effort you put into documenting the series of commands that create your project is significant. The developer is rewarded by impending document obsolescence as soon as they publish it.
- .img files are too large for reasonable Internet distribution
- It’s hard to find project guides with instructions that are up-to-date
- Pi project guides have either tedious manual steps or depend on dangerous solutions for automation
- Deploying firmware to multiple pi’s simultaneously is a serial operation
- Documenting all the components of your project during image development based on history files is a pain
A solution that addresses these problems should have the following properties:
- It should be inexpensive to share your project with others
- It should be easy to keep your project updatable so that future developers can build upon your work
- The solution’s build process should integrate the most up-to-date builds of open source binaries & packages throughout your project to repair vulnerable dependencies
- It should be push your projects to multiple devices simultaneously
- It should be easy to review & document your project’s components
I struggled with developing a firmware for the Raspberry pi for several months while developing a security testing tool for IoT vendors. Based on the lessons I learned, I developed a better solution for publishing firmware to linux-capable embedded devices. This solution makes it easy for others to rebuild your system from scratch and consequently collaborate with other developers. Additionally, it enables re-use of previous recipes for rapid development of new prototypes. This project also makes it possible for developers on low bandwidth connections to reduce the size of files necessary for enabling build collaboration. Not everyone has multi-gig fiber Internet access!
I present to you the Builder Hotspot. You can use this solution in your development process to create a “build recipe” through Ansible playbooks. The build recipe can be published and executed almost as simply as a shell script by users who want to try your project. You’ll save on bandwidth for file hosting and make specialized OS configuration collaboration for raspberry pi projects easier & more transparent.
The builder hotspot is a Raspberry Pi firmware image that turns your Raspberry Pi into an ansible-powered developer wifi network which deploys build images to multiple devices simultaneously. It speeds up firmware build times by caching apt-get packages to reduce unnecessary downloads and supporting concurrent pushing of recipes to multiple devices.
Using Ansible to build firmware on multiple raspberry pis
Apt-cacher-ng caching
Learn more at my firmware page:
https://firmware.patrickmccanna.net
