blog

April 29, 2022

I’m frequently receiving requests for writing advice due to the clarity of my writing. I found an article on hackernews that resonates with me greatly. If persuasive writing is a skill you hope to develop, you should read this: http://www.covingtoninnovations.com/mc/WriteThinkLearn.pdf

In particular, the following ideas reflect a motivation I always try to capture in my writing:

“I’m not giving this presentation (or writing this paper) because I’m important. I’m doing it because you’re important.”

“I’m not going to demand that you put up with my quirks (bad spelling, bad organization, sloppiness [ME: Lolcat memes]). I’m going to package the information so that it enters your heads as easily as possible.”

Remembering that you are writing to help others seems to have a consistent impact-multiplier effect. I encourage you to be humble and consider how to invoke this motivation in your writing style.

April 13, 2022

Let’s canter

March 2, 2022February 24, 2022

🏂

March 1, 2022March 3, 2022

👈🏻 Some tips for those transitioning from Cron jobs to SystemD timers for scheduled tasks.

If you’re a greybeard who is moving away from cron jobs into the world of SystemD Timers, you might find this helpful. I burned a lot of time learning systemd timers for the first time this week and I want to highlight something for others who have never used SystemD to scheduled tasks.

This tutorial is great: https://www.fosslinux.com/48317/scheduling-tasks-systemd-timers-linux.htm. Follow it to the letter and you should be fine. You might be like me and miss a letter or two. I’d like to save you from some painful debugging & searching.

The general process for scheduling a task with systemd consists of the following steps:

Create a service file that declares the shell script/binary that should be executed. This service file will express the system state target for when your task should be allowed to execute. A common target is “multi-user.target”. I’ve had some services that did not launch at multi-user runlevels. If you want to be very conservative, you can specify a target of “graphical.target”. You can see options for your targets using the command systemd-analyze critical-chain. Additionally, the service file expresses a “type” of service: either simple, exec, forking, oneshot, dbus, notify or idle simple, exec, forking, oneshot, dbus, notify or idle. The useful information I’ve had to collect to deciding the type of service are 1) Determine if the script/binary you call forks and 2) Identify if your binary/script/ provides a nonzero return code during execution. You can find the return code of a script using the linux command echo $?. Check out https://www.freedesktop.org/software/systemd/man/systemd.service.html for more details on the types you can declare.
Create a timer file that declares the schedule on which the script/binary should be executed. This timer file will express the calendar on which the service is launched. A counterintuitive aspect of scheduled tasks in system d is that it is the timer file that declares the service it manages.
Enable your service. The systemctl enable command is used to enable a service. Enabling a service means that it will be available after reboot. If you want to launch the service immediately, you can use the command systemctl --user enable [servicename.service] --now
Enable your timer. The systemctl enable command is used to enable a service. Enabling a timer means that it will be available after reboot. If you want to launch the timer immediately, you can use the command systemctl --user enable [timername.timer] --now

You will in all likelihood write a valid .service file for launching commands & .timer file for scheduling the tasks. Systemctl –user enable TheTasks.service will appear to return fine and systemctl –user start TheTasks.service will execute your task- but you may find that the service doesn’t run with the schedule. When you check the status of TheTasks.service, you’ll see something like:

systemctl status TheTasks.service
● TheTasks.service - Service that does Tasks
   Loaded: loaded (/home/me/.config/systemd/user/TheTasks.Service; enabled; vendor preset: enabled)
   Active: inactive (dead) Since [$dateOfLastReboot]
Process: 883 ExecStart=/usr/bin/script.sh (code=exited, status=0/SUCCESS)

Inactive (dead) means it won’t launch. You’ll second guess yourself and wonder if you’re writing the timer correctly. You probably are- but if you have doubts, definitely checkout the systemd-analyze calendar command to confirm that you’re setting it correctly.

systemd-analyze calendar 'Sun,Thu *-*-* 15:00:00 America/Los_Angeles’
  Original form: Sun,Thu *-*-* 15:00:00 America/Los_Angeles
Normalized form: Thu,Sun *-*-* 15:00:00 America/Los_Angeles
    Next elapse: Sun 2022-02-27 15:00:00 PST
       (in UTC): Sun 2022-02-27 23:00:00 UTC
       From now: 18h left

Good news: you are scheduling your tasks correctly. Bad news, searching this error on google is not fruitful. inactive (dead) can happen for a lot of reasons. But one reason which isn’t well covered:

YOU PROBABLY FORGOT TO ENABLE THE .TIMER:

systemctl --user status TheTasks.timer
● TheTasks.timer - Weekly timer that launches tasks
     Loaded: loaded (/home/me/.config/systemd/user/TheTasks.timer; disabled; vendor preset: enabled)
     Active: inactive (dead)
    Trigger: n/a
   Triggers: ● TheTasks.service

So you can go ahead and enable that:
systemctl --user enable --now TheTasks.timer
And if you check the status a second time:

systemctl --user status TheTasks.timer
● TheTasks.timer -  Weekly timer that launches tasks
     Loaded: loaded (/home/pi/.config/systemd/user/TheTasks.timer; enabled; vendor preset: enabled)
     Active: active (waiting) since Sat 2022-02-26 20:20:57 PST; 5s ago
    Trigger: Sun 2022-02-27 15:00:00 PST; 18h left
   Triggers: ● TheTasks.service

You’re in great shape. Hope that helps.

February 25, 2022May 10, 2023

BuilderHotspot

A new way to develop & publish SingleBoardComputing IoT Projects

I want to share a better way to accelerate & automate the development of Single-Board-Computing projects with you. (e.g. the Raspberry pi)

The Raspberry Pi is an optimal platform for hobbyists who want to build prototypes and learn about computers. The way that we share Rasbperry pi projects with others on the Interenet is not optimal.

What are the current problems facing people who want to share their Raspberry Pi Project?

Publishing a full firmware image to share with others is hard for a variety of reasons:

Firmware images for linux builds are large (For example, The Quake On Lan firmware is 1.38 gigs. https://www.quakeonlan.net)
Large files have greater hosting providers costs (Note that the Quake on Lan team manages download costs by asking people to pay/register for access to faster downloads https://ufile.io/jvp3et41)
Projects that go viral have unpredictable and potentially significant hosting costs (See above for large file-distribution coping examples)
Firmware build process requires perfect execution. Failing to implement perfect, repeatable execution can result in exploitable secrets lingering in the filesystem or History files. It also puts you at risk of difficult to discover configuration mistakes. Unit tests must be written to discover & remove private material from final build image
Iterative image development is error prone. If a DD operation against a firmware image results in block errors, these can be difficult to correct in successive builds, costing you disk space on the final image. Sharing firmware between developers carries undiscovered mistakes forward into successive builds.
Distributing large files for incremental updates slows down development. Developers have to wait for the latest firmware image to finish downloading before they can commence making modifications.
Build procedure documentation goes stale. (Google “Raspberry Pi Wifi hotspot”. Note that the responses reference IP Tables in their configuration, which is no longer included in Raspbian (https://thepi.io/how-to-use-your-raspberry-pi-as-a-wireless-access-point/)
Human executed installation of software risks mistakes in the final build.

Embedded IoT developers must first come up with a prototype that works. This can involve a lot of trial and error that will leave your environment’s history command filled with both useless or wrong commands as well as the ones that are correct. When you’ve finished making one working prototype, you need to go through the process a second time, while only limiting yourself to the necessary commands for creating the device. Those exact and reduced commands must be documented. Next you have to test your documentation for accuracy. Then you’re finally ready to publish your build process. That decision to share your your project started as a simple act of charity. It becomes a significant investment of time and money if you’re going to do it well. If you decide to skip documenting your steps & instead publish a firmware image of your project, you’re going to pay bandwidth bills for files that usually start at 2 gigs or greater. In that situation, you’re asking users who might have slow internet connections to download very large files.

There are problems for the people who will read your guide as well. Users who follow Raspberry pi project guides spend almost as much time troubleshooting stale documentation as they do troubleshooting their actual device. If someone follows your instructions 2 years after you’ve published, the user may find that linux has moved forward and your commands don’t work. The effort you put into documenting the series of commands that create your project is significant. The developer is rewarded by impending document obsolescence as soon as they publish it.

.img files are too large for reasonable Internet distribution
It’s hard to find project guides with instructions that are up-to-date
Pi project guides have either tedious manual steps or depend on dangerous solutions for automation
Deploying firmware to multiple pi’s simultaneously is a serial operation
Documenting all the components of your project during image development based on history files is a pain

A solution that addresses these problems should have the following properties:

It should be inexpensive to share your project with others
It should be easy to keep your project updatable so that future developers can build upon your work
The solution’s build process should integrate the most up-to-date builds of open source binaries & packages throughout your project to repair vulnerable dependencies
It should be push your projects to multiple devices simultaneously
It should be easy to review & document your project’s components

I struggled with developing a firmware for the Raspberry pi for several months while developing a security testing tool for IoT vendors. Based on the lessons I learned, I developed a better solution for publishing firmware to linux-capable embedded devices. This solution makes it easy for others to rebuild your system from scratch and consequently collaborate with other developers. Additionally, it enables re-use of previous recipes for rapid development of new prototypes. This project also makes it possible for developers on low bandwidth connections to reduce the size of files necessary for enabling build collaboration. Not everyone has multi-gig fiber Internet access!

I present to you the Builder Hotspot. You can use this solution in your development process to create a “build recipe” through Ansible playbooks. The build recipe can be published and executed almost as simply as a shell script by users who want to try your project. You’ll save on bandwidth for file hosting and make specialized OS configuration collaboration for raspberry pi projects easier & more transparent.

The builder hotspot is a Raspberry Pi firmware image that turns your Raspberry Pi into an ansible-powered developer wifi network which deploys build images to multiple devices simultaneously. It speeds up firmware build times by caching apt-get packages to reduce unnecessary downloads and supporting concurrent pushing of recipes to multiple devices.