An Interactive Init System

2021/07/10

... yet another thing in UNIX that is stupid, along with how to fix it.

How do you start a program?

Well, this must be a trick question.

You mean... from the command line?

... type its name? and hit enter? Am I answering the right question here?


~ $ program_name
Hello world, this is program_name v0.1 beta.
outputs!
###########
###########
###########
###########
###########
~$

How do you start a program in the background?

OK I still don't understand what's going on here. Are we seriously writing a blog post on this?


~ $ long_running_program_name &
~ $

... okay, I might have been a little bit too optimistic here, in practice it's probably more like...


~ $ long_running_program_name &
~ $ Hi! you launched long_running_program_name!
logs
more logs
even more logs
Listening on http://localhost:8080.

... but I think you get the point. Maybe you should pipe the output into a file. Or you could just use screen or tmux, launch it in the foreground and detach? That way you can look at its output any time you want.

Same thing but now it's persistent?

What do you mean by "persistent"? Ohh so it should survive a restart. Um. Well. Soooo...

Okay, this is hard.

So which init system were you using again?

System V?

Well, you'll need to make an init script. Well, yes, it has a lot of "start)" and "stop)" in it, but you can just copy another one. And then you swap out the interesting parts. And then you start it with /etc/init.d/long_running_service start. Except... well, does your program detach from the terminal? Because, y'know, otherwise it won't work.

Well, just make it detach from the terminal then.

... sure, yes, there is nohup. So you use that, and then you create a pidfile... well, it's a file. That you create with your process id in it. So that you can find it if you want to stop it. Does your program create one? No?

... just make it create one?

... or, whatever, yeah, look it up on the internet how to do this. I don't care. So you're almost done, so if you ignore all the issues caused by different environment variables, you can just run your init script via sudo, and... what? what do you mean by "logging"?

Yes, I know that you can look at what it prints out with screen, but you're writing a serious system service now. It should be able to print stuff to a log file. Or... at least syslog. It's easy, you just include...

... wait, no. You don't want that. Yes, sure, it solves some of this, but it'll eat your soul. Or at least all distros. Just don't. Why don't you just use...

s6 init?

It's all simple and UNIXy. See, you set up this directory. That has a subdirectory for each service. And then each of them has a run file, you put your command in them, and then you run s6-svc -u [servicename]. Yes, "u" is for "up". No, it doesn't have longer-named commands. And then you launch s6-svscan from your actual init system. If you don't want to swap out your existing init system.

Ohh, logging? Well, you can just create a subdirectory of that other subdirectory I just mentioned. And also add a "run" file. Then, it's s6-log, which has these 6 easy-to-remember, single-letter options. And its own control syntax, with these other convenient, single-letter control characters.

... yeah, and then you edit inittab... What? No. It's not better. Think of your soul. You really don't want to be yet another person who uses...

systemd?

... sure. I give up. You win.

... and no, I'm not writing yet another HOWTO for systemd services. You can look up syntax for unit files. It also does logging. It's mildly less horrible than most of the above, in exchange for your soul and your freedom. But it's still a lot more complicated than it should be.

The point is...

All of the above are a lot more complicated than they should be. And they're significantly different from the default, "type the name of the program to launch it" baseline.

Could we do better?

screen for system services

Imagine the following.


~ $ sudo svcrun long_running_program_name --fancy_parameter=123
... starting service "long_running_program_name"; child tree has PID 1234
logging to /var/log/long_running_service/current.txt, rotating every 1 day or 10 MB
Press Ctrl-C to detach.
--------------------------------
Hi! you launched long_running_program_name!
using fanciness=123.0
logs
more logs
even more logs
Listening on http://localhost:8080.
^C
~ $

... but imagine that it's still running in the background.

"svcrun" is our made-up tool. It doesn't currently exist, but it should be possible though to write one. It:

launches the program requested, passing through the parameters
... in a properly sanitized environment, as a child of a service process, not the shell launching it
it lets you see all its output right away (essentially it's a "tail -f [logfile]"; the actual process isn't printing here)
just like with s6, no need for pidfiles (since it's our child)
just like with s6, no need for syslog / etc (since we can capture its stdout / stderr)

But, most importantly, it is also ensured that we restart this the next time the machine starts up.

If we try starting it the second time:


~ $ sudo svcrun long_running_program_name --fancy_parameter=123
... service "long_running_program_name" already running; child tree has PID 1234
logging to /var/log/long_running_service/current.txt, rotating every 1 day or 10 MB

Restart [R] / Attach [A]? A

Press Ctrl-C to detach.
--------------------------------
[2021-07-10 12:30:01] Hi! you launched long_running_program_name!
[2021-07-10 12:30:01] using fanciness=123.0
[2021-07-10 12:30:01] logs
[2021-07-10 12:30:01] more logs
[2021-07-10 12:30:02] even more logs
[2021-07-10 12:30:02] Listening on http://localhost:8080.
^C
~ $

... just like as if it was screen. Except it does things the right way, it's safe to run as root, and it's not taking over the entire terminal session (ncurses-style) but stays in per-line mode.

It can probably have some config files, designed for editability both by the above tool and by hand. The point is though that... you shouldn't normally need to edit config files, and then try starting the service, looking at logs, concluding that you have the wrong parameters, re-editing the config file, re-launching the service, etc etc.

Maybe I could actually write such a thing?

This is post no. 17 for Kev Quirk's #100DaysToOffload challenge.

... comments welcome, either in email or on the (eventual) Mastodon post on Fosstodon.