A user interface design for a scrolling log viewer with varying levels of importance (by alaric)

Like many people involved with computer programming and systems administration, I spend a lot of time looking at rapidly scrolling logs.

These logs tend to have lines of varying importance in them. This can fall into two kinds, that I see - one is where the lines have a "severity" (ranging from fatal errors down to debugging information). Another is where there's an explicit structure, with headings and subheadings.

Both suffer from a shared problem: important events or top-level headings whoosh past amidst a stream of minutae, and can be missed. A fatal error message can be obscured by thousands of routine notifications.

What I think might help is a tool that can be shoved in a pipe when viewing such a log, that uses some means (regexps, etc) to classify log lines with a numerical "importance" as appropriate, and then relaying them to the output.

However, it will use terminal control sequences to:

1. Colour the lines according to their importance
2. Ensure that the most recent entry at each level of importance remains onscreen, unless superceded by a later entry with a higher importance.

The latter deserves some explanation.

To start with, if we just have two levels of importance - ERROR and WARNING, for instance - it means that in a stream of output, as an ERROR scrolls up the screen, when it gets to the top it will "stick" and not scroll off, even while WARNINGs scroll by beneath it.

If a new ERROR appears at the bottom of the screen, it supercedes the old one, which can now disappear - letting the new ERROR scroll up until it hits the top and sticks.

Likewise, if you have three levels - ERROR, WARNING and INFO - then the most recent ERROR and WARNING will be stuck at the top of the screen (the WARNING below the ERROR) while INFOs scroll by. If a new WARNING appears, then the old one will unstick and scroll away until the new WARNING hits the top. If a new ERROR appears, then the old ERROR and WARNING at the top will become unstuck and scroll away until the new ERROR reaches the top.

So the screen is divided into two areas; the stuck things at the top, and the scrolling area at the bottom. Messages always scroll up through the scrolling area as they come, but any message that scrolls off the top will stick in the stuck things area unless there's another message at the same or higher level further down the scrolling area. And the emergence of a message into the bottom of the scrolling area automatically unsticks any message at that, or a less important, level from the stuck area.

That way, you can quickly look at the screen and see a scrolling status display, as well as (for activity logs from servers) the most recent FATAL, ERROR, WARNING, etc. message; or for the kinds of logs generated by long-running batch jobs, which tend to have lots of headings and subheadings, you'll always instantly see the headings/subheadings in effect for the log items you're reading.

This is related somewhat to the idea of having ERRORs and WARNINGs be situations with a beginning and an end (rather than just logged when they arise), such as "being low on disk space"; such a "situation alert" (rather than an event alert, as a single log message is) should linger on-screen somewhere until it's cancelled by the software that raised it emitting a corresponding "situation is over" event. Also related is the idea that event alerts above a certain severity should cause some kind of beeping/flashing to happen, which persists until manually stopped by pushing a button to acknowledge all current alerts. Such facilities can be integrated into the system.

This is relevant for a HYDROGEN console UI and pertinent to my previous thoughts on user interfaces for streams of events and programming interfaces to logging systems.

The joys of compressed air (by alaric)

I've always had a hankering for pneumatic tools. The idea is that compressed air is used as a power source for hand tool, rather than an electric motor. This has various advantages - the tool is lighter (motors are heavy), it's safer (electric motors, when stalled, produce a sharp increase in torque that can make the thing break your wrist if you're not lucky, and the motor can then burn out), and it's cheaper (air drive thingies are cheaper than motors) and simpler. This makes a wider range of tools practical and affordable, from drills and power screwdrivers to stranger things like pneumatic files and caulking guns. And compressed air has some unique uses, too - tyre inflaters, blowers for clearing dust away, and spraying liquids for painting or cleaning.

The downside, however, is that you need a source of compressed air. Thankfully, these have been getting more popular in the hobbyist market, and therefore cheaper, and got a bonus from work, and I've wanted one for years, and so, POW! I now own an air compressor!

It's only a small one. I spent just over a hundred pounds, which is more than I've spent on something purely for fun (it's a tool, but not one I need) in ages. I picked up a set of basic air tools with it - a tyre inflater, a blower, a paint sprayer, and a wash sprayer. Sarah's keen on using it to paint things, so once she's had a go with the spray gun, I'm planning on getting her a more artist-grade air brush with fine controls, and she's getting me a pneumatic ratchet (that can do and undo screws and bolts) as a first "proper" tool.

I'm very tempted by a pneumatic nail gun, because they can drive a nail into wood in a single "phut" compared to having to hammer one into place (and the ever-present risk of it going wonky halfway through), which is a huge labour saver - but I don't actually use nails all that often. Perhaps in the next round of workshop roof fixing...

I thought it was broken when I first unpacked it; the manual had a series of steps to follow when first setting it up, and when switching it on (check the crankcase oil level, set the valves to a safe position, etc) - I followed them all and turned the power on, and the motor started roaring away and the tank pressure rising. But the pressure wouldn't go above two bar, and when I turned the motor off, all the air hissed out again. It was getting late so I got ready to go to bed rather than playing more, but while flicking through the manual's troubleshooting section (it looked like a failure of the non-return valve or the safety valve, and I resolved to check both in the morning), I noticed that the instructions for AFTER you've used the compressor included opening a cylinder drain valve to let out any condensation... I felt underneath and, sure enough, the drain valve was wide open! Checking that's closed should be part of the starting process, but wasn't in the instructions...

In the meantime, I've been having fun with the kids making balls levitate in the air stream from the blower 🙂 We can float a ball-pond ball a good twenty to thirty centimetres above the blower nozzle, at an angle of up to forty-five degrees. and 1.5bar; when I have a face shield on and no children crowding around I might try cranking the output pressure up to the full 6-8bar the machine can produce and see if I can float balls at steeper angles, or heavier objects...

Home Automation: Phase 1 (by alaric)

I've always had a nerdish fascination with home automation.

There's been a recent trend towards the "Internet of Things" (a.k.a. "IoT"), which is related, but different. The IoT seems to focus more on home devices talking to centralised Web services, which is a hateful model (we all know why: those central services are outside of your control, so unless you're paying a subscription, you are the product rather than the customer; and if they're shut down, your devices become useless; and they can leak your private information whenever they want; and they can take control of your home; and it all falls apart if your Internet connection goes down; and I'm sure there's others).

No, I want to have my house under computer control - but with those computers under MY control. This is something I've planned for for ages, but as with all hardware projects, getting started is tricky; I need to commit to a final design and then afford to buy the parts, and that's scary because I might find out that the parts don't quite work together in the way I wanted. Unlike with software, hardware hacking requires up-front commitment of resources, that can't be backed out of. Scary!

So the trick is to split the thing into small parts, with flexible interconnections, so I can iteratively prototype parts and then connect them up in due course. And this last week I took the first step - building an announcement system in the living room.

Read more »

Thoughts on Programming and Tracing (by alaric)

I was recently pointed at this interesting article: Learnable Programming.

It's a good read, overturning many assumptions the software industry has picked up over the years, and propagated without thought since.

The first part suggests allowing a programmer to trace the flow of execution of a program graphically, using an interactive timeline. My first thought was that this was all well and good, but would rely on every library in the language annotating every operation with information about how to present it - producing the little thumbnails to go in the timeline, or exposing numeric values that can be plotted onto charts. Also, highlighting the "current" drawing operation in red on the canvas relies on those operations being things that affect a canvas; more abstract operations, such as writing to a database (or even generating images to be encoded directly into a file rather than onto the screen) would require a more explicit "object preview".

However, those are not insurmountable goals. And, perhaps, things that can be built on top of my ideas about logging and tracing, making it possible to use such an interface to go through traces of execution captured from production servers, rather than just within a cute live-coding IDE; the trace entries generated by operations in your libraries could, with the help of a meta-library of trace visualisation rules, generate those little thumbnails. However, it would need to be augmented with dynamic scope information provided by the programming environment itself to know which line of code caused the trace event; the kind of thing one finds in a stack trace.

He asks "Another example. Most programs today manipulate abstract data structures and opaque objects, not pictures. How can we visualize the state of these programs?"; so I suggest that the abstract data structures and opaque objects be annotated with code that summarises their state. Many languages have a notion of "return a string representation of this object", generally aimed at debug logging - Python's repr() versus str(), for instance. Perhaps if we moved to expecting objects to return HTML representations of themselves, we could take a step in that direction.

The second part (and I'm taking some temporal liberties here, as some concepts I've included in the first part are touched upon in the second and vice versa) is also inspiring; it looks at the bigger picture, considering how libraries and code-editing environments can be designed to make it much easier for programmers to identify what operations their libraries are making available to them, rather than requiring the first step to be the reading of documentation. It touches on topics such as the dangers of mutable state (preaching to the converted here!), and the choice of library function names to make code using them clear (I'm also a big fan of smalltalk / Cocoa-style function call syntax, and how it might be brought into the Lisp family of languages...)

I've written before that I think modifying software should be a much more widely-practiced activity; and I think that should be achieved through removing unnecessary obstacles, rather than forcing everyone through complicated programming classes. I'm always interested in more thoughts on how to make that happen!

Look upon my works, ye mighty, and despair (by alaric)

I took it upon myself to translate the following into Lojban:

I met a traveller from an antique land
Who said: Two vast and trunkless legs of stone
Stand in the desert. Near them, on the sand,
Half sunk, a shattered visage lies, whose frown,
And wrinkled lip, and sneer of cold command,
Tell that its sculptor well those passions read
Which yet survive, stamped on these lifeless things,
The hand that mocked them and the heart that fed:
And on the pedestal these words appear:
"My name is Ozymandias, king of kings:
Look on my works, ye Mighty, and despair!"
Nothing beside remains. Round the decay
Of that colossal wreck, boundless and bare
The lone and level sands stretch far away.

I decided I wanted to aim for an idiomatic Lojban translation, rather than a word-for-word translation of the English (which would probably result in quite clumsy Lojban). However, this was a challenge, as it would mean making up new good-sounding idioms for things that I either didn't know, or hadn't actually developed as idioms in Lojban yet!

Here's what I came up with. There's plenty of problems with it, which I'll explain below, in the hope that jbopre will read this and suggest improvements. However, even if you don't know Lojban, please read on; I think you might find the breakdown of it (I give literal English translations of it) interesting:

.i .u'e re lo barda tuple ku se stuzi le cantu'a
.i cpana le canre fa lo se porpi be lo flira
noi .iiru'e lo du'u ke'a frumu ku
     .e lo du'u ke'a tolpluka turni cisma ku
   nibyti'i lo du'u lo zbasu ku pu te smuni
      le se cinmo poi lu'e ke'a za'o renvi
           zi'e poi fu lu'e ke'a xraci'a fi le na jmive
.ije rakci'a fi le zbepi
     fe lu ga'isai mi'e la'oi .ozymandyus.
           .i mi turni lo turni
           .i .a'onaidai ko catlu tu'a mi doi tsali li'u
.i zo'e bi'unai po'o renvi
.i lo .o'ebe'udai pinta canre ku po'o diklo le barda se daspo

Read more »