A user interface design for a scrolling log viewer with varying levels of importance (by alaric)

Like many people involved with computer programming and systems administration, I spend a lot of time looking at rapidly scrolling logs.

These logs tend to have lines of varying importance in them. This can fall into two kinds, that I see - one is where the lines have a "severity" (ranging from fatal errors down to debugging information). Another is where there's an explicit structure, with headings and subheadings.

Both suffer from a shared problem: important events or top-level headings whoosh past amidst a stream of minutae, and can be missed. A fatal error message can be obscured by thousands of routine notifications.

What I think might help is a tool that can be shoved in a pipe when viewing such a log, that uses some means (regexps, etc) to classify log lines with a numerical "importance" as appropriate, and then relaying them to the output.

However, it will use terminal control sequences to:

1. Colour the lines according to their importance
2. Ensure that the most recent entry at each level of importance remains onscreen, unless superceded by a later entry with a higher importance.

The latter deserves some explanation.

To start with, if we just have two levels of importance - ERROR and WARNING, for instance - it means that in a stream of output, as an ERROR scrolls up the screen, when it gets to the top it will "stick" and not scroll off, even while WARNINGs scroll by beneath it.

If a new ERROR appears at the bottom of the screen, it supercedes the old one, which can now disappear - letting the new ERROR scroll up until it hits the top and sticks.

Likewise, if you have three levels - ERROR, WARNING and INFO - then the most recent ERROR and WARNING will be stuck at the top of the screen (the WARNING below the ERROR) while INFOs scroll by. If a new WARNING appears, then the old one will unstick and scroll away until the new WARNING hits the top. If a new ERROR appears, then the old ERROR and WARNING at the top will become unstuck and scroll away until the new ERROR reaches the top.

So the screen is divided into two areas; the stuck things at the top, and the scrolling area at the bottom. Messages always scroll up through the scrolling area as they come, but any message that scrolls off the top will stick in the stuck things area unless there's another message at the same or higher level further down the scrolling area. And the emergence of a message into the bottom of the scrolling area automatically unsticks any message at that, or a less important, level from the stuck area.

That way, you can quickly look at the screen and see a scrolling status display, as well as (for activity logs from servers) the most recent FATAL, ERROR, WARNING, etc. message; or for the kinds of logs generated by long-running batch jobs, which tend to have lots of headings and subheadings, you'll always instantly see the headings/subheadings in effect for the log items you're reading.

This is related somewhat to the idea of having ERRORs and WARNINGs be situations with a beginning and an end (rather than just logged when they arise), such as "being low on disk space"; such a "situation alert" (rather than an event alert, as a single log message is) should linger on-screen somewhere until it's cancelled by the software that raised it emitting a corresponding "situation is over" event. Also related is the idea that event alerts above a certain severity should cause some kind of beeping/flashing to happen, which persists until manually stopped by pushing a button to acknowledge all current alerts. Such facilities can be integrated into the system.

This is relevant for a HYDROGEN console UI and pertinent to my previous thoughts on user interfaces for streams of events and programming interfaces to logging systems.

You Fib, Voting and Representation (by sarah)

A dark malaise is pouring over my country
The hate is bile rising choking community
Dear politicians You Fib
Not even the old skool twist of the figures
You simple do not check them in the first place
And it scares me how reasonable you can appear
How you have lulled the desperate and greedy
How you have twisted the heart of my land
I fear the outcome of those he seek the easy answers
Not like those other parties
Unlike them and yet... ah yes there
Hate of the different just because it is not you

Choose wisely when voting - remember MEP is proportional representation so tactical voting is stupid - you get what you vote for - also think careful about the council elections and who you want running your area and the knock on effects. PAX

Though I don't agree with it, an out of Europe policy is a valid position; I just wish that there was someone to represent my friends who feel this way without all the other baggage. I think there is a minority of those that think this is what they are working for and getting, but evidence is against it and that is sad. I see images of people trying to discuss the actual facts and figures of Europe and it sinks into obscurity due to the racism, sexism and other such topics. They do not have anyone to represent them in reality and that is wrong.

Of course there isn't really anybody to represent me and Al either, and here we hit a crux of politics - are we voting for parties or people?

If it is a party then I want a big list of all the parties and policies and to pick what I want from that - if it is people then what they do and say is incredibly important and the idea that their words and actions are not party policy becomes void, especially if they say something or do something and retain their position of power.

Given the choice I personally would probably go for Pirate Party, Green, TUSC (trade unionists and socialist coalition followed by lib dem - this is from having been lib-dem/green depending on what the vote was for and who was on offer. Sadly my choices are very limited and options here seem to be lots of far right groups, which frankly scare me.

I think if UKIP had turned into the party that was planned, then I wouldn't be worried, but that is not what I am seeing - I am seeing xenophobic, racist, sexist, homophobic view and hate of the vulnerable and different.

And that is without the privitization of the NHS and business practices - of course, I concede that privitization of the nhs is a valid political standpoint; just not one I agree with.

And of course the big three (Conservatives, Labour and Lib Dem) are not innocent at all in this - they have candidates and councillors that have said awful t hings, though they tend to lose their positions (of course the media could be covering stuff up and running a smear campaign but actually they seem to have been mostly supporting UKIP and so on. We have been astonished that there have not been any balanced pieces with the upcoming elections.).

Politics, like everything else, depends on your own story as to what you see - people can see the same interview and get different things out of it. One side sees a victory, the other evidence of bullying and so on. It is a mine field - I just wish people would actually think for themselves - go and research and think and look at their own biases - and we all have our own.

A hundred years ago women were still dying to allow me the vote - something which at least one UKIP member would have taken away.

Personally I want to stay in Europe, for human rights reasons - ie maternity leave, holiday pay etc... and I believe it can work for economics too - happy workers are hard workers (some tweaking is often needed depending on job type etc... but it pretty much holds.) And even with the risk of those I urge to vote, even voting UKIP. I will still do it as I believe voting is important (even if voting methods and politics in general have huge flaws).

At the same time I am actually sick of political canvassing - I think it kind of shouldn't be allowed - that there be one website that contains the policies (in accessible ways so explained in videos etc...) of ALL the parties. One day I had the BNP, UKIP and British Democrats in a row - this then caused me to snap at the little old man who was with TUSC who's policies I quiet like - canvasing fatigue I'm calling it!

Of cause there is the apathy issue in the UK were politics is concerned as well and therefore how you would get people to actually go and look.

I actually don't really have much of a point in this post other than - go and vote, think about who you are voting for though especially with Europe stuff as it is proportional representation and so there is no protest vote - you get what you vote for. Also though UKIP in general scare me, I feel sorry for the minority who seem to actually be trying to discuss political points and get derailed both within and without and those drawn to the party inside and outside of London seem to vastly differ.

ie political representation in this country sucks as does the actual voting system, but please still vote.

The scariest thing I've seen so far was the BNP youth video - truly scary - I kind of don't want to put it on here - even if it turns out I agreed with one policy :/

This is me looking at everything with the coolest, rationalist head I have - emotionally I want to just rage about Nazi attitudes and scapegoats.

Other good articles on voting:

Go Vote Dammit

Voting For Tomorrow

The joys of compressed air (by alaric)

I've always had a hankering for pneumatic tools. The idea is that compressed air is used as a power source for hand tool, rather than an electric motor. This has various advantages - the tool is lighter (motors are heavy), it's safer (electric motors, when stalled, produce a sharp increase in torque that can make the thing break your wrist if you're not lucky, and the motor can then burn out), and it's cheaper (air drive thingies are cheaper than motors) and simpler. This makes a wider range of tools practical and affordable, from drills and power screwdrivers to stranger things like pneumatic files and caulking guns. And compressed air has some unique uses, too - tyre inflaters, blowers for clearing dust away, and spraying liquids for painting or cleaning.

The downside, however, is that you need a source of compressed air. Thankfully, these have been getting more popular in the hobbyist market, and therefore cheaper, and got a bonus from work, and I've wanted one for years, and so, POW! I now own an air compressor!

It's only a small one. I spent just over a hundred pounds, which is more than I've spent on something purely for fun (it's a tool, but not one I need) in ages. I picked up a set of basic air tools with it - a tyre inflater, a blower, a paint sprayer, and a wash sprayer. Sarah's keen on using it to paint things, so once she's had a go with the spray gun, I'm planning on getting her a more artist-grade air brush with fine controls, and she's getting me a pneumatic ratchet (that can do and undo screws and bolts) as a first "proper" tool.

I'm very tempted by a pneumatic nail gun, because they can drive a nail into wood in a single "phut" compared to having to hammer one into place (and the ever-present risk of it going wonky halfway through), which is a huge labour saver - but I don't actually use nails all that often. Perhaps in the next round of workshop roof fixing...

I thought it was broken when I first unpacked it; the manual had a series of steps to follow when first setting it up, and when switching it on (check the crankcase oil level, set the valves to a safe position, etc) - I followed them all and turned the power on, and the motor started roaring away and the tank pressure rising. But the pressure wouldn't go above two bar, and when I turned the motor off, all the air hissed out again. It was getting late so I got ready to go to bed rather than playing more, but while flicking through the manual's troubleshooting section (it looked like a failure of the non-return valve or the safety valve, and I resolved to check both in the morning), I noticed that the instructions for AFTER you've used the compressor included opening a cylinder drain valve to let out any condensation... I felt underneath and, sure enough, the drain valve was wide open! Checking that's closed should be part of the starting process, but wasn't in the instructions...

In the meantime, I've been having fun with the kids making balls levitate in the air stream from the blower 🙂 We can float a ball-pond ball a good twenty to thirty centimetres above the blower nozzle, at an angle of up to forty-five degrees. and 1.5bar; when I have a face shield on and no children crowding around I might try cranking the output pressure up to the full 6-8bar the machine can produce and see if I can float balls at steeper angles, or heavier objects...

Home Automation: Phase 1 (by alaric)

I've always had a nerdish fascination with home automation.

There's been a recent trend towards the "Internet of Things" (a.k.a. "IoT"), which is related, but different. The IoT seems to focus more on home devices talking to centralised Web services, which is a hateful model (we all know why: those central services are outside of your control, so unless you're paying a subscription, you are the product rather than the customer; and if they're shut down, your devices become useless; and they can leak your private information whenever they want; and they can take control of your home; and it all falls apart if your Internet connection goes down; and I'm sure there's others).

No, I want to have my house under computer control - but with those computers under MY control. This is something I've planned for for ages, but as with all hardware projects, getting started is tricky; I need to commit to a final design and then afford to buy the parts, and that's scary because I might find out that the parts don't quite work together in the way I wanted. Unlike with software, hardware hacking requires up-front commitment of resources, that can't be backed out of. Scary!

So the trick is to split the thing into small parts, with flexible interconnections, so I can iteratively prototype parts and then connect them up in due course. And this last week I took the first step - building an announcement system in the living room.

Read more »

Thoughts on Programming and Tracing (by alaric)

I was recently pointed at this interesting article: Learnable Programming.

It's a good read, overturning many assumptions the software industry has picked up over the years, and propagated without thought since.

The first part suggests allowing a programmer to trace the flow of execution of a program graphically, using an interactive timeline. My first thought was that this was all well and good, but would rely on every library in the language annotating every operation with information about how to present it - producing the little thumbnails to go in the timeline, or exposing numeric values that can be plotted onto charts. Also, highlighting the "current" drawing operation in red on the canvas relies on those operations being things that affect a canvas; more abstract operations, such as writing to a database (or even generating images to be encoded directly into a file rather than onto the screen) would require a more explicit "object preview".

However, those are not insurmountable goals. And, perhaps, things that can be built on top of my ideas about logging and tracing, making it possible to use such an interface to go through traces of execution captured from production servers, rather than just within a cute live-coding IDE; the trace entries generated by operations in your libraries could, with the help of a meta-library of trace visualisation rules, generate those little thumbnails. However, it would need to be augmented with dynamic scope information provided by the programming environment itself to know which line of code caused the trace event; the kind of thing one finds in a stack trace.

He asks "Another example. Most programs today manipulate abstract data structures and opaque objects, not pictures. How can we visualize the state of these programs?"; so I suggest that the abstract data structures and opaque objects be annotated with code that summarises their state. Many languages have a notion of "return a string representation of this object", generally aimed at debug logging - Python's repr() versus str(), for instance. Perhaps if we moved to expecting objects to return HTML representations of themselves, we could take a step in that direction.

The second part (and I'm taking some temporal liberties here, as some concepts I've included in the first part are touched upon in the second and vice versa) is also inspiring; it looks at the bigger picture, considering how libraries and code-editing environments can be designed to make it much easier for programmers to identify what operations their libraries are making available to them, rather than requiring the first step to be the reading of documentation. It touches on topics such as the dangers of mutable state (preaching to the converted here!), and the choice of library function names to make code using them clear (I'm also a big fan of smalltalk / Cocoa-style function call syntax, and how it might be brought into the Lisp family of languages...)

I've written before that I think modifying software should be a much more widely-practiced activity; and I think that should be achieved through removing unnecessary obstacles, rather than forcing everyone through complicated programming classes. I'm always interested in more thoughts on how to make that happen!