Backup system progress (by alaric)

I mentioned an intent to build a backup system this Christmas. Well, I've not had two days to work on it yet; I've just had an hour and a half one evening, in which I've written the local-filesystem backend...

But last night I had a dream about it. In particular, I dreampt of visiting an old friend I've neglected to keep in touch with lately; and at her house she had a rather snazzy little mini-laptop, like an EeePC or something, which had an eSATA connector into which she plugged some humungous (in terms of gibibytes; it was the size of a double CD case, physically) external hard disk upon which she had a filesystem that seemed to work very much like Fossil/Venti, the original inspiration for my backup system - or perhaps like a giant personal Git repository.

In particular, one feature that cropped up in the dream was that the filesystem had a magic file on it which was an RSS feed of recent changes to files.

Which got me thinking about features for version 2 of my backup system (if I get to finish version 1, that is!). My focus is on offline use, for batched backups, but the filesystem in my dream was being used online.

For a start, we could have a log of changes to tags, as in my system tags are the "roots" of the archive system. The creation of new tags, the deletion of tags, or the updating of a tag to point to a new snapshot would all be logged. This could then be used to create the RSS feed.

Secondly, it shouldn't be too hard to write a FUSE interface to the thing for read-only access, presenting a root directory containing an RSS file generated from the log, along with a directory for each tag, which in turn contains a current subdirectory containing the current value of the tag as well as dated subdirectories containing all the past contents of the tag, in ISO date format so they sort correctly. And perhaps an RSS file just listing the history of that tag.

But then the next cool thing would be to allow write access, by using a local disk directory as a staging area, so the current subdirectory can be written to (with the changes being spooled to the local disk). Then when a commit command is given, those changes are merged into current. Which would require a new underlying operation to merge changes in, rather than taking a new snapshot; the difference being that any files or directories missing in the directory tree being snapshotted are 'inherited' from a previous snapshot already in the system, with some mechanism to reflect deletions.

Anyway, unrelated to the dream, it also occurred to me that it'll be neat to support replicated archives; my implementation of the backend architecture will make it easy to take a set of backend instances and merge them into one, with every write going to all of them and reads serviced from the first one that succeeds. It'll also be easy to support staged archives, where a number of read-only backends are checked for existing blocks, but all new blocks go to a nominated writable backend, with another backend adapter. That will allow for generational backup systems, where a local disk is filled up with backups until it reaches a size limit, whereupon its contents are shipped off to a DVD writer (keeping a local cache of what blocks where on there, so the DVD need not be put in unless the contents of blocks are actually needed).

But, all idle speculation for now. I still need another day and three quarters to implement the core of the thing...

Design patterns (by alaric)

I'd thought I'd had this little rant before, but as I went to find the post I'd had it in to link to it from a big ranty post I'm working on, I couldn't find it.

So here it is.

My opinion of design patterns.

Design patterns are common structures that appear in computer programs, but that can't themselves be abstracted into a reusable component.

That's fair enough. I have the "gang of four" book of design patterns, that started the craze, and I think they're all good design patterns in mainstream OO languages; things that can't be actually encapsulated as a class or whatever, but that keep cropping up.

However, the problem is that the patterns community has somehow ended up with some dogmatic folks in it. Who take phrases such as "Design patterns are common structures that appear in computer programs, but that can't themselves be abstracted into a reusable component" - and then attack anybody who claims to have encapsulated a design pattern into a reusable component.

Because design patterns are really only valid in relation to a set of languages. For example, in assembly language (or even traditional BASIC with GOSUB), procedure calls are a design pattern. There is no language construct for a procedure; instead, you have a GOSUB or CALL to a given address, that explicitly saves the calling location somewhere and then jumps, and a RETURN to go back to that saved location. The GOSUB...RETURN pairing is put in there by the programmer; it's a design pattern.

But then higher-level languages have explicit procedure abstractions, and all the fun they entail.

Even object orientation is a design pattern in languages like C. "Put function pointers in a struct, and call them with the struct pointer passed as the first argument". Or including a pointer to a struct of function pointers in every data struct (and call it a virtual method table). But go to Java and that's an explicit language abstraction facility.

The gang-of-four book is taken as the authoritative book of common design patterns (although it is, in my opinion, a book of design patterns for mainstream OO languages). And design patterns are things that cannot be expressed as native abstractions in code. THEREFORE, fools think, it's impossible to write a design pattern in code. And anybody who claims to have done so needs shouting at.

Sadly, as demonstrated in Peter Norvig's presentation on design patterns in dynamic languages, more advanced languages can express abstractions that others can't, and there are plenty of more advanced languages than C++ and Java.

In fact, I think that the appearance of design patterns in a language is probably a hint that the language needs to be more powerful - and suggests how to make it so. What sort of abstraction would make it possible to encode your design patterns?

And so, as I read "Modern C++ Design" by Andrei Alexandrescu (a book on doing good things with C++ templates), I come across the following phrase:

Eventually, Andrei turned his attention to the development of template-based implementations of popular language idioms and design patterns, especially the GoF [gang of four] patterns. This led to a brief skirmish within the Patterns community, because one of their fundamental tenets is that patterns cannot be represented in code. Once it became clear that Andrei was automating the generation of pattern implementations rather than trying to encode patterns themselves, the objection was removed...

Bleargh. It sucks when people get all religious about the interpretation of a given piece of "scripture". Even when people manage to work around it by using illogical arguments ("No, no, it's not a representation of the pattern in code, it's a bit of code that automates the creation of implementations of the pattern..."). Sadly, in other realms, this kind of thing can lead to people being burnt at the stake.

Sump plug woes (by alaric)

It's time I changed the oil (and the oil filter) in the van, since it's been over a year now.

So I bought myself some new oil, and a new oil filter, and prepared to do the deed according to the instructions in my Haynes manual.

The first step is removing the plug in the bottom of the oil sump, that lets the oil out. It looks like this:

Naturally, when you remove it, lots of hot oil comes running out (you do this with the engine warm so the oil flows well). So you do it with a bucket underneath!

However, I've not gotten as far as that step yet, because the head of the sump plug (which is hexagonal, like a bolt) is rounded. You can see in the pictures how the corners of the hexagon have all come off. They're shiny where I've been trying to grab them with a spanner, which just rotates when I apply enough force... and it doesn't take very much force.

So I went out and bought a special thing for undoing damaged nuts and bolts. It'll make a mess of the sump plug, so I bought a new sump plug, too. The magical nut remover looks like this:

The spiral grooves inside are rifled so that when you put it onto a nut and turn it (such as with the big adjustable spanner in the background), the points grip into the metal of the nut, and the spiral pulls the socket onto the nut as it twists it out, so that it can't push the socket off. Most stripped nuts and bolt heads are rounded towards the top, so tend to push spanners and sockets off as they are twisted; this thing pulls itself onto the nut with the spiral shape, thus counteracting this.

However, my sump plug bolt head is already far enough gone that the special socket rotates on it (shaving off little curls of shiny metal as it goes) when I turn it, again with surprisingly little force - but the next size of socket down doesn't fit onto it...

Perhaps the sump plug was made out of lead or aluminium or something?!?

I guess the next thing to try would be getting under there with a file and shrinking the head down to the next size by putting new flats on it.

But, since it's freezing outside and I've nowhere inside to work on the van, and we need the van ready to do a long drive in a couple of days (which I'd rather not do with the current black oil in, in this punishing weather), I think I'm going to take it to a garage and ask them to sort it out for me... I can give them the oil, the oil filter and the replacement sump plug I've already bought and just ask them to do the hard part!

How to use a Maplin USB microscope on a Mac (by alaric)

We received a USB microscope from Maplin for Christmas, from my father and stepmother. It's basically a webcam engine coupled to a different optical system; so there's a small hand-held device on a cable that plugs straight into a USB port. It has inbuilt LED lights, so you push the end right up against a surface, and the casing of the unit holds the surface at the correct focussing distance while illuminating it. There's a manual control to focus, but once it's set, you don't need to fiddle with it.

It's Maplin code N87FX.

Anyway, it claims to work under various flavours of Windows, and certainly doesn't work out of the box on a Mac - but a bit of research (see the USB vendor/device codes in System Profiler, looking them up at http://www.linux-usb.org/usb.ids to find the generic name of the device, googling from there) led me to http://www.pc210.com.cn/downloadhtml/sonix201_PC_Camera_driver_for_MAC_405.html where a suitable driver can be downloaded. The RAR file wouldn't expand under Stuffit, but unrar from pkgsrc on my NetBSD system opened it fine, to reveal two zip files (duh) each containing a .dmg file (duh) which had a different driver in each... I just installed both.

Skype then immediately picked up on it as a webcam. But Photo Booth refuses to talk to anything other than the inbuilt iSight on my laptop, but then I found that the driver installs had also installed an app called Webcam Monitor, which is a simple wrapper around the underlying Quicktime video capture APIs, adapted from Apple's reference source code. It can record video, or a simple ⌘-C will copy a picture to the clipboard for Preview.app to save. It'd be nice to find something where I could just press a capture button and have a snapshot appear in a list which I can then choose a name for and save as a .PNG to automate this, though. Perhaps I ought to do my own quick adaption of Apple's reference source code...

Observe, my stubble!

Puzzling properties of my mind (by alaric)

On Saturdays, I take the train home from London.

I was feeling particularly groggy last Saturday; I was tired for some reason, I had a lot to carry (three bags, two heavy ones and one bulky one full of clothes), and I foolishly decided to pack and get straight on the Underground without going out to buy and then eat breakfast, in my desire to get home to my wife and child soon.

So after climbing the steep hill between my lodgings and the underground station I was already running a bit short on blood sugar. Which is probably why, while navigating a route around the underground lines they had closed for maintenance over the weekend, I got on the Victoria Line in the wrong direction, and nearly got on another line in the wrong direction while making my way back.

I grabbed some food at Paddington and leapt onto the next homewards-bound train; there were no free seats so I crouched in the vestibule with my bags, wolfing down my food, then leaning blearily against the wall while my body verified it really was food, so that it could afford to release my final reserves of nutrients to keep me going while it processed the new bounty (although, as it made it abundantly clear to me, it would rather I slept while it did this).

I got off at Swindon to change for the Stroud train; it was 2:35pm or so, and the next train to Cheltenham Spa was at 3:00pm or so, so I rang home and told them to pick me up from Stroud at 3:30pm, then went and sat down.

Some ten minutes later I glanced up at the indicator board, and saw that the next train was actually at 3:15pm. Odd - I must have misread it. I rang Sarah up and said I'd be a quarter of an hour later. It was still a Cheltenham Spa train from platform 2, though.

So at 3:10pm I went to platform 2, which was a little terminating platform used for the branch line up to Cheltenham, and got onto the train. It pulled off, and the inspector came to check tickets; when I showed him mine, he said "But this train doesn't go to Stroud. We're going to Westbury! You'd best get off at Chippenham and get the next train back to Swindon!".

Bleargh.

Well, I did that, and made it onto the correct train this time, and got to Stroud at about 4:45pm instead of 3:45pm; I wonder what time I'd have made it home if I'd not gone the wrong way on the Underground, too.

But through this whole day of bleary groggy failing to manage to comprehend even something as simple as the rail system, what did I while away my long waits doing?

Analysing distributed algorithms. A specialist topic far more involved than rail route planning.

How I was finding that a welcome rest from trying to figure out which train to get on, I don't know.

Or does this just say something about our rail system?