How to use a Maplin USB microscope on a Mac (by alaric)

We received a USB microscope from Maplin for Christmas, from my father and stepmother. It's basically a webcam engine coupled to a different optical system; so there's a small hand-held device on a cable that plugs straight into a USB port. It has inbuilt LED lights, so you push the end right up against a surface, and the casing of the unit holds the surface at the correct focussing distance while illuminating it. There's a manual control to focus, but once it's set, you don't need to fiddle with it.

It's Maplin code N87FX.

Anyway, it claims to work under various flavours of Windows, and certainly doesn't work out of the box on a Mac - but a bit of research (see the USB vendor/device codes in System Profiler, looking them up at http://www.linux-usb.org/usb.ids to find the generic name of the device, googling from there) led me to http://www.pc210.com.cn/downloadhtml/sonix201_PC_Camera_driver_for_MAC_405.html where a suitable driver can be downloaded. The RAR file wouldn't expand under Stuffit, but unrar from pkgsrc on my NetBSD system opened it fine, to reveal two zip files (duh) each containing a .dmg file (duh) which had a different driver in each... I just installed both.

Skype then immediately picked up on it as a webcam. But Photo Booth refuses to talk to anything other than the inbuilt iSight on my laptop, but then I found that the driver installs had also installed an app called Webcam Monitor, which is a simple wrapper around the underlying Quicktime video capture APIs, adapted from Apple's reference source code. It can record video, or a simple ⌘-C will copy a picture to the clipboard for Preview.app to save. It'd be nice to find something where I could just press a capture button and have a snapshot appear in a list which I can then choose a name for and save as a .PNG to automate this, though. Perhaps I ought to do my own quick adaption of Apple's reference source code...

Observe, my stubble!

Puzzling properties of my mind (by alaric)

On Saturdays, I take the train home from London.

I was feeling particularly groggy last Saturday; I was tired for some reason, I had a lot to carry (three bags, two heavy ones and one bulky one full of clothes), and I foolishly decided to pack and get straight on the Underground without going out to buy and then eat breakfast, in my desire to get home to my wife and child soon.

So after climbing the steep hill between my lodgings and the underground station I was already running a bit short on blood sugar. Which is probably why, while navigating a route around the underground lines they had closed for maintenance over the weekend, I got on the Victoria Line in the wrong direction, and nearly got on another line in the wrong direction while making my way back.

I grabbed some food at Paddington and leapt onto the next homewards-bound train; there were no free seats so I crouched in the vestibule with my bags, wolfing down my food, then leaning blearily against the wall while my body verified it really was food, so that it could afford to release my final reserves of nutrients to keep me going while it processed the new bounty (although, as it made it abundantly clear to me, it would rather I slept while it did this).

I got off at Swindon to change for the Stroud train; it was 2:35pm or so, and the next train to Cheltenham Spa was at 3:00pm or so, so I rang home and told them to pick me up from Stroud at 3:30pm, then went and sat down.

Some ten minutes later I glanced up at the indicator board, and saw that the next train was actually at 3:15pm. Odd - I must have misread it. I rang Sarah up and said I'd be a quarter of an hour later. It was still a Cheltenham Spa train from platform 2, though.

So at 3:10pm I went to platform 2, which was a little terminating platform used for the branch line up to Cheltenham, and got onto the train. It pulled off, and the inspector came to check tickets; when I showed him mine, he said "But this train doesn't go to Stroud. We're going to Westbury! You'd best get off at Chippenham and get the next train back to Swindon!".

Bleargh.

Well, I did that, and made it onto the correct train this time, and got to Stroud at about 4:45pm instead of 3:45pm; I wonder what time I'd have made it home if I'd not gone the wrong way on the Underground, too.

But through this whole day of bleary groggy failing to manage to comprehend even something as simple as the rail system, what did I while away my long waits doing?

Analysing distributed algorithms. A specialist topic far more involved than rail route planning.

How I was finding that a welcome rest from trying to figure out which train to get on, I don't know.

Or does this just say something about our rail system?

A stressful upgrade – and a design for a better backup system (by alaric)

My server cluster has been having lots of outages lately - which I traced down, after much experimentation, to probably being the ethernet driver in NetBSD 3.1 (which I was running), since the machine in question (as luck would have it, the NFS/NIS server; mental note, make the other server be a NIS slave so it can run on its own...) seemed to just disappear from the network but be perfectly happy if spoken to over a serial console - but ifconfig wm0 down ; ifconfig wm0 up would then hang it.

So, since a machine with the same Ethernet interface but running NetBSD 4.0 was running fine, and I could see there'd been a lot of commits to the driver between the versions, I decided to upgrade it.

Easy enough, right? Stick in the NetBSD 4 boot CD, boot, select Upgrade. But then problems struck; my /usr and /var are on a RAIDframe mirror set, and the install kernel didn't have RAIDframe installed. So I just let it install into /usr and /var directories under the root directory, and then booted into the new system, mounted the RAID, and copied the new /usr over my old one in order to upgrade all the binaries while leaving /usr/pkg and /usr/local untouched.

Read more »

Block device snapshots in NetBSD (by alaric)

fss is a neat feature in NetBSD that I hadn't noticed (and I suspect few people have). It's a filesystem snapshot driver.

You attach it to a mounted filesystem, and whenever a block in the underlying disk partition is written to, it copies the old block to a snapshot file. It also creates a snapshot block device, and any reads to that device look in the snapshot file first, then look in the underlying disk partition if there isn't a corresponding block in the snapshot file. End result? The snapshot block device looks like a frozen-in-time copy of the underlying disk partition, without the space and time requirements of actually copying the whole thing.

It works like this:

        -bash-3.2$ cat /home/alaric/test.txt
        THIS IS A TEST
        -bash-3.2$ sudo fssconfig fss0 / /tmp/back
        Password:
        -bash-3.2$ sudo mount /dev/fss0 /mnt
        -bash-3.2$ cat /mnt/home/alaric/test.txt
        THIS IS A TEST
        -bash-3.2$ cat > /home/alaric/test.txt
        THIS IS NOT A TEST
        -bash-3.2$ cat /mnt/home/alaric/test.txt
        THIS IS A TEST
        -bash-3.2$ cat /home/alaric/test.txt
        THIS IS NOT A TEST
        -bash-3.2$ mount
        /dev/wd0a on / type ffs (local)
        kernfs on /kern type kernfs (local)
        procfs on /proc type procfs (local)
        /dev/fss0 on /mnt type ffs (local)
        -bash-3.2$ sudo umount /mnt     
        -bash-3.2$ sudo fssconfig -lv
        fss0: /, taken 2008-11-29 01:21:23, file system internal
        fss1: not in use
        fss2: not in use
        fss3: not in use
        -bash-3.2$ sudo fssconfig -u fss0
        -bash-3.2$ cat /home/alaric/test.txt 
        THIS IS NOT A TEST
        -bash-3.2$ sudo rm /tmp/back
        override rw-------  root/wheel for '/tmp/back'? y

...taking it from me that there were no long pauses after any command in that sequence; no operation took time noticeably proportional to the 8GB size of the disk partition.

I created the snapshot file on the same filesystem I was snapshotting, which is only allowed for the ffs filesystem due to needing special locking; but it can snapshot any filesystem if I put the snapshot log on a different filesystem.

This is useful for various things:

1. Consistent backups. As a backup, that might take hours, runs, the filesystem is changing underneath it, meaning that you can end up with a broken backup when your applications change groups of files together; the backup can end up with non-corresponding versions of things. Imagine running a backup while reinstalling a load of applications and their shared libraries, and it getting a mixture of old and new versions of libraries and binaries. Ick.
2. Short-term onsite backups. Imagine running a snapshot every hour, from a cron job, and deleting the oldest snapshot, so you have four hourly snapshots on the go. If you do something stupid, you can go back and retrieve old versions of your stuff. Or perhaps a week's nightly snapshots. Not a backup in that it won't protect against system failures, but it's the kind of backup you can go back to when you mess something up at the filesystem level.
3. Trialling potentially disastrous operations, like major software upgrades. Take a snapshot beforehand. If it fails, then copy the afflicted files back from the snapshot.
4. Security auditing. Take regular nightly snapshots, then you can compare them to the live system to see what's changed, to help analyse successful breakins.

There is one caveat: a snapshot taken from a mounted filesystem will, when itself mounted, of course give you a log warning:

  /dev/fss0: file system not clean (fs_clean=4); please fsck(8)
  /dev/fss0: lost blocks 0 files 0

...and you can't fsck it since it's read-only, so you might run into trouble with that, but I think a good sync before taking the snapshot should make the window of opportunity for problems quite small.

This is really neat stuff. It's been in since NetBSD 2.0, and is still marked as experimental - so more people need to try it out, find any bugs, and otherwise confirm it works fine so it can lose that experimental tag 😉

I’m missing Scheme (by alaric)

I've not done any Scheme programming for ages. In fact, the past few months have been quite a haze of relentless hard work; I'm liking what I'm actually doing for a living, except I've been doing rather a lot of faffing about recruitment rather than actually doing it lately.

I'm having to spend half of my week in London, and the other half working from home - but with Sarah away that half of the week doing her course, I'm working from home alone by day and looking after Jean in the evening, while having my working day bracketed by taking Jean to and from nursery, which is a half hour round trip each way. All the thrill of commuting without the fun of working somewhere different to where you sleep, or with people.

So, no programming-for-fun lately! But that can't last forever, since trying to stop my mind from going exploring for too many months in a row is always rather futile.

So I came across Ventonegro's post on and-let* and it set me thinking. The Lisp family of languages (which includes Scheme) are renowned for their macros, which are the key rationale for the minimalist syntax; without things like if holding a special place in the language, user-written macros are just as powerful as anything that comes built into the language. This lets you extend the language with features that you'd be mad to build into a language core, but which are nonetheless useful reusable constructs, such as and-let*.

As an aside, let me just explain and-let* - the name is a terse mnemonic that makes sense to Schemers and nobody else, but it's a way of compactly writing bits of code that attempt to compute something in steps, where the trail might end at any step and fall back to some default. The example Ventonegro gives is rather good:

  (define (get-session request)
    (and-let* ((cookies (request-cookies request))
               (p (assoc "session_id" cookies))
               (sid-str (cdr p))
               (sid (string->number sid-str))
               ((integer? sid))
               ((exact? sid))
               (sp (assq sid *sessions*)))
      (cdr sp)))

Which translates to:

• If there are cookies available
• And there is one called session_id
• And parsing it as a session id succeeds
• and the session id is a number
• and that number is an integer
• and that number is exact (eg, 3 rather than 3.0)
• and that number is the ID of an existing session
• ...return that session

A few languages happen to make that pattern easy to write natively by putting assignments inside an and, as Peter Bex points out, but with Lisp you don't need to rely on the that piece of luck; you can roll your own luck.

There's a whole library of useful macros and combinators (another handy higher-order programming tool) in most Scheme systems, and any your system lacks can be copied easily enough. But it occurs to me that there's very few educational resources on actually using them. I think a definite theme, if not a whole chapter, in a "practical Scheme" book would have to be the introduction and then applied use of such handy macros (and a damned good reference guide to them all), because reading the definition of and-let* failed to really fill me with inspiration for situations I'd use it in. While reading Ventonegro's example reminded me of some ugly code I'd written that could be tidied up by just using and-let*.

It's great to be able to assemble your own syntactic tools, but presenting them as one unorganised mass will just make your language seem as complex and messy as C++ and Perl combined; yet sticking only to the core base language and expecting programmers to spot their own patterns and abstract them out results in duplication of effort, and every piece of source code starting with a preamble full of generally rather general macros, neither of which are good. Rather than choosing a tradeoff between the two, as static programming language designers are forced to, we need to find a way of cataloguing such tools so that they can easily be split by category, and by priority; so the ones that are most widely useful can be learnt wholesale, and ones that are more useful in certain niches can be glanced at once then gone back to and studied if required.