Failing (by sarah)

I've failed at morning - there is pink tooth paste on our bed and Mary only half dressed in the car for school 🙁 Still I think that is the first proper melt down we have had so far this term with getting ready in the mornings so... getting there - of course Jean has taken the bright turquoise coat to school so I am sure we will be getting a letter or she'll be getting a detention or something about that as she's only allowed black ones :/ To be fair her black one is at the school drying out from yesterday still but she was supposed to take my coat!

My chest is still bad and I've failed to finish decorating the girls rooms. I've now been ill since the 2nd of November and am BORED. Hoping it will sort itself asap.

Yesterday was a System of A Down and Cradle of Filth et al kind of day, Jean came home sans coat and bag as they were too wet and she'd been lent other things in their sted. Which was a relief as I thought she'd gone out in the torrential rain without anything!

Today I wonder if she's remembered her jujitsu bag as she's going straight from school with her friend and I forgot to remind her - I can see the bag from here but that does not mean she hasn't got the trousers and t-shirt with her - the jacket is just too bulky for her to carry in with all her school things.

Maybe I'm letting her down by not sorting it all out, by not driving etc - I hope she is just becoming independent. She's actually pretty epic at organising herself considering she is organising herself and she is mine and Al's daughter and she is only 11.

Rain like this always makes me worry - in 2007 before Gloucester but in Gloucestershire we were flooded and ended up being out of our home for about a year. Rains since have caused issues with the new houses roof etc... and though I know that means it's now a good roof... the fear is there tangled in my brain, if it rains heavy I feel I should go and just check that things aren't flooding - because you know I'd be able to do something about it if it where :/

I'm all mouth ulcery - and run down... thinking it's the aneamia, thinking it's still on running issues of having gotten low levels of gluten etc... over the summer etc... but I don't know.

There are happy things to write about just feeling a little deflated so thought I'd share what was going on. Alaric's new job is great - he's loving it but due to traffic in Cheltenham he is not getting home until gone 7 at night and because he does school run in the mornings he's gone for like 12 hours to do do his 9 hr job. This is the first time I've been on my own on my own in the house everyday since having kids... there has always been a kid about and/or a husband. It's weird because instead of the relief of them having their 2 days in nursery it's like... the house is EMPTY.

I think I'm getting less done but I also think I'm getting more done as I am doing the new rest regime from the doctor to try and get the head bang healing properly.

Tomorrow there is coffee with a friend and at some point I need to go and pick up some bits from another... I have stalls to organise for Salaric Craft and The WigglyPet Press for December and I need to decide weather to shut down my Patreon account due to the fact I think I'm going to end up triple taxed on income that otherwise would be taxed once max.

It's a shame I like the platform... :/

Anyway I will now go and up load pics so I can get back to cutsie blogging and political rants.

Towards the Family Mainframe (by alaric)

Last September, I posted progress on the construction of our domestic mainframe. To recap, the intent is to build a dedicated home server that's as awesome as possible - meaning it's reliable, safe, and easy to maintain. That rules out "desktop tower PC in a cupboard" (accumulates dust bunnies, gets too hot, easily stolen, prone to children poking it); "put a 19" rack somewhere in your house" is better, but consumes a lot of floor footprint and doesn't fix the dust bunny problem. So I've made my own custom steel chassis; fed cold air at pressure via a filter, incorporating a dedicated battery backup system, locked and anchored to the wall, and with lots of room inside for expansion and maintenance.

Since that blog post, I've finished the metalwork, painted it with automotive paint using a spray gun (which was a massive job in itself!), fixed it to the wall, and fitted nearly all of the electronics into it.

A significant delay was caused by the motherboard not working. I sent it back to the shop, and they said it was fine; so I sent the CPU back, and they said THAT was fine; so I sent both back together and it turned out that the two of them weren't compatible in some way that was solved by the motherboard manufacturer re-flashing my BIOS. That's now up and running; I was able to use the HDMI and USB ports on the outside of the chassis to connect up and install NetBSD from a USB stick, then connected it to the network and installed Xen so I can run all my services in virtual machines. It's now running fine and everything else can be done via SSH, but the HDMI and USB ports are there so I can do console administration in future without having to open the case (unless I need to press the reset button, which is inside).

The one thing it's lacking is the management microprocessor. I've prototype this thing on a breadboard and written the software, but need to finish off the PCB and cabling: but it will have an AVR controlling three 10mm RGB LEDs on the front panel, and three temperature/humidity sensors in the inlet and outlet air (and one spare for more advanced air management in future). But the idea is that the three LEDs on the front panel will display useful system status, and the environment sensor data will be logged.

Here's what it looks like from the outside; note the air inlet hose at the top left:

The socket panel on the left hand side worked out pretty well - 240v inlet at the bottom, then on the aluminium panel, three Ethernets, HDMI, and USB (my console cable is still plugged into the HDMI and USB in the photo, which won't usually be the case):

And here's the inside, with lots of space for more disks or other extra hardware; the big black box at the bottom is the battery backup system:

Now I have Xen installed, I'm working on a means of building VMs from scripts, so any VM's disk image can be rebuilt on demand. This will make it easy for me to upgrade; any data that needs keeping will be mounted from a separate disk partition, so the boot disk images of the VMs themselves are "disposable" and entirely created by the script (the one slightly tricky thing being the password file in /etc/). This will make upgrades safe and easy - I can tinker with a build script for a new version of a VM, testing it out and destroying the VMs when I'm done, and then when it's good, remount the live data partition onto it and then point the relevant IP address at it. If the upgrade goes bad, I can roll it back by resurrecting the old VM, which I'll only delete when I'm happy with its replacement. This is the kind of thing NixOS does; but that's for Linux rather than NetBSD, so I'm rolling my own that's a little more basic (in that it builds entire VM filesystems from a script, rather than individual packages, with all the complexities of coupling them together nicely).

I'm using NetBSD's excellent logical volume manager to make it easy to manage those partitions across the four disks. There are two volume groups, each containing two physical disks, so I can arrange for important data to be mirrored across different physical disks (not in the RAID sense, which the LVM can do for me, but in the sense of having a live nightly snapshot of things on separate disks, ready to be hot-swapped in if required). I still have SATA ports and physical bays free for more disks, and the LVM will allow me to add them to the volume groups as required, so I can expand the disk space without major downtime.

So for now it's just a matter of making VMs and migrating existing services onto them, then I can take down the noisy, struggling, cranky old servers in the lounge! This project has been a lot of work - but when I ssh into it from inside the house (over the cabling I put in between the house and the workshop) and see all that disk space free in the LVM and all the RAM waiting to be assigned to domU VMs that I can migrate my current services to, it's all worth it!

The family mainframe (by alaric)

I'm in the process of consolidation the home fileserver and the public Internet server - currently two separate bits of hardware - into a single physical device, virtualised to support multiple indepedent machine images. Having a single family mainframe will simplify the management of the complex web of computers and services that support our digital life.

For various reasons, the best place to build such a thing is at the office end of my workshop. Even though it's at the "clean" end, this is still a room that is prone to having fine conductive dust in the air, varying humidity and temperature, and (heaven forbid) a leaking roof. Also, I want a case with extensive room for upgrades, and which makes it easy to replace parts. Having used 1U rack-mounting servers for quite some time, I am sick of highly compact servers that are difficult to work with, requiring extensive dismantling to get to parts.

Clearly, I needed a rather special chassis for this new family mainframe, so I bought a load of steel, picked up my tools, and got to work. I've been working on this for months; I initially cut up the metal at home, then visited a friend's workshop to borrow his pneumatic rivet gun and his MIG welder. Since obtaining my own TIG welder, I've been able to continue at home.

The chassis is nearly structurally complete; this weekend, I've been attaching mounting brackets inside it for everything to attach. All that remains is to finish welding the upper panel on, then the whole thing can be cleaned and galvanised, and the exterior painted. Then I can fix it to the wall and start fitting the electronics and electrical systems!

The first thing I did this weekend was to fit mounting brackets for the processor frame. This is taken from a standard ATX case, and is the base plate with standoffs to mount the motherboard, the frame to attach expansion cards to, and the frame to hold the PSU. This is screwed into the chassis, so that I can use an existing frame (rather than having to make one myself), and so I can replace it if needed. The frame is held in place by two locating pins that fit into holes in it, and then two screws through the upper-left corner (I drilled and tapped holes in the top left bracket), and a little spacer at the top right to stop it from flexing:

With the frame in place, it looks like this:

Next came the expansion frames. I may need to add additional hardware inside the chassis in future, but once it's holding a running server and painted and bolted to the wall, I can't really take it down to weld additional brackets into. So I cut off one-inch lengths of square tube, drilled and tapped a hole in the centre of one side, and welded them to the inside of the chassis. I drilled holes in the ends of strips of steel, so they screw into the pairs of brackets, creating a metal strap that can be removed, things mounted onto (via welding etc), and then screwed back into place, without causing major disruption. There are two - one beneath the process frame, above where the UPS will go; and another right at the top, above the environment management system.

Here's the upper one:

And here's the lower one:

The welds were quite difficult, as I had to reach right down into a corner of the chassis. As such, they were either OK or awful, depending on whether I had to use my right (dominant) or left hand:

I also cut and drilled some mounting flanges, which will be what are used to bolt it to the wall:

When I made the sides of the chassis, I welded angle iron onto them, in order to attach said flanges:

(Note the plasma-cut hole, which will be where a removable plate with sockets for Ethernet, VGA, and USB will go).

The mounting flanges are quite thick (the wall is rough and bumpy, so the chassis needs to be spaced slightly from it), so it was good fun welding them to the much thinner angle iron. I think I did an OK job:

Then I mounted the internal frame for mass storage devices, which goes above the processor frame, below the environment management system. It's a metal plate drilled for lots and lots of 3.5" disk drives, which attaches (with screws) to brackets I welded into place:

With all the internal stuff done, I started to weld the top panel in place, which I'd avoided in order to enable me to get access into the top:

Annoyingly, I ran out of argon while doing the tack welds. A TIG welder without shielding gas is a lot like a plasma cutter, and I burnt a nice hole in a shower of sparks. It's only a small hole, so I'll be able to weld over it when I finish the job off.

Unable to do the final welding, I drilled a hole in the eaves, where clean outside air will be drawn in through a duct into the environment management system:

I also hefted the entire thing up to the wall where it will be mounted, propped it in position, leveled it, and drilled through the holes in the flanges to make the holes that will be used to anchor-bolt it in position:

Workshop progress (by alaric)

Setting up my workshop has been a long battle, thanks to the roof leaking (and everything getting covered in slime mold because of the damp, and ivy growing in through the holes in the roof and shedding sticky sap and dead leaves on everything, and a cat getting in there and pooing on the floor...). Things got rusty, everything got grimy, and stuff was moved around willy-nilly to get it out of the water; and then because everything was in the wrong places, stuff couldn't be put away properly, and so things ended up piled wherever they could go. This was quite distressing for me; by nature, I'm a person who makes things, but I've not been able to do anywhere near as much making as I'd like for years, because I didn't have a nice place to work and didn't have easy access to my equipment.

But, now the roof is fixed, and everything's had about a year to dry out. I've had time to continue painting the floor (it's all now covered in paint, but some parts need a second coat); I've cleaned and tidied;0 and thrown away wood and metal that was too rusted to be of any use, and scraped up the remains of cardboard boxes that had dissolved into fungus, and found and removed all the cat turds and scrubbed the floor with disinfectant; thrown away computer parts that were covered in disturbing fungal blooms; and cleaned and tidied the computer desk. I've made racks to store the wood and metal stocks that are still usable, and put down linoleum under the computer desk so the chair can roll around freely (it didn't do too well on the rough concrete) and I have a nice surface to rest my feet on.

I put up a bracket with hooks for my boiler suit and lab coats, by the door, as the place they used to hang now interferes with the material racks:

I've got the compressor installed under the workbench, rather than kicking around the floor:

And my computer desk is all set up nicely; you can't really see it here, but there's a desktop PC, with a nice set of speakers and an amplifier so I can listen to music - or the audio from my wide-band scanner, seen to the right of the monitor, which picks up the FM broadcast band nicely:

What's next? I still need to assemble some shelving, as there's still stacks of flimsy plastic crates holding a lot of stuff. And put a second coat of paint on some bits of the floor (which will be easy to get to when the crates are gone). And I've got a metal garden waste incinerator I was going to turn into a furnace (but which is far too large, and now I've helped somebody else build a furnace that he lets me use), which I need to find a new home for. And I need to find a place to store the festival trolley, which currently kicks around on the workshop floor (getting in the way and offering plenty of shin-scraping opportunity). And I need to finish the meter-and-a-quarter-high heavy steel server chassis standing in the middle of the room (which will be fixed to the wall next to my computer desk when it's finished).

I want to make a new welding bench, too - my current one is curved, as (in the wildness of youth) I tried to put far more welds between the top surface and the frame than was needed, causing it to warp. This means it's very hard to make flat things, as they don't lie flat when I'm lining them up to weld. And, as it stands on four rather thin angle-steel legs, on a rough concrete floor, it wobbles, so isn't much use to mount a vice on. Speaking of vices, while clearing up in the workshop, I came across this beast:

It's a "leg vice"; the long metal leg should be embedded into the ground to steady it. It's a blacksmith's tool, intended to hold something while it's battered with a hammer; thus the exceptionally sturdy construction. If I bolted it to my current bench, then the bench would fall over if I tried to use it in earnest. We found it in the stable where we lived before, which is the ancestral home of one side of my family; I was given it as I thought I could try and get it working, although I was expecting a long task ahead of me to rebuild seized parts. Thankfully, the screw thread was in perfect condition, and penetrating oil and elbow grease got the joint un-seized, and it's now working nicely. It still needs some rust removal to make it more pleasant to touch and to avoid contaminating everything I clamp with rust, but that won't take long.

I costed up materials to make one with a more rigid frame, bolted to the wall at the back and with two legs at the front so that it wouldn't wobble (and, with the wisdom of experience, only stitch-welding it to the frame, as that'll provide more than enough strength without curving it into a bow); making that will cost a little over sixty pounds (plus welding consumables and electricity). The design includes a mounting point for the leg vice, so that it protrudes out into the room (with the top of the vice level with the top of the bench, so it's not in the way of large things going on the bench), and converting the old bench into a shelf under the new bench to provide much-needed handy storage for grinding tools and welding clamps.

However, having just spent a bunch of money on welding gear I'm not going to be in a position to splash out sixty quid on a welding bench for another month or so (let alone buying metal for other metalwork projects needed around the house, such as a set of railings for the front of the house...). Thankfully, we've found a local scrap metal merchant who are willing to let us rummage around for cheap metal! When we get a chance, I'm going to head over there and see if they have any steel plate for the top and box/angle section for the frame and legs... It'll be nice to have a good welding project to focus on with my new TIG welder!

Cabling my workshop (by alaric)

As a child, I read a novel called Triplanetary, which featured a character called Gray Roger. Roger was the evil master of a space-born flying citadel, and usually sat in an office, at a massive control desk. Through this desk, he controlled the massive machinery driving the citadel and its tremendous weapons.

This image (and many others like it) meant a lot to me; for as long as I remember, I have wanted a Secret Lair with Control Panels.

When we bought our house, one of the big selling points for me was a small building at the end of the garden, which I turned into my workshop. The far end has my computer desk and my electronics desk, and the end by the doors is a messy area for metalworking.

However, the years since have been blighted by problems with the roof leaking, which has damaged furniture and tools, and caused horrible mold to grow on everything. Last summer, with the help of a friend, I properly felted the roof, which seems to have solved the leaking. There is still work to be done cleaning the interior and re-painting stained floors, but the inside has now dried out.

However, even aside from the roof problems, I had no Internet connection to the workshop. Wifi couldn't reach from the house, and wouldn't be sufficient for my needs. So even with the place drying out, I've not had a place I can sit and work with a computer at home; everything was done with laptop on lap on the sofa, or sat at the kitchen table, or similar - hardly comfortable, while I have a proper desk and office chair sitting out there!

So I have long wanted to run Ethernet cables down to the workshop, and haven't had a chance to organise it while I'm focused on trying to fix the roof.

But the colocation deal I use to host my Internet server is coming to an end, and rather than moving to a new one, I'm going to run the server from home. We have reliable, low latency, high-speed fibre broadband from Andrews and Arnold, and a server hosted at home will be one I can attend to for maintenance, upgrades and emergencies; having the current one based in London is somewhat inconvenient!

As there's nowhere good in the house to keep servers, they need to go in the workshop - so I now have a deadline to get Ethernet out there. And over the past few weekends, I have done just that, with much help from Jean (a lot of cabling work cannot be done single-handed; long bits of trunking need to be held up at both ends before being stuck down, and pulling cable through conduit requires somebody pulling at one end while somebody else feeds the cable in carefully at the other).

We had to use no less than three different cabling techniques, as the cables pass through several different regions.

It all starts in the cupboard under the stairs, where the PPPoE termination of the fibre broadband from the phone network is. I've since mounted the patch panel onto the wall and tidied up the cabling, but here's how it looked earlier:

The cables go through the wall (which was once the external wall of the house before it was extended, so required drilling with a massive diamond core drill) into the library, where I've put a neat white box over the hole:

The trunking goes off to the kitchen wall. The hole through that wall is smaller, so can be hidden behind the trunking at the left. In the kitchen I used smaller trunking that can't cover the hole, so another white box covers the hole on this side:

Across the garden, we had to switch from trunking to waterproof conduit, all sealed together with funky glue, and access boxes with waterproof gaskets:

Then inside the workshop, I used suspended cable trays. I love suspended cable trays - I can just lay extra cables up there whenever I need, which is of course a great feature for a Lair full of Controls:

Finally, the cables come down from the tray system into the comms cabinet:

Last night, I tested all four cables with a cable tester, and all pairs are OK. I've put the desktop PC I built (but never got to use) in the workshop and hooked it up to one of them, and it's working fine at Ethernet frequencies, too!

Having a desktop machine is a great boon, even if I'm sat in the house on my laptop - I bought a cheap, light, laptop with a view to using it as a remote terminal to the much beefier desktop machine where possible.

And, of course, no Lair is complete without a way of disposing of enemies. Having had enough of water around the place already, I opted against the traditional shark pits, and instead went for a cage full of dinosaurs instead.

And so now, I can sit in my bed with my laptop, and make the computer do things in the cold, dark, workshop at the end of the garden. Muhahah.