Intelligence, Knowledge, Wisdom (by alaric)

I see a lot of confusion between intelligence and knowledge. There's this cultural perception that a well-educated person is intelligent - and wise to boot. However, I've met plenty of people who are incredibly intelligent, yet know very little; or people who have memorised lots of information, but don't really understand it because they're not very intelligent.

So there's definitely a distinction between intelligence and knowledge, and I think there's also a third intellectual strength: wisdom. Note that I'm only analysing core intellectual strengths, too - skills like empathy are also important, but beyond the scope of this article; and thinks like mathematics are a matter of training the mind to do certain things quickly that can only be done if you're intelligent enough to master it in the first place, and have the knowledge - many skills, such as mathematics, being able to paint or to accurately outguess defenders and get a football into a goal, are also mental skills, but ones that are learnt by building upon the basic attributes of knowledge, intelligence, and wisdom.

(Players of role playing games will now recognise what got me thinking about all this in the first place)

Knowledge is stuff you can memorise directly from some external source. It might just be facts - the name of the first president of the United States, the melting point of lithium, who won the last World Cup.

Intelligence is a bit harder to define. It's about being able to perceive patterns, I think. An IQ test only really tests certain aspects of intelligence, but I think it's mainly pattern recognition and being able to hold complex structures in your short-term memory that you can then recognise the patterns in.

This can be likened to being able to gain a high-altitude view of things in your mind. The gifted artist looks at people's perceptions of the world, and sees them as all parts of a bigger picture; by spotting the bits of the picture that few people are seeing, and then managing to portray that to people (which is a skill), they manage to surprise and excite us, and to broaden our horizons by drawing our attention to things we had overlooked. The gifted mathematician looks at the properties of the Lambda calculus and of Hilbert systems and notices a shared pattern, and thus comes up with the Curry–Howard correspondence, and thereby realises profound facts about the processes of computation and reasoning that are driving the development of programming languages into the future.

And, yet, I have met people with great knowledge, and great intelligence who, nonetheless, are definitely fools.

There is a certain stereotype of the short-sighted baffled boffin; the kind who invents the nuclear weapon, then after it is used in anger, splutters "But... but... I thought it would be an end to warfare! I didn't think anybody would be so stupid as to use it!". The kind who sits there cranking out great work in their narrow field, yet without even being able to comprehend a reason why, yet alone wanting to.

And, also, I've met plenty of people who aren't intelligent, know very little, yet somehow manage to find their way peacefully and happily through life, bringing something undeniably good to everything they are involved with. They clearly have some positive core attribute, but what is it?

This is why I introduce the concept of wisdom. My hypothesis is that wise people have a grasp of certain fundamental patterns that underlie everything. Not the specific patterns that intelligence focuses on; more things like 'when two powerful forces are in opposition, things can slip and suddenly come out sideways'. This basic principle applies to physical forces, as well as to things like ideologies in opposition. They're patterns, and it's easy to mistake them for the rules that intelligence seeks to understand within complex systems; but there's some important differences. The patterns wisdom finds are broad. To find them, you need to look at a lot of things, rather than to look deeply at one. You can look at these things shallowly, and indeed, doing so can help you to spot the patterns without access to intelligence, by letting you "see the wood for the trees" rather than being tangled in details.

Also, they are weak correlations. They point to vague tendencies of systems, rather than to definite rules. They do not apply in all cases, even. They are more gut feelings, or intuitive hunches.

A truly great scientist combines all three attributes. They have knowledge of their field, the intelligence to understand it well enough to spot the rules, and wisdom that provides them with hunches; certain properties of a physical system may, based on past experience of such properties, lead the scientist to wonder if those properties will be conserved under rotation? Then they can use their knowledge and intelligence to do the maths and work it out, which may lead them to an interesting conclusion.

Similarly, the great artist has knowledge of the world, intelligence to spot interesting concepts - and wisdom that lets them guess how the viewers will react to their work. Nobody can know how the world will respond to something new, as we cannot know what's going on in other people's heads. No matter how intelligent you are, you'll never be able to reason the behaviour of millions of people. The best we can do is to draw on wisdom, to form hunches.

There is a stereotype of wisdom, but it's often confused with intelligence or knowledge. Sherlock Holmes is, perhaps, the stereotypical intellectual genius; his knowledge and intelligence are focussed on, but there's clearly wisdom as well. The purest expression of wisdom we find in popular media is the "wise old sage" stereotype. They're typically portrayed as old-fashioned and technophobic; they rarely exhibit vast stores of knowledge, or even great intelligence. They're usually an old-looking wispy-haired white male in a robe, spouting seemingly meaningless phrases that nonetheless turn out to be strangely insightful and useful.

This is a bit of a caricature, but with some vague connection to the reality, I think. A purely wise person would need to have been somewhat isolated from modern life in order to avoid ending up gaining knowledge, and the absence of knowledge would give them precious little complex mental structures to practise intelligence upon. But a long life would give an active mind time to figure out the deeper patterns, and build up wisdom. However, unless that wisdom didn't involve people much, I think a truly wise person would tend to have better communication skills than the traditional portrayal!

High power LEDs (by alaric)

I've got a few 3W RGB LEDs that I've been meaning to play with, so over the Christmas break, I decided to hook 'em up to the bench PSU and have a play.

As I have but one variable bench PSU with current limiting, I could only easily light one LED at a time. I didn't have big enough resistors to build individual LED current regulation circuits - I just set the current limit on my PSU to 0.35A and cranked the voltage up until it maxed out, hooked up to one LED in turn.

They are certainly dazzlingly bright:

Since the green and blue LEDs both have the same forward voltage, I figured I might be able to drive them together by using a pair of resistors as a current splitter, and setting the PSU for 700mA, thus ensuring that 350mA went to each LED.

However, my 0.25W resistors started to smoke when I got to about 400mA, so I shut it off - if one of the resistors burnt out then the entire 400mA would go into the surviving LED, overloading it (until its resistor also burnt out), and possibly making the thing explode. I ended up with a nice pair of burnt-out resistors:

Which is a shame, because I'd love to see how bright the thing is at maximum, with all three LEDs going!

My lab partner was most impressed, and asked me lots of questions about current and voltage; I had to resist her demands to keep making things, so I could go inside and write this blog post:

BlackBerry (by alaric)

Many moons ago I did some work writing apps for BlackBerries. I liked the things at the time; they seemed to be well-built, both from the hardware and software angles.

So when my mobile phone contract came up for renewal (meaning I can get a free new phone if I sign up for another two years), with my existing phone falling to bits and rather crashy, I was pleased to find that I was eligible for a free BlackBerry 8520!

The device is a nice evolution of the BlackBerry I was using back in 2004 or so; rather than the thumb-activated scroll wheel we now have a two-dimensional scroll thing that works like a trackball, but seems to really be the innards of an optical mouse, set up so it sees my thumb moving over a small plastic window. This works well, takes up little space, and has no moving parts apart from the click action when it's pressed in to select something. The one downside of the new hardware is that my original Blackberry had a reflective LCD; it had an optional backlight, but spent most of its time with it switched off, simply reflecting the light incident upon it (in colour!). This didn't make for vibrant, saturated, hues in photos, but it did save a lot of power, and meant that the screen was highly readable in the brightest sunshine.

There's a few rough edges in the software; my model lacks a GPS, and the supplied maps application lets me enter my home and work locations by typing in an address, then gives me the option to locate that from the current GPS position (which of course fails) or to look up the address - which also fails, claiming the state/province cannot be found. If I just put the postcode into the address and nothing else, it works - but only matches on the first part of my postcode, getting a location that's some distance away in my rural area. I'd like to have an option to choose the location I've scrolled the map to by hand, geocoding that doesn't suck, and no menu options about GPSes when I have no GPS, please.

The mail system tries to auto-configure itself. Which is a blessing, and a curse. I bet it's a blessing for many users, and their IT departments, that they can just enter their email address and password, and have the rest fetched. However, I have a funny mail setup; I have lots of different IMAP mailboxes on the same server, with usernames like "alaric-work". And there happens to be a POP3 daemon listening on the machine that hosts my employer's web site. So when I put in my work email address, it notices that the domain part of the email address has an A record, and it has a POP3 server, and that POP3 server has a user called alaric (which is the user part of my work email address) which it can log in as with that password - so it goes ahead and makes me a POP3 account... with the wrong username and wrong mail server. Which would be OK apart from the fact that there's no way of changing the protocol or username on an existing mail account.

The trick, it turns out, is to deliberately put the wrong password in on the initial setup screen. This causes the POP3 login attempt to fail, and the subsequent IMAP one (in order to be compatible with Outlook's autodetection, it tries POP3 before IMAP!) too; it then says it can't automatically configure me, and asks me to list username and mail server. It then proceeds to guess IMAP somehow (it still didn't ask me, and the machine has POP3 as well as IMAP on it), and pow, my IMAP account is set up.

Personal Information Management (by alaric)

One of the neat things computers have become able to do as they become more "personal" - eg, we spend more of our lives operating through them, and they become more portable - is personal information management (PIM).

I remember PIM apps in the early 1990s, running under MS-DOS. Quitting whatever you were doing and starting up a separate app to look at your calendar was a bit unwieldy so they didn't do so well, except perhaps Borland Sidekick, which used some clever tricks to pop itself up over running applications.

But nowadays we have systems like Apple's PIM components in Mac OS X; an address book, todo list manager, crypto keyring and calendar provided with the OS, with nice interfaces for using them yourself and an API for all applications to use the same databases. Mac OS software seamlessly uses the inbuilt PIM infrastructure wherever applicable, for the good of all. It's about as good as it gets, so far.

But it's not perfect.

For a start, the task list is a bit primitive. When I had a Mac, I used the excellent Things.app to manage my tasks, as it supports projects and roles and all that stuff, which helps to keep my hectic life compartmentalised. Since it has a much more rich data model than the OS' inbuilt task list, it has to have its own database to keep it in, but it integrates with the inbuilt one as well as it can. My tasks in Things.app get added to the native task list, without their extra information; and new tasks I add to the native task list appear in Things.app, in the 'inbox' area for unclassified tasks, awaiting my attention to move them to a project.

It'd be nice if the underlying PIM database was flexible, allowing arbitrary properties to be added to objects. Then the native task list viewer could share the same actual task list as third party apps, and it would just ignore the extra information about projects.

But even that would suck a bit. Imagine I also had a task manager app that synched to my smartphone, and let me tag each task with the physical locations I can do it from (eg, DIY must be done from home), so an app on the smartphone can show me tasks filtered by location. I'd then need to juggle two task list apps; both would be a superset of the basic task list app (and would therefore duplicate all the basic display-a-task, deal-with-due-dates, etc logic), but each adding their own extra features. Altering all of the properties of a task would involve finding it in two separate apps. Et cetera.

As it happens, I'm also interested in making more use of knowledge representation techniques in software. Knowledge representations such as Resource Description Framework or Horn clauses have the useful property that information from different sources can be merged, as long as the names of things are agreed upon. They work by storing information, not in tables (like the relational model), nor as a graph structure (as the in-memory data model of most programming languages), but as an unstructured list of statements about objects.

The objects can be literal values - strings, numbers, that sort of thing - or symbols of some kind, used to represent objects that don't (or might not) exist directly within the computer, such as people or concepts. RDF uses URIs as its symbols; Horn clauses (being a mathematical notation) are a bit vaguer as to what a symbol is. Either way, they have to be some identifier for the abstract objects.

Each statement links a number of objects, with a relationship (which is itself an object, usually constrained to be a symbol).

So say we have objects called alaric and cheese (the latter representing the general concept of cheese, rather than any particular lump of cheese), and an object called like, representing the concept of liking something. We might write something like:

  (like alaric cheese)

...to mean "alaric is related to cheese by like". As it turns out, just about anything can be represented as such statements. From the relational model, a table can be converted into a symbol used as a relationship (the table name will usually do), and each row into a relationship of the objects that are the values of the fields of that row (likewise, we could have a relational table called LIKES that lists the IDs of objects that like other objects). Relational models usually use arbitrary integers as the "symbols", and automated mapping into knowledge representations is hard because it's not always explicit if an integer column is an identifier, or the identifier of what (as it could be a foreign key), or just a price or some other actual integer quantity. But with a bit of human guidance, it can usually be done.

However, note that in a relational model, that LIKES table would have two foreign keys into some other table that lists all the objects that can take part in liking. It'd be impossible to say that alaric likes like itself (it's nice to like things!), since like is a table rather than a row in whatever table the foreign key pertains to. The relational model has a very static type system, as people who have tried to map class hierarchies into it often find.

Now, the fun thing about knowledge representations is that the objects are implicit. In a relational system there'd be a table of People or whatever, giving each person an arbitrary integer as a primary key then listing details of that person. You can point at rows in this table and say "there are the people".

But a knowledge model just has lots of facts about each person, sort of spread around. There's no place you can point at and say "there's the person". If you want a list of all the people, then you need to look for all statements saying (is-a-person X) ("X is a person"), which is as close as we get to assigning types (not that (is-a-person X) can't coexist alongside statements such as (is-a-customer X); an object can have lots of types, all overlapping). If you want to delete an object, you need to scan the knowledge base for all statements referring to that object, and delete those.

If my personal information was stored in a knowledge base, then Things.app could share the same basic relationship objects as the inbuilt task list. (is-a-task foo), (title foo "Feed the cat"), (is-urgent foo), (due-on foo (date 2009 12 16)), but also add its own: (is-a-project bar), (title bar "World domination"), (is-part-of foo bar).

Note the use of generic relationships - title gives any object a title. is-part-of is a generic containment relationship. This means that even without knowing about projects and task lists, software can tell that things have names, and that there's some kind of containment relationship to explore. The names we give objects - foo, bar in my example - are arbitrary; so for objects that don't have a natural name, they could just be random strings, like UUIDs; RDF even lets such objects have no name and be referred to implicitly. Objects we want to use widely (such as relationships) will benefit from nice names, however.

It's easy to merge knowledge bases, too. Just pour all the statements into one file. You have to be careful of the same object having a different name on each side, obviously, and how to fix that can only be handled on a case-by-case basis; RDF has the concept of "inverse functional properties" that are meant to uniquely identify an object (such as email addresses for people) that can be used to detect and automatically merge things, but it's not applicable to all situations.

If I merged my task lists from my laptop and my phone for the first time, for example, I'd now have more objects satisfying the query (is-a-task X); they'd be seamlessly merged. If I had synched my devices so they both had the task called foo above, and on one I added some extra statements about foo, they'd be seamlessly merged in. It gets more fun when there's collisions - if I change an existing statement, for example. If my knowledge base is sophisticated and puts unique IDs and timestamps on each statement then it can spot that it's a change and handle that in the merge; if not, then I might end up with both the old and new statements, and the application has to decide how to resolve that.

But all of this doesn't solve the larger problem: if I have several different apps, each of which add more behaviour to tasks, I still need to find the same task in both of them to see all a task holds.

So let's get rid of the apps. Can we make a general "knowledge browser" that's good enough to just edit the knowledge base directly?

This task can be helped a lot by having statements about relationships. I might install a package of pre-written knowledge that states that the like relationship normally relates a person to another person, or concept. It might also state that (like X Y) can be written in English as "X likes Y" (and another rule can state that an object X can be written in English as the title of X, if X has one). All of this can just be more knowledge in the knowledge base. A universal knowledge editor could use these statements to build a user interface, in a very similar vein to a browser using CSS to display some arbitrary XML document.

And yet, I'd still be able to jot down my own made-up relationships. (is-on-my-christmas-list bob). There might be no vocabulary statements defining what this Christmas list thing is about, but a universal editor might well list "is-on-my-christmas-list" when I look up Bob, and would thereafter be able to tab-complete "is-on-my-christmas-list", having seen it already in the knowledge base; it might even notice that existing objects tagged with "is-on-my-christmas-list" are all also tagged "is-a-person" or "is-a-company", and use that to guide editing - but that's a bit more advanced. So I can install pre-written vocabularies that make everything nice, or I can just make my own up as I go along, and maybe write vocabularies for them later.

Knowledge base systems can also be fed rules. Say I have an appointment database, with appointments of the form (is-appointment foo) (on-date foo (date YYYY MM DD)), etc.

I might write a rule of the form "(is-appointment X) and (on-date X (date YYYY MM DD)) and (title X (printf "%s's birthday" Z)) if (birthdate Y (date SOME-YYYY MM DD)) and (title Y Z)". Then anybody (or anything) that has a birthday in the system will cause an appointment to appear called "foo's birthday" on any date that has the same month and day as the birthday.

What if we know the month and day of somebody's birthday, but not their year of birth? We can just write: (birthdate foo (date _ 04 03)) (where _ is a special variable symbol that means 'unknown'). Since searching a knowledge base is a matter of pattern matching, that will match a query for (birthdate Y (date SOME-YYYY MM DD)), and not bind a value to SOME-YYYY.

Finally, they can also read from external data sources. I might tell my knowledge base that (exists X) if X is a pathname that refers to a file or directory that exists, that (is-part-of X Y) if X and Y are pathnames that refer to files or directory that exist, X is a directory, and Y is directly within that directory, that (title X Y) holds if X is a pathname and Y is the last part of it (the filename), etc. The rules can work both ways - if the knowledge manager is directly told (is-a-directory X) and it isn't, then the extension rules can tell it how to create it.

Then, suddenly, my home directory becomes knowledge, too. I can add statements saying that a given directory tree pertains to a given project. Then my knowledge browser UI can tell me about the files I've worked on as part of a project. Perhaps I could teach it how to open up applications to edit different file types. Or teach it how to read the files itself and understand their structure and turn it all into more knowledge.

Now extend that to email messages, browser bookmarks, my phone's call history, my IM history, my Twitter account...

On being oddly dressed (by alaric)

Last Friday, I stood up in front of a hundred or so people and gave a five-minute talk on some software I've spent two years of my life writing. However, I wasn't particularly self-conscious about the fact that I was oddly dressed.

For me, clothes are about:

1. Keeping me warm
2. Carrying my stuff

It's not that their appearance doesn't matter to me - I don't want to be wearing shabby or tatty clothes. I don't want to wear garish bright colours. I like my clothes to more or less match, so I tend to choose solid dark colours when I buy myself clothes, as they're easy to look smart in.

But every now and then I get a comment from somebody that I must be a bit weird to go around wearing a podbelt and an assault vest... and when the weather's bad, I got outside in a full-length heavy cloak. Luckily, saying that sort of thing disqualifies people from me being too interested in their opinions, so it doesn't particularly bother me.

I like carrying lots of stuff with me. I'm equipped for every eventuality. When people get things in their eyes, I'm there with a mirror and tweezers. I have the obligatory geek multi-tool, of course. My first-aid kit has brought comfort to many a cut finger. My little lengths of string have jerry-rigged many a repair. I always have a torch, a compass, a pen, a notepad, a monocular, and a laser pointer to hand; so I can navigate, find things in the dark, read small text on a projector from the back of the room (and then point to the thing I'm asking about with the laser). If a button comes off of something, I sit down, take out my sewing kit, and fix it. In my laptop bag is a pouch full of cables and adapters, which has saved the day on many a late-night data-centre emergency. When it's raining so hard that people are cowering in shop doorways, my cloak keeps me dry; at the conference on Friday, when there were no seats left, it folded up tightly and became a low stool so I could sit comfortably. A week or so ago, when I was driving home from London very late one night and became too tired to continue, I pulled into a dark lay-by and slept underneath it, warm and comfortable even when the temperature plummeted before dawn.

I'm not just hoarding gadgets for the sake of it - I do assess the trade-offs of every extra bit of weight to carry around. Weight in the podbelt isn't an issue as it carries very nicely on my hips, I barely notice the weight of it, but space there is at a premium. Weight in the assault vest is more of an issue, since it pulls at my shoulders. I've tried having just a podbelt, but it's not good to wear while sitting down, so I tended to take it off and sling it over the back of the chair, which makes things harder to get at; and I've tried just having an assault vest, but weight was a problem. The current combination means I can keep lightweight things I often need while sitting down (mobile phone, pens, pads, business cards, laser pointer, etc) on me all the time, while weightier things I tend to need more on the move (keys, wallet, tools, first aid gear) in the podbelt. I have optional extra things I add for specific "missions" that I wouldn't want to carry all the time, too - I have a special tool jacket with loops for screwdrivers and the like which I wear if I'm doing DIY in awkward locations, an extra assault vest with more specialist stuff for when I'm busy being a Cub leader, a black lightweight mesh one with large pockets for hiking (the large pockets accept good quantities of food, GPSes, and the like), a water flask that goes on the podbelt, and a spare podbelt pouch that I'm going to assemble a survival kit in: emergency rations, a survival blanket, that sort of thing.

When I've explained this to people who question the amount of stuff I carry, they say "But what are the odds of all these things happening?". But they happen all the time! So I'm happy being prepared for anything... it makes life a lot less stressful. My clothes and their pockets become an extension of my body; we are, after all, all cyborgs.