I'm Not Gay

I'm now the linux bod at work, and I've been therefore having to get to grips with Fedora. Fedora uses "yum" as its package manager, but I've never used it before. So I needed to get to grips with it and fast. What better way than to read the associated manual pages, which I did. But I didn't half feel gay typing

man yum

By the way, on a totally unrelated topic my very non-gay stag-do will be on Saturday the 30th of September in Aberystwyth. If I've not emailed you and you want to go then get in touch with me. It's looking most likely that we'll spend the day either playing snooker/pool or bowling, followed by a night of drinking games. I'd like to go paintballing if anyone else is up for it, but I can't do that without suitable numbers.

Pirates Of The Carribbean

Bloody hell what a long film. I've got nothing against films being long when they need to be, but there seems to be an alarming trend these days for films to be longer and longer and longer without rhyme or reason. I might try the blogging equivalent: Spew out reams and reams of unimportant bilge, oh wait- I already do that. But that's not the point.

The Lord Of The Rings films had to be long. There was so much stuff needed to make the film make sense that they couldn't really have been any shorter. In fact the whole thing made more sense to me when I watched the 3 hour versions, loads of important stuff got cut from the theatrical cut due to time limitations.

But The Pirates Of The Carribbean just did not need to be long. Tonnes of stuff in that film was a pointless waste of my, and everybody else's time.

As well as becoming too long, there is also an infuriating tendency to make films in two or three parts now. I've sat through nearly 3 hours of nonsense just to see how the film ends, and it doesn't. It stops at a blatant "we're going to make another ridiculously long film for you to grow old watching" point. Someone said, prior to me going to see the film, that the end "hints at a third film". Those were the exact words they used. Hints? Hints?! That bit of hinting was about as subtle as a brick. The film doesn't so much end as stop. We see all the important characters in their different locations. Then the titles. Where's the conclusion? There isn't one, is the answer. You'll have to watch the third film for that.

Personally I couldn't give a flying fuck about what happens to any of the characters. They all either annoy me or bore me. None of them do anything to make me have any sympathy for them, and the plot was pointless. I doubt I'll be wasting another £6.25 on the next film.

We also saw Superman Returns last week. That was also quite disappointing. Superman? Superdad more like. But that didn't make much sense either as he has miraculously become younger in this latest film. Clearly Superman has a special ability to produce children and look younger as a result. As opposed to humans who usually instantly age by 20-30 years when they have kids.

Before Superman began there was a trailer for Lady in the Water. I nearly had a panic attack in the cinema during the trailer. As I've mentioned briefly on my blog before I have a very strange phobia to do with the grates in the bottoms of swimming pools. They terrify me. I couldn't really tell you why, they just do. Lady in the Water seems to revolve around a woman who appears in a swimming pool, whose home world can be found by pulling up the grate in the pool and swimming down a long dark tunnel. Unfortunately there's also a scary wolf-like thing chasing them and god knows what else.

I felt like the flesh was going to crawl clean off my bones and make a break for the back of the cinema. Liz really wants to see it. I'd probably really want to see it too if I could get past the swimming pool thing. I just know that if I were to watch this film I'd never be able to get in a swimming pool again. :-/

Crazy Patents

I stumbled across this list of useless and crazy patents the other day. There are some real gems in there, including:

Why would someone patent a method of swinging on a swing? Why would the patent office even let them patent that!? Where's the inventive step that is required in order to be patentable? Where's the market value? Surely people have been swinging from side to side for hundreds of years? Why am I asking so many questions?

"Pro-Music" - An Anti-Piracy and Pro-Money Website

This bullshit makes me so angry. If the music industry tried producing a quality product at a sensible price, tried paying the artists fairly, allowed consumers to download in a variety of formats, didn't shove awful copy protection into the product which breaks the standards of the files downloaded, didn't charge ludicrous amounts for CDs and didn't force people to upgrade to a new medium every 5 or 6 years ... then maybe I'd have sympathy for them.

The truth of the matter is that I would love to be able to legally download music. I've been waiting for the music industry to embrace this new technology, but they haven't and other people have. Now they can't stand to lose out, but it was their own inaction that lead to this situation.

I don't want to pay as much for a downloaded, low quality MP3 as I would have paid for the album. I don't want to grapple with the product when I want to listen to it because it's full of copy protection. I don't want to waste time ringing the company I bought the music from to obtain a new licence becaue I want to listen to it on a computer other than the one I downloaded it to. Why the hell should I?

http://www.allofmp3.com is the site that came closest to doing the job properly. They ask for sensible prices, and they provide the music in any format you could want it in. Best of all they don't assault their downloads with copy protection bullshit. Surprise surprise, the music industry is fighting them and they will likely be shut down within a year.

If you haven't already listened to Larry Lessig's talk about Free Culture then I suggest you do so now. (Thanks to Saad for sending me this link a year or two ago). You can also read the book "Free Culture". His talks, book, and everything else he has made public is available for free. You can even make "remixes" (any derivativeve work based on his works) for free under the terms of the Creative Commons license. His thinking outlines a radical change to the media industries, one that would benefit everyone. Everyone, that is, except for the fat cats who currently run the assorted media companies that make up the media industry.

If you fancy downloading some high quality, copy-protection free music of any genre you could possibly think of, and you want to do so for free, then head over to CC-Mixter. Now that's a site that really is pro-music.

2 Month Countdown - yowsers!

Only 2 months to the wedding now. 2 months! Yikes! We'd better get the rats measured up for bridesmaids outfits.

Okay that was possibly a little too weird.

Anyway the reason for this blog was more to talk about a point Liz made on her blog recently:

Read Full Post...

She makes a good point. It's strange, at least in my eyes, how men aren't supposed to be interested in any of the proceedings but women are. I've only seen one wedding magazine aimed at blokes and it had a very laid back attitude to the whole thing, where as by contrast all the others were clearly aimed at women (their style gave them away very quickly) and the only parts that are for blokes are written as "pull out pages" or "cut-out sections" which the over-busy wife-to-be can rip out and pin to his underpants while she's ironing, like we have to be tricked into giving a shit about our weddings.

But hey let's be fair. We no doubt went to all the effort of "wooing" our fiancées in the early days of our relationships. Undoubtedly we will have paid for all the romantic meals. We'd have been the ones who proposed, and who bought that £2,000 ring she now sports. And, most importantly of all, we will be the ones who will be out earning the money so she can go to the shops and buy herself shoes. So it's all perfectly fair really.

Ultimately I shall be a guest at my own wedding. It's kind of a sad fact that, because society sees weddings as being about the excited bride desperately running around to make sure the petunias look absolutely perfect while her cocky, possessive groom couldn't give a fuck about how - or indeed - who anyone is, or how the preperations were made because that's not macho enough, it leads to a lot of people thinking that's how it's SUPPOSED to be.

I've seen it at every wedding I've been to. Aunties marvelling over the bride's dress; the proud father having his picture taken with his daughter; the mother of the bride clucking around sorting out the last minute preperations and ensuring that everyone eats their fill of the buffet. The most attention the groom gets is when the father of the bride does the usual oh-so-comical "you'd better look after my daughter or you'll have me to answer to" routine. Like nobody's heard that one before. I suppose I get to make a speech though, so that's cool. I'll get tanked up on the free beer and call everyone a bunch of freeloading bastards. That'll give 'em a wedding to remember.

Or perhaps not.

I'm certainly glad we did the "pre-marriage counselling" thing with the church. It was a bit American, but it did raise some good discussion points that could, potentially, have led to conflict later on. The average couple spends approximately £17,000 on their first wedding (as most people have more than one these days). I don't have the figures for the average spending per couple on ensuring their marriage will remain strong, but I'm 99.9% certain it would be a shocking statistic if I had it.

Talking of the costs, even the "wedding on a shoe-string" articles expect you to be spending around £9,000 minimum. Ours will be clocking in at just over £4,000 (quick estimate) and that's including the honeymoons (we're having two). We could have probably done it even cheaper if we'd wanted, but it's nice to splash out a bit, and we are snowboarding for the second honeymoon which isn't a cheap pass-time.

I have come to really hate the whole commericalism of weddings. Ironically the way prices get hiked up as soon as you use the magic "w" word just cheapens the whole thing. When all the importance of a wedding is in the one-time goods you buy or hire for the day, is it any wonder that divorce rates are so high?

FIRE!!

And the problem with Rainbow Tables...

Thursday night's viewing of the film La Promesse was interrupted by a crackling, popping sound. Out of curiosity I opened the front door and looked outside, and was alarmed to see 6 foot high flames across the road. Some people from a few houses down were trying to put it out with 500ml measuring jugs (I kid not).

I rushed back inside, filled the washing bowl full of water and ran outside to help put it out. Liz followed me out with a wok full of water (the next largest item she could find). We soon had put it out, but it was a bit of a shock. The cause of all this drama? Some tit had thought it would be a good idea to discard their cigarette end over the fence into the long dry grass on the other side. Genius.

Anyway, today I've been considering the theory of Rainbow Tables some more.

One of the interesting problems that can arise is that chains within the table can collide and marge. Referring back to the example chain I created yesterday:

ABCD --HASH--> 1234

1234 --REDUCE--> M8UK

M8UK --HASH--> 6682

6682 --REDUCE--> YGA9

YGA9 --HASH--> 9102

9102 --REDUCE--> A00C

A00C --HASH--> 0176

0176 --REDUCE--> HHUA

...

KPUH --HASH--> 9091

9091 --REDUCE--> FRIN

FRIN --HASH--> 7682

7682 --REDUCE--> BVCR

Let's say for example we begin creating a new chain with the starting password of "T6YH" and this gives a hash of 3271. When we apply our reduction function it could, perhaps, lead to a password of "KPUH". This collides with the chain above. While the collision itself isn't too much of a problem, the fact that the two chains will now merge IS a problem as we are wasting a lot of space in the table that could be put to better use.

To get around this multiple tables can be used, and a different reduction function can be specified for each table. That way when collisions do occur the chains will not merge.

A related problem is that of false alarms. We have already seen that collisions can occur. What if a collision occurs and causes a match on and end-point of a chain? For example we are trying to crack the hash "3331" and the reduction algorithm produces a password "BVCR". This collides with the end-point of the hypothetical chain we generated yesterday. The crack algorithm is now going to start searching the 10,000 length chain that starts at ABCD and ends with BVCR, because it has matched the end-point but, in reality, the password we are attempting to crack cannot be found in this chain at all. This called a false alarm and can dramatically increase the time needed to search a table.

No mention of a cracking technique is complete with a mention of how to circumvent that form of attack, so here's how to stay safe: Firstly ensure that your password is of at least length 8 nad contains alpha upper, alpha lower, numerical, punctuation and special characters. If you have trouble choosing long passwords then choose two short passwords and break them up with a special character or punctuation. E.g. Choose a sentence such as "when the sun doth shine" and take the first letter of each word: wtsds. Replace suitable characters with numbers: wt5d5. Upper-case a character: wT5d5. Now do the same with another sentence: "and the beast said 'be you angels?'" atb5Bua? and then append the two passwords with, say, a > between them.

wT5d5>atb5Bua?

14 characters with upper, lower, numerical, punctuation and special characters. You can't get much stronger than that. It would cost a LOT of time and money to crack a password of that length. I often use this technique to generate very strong passwords, appending passwords I use regularly with extra punctuation between them.

In terms of designing systems that are not at risk from Rainbow Tables, simply ensure you use a good random salt. Say a user provides a password of "tImg.8p!". A hacker would need to search approximately 96^8 passwords but will, on average, only need to search half of those passwords to crack it. That's 3,606,947,894,919,168 (3 thousand billion) passwords to try.

If you add a 4 character salt to it: "tImg.8p!*S@!t" then that's (96^12)/2 average number of passwords to test. That's 306,354,878,664,883,681,886,208 (306 thousand trillion) passwords to test. That's far beyond what's feasible to hack.

Anyway I'm bored of Rainbow Tables now. Time to start finding something else of interest.

You are not a unique and special snowflake

You may very well be sharing your DNA with someone else

...But first, all about Rainbow Tables

I've been thinking a lot recently about big numbers. I'm not sure why, it's all sort of an introspective thing I suppose. I've always been fascinated by AI (and still hope to get a job involving me in AI at some point in the future) and, of course, part of AI is working out how to deal with massive amounts of data. The other day I started looking at passwords and cryptography. I got wondering "why 8 characters?" After-all, the general advice for a strong password is 8 characters minimum, but why 8? So I worked out an algorithm that could determine the approximate time needed to crack a password based on various parameters. It turns out that a government could probably crack a 7 character password in under a month, where as an 8 character password would take over a year (if they've got a lot of power available).

This led me to start looking at Rainbow Tables. Rainbow Tables are a form of time-memory trade-off hacking. The idea is like this:

A 1 way hash can turn a password into a hashed value from which it is not possible to determine the original starting password. However, if we take every single possible password, feed it through the hash and store the value then we can look the password up from the hash. The problem is that the amount of storage needed for this is vast.

Rainbow Tables can massively reduce the amount of storage needed. This is done by storing the passwords and hashes as chains. To do so, a second function known as a reduction function is generated which can take a given hash and generate a password from it. It's not a reverse of the original hashing function (if we had such a thing then we wouldn't need Rainbow Tables now would we?). This reduction function can then be used to generate the chains. This is now probably best explained with an example.

Say we start with the password ABCD. We apply the hashing function to it to get a hash of 1234.

ABCD --HASH--> 1234

Now we feed the hash into the reduction function to generate a new random password.

1234 --REDUCE--> M8UK

This new password is fed back through the hash function, and the resulting hash is fed back through the reduction function.

M8UK --HASH--> 6682

6682 --REDUCE--> YGA9

This is repeated some predefined number of times. From what I've read, the standard is 10,000 times.

YGA9 --HASH--> 9102

9102 --REDUCE--> A00C

A00C --HASH--> 0176

0176 --REDUCE--> HHUA

...[many iterations later]...

KPUH --HASH--> 9091

9091 --REDUCE--> FRIN

FRIN --HASH--> 7682

7682 --REDUCE--> BVCR

So we now have a chain which starts with the password ABCD and ends with the password BVCR. All the passwords and hashes generated by the algorithm to get this new password are discarded, and the start and end are stored in the database. How is this useful? Well we are storing only 1 entry in the database per 10,000 passwords. So we have massively reduced the need for storage. This might seem a little pointless at first sight as at no point are we storing the actual hashes of any of these passwords, but the point is we have the means to recreate those intermediate passwords and hashes.

As an example, let's say that we want to crack the hash 0176. This is fed through the algorithm that generates the chains and each password is tested against the rainbow tables. If a match is found then we can find the hash:

0176 --REDUCE--> HHUA [not in our table]

...[many iterations later]...

KPUH --HASH--> 9091

9091 --REDUCE--> FRIN [not in our table]

FRIN --HASH--> 7682

7682 --REDUCE--> BVCR [found in the table]

Having found BVCR in the table, we look up the start of the chain. Remember we stored ABCD alongside BVCR. Now we begin reconstructing the chain that led to BVCR until we find the password that, when hashed, provides us with the hash value we are trying to crack.

ABCD --HASH--> 1234 [not the hash we want]

1234 --REDUCE--> M8UK

M8UK --HASH--> 6682 [not the hash we want]

6682 --REDUCE--> YGA9

YGA9 --HASH--> 9102 [not the hash we want]

9102 --REDUCE--> A00C

A00C --HASH--> 0176 [matches the hash!]

And thus the password is cracked. To do this for the entire 8 character "strong password" space would take approximately 3 months if distributed over 40,000 dedicated PCs. That's a lot of passwords to hash.

As a part of all of this I also got thinking about something that occurred to me back in secondary school when I was doing Biology. DNA can be configured in any of 3 billion possible ways. That's 3,000,000,000 for those of you who can't imagine that in numerical form. Nine zeros; that's a big number. Obviously not every configuration is a good configuration, but there are certainly a lot nevertheless.

I remember when we were studying DNA, and this fact was mentioned, I thought to myself "3 billion is a very big number, it's no wonder DNA matching is used to catch criminals". And then something else occurred to me. There are approximately 6.5 billion people in the world. 3 billion possible configurations of DNA, 6.5 billion people. Now I'm no mathematical genious, but even I can see that this means there have to be duplicates out there. In fact it's very likely, from a statistics point of view, that there is at least one other person in the world who shares my DNA. There's potential for "the perfect crime" in there somewhere.

And this brings me all the way back to the title of this blog. "You are not a unique and special snowflake." And it's so very true, you're not. There's a very real chance that someone, somewhere, has the same DNA as you.

Almost Impoverished

You know that old joke that's usually seen in cartoons and comics where somebody opens their wallet and moths fly out? A variant of this is where they turn their pockets out to find some change and moths fly out...

Well, last night, a sodding caterpillar crawled out of my wallet. Clearly it's time to start looking for a slightly better-paid job.

Gale Force Grasshopper

One lucky grasshopper got to experience winds today in excess of those experienced in a hurricane. How can I know this? Because it was sitting on the windscreen wiper on my car while I was driving at ~90mph on the motorway, its little antennas flapping wildly in the wind. Strangley it didn't seem at all perturbed by this, and even had a bit of a wash. When the traffic all ground to a halt it waddled across the car onto the bonnet. When it got there, however, the traffic started moving again so it stayed where it was (on a perfectly smooth surface) and braced the wind. When I had to stop again because of the traffic it hopped off in the middle of the motorway where it probably got flattened.

A 90 mph wind is faster than the winds experienced in a hurricane. It's off the Beaufort wind scale, which only goes up to force 12 (which is 73-83 mph winds) so I would estimate that the grasshopper experienced winds of approximately "force 13" or "force 14", if there were such a thing. That's some serious wind to a creature that's only 1.5-2cm long, hanging onto a smooth surface. They have one hell of an impressive grip.

RIP George

This week is turning out to be a bit of a nightmare.

We lost another one of the mice during the night. :-( George suddenly seemed quite slow and unwell last night and was suffering from a slight sniffle; By that point it was too late even for an emergency vet: He didn't make it through the night. We now only have one mouse. He shall be going straight to the vets tonight for a health-check and we shall bury George afterwards.