MapReduce explained in 41 words

Goal: count the number of books in the library.

Map: You count up shelf #1, I count up shelf #2.

(The more people we get, the faster this part goes. )

Reduce: We all get together and add up our individual counts.

more ...





Hadoop's MapWritable sometimes a performance hog

I've been using Hadoop a lot lately for a stealth mode project I'm working on. One of the big lessons I'm learning is that where medium to big data is concerned, data formats matter a lot. Where small filesizes are concerned, there is little harm in slinging ...

more ...

Structural Shift in the Economy

It's becoming a more or less accepted economic theory that a large part component of our current recession is structural. This means that during the housing bubble we had lots of people in housing related jobs (construction worker, realtor ), but once the bubble burst those jobs ceased to exist ...

more ...

Hadoop error - HTTP Response Code 503

I recently had a power failure, which resulting in my hadoop cluster shutting down. No matter, hadoop came back after a little while.

However, I ran into problems immediately after restarting it:

1
2
3
4
5
6
7
8
$ hadoop fsck /
Exception in thread "main" java.io.IOException: \
                    Server ...
more ...