Introducing Hobo

In this post I'd like to introduce Hobo, a fast in-memory index for your data. Hobo is not a database, it's an external index for data you have stored elsewhere. Hobo is used for two primary purposes - the first is to answer queries of the form "find me all items …

more ...

A simple trick to speed up complex Postgres queries on EC2

A major problem with running Postgres on EC2 is that EBS performance often sucks. In addition to performing poorly, EBS also uses the network connection, which can be undesirable. Ephemeral storage is provided, and tends to have better performance characteristics, but unfortunately it lacks durability.

However, we can still use …

more ...

Networking problems while cloning Ubuntu VM

I've run into a curious error while cloning Ubuntu VM hard drives. After cloning, the network card no longer works. However, if I clone the VM completely (including the mac address), there is no problem. The reason for this is that Ubuntu, upon installation, stores the mac address of the …

more ...

The Calories In/Calories Out model explains weight stability

A fairly recent trend in discussions of obesity is to focus on weight stability. Weight stability is a phenomenon by which a human maintains roughly the same bodyweight over a long period of time. Karl Smith recently brought it up, for example:

...its hard to square [theories of obesity based …

more ...

TechCrunch messes up the math on sexism

Eric Ries over at TechCrunch recently wrote an article discussing racism/sexism in Silicon Valley and the technology industry. The article discusses differences in aptitude between men and women, and attempts to downplay them as the cause of a lack of women in technology (and at YC-funded companies, specifically):

Could …

more ...

Why Americans don't hire servants

Recently, Bryan Caplan criticized the ZMP theory of unemployment on the following grounds:

Most economists aren't entrepreneurs. The fact that we can't think of a productive job for a low-skilled worker is weak evidence for ZMP. But if even ivory tower economists can think of productive jobs for low-skilled workers …

more ...

Did immigrants (and Simpson's Paradox) cause the Great Stagnation?

Many people have commented on the fact that, after adjusting for chained CPI, median income has not risen significantly since the 1970's. Tyler Cowen points to this as evidence for his theory of the "Great Stagnation", which purports that the economy has grown more slowly during the latter parts of …

more ...

MapReduce explained in 41 words

Goal: count the number of books in the library.

Map: You count up shelf #1, I count up shelf #2.

(The more people we get, the faster this part goes. )

Reduce: We all get together and add up our individual counts.

more ...