I’ve been lately working on trying to generate heatmaps from an assortment of data. To say it has been tough is an understatement – I have tried Python, PHP, and Ruby libs/classes, and all have been too slow, too weak, and/or plain ugly (high peak is red, not blue nor fire-colored).

In the case of local, it is a combination of:

• Generating data in the format you need (x,y) coordinates with a related ‘value’ (the higher the value, the more obvious in the heatmap)
• Utilizing the data and being able to quickly and efficiently plot it onto an image
• The ability to create multiple levels and multiple images. For example, if using Google Maps, there are at least 5 levels that are important enough. Each tile image is 256×256, and each zoom increases the # of tiles by a factor of 4 (when you zoom in, what was one tile becomes 4). This is arguably the hardest part – eg if you only create by 256×256 grids, you miss location information just out of the grid. There might be a heavy heat-spot you might miss. And generating one big image is out of the question as you would exhaust memory. Using rough math, and exactly full zoom out, we have 12 256×256 grids. Another level of zoom, you end up with 48 images. Another zoom you end up with 192 images. Another zoom nets you 768, and one more gets you at 3072 images. One image that is 3072 tiles of 256×256 is 201,000,000 pixels. Even 768 tiles ends up being 50 million pixels. One image = no go.
• Conversion of (latitude,longitude) point into X,Y coordinates. In itself not difficult, but when you are trying to keep track of grids, and minimize CPU time spent, it becomes much more difficult.

Anyway – more than anything, this post was just on a challenge I’ve been working on. I haven’t solved it, but what I love is how much it brings local to life, and makes for another way of looking at data. For example, fast food joints in California:

Ideally I can get this damn fast enough that I can generate this in real-time and release as a lib. If not – oh well, a very interesting learning experience.