Niko's Project Corner

Efficient in-memory analytical database

Tra­di­tional databases such as MySQL are not de­signed to per­form well in an­alyt­ical queries, which re­quires ac­cess to pos­si­bly all of the rows on se­lected columns. This re­sults in a full table scan and it can­not ben­efit from any in­dexes. Column-ori­ented en­gi­nes try to cir­cum­vent this is­sue, but I went one step deeper and made the stor­age column value ori­ented, sim­ilar to an in­verted in­dex. This re­sults in 2 — 10× speedup from op­ti­mized colum­nar so­lu­tions and 80× the speed of MySQL.

Real-time Heat Map Generator

I wanted a real-time map gen­er­ator to vi­su­al­ize re­gional prop­erty price changes based on cho­sen time in­ter­val. I didn't want to re­sort to pre-gen­er­ated tiles, be­cause this would pre­vent user-cus­tomized out­put and limit con­fig­ura­tion op­tions. To get the best per­for­mance, I im­ple­mented a FastCGI pro­cess in C++ with a REST­ful in­ter­face to gen­er­ate the re­quired tiles in par­al­lel. The re­sult­ing pro­gram can gen­er­ate a cus­tomized 1280 × 720 res­olu­tion JPG in 30 mil­lisec­onds and equiv­alent PNG in 60 mil­lisec­onds.

HTTP API load tester

When de­vel­op­ing REST­ful APIs, it is im­por­tant to know how many re­quests per min­ute the end point is able to serve. Be­cause of my in­ter­est in Ng­inx, FastCGI and multi-threaded C+++, I de­cided to de­velop my own in-broser HTTP load tester which sup­ports easy con­fig­ura­tion, any num­ber of par­al­lel load-gen­er­at­ing worker threads and real-time graph­ing based on jQuery pow­ered High­Charts li­brary.