Niko's Project Corner

Server monitoring and analytics

Description Server data and S.M.A.R.T. logging and reporting system
Languages PHP
Tags Elas­tic­search
Duration Spring 2014
Modified 26th April 2014

There al­ready ex­ists many server mon­itor­ing and log­ging sys­tems, but I was in­ter­ested to de­velop and de­ploy my own. It was also a good chance to learn about Elas­tic­Search's ag­gre­ga­tion queries (new in v1.0.0). Orig­inally Elas­tic­Search was de­signed to provide scal­able doc­ument based stor­age and ef­fi­cient search, but now it is gain­ing more ca­pa­bil­ities. The pro­ject con­sists of a cron job which pushes new met­rics to Elas­tic­Search, a REST­ful JSON API to query statis­tics on recorded num­bers and plot the re­sults in a browser (based on High­Charts).

The cron job is sched­uled to pro­duce a new dump of com­puter's sen­sor data, disk S.M.A.R.T. data, tem­per­atures, disk us­age and so forth into a text file on the disk. Then it is parsed by PHP, con­verted into a "doc­ument" and stored into Elas­tic­Search. The Elas­tic­Search's "schema" is de­signed to sup­port log items from mul­ti­ple ma­chi­nes, mul­ti­ple disks / ma­chine and so forth.

Figure 1: Ag­gre­gated hard disk data (tem­per­ature, start-stop count and power on hours) at 5 min, 6 hours and 7 days in­ter­vals. For each data the min, av­er­age and max are cal­cu­lated.

Ex­am­ple graphs are shown in fig­ures 1 and 2. These are gen­er­ated by a static HTML5/JS file, and the UI gen­er­ation is fully driven by API's re­sponses. The ex­posed API con­sists of REST­ful URLs such as /api/smart/6h to get S.M.A.R.T. ag­gre­gated at 6 hour in­ter­vals.

Figure 2: This graph vi­su­al­izes that how big frac­tion of the time disk has been spin­ning (blue bar) or idle (green bar). Not hav­ing the disk al­ways spin­ning can save en­ergy and in­crease disk's life span, but the num­ber of spin-downs and spin-ups should also be min­imized.

HTTP re­quests are han­dled by a PHP script which con­structs the cor­re­spond­ing Elas­tic­Search ag­gre­ga­tion query, trans­forms the re­sponse into the de­sired struc­ture struc­ture and gen­er­ates the JSON re­sponse. Elas­tic­Search is able to ag­gre­gate 500k doc­uments in 150 mil­lisec­onds, thanks to good caching and an ef­fi­cient im­ple­men­ta­tion. It will be in­ter­est­ing to use it in fu­ture pro­jects as well. This li­brary is avail­able at­ticaggre­ga­tor-php.

Related blog posts: