Niko's Project Corner

Service discovery with Docker, Consul and Registrator

Tra­di­tion­ally com­put­ers were named and not eas­ily re­placed in the event it broke down. Server soft­ware was lis­ten­ing on a hard-coded port, and to link pieces to­gether these ma­chine names and ser­vice ports were hard-coded into other soft­ware's con­fig­ura­tion files. Now in the era of cloud com­put­ing and ser­vice ori­ented ar­chi­tec­ture this is no longer an ad­equate so­lu­tion, thus elas­tic scal­ing and ser­vice dis­cov­ery are be­com­ing the norm. One easy so­lu­tion is to com­bine the pow­ers of Docker, Con­sul and Reg­is­tra­tor.

Nginx docker image for easy file access via HTTP

Of­ten I find my­self hav­ing a SSH con­nec­tion to a re­mote server, and I'd like to re­trieve some files to my own ma­chine. Com­mon meth­ods for this in­clude Win­dows/Samba share, SSHFS and up­load to cloud (which isn't triv­ial to do via plain cURL). Here an easy-to-use al­ter­na­tive is de­scribed: a sin­gle line com­mand to load and run a docker im­age which con­tains a pre-con­fig­ured Ng­inx in­stance. Then files can be ac­cessed via plain HTTP at the user-as­signed port (as­sum­ing fire­wall isn't block­ing it).

Scalable analytics with Docker, Spark and Python

Tra­di­tion­ally data sci­en­tists in­stalled soft­ware pack­ages di­rectly to their ma­chi­nes, wrote code, trained mod­els, saved re­sults to lo­cal files and ap­plied mod­els to new data in batch pro­cess­ing style. New data-driven prod­ucts re­quire rapid de­vel­op­ment of new mod­els, scal­able train­ing and easy in­te­gra­tion to other as­pects of the busi­ness. Here I am propos­ing one (per­haps al­ready well-known) cloud-ready ar­chi­tec­ture to meet these re­quire­ments.