As you may recall from last time, we have been doing some infrastructure upgrades to bring down costs and improve reliabilty. While the change in deployment config had some unintended negative consequences, the changes should be big improvements going forward. This is a summary of those changes.
Mostly, we have been working on Dockerizing everything and moving subservices to Amazon's Elastic Container Servce (ECS). Docker is useful for isolation and reliable builds across platforms. This change was motivated by the need to share the codebase; in an effort to streamline development, we have been talking to an outside expert, and needed a more reliable means to bring the entire (substantial) codebase up. Dockerizing our database, redis, workers, servers, etc took a bit of effort, but now we can bring up everything with
docker-compose up instead of the previous litany of incantations.
These changes also let us leverage ECS to bring down costs. We were using hosted deployments before to cut down on our maintenance & management burden (because our team is small). But with Dockerized everything, we can host our services more easily in Amazon's cloud without having to handhold them.
As we shift back from architecture management to active dev, we are looking to make the replay parsing pipeline faster and extract more data. Look for that soon!
July 12, 2016, 2:29 p.m.