Canary Deployment: What you need to know

What is Canary Deployment?

Canary deployment is a risk mitigation strategy for software releases. it allows applications developers to limit the damage caused by the release of faulty software updates and roll back such faulty updates quickly and safely without compromising the entire software assets.

Canary deployments allow an incremental release of software components, it allows features to be updated/ rolled out in phases. the software releases can then be monitored and user data collated and analysed to then determine if features should be fully deployed or rolled back. The canary version of software’s with all the required application code and dependencies are mainly used by developers to test out new features and upgrades to see how they handle the production environment.

Why should I use Canary Deployment?

if you are a software developer in continuous integration and continuous delivery/continuous deployment and you would like to ship features to a small percentage of your user and monitor their behaviours in real-time, then canary deployment will work best for you.

You can trunk a portion of your users (Beta testers) through the canaries while others users are allowed to be on the stable version. The beta testers on the canaries’ version of your software/ applications then become your early warning system, you can use them to identify broken features, broken process flow, bugs, and features incompatibility before the applications get to the entire user base. if an unstable version of your product is shipped to all your users, all of them will suffer, however, if only 2-5% received the unstable version, most users will not even know and you can safely roll it back quickly limiting the damage caused.

With canary deployments, software updates can be shipped faster, as buggy updates or releases can be quickly detected and fixed, this gives developers the liberty to deploy as many features as possible as they are built. This is extremely good for a software companies' bottom-line as not only will you build and ship features faster, you will also be able to lower the cost and mitigate impacts of a release failure.

Canary deployments significantly reduce negative feedback on new application features delivered to the production environment. Development teams get real-time usage data quickly, which allows them to refine and integrate the next round of application features faster and more effectively. Shorter development stages like this are one of the hallmarks of continuous integration/continuous delivery processes.

Challenges of Canary Deployment technique

Canary deployments require more administration, you are going to need to use a load balancer to separate users for a canary deployment, this often requires more resources and need for additional administration requirements. In this instance, you will have to create a two-production environment (Primary & Secondary) and the secondary backend that will run alongside the primary production environment.

You will have two codebases, two app servers, potentially two web servers, and networking resources to maintain. When software’s are installed on local machines (computers or mobile devices) canary deployment can be challenging when new releases are to be tested on software’s distributed in user computers or mobile devices. in this scenario, there is less control over when the users will upgrade to the new version, if your canary deployment is packed in that latest release, your new application feature may not get installed on as many user systems as you need to get good test results.

Another disadvantage of canary deployment is that users are still exposed to risk, despite that only a few users are exposed to this risk, the risk of end-user exposed to an untested update could be potentially damaging, developers are advised to measure the risk and if they are significant, robust internal testing should be considered.

How to implement a canary deployment

Measurement metrics & Group Selection

The most important decision when implementing canary deployments is to determine the metrics you would like to measure. The core purpose of the deployment is to determine if the updates of the new features are safe. The metrics of measuring the feedbacks have to be in place and metrics to be measured can include the following

1. error counts
2. CPU usage
3. Memory usage
4. Latency
5. User interactions with features etc

Then you have to select the ideal canary group for your testing, this can comprise of any or multiples of the following user groups; highly engaged Early Adopters who are usually your beta testers, you can also choose to deploy canary version by region, for instance, users in Europe or Australia can be exposed to the canary version via IP Routing or the developer can deploy canary version to internal users for testing before scaling to the wider user base.

RESOURCE REQUIREMENTS FOR CANARY DEPLOYMENTS

Developers can use either of the following resources to deploy canaries. Depending on the resources available, users can be partitioned using the resources to obtain real-world user data needed to test outlined metrics.

Load Balancers:

Load Balancing is the distribution of a set of tasks over different computing units (or related resources), to make the overall process easier to execute and much more efficient. some developers implement canary testing by deploying load balancers to route users to different versions of the applications.

This is done by having two load balancers for the application, Load balancer 01 will receive 95 per cent of the traffic, and load balancer 02 will get the remaining five per cent.

Feature Flags:

Another cost-effective way to deploy canary is through feature flags embedded in the code of the application, they are easy and quick to deploy. this is more like inserting a conditional "IF STATEMENT" in a code from which users take different code paths at runtime depending on a condition or conditions set by the development team. In a canary deployment, that condition is whether the user is in the canary group or not.

Kubernetes:

By managing Kubernetes, Canaries can be deployed such that only a portion of the live users are allowed unto the canaries’ version (New version) whilst the production version is deployed to the majority of the users.

The new version of the application is gradually deployed to the Kubernetes cluster while getting a very small amount of live traffic as the canaries’ versions are deployed. more user data are collated, analysed and if no problem is detected, users are incrementally connected to the updated version. in the end, all traffic is directed to the new updated version which now becomes the main version.

If you enjoyed this guide to canary deployment then why not check out our tools guide on open source SIEM next?