Glossary

Progressive Delivery

Progressive delivery is the logical next step for teams who have already implemented agile development, scrums, a CI/CD pipeline, and DevOps. It includes many modern software development processes, including canary deployments, A/B testing, and observability. It is essentially a modified version of continuous delivery – in fact, before the term…

A/A Testing

A/B testing is the process of split testing two different variations of a web page or feature by serving different versions of the feature to specific percentages of users, gathering data over time until the sample size is large enough, then finding whether there are significant results for a key…

Canary Deployment

A canary deployment, or canary release, is a deployment pattern that allows you to roll out new code/features to a subset of users as an initial test. Implement Canary Releases The initial steps for implementing canary deployment are: create two clones of the production environment, have a load balancer that…

Blue/Green Deployment

Blue/green deployment is a continuous deployment process that reduces downtime and risk by having two identical production environments, called blue and green. (The names blue and green aren’t special or important – this process is also called red/black deployment or A/B deployment but for the purpose of this post, we’ll…

A/B/n Testing

A/B/n testing is the process of A/B testing with more than two different versions. The little “n” doesn’t refer to a third test, but to any number of additional tests: A/B/n encompasses A/B/C, A/B/C/D, or any other type of extended A/B test. Despite these additional variations, though, A/B/n testing works…

Multivariate Testing

Multivariate Testing is a method of experimenting with different variations of particular elements in a feature implementation, such as the headline, images, copy, etc in a landing page or application launch screen, or other critical moments of truth in a customer journey, in order to determine which variations of said…

Multi-Armed Bandit

In general, a multi-armed bandit problem is any problem where a limited set of resources need to be allocated between multiple options, where the benefits of each option are not known or are incompletely known at the time of allocation, but can be discovered as time passes and resources are…

Feature Rollout Plan

A feature rollout plan is a process that allows the introduction of a set of new features to a group of your user base. A good rollout plan gives developers control of the releases in the development cycle of a limited set of features. This way is possible to ship…

Mobile A/B Testing

A/B testing in general is the process of testing two variations of a resource by showing different versions to different users, then comparing the test results (aka, the differences in user behavior between the two groups) for statistical significance. The process is essentially the experimental method as applied to software…

Continuous Delivery Tools

Continuous delivery is the process of systematically keeping code deploy-ready at all times. This shift in release process requires several changes in the operations of the development team, from improving automated testing to implementing Agile and DevOps processes. But above and beyond these changes, having the right tools to facilitate…