A

A/A Testing

A/B testing is the process of split testing two different variations of a web page or feature by serving different versions of the feature to …

A/B Testing

A/B testing, otherwise known as split testing, is the process of testing two different versions of a web page or product feature in order to …

A/B/n Testing

A/B/n testing is the process of A/B testing with more than two different versions. The little “n” doesn’t refer to a third test, but to …

B

Blue/Green Deployment

Blue/green deployment is a continuous deployment process that reduces downtime and risk by having two identical production environments, called blue and green. (The names blue …

C

Change Advisory Board

A change advisory board (or CAB) is a collective of representatives from different departments within the company who run that company’s formal change management processes. …

Chaos Engineering

In any complex system, failure is inevitable. Chaos engineering, or chaos testing, embraces this reality, allowing you to introduce failures in a controlled way.

Client Side Testing

Client-side testing refers to any type of testing – commonly A/B testing, but also multivariate testing or multi-armed-bandit testing – that occurs in the user’s …

Configuration Drift

More and more companies are beginning to understand that using a staging environment to test features causes more harm than good. Because this process separates …

Controlled Rollout

A controlled rollout is a feature rollout with highly granular user targeting. It allows you to release new features gradually, ensuring a good user experience …

D

Dark Launch

Dark launching is the term for releasing features to a subset of your users, seeing how they respond, and making updates to your features accordingly. …

Data Pipeline

Data pipelines automate the flow of data from one point to another. In a data pipeline, you start with defining how data is collected, and …

F

Feature Experimentation

Ordinary software development is widespread with guesswork. Some amount of data, plus some amount of the product manager’s hunches, drives the decisions of what new …

Feature Flag Management

Feature flag management is the process of, well, managing feature flags. While feature flags are extremely useful for many purposes, they require an organized process …

Feature Flags

A feature flag, or feature toggle, is a software development tool used to safely activate or deactivate features for testing in production, gradual release, experimentation, and operations. 

Feature Flags Framework

The feature flags framework is a revolutionary method of software development that allows individual features of a software product to be individually enabled or disabled. Feature flags allow features to be centrally managed from outside of the application, meaning they can even be turned on and off after they’ve already been rolled out to end users.

Feature Flags in DevOps

DevOps is a software development philosophy that combines the Agile development approach focusing on rapid delivery with a collaborative effort between the development and operations staff. This collaboration is key to creating a more efficient and adaptive product lifecycle, and formalizes the increased role that operations teams play in the service-focused landscape of today.

Feature Toggles

Feature toggles, also known as feature flags, are components of software development that allow specific features of an application to be activated or deactivated at will. This allows developers to safely “toggle” new features on or off for testing. Feature toggles have many uses beyond just testing, such as the targeted release of new features, experimentation, operations, or activation of premium features.

H

J

Jira Feature Flags

One of the biggest challenges that software development teams encounter when adopting feature flags is feature management, or how best to track new features across …

K

Kill Switch

In software development, a kill switch is a button that disables a feature if needed.  This enables stakeholders to turn off broken features in production …

M

Mobile A/B Testing

A/B testing in general is the process of testing two variations of a resource by showing different versions to different users, then comparing the test …

Multi-Armed Bandit

In general, a multi-armed bandit problem is any problem where a limited set of resources need to be allocated between multiple options, where the benefits …

Multivariate Testing

Multivariate Testing is a method of experimenting with different variations of particular elements in a feature implementation, such as the headline, images, copy, etc in …

O

Observability

Observability is defined as the ability of the internal states of a system to be determined by its external outputs. With the unknown unknowns of …

P

Progressive Delivery

Progressive delivery is the logical next step for teams who have already implemented agile development, scrums, a CI/CD pipeline, and DevOps. It includes many modern …

S

Server Side Testing

Server-side testing refers to any type of testing – commonly A/B testing, but also multivariate testing or multi-armed bandit testing – that occurs on the …

Simpson’s Paradox

Simpson’s Paradox, otherwise known as the Yule-Simpson Effect, is a reversal paradox where the correlation found in each of several groups either disappears or even …

Smoke Testing

Smoke tests are a type of regression test which ensure that your most important, critical functional flows work. These tests should not encompass your entire …

T

Type I Error

A type I error (or type 1 error), also called a false positive, is a type of statistical error where the test wrongly gives a …

Type II Error

A type II error (type 2 error) is one of two types of statistical errors that can result from a hypothesis test (the other being …

U

Usability Testing

A key part of the software development process, usability testing provides invaluable feedback on the user experience of a product. Usability testing means conducting real-world …