In our last blog about controversies, we talked about having clear goals. Controversies are problematic when they challenge what you think a project should achieve. That’s why having clear goals is essential. It’s something that your founders or senior management team need to define clearly. Your analytics team can build a metrics system to measure those goals.
Experimentation and A/B testing are great tools when you have a well-understood goal. But, you may disagree on the best way to get to it. Try different approaches and implementations. However all, some, or none of your ideas may move the needle towards that shared goal.
Team members may have disagreements, but A/B test results for each idea will put everyone on the same page. Experimentation leverages an agreement on principles into an agreement on implementation.
Formalize What You Agree On vs. Not
Disagreements on what the core objectives are for an organization are fundamental. If a project and the people involved get pulled in different directions, things will get difficult quite fast. When left unresolved, they are toxic to an organization. Do not dismiss disagreements. Investigate what could be reasonable goals.
Publicly Set Goals
Ask stakeholders what they are hoping to achieve, and what good looks like to them. Highlight the commonalities. Thread different steps of a process as a coherent quilt. Have people see the bigger picture regularly and how they contribute to a necessary step. Have them see how others contribute, but be mindful of contradictions.
If you see people pulling in opposite directions, and you feel stuck, ask about scenarios that split the difference. Ask about counterfactuals: unrealistic, exaggerated examples that serve one objective strictly but not the other at all.
For example, offering a higher level of customer service is something that is often debated. It looks like a cost center, therefore doing less should mean more profit. However, some stakeholders want to maintain strong services. Marketing cares about preserving the brand. Legal might need an eye on the claims made.
Let’s take the extreme form of that argument, a counterfactual. Would you cut customer service entirely, if that meant saving on costs? Probably not. Why not? Because short-term profits are likely not your only goal.
Your goal should include and be mindful of the benefits that customer service is providing. Why do we care about branding? Are we carrying legal risks? Stakeholders likely know this but haven’t had a chance to write it down, express it clearly, and thread the connection with their own work.
Explicit empathy for well-understood challenges, and clearly expressed objectives help define a shared goal. Confirm whether people still disagree on the overall goal, or whether opinions have changed on how to reach it. If they still disagree on what the wider goal looks like, it’s worrisome.
In most cases, disagreements are about how to get there, and which effects are strongest. Does investing the same amount in advertising or customer service have the same effect?
Documented Hypothesis
Expect disagreements on how to reach those goals. They are a normal process of establishing strategy. Thankfully, experiments can resolve them. Formalize a position with a written thesis and hypotheses:
Customers love us, but they use our service occasionally, not enough for us to remain on top of their minds. Having a content-driven effort to catch their attention, like a magazine, could be cheaper than re-acquisition cost.
If that assumption is true, everyone will celebrate having cheaper re-activation costs. But the marketing team, in charge of acquisition and re-acquisition, might fear that part of their budget will be re-allocated. With higher user retention, the lifetime of a user is longer. Their estimated lifetime value is higher. The marketing team can therefore spend more on first activations.
On the other hand, if keeping users engaged with an editorial approach proves unsuccessful then the disagreement resolves. Let’s keep re-activating users as long as the attempt is credible. Senior management should praise the effort, highlight the lessons learned, and encourage more exploration. Both supporters and doubters of the project will come back, agreeing on the overall goal. They will discuss more ideas to retain customers’ attention, knowing those will be judged on their merit.
Failing Convincingly
Some companies promote that pattern of learning by hypothesis by encouraging employees to fail convincingly. The encouragement to fail can seem counter-productive. Of course, we want to win, however, values are what your organization does that others don’t. Failing rarely comes without lessons. Iterations will lead to success.
What matters is that the project team implements a promising idea without fear of being punished for failing. Success comes from a mix of good ideas and bad ones that help to point toward better solutions. Both require creativity, iterations, and welcoming new suggestions.
Encourage stakeholders to share alternatives. Those will turn into hypotheses that will feed your experimentation program. Promote communicating how an alternative strategy could support a shared goal. Encourage setting a simple test to prove or disprove every promising idea.
Praise employees who gracefully acknowledge their thesis was off and drive reusable lessons from it. Sharing those lessons and connecting them explicitly to subsequent decisions will weave controlled, well-understood failures into supporting your success.
Overarching Goal Before Hypothesis
Debates and successive learning only happen if everyone agrees on the overarching goal. Running experiments without that agreement and clarity might lead to inconclusive results. You want falsifiable assumptions, an undisputed sign that a change is effective.
Worse, an experiment leads to significant results, but the evaluation is ambiguous. For example, removing a warning on the purchase page increases conversion and profit, but it also reduces retention.
Does everyone, seeing those results, agree on whether to move forward? If not, you are in a dangerous place. Experiments can’t bring the debate back to an agreement. Instead, you risk an ongoing disagreement. To prevent this, senior executives should attend sessions where teams announce the results of their experiments.
There, the organization publicly agrees on roll-out decisions. This meeting allows executives to re-explain the reasoning behind each decision. Doing it for every ambiguous result might feel repetitive. But, it helps drive the understanding of the organization’s priorities. It also fosters adhesion to company values.
Arbitrage
What happens when a change increases profit but degrades the experience for some customers? A delicate decision between two possible directions must be made—an arbitrage if you will. Let’s examine these two possible scenarios:
Scenario 1
During test review sessions, senior executives can remind everyone that being “customer-first” isn’t just a generic poster that hangs in the lobby. Instead, customer experience is a major priority. Therefore, the decision to limit customer services cannot be rolled out as per recommendations. For the next series of projects, they will investigate what customers desire in an attempt to improve their experience through the features released.
Scenario 2
In another organization, senior executives can use the same opportunity to celebrate profit increases. They can praise the team for their work and grant bonuses, publicly overlooking the impact on satisfaction. Teams will notice and ensure any changes implemented would lower costs. As a reaction, the following cycle of expectations, suggestions, results, approvals, and feature releases would focus primarily on profits—solidifying corporate values around bottom lines.
Presenting arbitrages reinforces the understanding of how metrics influence each other. Can you increase engagement without lowering conversion? Does increasing basket size always lower satisfaction? Reconciling controversies like these are key to understanding what can be achieved across an organization. This process is a touchstone for developing corporate values and more. They help set realistic goals that teams are meant to pursue.
Thank you for reading our three-part blog series: “How to Make Controversy Good for Business, Part 2.”
Part 3 is coming soon to a Split blog near you.
Get Split Certified
Split Arcade includes product explainer videos, clickable product tutorials, manipulatable code examples, and interactive challenges.
Switch It On With Split
Split gives product development teams the confidence to release features that matter faster. It’s the only feature management and experimentation solution that automatically attributes data-driven insight to every feature that’s released—all while enabling astoundingly easy deployment, profound risk reduction, and better visibility across teams. Split offers more than a platform: It offers partnership. By sticking with customers every step of the way, Split illuminates the path toward continuous improvement and timely innovation. Switch on a trial account, schedule a demo, or contact us for further questions.