It is easy to call activity an experiment. Change a headline, send a campaign, add a referral prompt, and watch a chart for a few days. If the number rises, call it a win. If it falls, call it a loss. Then move on.

That is not a learning system. It is a sequence of actions with labels attached.

A useful experiment starts with uncertainty that matters. It says what the team believes, who it expects to change behaviour, why that behaviour may change, and what evidence would cause the team to update its view. The result can still be a disappointing metric. It is not a failed experiment if it produces a clear decision.

Write the hypothesis in a way that can be wrong

\u201cImprove activation\u201d is an ambition, not a hypothesis. A better version is: \u201cIf new team owners see one completed example before they configure anything, more of them will invite a colleague in their first session because they can picture the outcome.\u201d

That statement has a proposed intervention, a defined audience, a behavioural mechanism, and a measure. Most importantly, it can be wrong. If the change does not improve invitations, the team can ask whether the example was weak, the audience was wrong, or the proposed mechanism was mistaken.

The UK Government\u2019s agile delivery guidance treats agile work as a discipline that includes user stories, prioritisation, planning, and measuring progress\u2014not simply shipping quickly. Growth work benefits from the same discipline. A test should be a small piece of evidence in a larger decision process.

Choose one primary behaviour

Experiments become unreadable when every available metric is treated as the goal. A new landing page can affect visits, sign-ups, activation, replies, and paid conversion. Those are not all equally useful for the specific question being tested.

Pick one primary behaviour tied directly to the hypothesis. Then name a small set of guardrails: perhaps unsubscribe rate, support volume, completion time, or the number of low-quality sign-ups. Guardrails do not turn an experiment into a giant dashboard. They stop a local improvement from hiding a serious cost.

For example, a referral prompt might increase invites while reducing task completion because it interrupts people too early. The invitation rate alone would make the test look successful. The completion guardrail would tell a more complete story.

Decide what happens before the result arrives

Teams often interpret results after they see them. That is how a weak signal becomes a strong narrative. Before running the test, write down three things:

  1. The minimum evidence needed to keep the change.
  2. The observation that would make you remove or rethink it.
  3. The next question you will ask in either outcome.

You do not need false precision to do this. A tiny early-stage product may not have enough traffic for a statistical conclusion. In that case, be honest about the limitation and combine numbers with direct observations: session recordings with consent, support replies, interviews, or a user\u2019s own explanation of what blocked them.

The key is to avoid declaring certainty you do not have. \u201cFour of six invited a teammate after seeing the example; we need a larger run before generalising\u201d is much more useful than \u201cthe experiment worked.\u201d

Keep the scope small enough to explain

Large changes create large ambiguity. If the team simultaneously changes the audience, message, price, onboarding flow, and sales follow-up, no one knows which element caused the result. A deliberately narrow test makes the learning portable.

That does not mean every experiment must be an A/B test. Sometimes the right move is five conversations with a clearly defined customer segment. Sometimes it is a manual concierge service that reveals whether anyone values the outcome before you build automation. The method should fit the uncertainty, not the other way around.

A simple experiment brief can fit on one page:

  • Question: What uncertainty are we reducing?
  • Audience: Who is included and who is not?
  • Change: What exactly will be different?
  • Primary behaviour: What action would support the hypothesis?
  • Guardrails: What harm or distortion would concern us?
  • Decision rule: What will we do with the evidence?

If the brief cannot be understood by someone outside the team, the experiment is probably too vague to interpret later.

Treat the result as a note, not a verdict

An experiment is one observation in one context. Seasonality, acquisition mix, novelty, and implementation details all matter. The right question after a result is not \u201cDid we win?\u201d but \u201cWhat did we learn that changes the next decision?\u201d

Document the answer while it is fresh. Include the original hypothesis, the actual change, the evidence, the caveats, and the action taken. This prevents teams from repeating the same dead ends and gives new teammates a view into why the product looks the way it does.

The most valuable growth experiments make future work less speculative. They narrow the field, clarify the customer\u2019s reality, and tell the team what to try next. A chart that moves is nice. A decision that gets smarter is the point.