Firebase A/B Testing

Firebase A/B Testing helps you optimize your app experience by streamlining the way you run, analyze, and scale product and marketing experiments. It gives you the power to test changes to your app's UI, features, or engagement campaigns to see how they impact your key metrics (like revenue and retention) before you roll them out widely.

A/B Testing works with FCM so you can test different marketing messages, and with Remote Config so you can test changes within your app.


Ready to get started? Choose a product:

Create Remote Config experiments Create messaging experiments

Key capabilities

Test and improve your product experience Create experiments with Remote Config to make changes to the behavior and appearance of your app across the variants in your experiment, and test which product experience is most effective at driving the results you most care about.
Find ways to re-engage your users by using the Notifications composer Use A/B Testing to help you find the most effective wording and messaging settings for bringing users into your app.
Safely roll out new features Don't roll a new feature out without making sure it meets your goals with a smaller subset of users first. Once you have confidence in your A/B Testing results, roll the feature out to all your users.
Target user groups Run targeted A/B tests using data about your app users. For example, you could target a subset of users running a specific app version, platform, language, or select users that match a Google Analytics user property value.

How does it work?

When you create an experiment, create multiple variants of a user experience and measure how well the variants perform toward a goal that you want to achieve (such as boosting in-app purchases). Your targeted user group can be defined by multiple criteria chained with "AND" logic; for example, you could limit the group to users of a particular app version who belong to both an Analytics audience such as "crashing users" that match a custom Google Analytics user property set by the client.

AB Testing experiments test Remote Config and messaging actions using Google Analytics to target users and measure outcomes.

With Remote Config, you can experiment with changes to one or more parameters to alter the behavior and appearance of your app. You could use this for subtle changes like tinkering with the best color scheme and positioning of menu options, or for more significant changes like testing a completely new feature or UI design. With the Notifications composer, you can experiment to find the right wording for a notification message.

Whether your experiment uses Remote Config or the Notifications composer, you can monitor your experiment until you identify a leader, the variant that best accomplishes your goal. You can start your experiment with a small percentage of your user base, and increase that percentage over time. If your first experiment does not reveal a variant that accomplishes your goal better than the baseline, you can start a new round of experimentation to find the best way to improve your app.

You can also track other metrics (app crashes, retention, and revenue) along with your goal so that you can have a better understanding of the outcome of your experiment and how it impacts the experience of using your app.

Implementation path

Add Remote Config or Firebase Cloud Messaging to your app If your app already uses Remote Config or Cloud Messaging (or both), you can skip to the next step.
Define the variants that you want to evaluate with an A/B test. Whether your change is subtle or the addition of a new UI or feature, if you can control that change using Remote Config, you can test multiple variants on that change with A/B Testing.

You can also use A/B Testing with the Notifications composer to test multiple variants on your re-engagement campaign before you roll it out to all users.
Define how you will measure success With an experiment that uses the Notifications composer, you can use an Analytics event to define the goal of your experiment and compare experiment variants. With a Remote Config experiment, you can use either an Analytics event or a conversion funnel to define the goal of your experiment.
Monitor your experiment to find the winning variant You can start your experiment with just a few users, and then roll it out to more users if early results look good. As you monitor your experiment, you will see whether some variants cause more app crashes or other impacts on the app experience, and you can also see which variant makes the most progress toward your goal.

Next steps