Now, we know App Store Optimisation (ASO) plays a huge part in the success of any mobile app. A good conversion rate on your listing and the effective discoverability of your app are key for successful acquisition. Failure to execute a well-thought ASO strategy can lead your app to plummet to the bottom of the app store listings and into the sunken place – and that’s the last thing app marketers want! That’s where Google Play Store Optimization experiments come in – these are a key tool in ensuring that you understand your audience, and which elements of your listing are working effectively to convert them to install your app.
Obtaining high conversion rates is one of the main factors to support your mobile app growth, especially if it’s from a first-time user install. This makes A/B testing a vital part of your ASO strategy as not only does it influence growth in general, but more importantly, it can supercharge your organic growth. This allows your app to climb up the app store ranking and alleviate the dependency on paid user acquisition.
For a long time now, app marketers have heavily relied on Google’s Play Store Experiments to run tests on their listings. Luckily to all ASO experts, in December 2021 Apple released a highly anticipated Custom Product Pages and Product Page Optimisation, allowing app developers to create additional versions of their app store product page and testing up to three alternate product page versions against the original.
But more on that later, let’s focus on Google first.
What are Google Play Store Experiments?
So glad you asked!
Google Play Store Experiments are an Android A/B testing function tool provided by Google on the Google Play Developer console. The tool allows developers to run multiple variants of a store listing to different users to determine which listing is received best. You’re able to test multiple factors of the listing such as the copy, images, the app icon, videos, etc. The tool also allows for these elements to be tested and monitored in the Google Play Store console to compare the controlled variant (that is the original listing) with the tests of different creatives and messaging to see which versions produce higher conversions.
Here is an example of various screenshots being tested:
Where Google Play Store Experiments fall short
Unfortunately, Google Play store experiments still have some flaws. They don’t feature an on-page behavioural analytics section to see video watch rates and screenshot gallery rates, which would be helpful. So yes, while Play Store experiments make it easier to see which variation is successful, they don’t explain why. Why did the copy from variation B perform better than the control and A? What aspect of the creative caused the installs to increase? This makes it hard to make data-driven decisions on which variation is the winner to support your CVR, so it’s important to set out some goals in advance. Luckily there are third-party A/B testing platforms that can be used to provide a better insight into your data.
How do you master and optimise your Google Play Store experiment strategy?
We summarised the top tips you should keep in mind to make your Google Play Store Experiments a success
1. Set an objective for your app store experiments
Like every marketing campaign, your experiment should start with setting a clear objective. Before you start A/B testing you should know what it is you would like to test and what your end goal is.
A/B testing usually begins with a hypothesis. An example would be “We will scale our conversion rate if we change our screenshots from horizontal to vertical”. By focusing on one element at a time, you narrow your scope and avoid overwhelming yourself with a macro look at the whole app. This also allows for more careful testing.
2. Exposure of your test
To have a successful experiment you need traffic for your results to be statistically significant. Timing is key here, it may not be the smartest idea to launch experiments on day 1 of your mobile app launch, especially when you are yet to gather benchmarks for your conversion rate. You’ll need to assess if you have the right levels of traffic for a sample size that will lead to conclusive and relevant results that you’ll have confidence in. In other words, do you have enough impressions to reach statistical reliability? It’s always useful to start off by using a sample size calculator to ensure that this is the case.
Once that has been qualified, exposure remains key. While the test audience is randomised, it’s also important that you give your experiment the chance to reach statistical significance via reach. Splitting a single test audience 50:50 makes a lot of sense, as it widens your experiment pool and increases exposure to the test variant – accelerating your journey to (hopefully!) conclusive findings.
3. Decide on the features to test
It is important to know what features you want to test. Every app is different, so what features you should choose to test vary. Below you can see the examples of the features you can experiment with within the Google Play store console:
It might make sense to start with creative experiments – but don’t forget to test your metadata too as this could have a significant impact on your conversion.
The results you receive from each variant help you to determine what creatives and messaging are more effective. This allows for better optimization of the app store. Carrying out successful experiments also helps build your ASO strategy that can give your app an advantage over another app in the same category.
4. Monitor your Google Play Store experiment results
Now that you have run your experiment and found the winning variant, you still need to monitor how it affects your installs. You might have an immediate good result; however, it could be short-lived, and the original listing may have performed better overall.
It is also important to consistently be testing your app listing to push your conversion as much as possible.
App Store Experiments on iOS15
Now, let’s move on to the Apple Store and the moment all ASO specialists have been waiting for. In July last year, Apple promised app developers two new iOS15 features: Custom Product Pages and Product Page Optimisation; both delivering significant improvements for ASO. Let’s take a closer look at how they can help mobile growth practitioners.
Custom Product Pages
Custom Product Pages (CPP) are the new feature for App Store product pages that allows you to create additional versions of your App Store listing to showcase different features or content within your app and share them with different audiences through unique URLs.
You may now customize screenshots, promotional text, and app previews for any of your page’s localisations. This means that users landing on your Custom Product Page (via a tap on the unique URL or ad) will view the custom variant of your product page with the custom content you created. This feature allows for up to 35 CPPs for each of your live apps.
Once the custom product page is complete, you can share its unique URL through social media advertising and user acquisition campaigns. This is especially useful if you’re planning to showcase seasonal or culturally relevant content.
Measuring the results
Another great addition that comes with the feature is that you can measure the performance of your custom product pages in App Analytics in App Store Connect. By clicking on the Acquisition tab, you can see the impressions, downloads, redownloads, and conversion rates, all of which provide valuable insight into the page’s success in driving more installs. But that’s not all you can do. You can also compare the performance of your custom pages against your default product page to see which one provides better engagement and higher in-app spend.
Apple Search Ads support
The feature comes with additional improvement, which will be released early this year. Apple Search Ads will support Custom Product Pages so you can use them to create tailored ad variants. This is an extremely useful feature in targeting different audiences that allows you to showcase the most relevant messaging and creatives for your target demographic.
Product Page Optimisation on iOS
Comparably to Google Play Store experiments, Product Page Optimisation (PPO) allows you to test different elements of your app store product page, such as icons, screenshots and app preview videos. It’s a great tool to understand what resonates with your audience, giving you the opportunity to test different hypotheses in the native Apple Store environment and to boost the conversion rate of your app.
You are able to create up to 3 alternative versions (‘treatments’) of your listing against the original, which are then shown to a percentage of selected at random iOS15 and later users. How you decide to split the audience (80/20 or 50/50) is totally up to you. Apple gives you the option to choose what percentage of traffic should be allocated to each treatment.
As with Custom Product pages, all results will appear in App Analytics in App Store Connect, enabling you to compare the number of impressions, installs and conversion rate for each of your listings in the App Store.
Summary
To summarise, Google Play Store Experiments should be included in your ASO strategy. They must be utilised to maximise your chance of converting potential users into installs. Ensure that your objective is clear, and you have your eyes on the prize. Remember to make your variants as different as possible while keeping in line with your branding so that you identify what users are looking for. Finally, once you have made a number of impressions, it’s time to start running your experiment!
With the release of new iOS 15 features – Custom Product Pages and Product Store Optimisation – mobile growth practitioners can now test different versions of their listings as well as create customised product pages and share their unique URLs in User Acquisition Campaigns. Both features are changing the game for experimentation on the App Store. You’ll have a greater chance of improving your CVR and gathering more data about your audience, helping you to showcase more relevant content for your key demographic.
Over the next few weeks, we will dive deeper into new iOS15 features, sharing our tips and best practices to consider for your ASO strategy in 2022. Stay tuned!
If you want to find out more about how improving the mobile customer experience can impact your app marketing campaign or you have any app marketing questions feel free to reach out to the Yodel Mobile Growth Team. Additionally, you can sign up to our newsletter for exclusive industry updates and app marketing insights. Want to find out more about optimising your app? Make sure to subscribe to our Mastering Mobile Marketing video series. Follow us on LinkedIn, chat with us on Twitter @yodelmobile, and join our #AppMarketingUK LinkedIn group.
