Allow Control Group to Have No Changes in Advanced Shipping Test
Previously, all groups in the Advanced Shipping Test were required to use a rate generated by ABConvert’s carrier rate. This meant that even the control group had to apply a test rate, making it difficult to compare against the store’s default settings.
What’s Changed:
• The control group can now retain the store’s original shipping settings instead of applying a test-generated rate.
• This allows users to accurately compare the effects of adding a new rate vs. keeping existing rates unchanged.
Example of Previous Behavior:
• Before this update, even the control group had to apply an ABConvert test rate.
New Behavior:
• Now, the control group can remain unchanged, and only the test groups will apply the experimental rates.
Why It Matters:
• This provides a true A/B testing experience, making it easier to measure the impact of adding a new rate versus making no changes.
Expand Shipping Test Tracking to Collect All Storewide Orders
Previously, the Shipping Test only tracked orders that used an ABConvert-generated carrier rate. Orders that selected other shipping methods were not included in the test data.
What’s Changed:
• We now collect all orders from the online store, regardless of whether they used a test-generated shipping rate or not.
Example of Previous Behavior:
• If a customer checked out using a non-test rate (e.g., free shipping or a manual shipping rate), their order was not included in the test results.
New Behavior:
• Now, all orders placed during the test period are tracked. This allows merchants to see the full impact of the test on overall order volume and conversion rates.
Why It Matters:
• Users get a more complete picture of order trends during a test period.
• Helps identify if adding a new shipping rate affects storewide order behavior beyond just the customers who selected the test-generated rate.