AstraPay

Testing What Makes Users Start KYC

I worked with UX Writing to test three KYC upgrade framings in a randomized new-build experiment. Monetary Highlight C beat Feature Highlight A at KYC Start (+1.22%, Holm-adjusted p≈0.016), but differences in final CVR between A, B, and C were not significant. We recommended C after reviewing the submission-volume objective and the limits of the evidence.

Role: Product DesignerKYCExperimentationUX WritingProduct AnalyticsConversion
SUPPORTING CASE STUDY · PRODUCTION EXPERIMENT

Three KYC messages. One decision under uncertainty.

With UX Writing, I tested whether a different reason to upgrade could get more people to begin KYC—while checking what happened all the way to submission.

Three distinct value propositions, not three cosmetic rewrites.

What the test actually supported
+1.22%C vs A at KYC StartHolm-adjusted p ≈ 0.016; significant
37.36%Highest final CVR: BDescriptive lead, not a proven winner
Not significantFinal CVR across A, B and CAll pairwise p ≥ 0.40
THE PRODUCT DECISION

Recommend C for stronger entry response without a detected final-conversion disadvantage.

B was slightly ahead on final CVR, but that difference was not statistically significant. The team prioritized submission volume, and the recommendation for C went through senior review and CMO confirmation.

C
Monetary highlight

A Rp100k voucher as a concrete reason to upgrade

Original experiment artifacts

Inspect the actual variant-comparison board and the production results. The selector above summarizes each framing; it does not replace or alter the original tested creative.

A, B and C were assigned to users on the new build by user ID modulo three. The old-build legacy cohort was concurrent but not part of that randomization. A separate comparison of the new build with legacy also included the Flexible KYC/Autosave release. It cannot establish that copy alone caused the observed rollout uplift.

Related projectFlexible KYC and Autosave →
The problem

The experiment ran on the new build—not on the legacy cohort

KYC completion was sitting around 30–33%. The homepage was one of the main entry points and its KYC message had been unchanged for a long time.

We prepared a new build for a production experiment without forcing the update. Users on the new build entered one of three variants, A, B, or C. Users who had not updated continued to see the legacy message on the old build as a concurrent control cohort.

A, B, and C were assigned in the backend using user ID modulo three. The legacy cohort was not part of the same randomization, so I separated evidence from the randomized arms from comparisons against the old build.

The question was simple. Which framing would make more users enter KYC and keep moving through submission?
Decision

Why B's descriptive lead did not settle the decision

Our KPI prioritized submission volume. Final CVR across A, B, and C was not significantly different, so B's small descriptive lead was not strong enough to decide the rollout on its own.

Variant B

Highest final CVR descriptively

B recorded a 37.36% final CVR, slightly above A and C.

B's final lead was not statistically significant, so I did not treat it as a proven winner.
Variant C, shipped

Strongest entry response with no detected downstream downside

C significantly outperformed A at KYC Start and stayed close to B. Downstream conversion did not show a significant decline after users entered.

With submission volume as the KPI, C gave stronger evidence for increasing entry without evidence that traffic quality deteriorated.
Product decision

A decision under uncertainty, reviewed with leadership

The result did not produce one clean winner. I took the decision case to the heads of Design, Data, and SysOps by separating what was statistically supported, what was only numerically highest, and how each signal related to the submission KPI.

The final recommendation was Monetary Highlight. It received final confirmation from the CMO and C replaced the legacy message as the main production direction.

B remained a useful fallback because it did not depend on a voucher and had the highest final CVR descriptively. That gave the team an option if the promotion ended or C performance declined.

The rollout followed the KPI and the strength of the evidence. B had the highest final number. C had the stronger entry evidence.
Combined KYC impact

The broader rollout result had more than one change

Variant C's new-build cohort and legacy old-build users were not randomly assigned across builds. Flexible KYC/Autosave also shipped on the new build, so the broader uplift must not be presented as a causal effect of the homepage message.

DERIVED · MoEngage funnel, unique users · 2 to 30 October 2025
New buildLegacy old build
01938.157.176.2Start after landingSubmit after landingOverall submitFunnel checkpointConversion %
TAKEAWAY

The new build stayed above legacy across every checkpoint. This uplift is read as the combined impact of the new KYC wording and Flexible Autosave.

View data
Funnel checkpointNew buildLegacy old build
Start after landing70.53%64.01%
Submit after landing40.28%34.12%
Overall submit37.53%34.12%

Legacy and new build are compared across the same funnel checkpoints. The new build also carried Flexible Autosave, so these differences are not used as isolated wording impact.