Google Play Closed Testing: What We Learned Building Kleos Music

A month ago, we started building Kleos Music with a simple idea: a local music player should not feel like a second-class version of a streaming app.

We wanted the music already stored on your phone to feel complete. Lyrics, artwork, metadata, widgets, notifications and search, without subscriptions, unnecessary features or an internet connection just to play your own music.

Then came the less glamorous part: Google Play closed testing.

Before an Android app can reach production, Google Play requires a closed test with at least 12 testers opted in for the required period. On paper, that sounds simple. In practice, finding 12 people is only the beginning.

For Kleos Music, we found most of our testers through tester-for-tester communities, or T4T, where developers test each other's apps, and through Reddit's r/AndroidClosedTesting. It was a practical way to get the first people using the app outside our own devices.

But there is an important difference between having 12 testers and actually testing an app.

A tester can opt in and never open the application again. Another might use it once and disappear. Someone else might find a bug and never report it. That is why relying on exactly the minimum number of testers is risky. People stop participating, forget about the test or simply become inactive. Having some margin above the minimum gives the test room to breathe.

More importantly, real testing changes the way you see your own product.

When you build an application yourself, you already know how it works. You know where the settings are. You know which button opens each screen. You know that a certain gesture exists because you implemented it yourself.

A tester does not have that knowledge.

That difference exposed things we would probably have missed.

Different Android versions behaved differently. Screens had different proportions and densities. Widgets needed adjustments. Media notifications had compatibility issues. Metadata matching behaved differently with unusual music libraries. Even something as simple as explaining how to manually scan a music folder turned out to be less obvious than we thought.

Every discovery became part of the development loop.

Use the app. Find something. Report it. Fix it. Build again. Test again.

During the closed test, Kleos Music went through build after build. Some changes were tiny. Others affected entire parts of the experience. The important part was not the number of builds. It was that every build represented something we had learned from real usage.

It also made us think differently about another Kleos project, Kleos Breathe.

Breathe taught us the other side of closed testing. The problem was not that we had discovered hundreds of bugs. It was almost the opposite: there was not enough engagement. There was little feedback, few issues being reported and therefore very little evidence that the product was connecting with testers.

That experience changed how we think about testing.

A closed test is not just a gate you have to pass before production. It can also be a filter for the product itself.

If people repeatedly do not use an application, do not come back to it and have little to say about it, the answer is not always to find more testers. Sometimes the product itself needs to be reconsidered.

That distinction mattered to us. Kleos Music generated questions because people were actually using it. Breathe generated much less signal because there was much less engagement. Both experiences were useful, but in completely different ways.

Google also asks developers to explain what happened during the closed test when applying for production access. That includes how testers were recruited, how they used the app, what feedback they provided, what changed as a result and why the developer believes the app is ready for production.

For us, those questions were not simply another form to complete. They were almost a summary of the development process.

We could point to real conversations with testers, real problems discovered on real devices and real fixes shipped in response.

That was probably the most valuable thing we got from Google Play closed testing.

It challenged assumptions we did not even know we had.

We assumed certain screens would scale correctly. They did not always.

We assumed people would immediately understand certain interactions. They did not always.

We assumed our metadata system would behave consistently across music libraries. Real-world collections proved more complicated.

And we assumed that if something made perfect sense to us, it would make sense to everyone else.

It does not.

That is why Android app testing is much more than checking whether an application crashes. A good closed test reveals the distance between what the developer intended and what the user actually experiences.

For kleosvic, that distance is where much of the useful work happened.

The goal was never simply to reach the 12-tester requirement and press the production button. The goal was to use the period before launch to make the product better.

We started the process thinking we were testing an app.

We ended it testing our assumptions about the app.

And that turned out to be much more useful.