Hi there! 👋 In this post, I’ll share my learnings from benchmarking an influence of KSP migration of an Android project. I will focus specifically on an error I initially made, as I hope it might be a useful insight for the community.

Glossary

• ABI change – a modification that breaks the binary interface of a class, such as adding a public method
• annotation processing – the process of interpreting annotations (e.g., @Module). This typically involves generating new code, as most annotation processors serve this purpose
• KSP (new), KAPT (old) – annotation processing engines
• Gradle Profiler – a tool for running builds multiple times, with the ability to apply modifiers to each run

Context

In one of my recent Android projects, I evaluated the impact of migrating from KAPT to KSP. Initial benchmarks suggested KSP significantly increased build times, but further analysis revealed an issue in the measurement process.

Error in Benchmarks

The only scenario that indicated a performance regression was the incremental build scenario, so I’ll focus on that.

The benchmark consisted of 10 measured builds, each with an ABI change applied to a database model. Since Day One Android uses Room, which relies on annotation processing, this change was expected to trigger processing.

However, I realized that when using KSP, the Kotlin ABI change triggered annotation processing – but with KAPT, it did not. This discrepancy meant the benchmark was flawed, as it was comparing two different behaviors – effectively an apples-to-oranges comparison.

Why this happened?

Two factors combined to invalidate the benchmark:

• KAPT was not triggered by the ABI change in the Kotlin file because it generates Java stubs, and those stubs were not part of the annotation processing.
• KSP was triggered because it works directly on Kotlin files and is more sensitive to actual ABI changes.

Let’s look how the generated code differs depending on the annotation processor used. The change we’ll be adding in both scenarios is:

As you can see, KAPT and KSP handle the incremental build very differently under the hood:

• KAPT – a Java stub is generated. This moves the public method, that was supposed to break ABI of the JournalEntryInfo, to a separate file. This results in the annotation processor ignoring any change to JournalEntryInfoKt as it’s a different file.
• KSP – it works directly on Kotlin files, without generating Java stubs. symbols file which contains inputs used for incremental compilation purposes¹, does contain the public method added for breaking ABI. This makes any changes to _m_99(...) method invalidating the annotation processor result state, causing the processor to run again.

Correcting the benchmark

To run the benchmark correctly – i.e., to force the annotation processor to run when an ABI change is detected – I temporarily converted JournalEntryInfo into a Java class. The Gradle Profiler modifies the body of a Java class rather than the file itself, making KAPT to recognize the change and triggering the annotation processor. This results in the following:

This ensures that both KSP and KAPT are rerun on each ABI-breaking change, allowing for a fair performance comparison between the two processors.

Supporting evidence

As I was looking for a way to introduce a custom modifier to Gradle Profiler, I’ve found the following GitHub issue

TIL

• Gradle Profiler’s ABI change won’t trigger the KAPT annotation processor if it edits a Kotlin file.
• KSP can be more sensitive to breaking ABI changes in Kotlin files because it doesn’t rely on Java stubs.
• Incremental compilation inputs for KSP can be traced through the symbols file.