Say goodbye to KAPT, use KSP to speed up Android compilation

1. KSP

When developing Android applications, many people complain about the slow compilation speed of Kotlin, and KAPT is one of the culprits that slow down the compilation. We know that many Android libraries use annotations to simplify template code, such as Room, Dagger, Retrofit, etc., and Kotlin uses KAPT to process annotations by default. KAPT does not have a special annotation processor, it needs to be implemented with APT, so it needs to generate APT parseable stub (Java code), which slows down the overall compilation speed of Kotlin.

KSP was born under this background. It is based on Kotlin Compiler Plugin (KCP) and does not need to generate additional stubs. The compilation speed is more than twice that of KAPT. In addition to greatly improving the build speed of Kotlin developers, the tool also provides support for Kotlin/Native and Kotlin/JS.

2. KSP and KCP

Kotlin Compiler Plugin is mentioned here. KCP provides Hook opportunities during the kotlinc process, during which it can parse AST and modify bytecode products. Many of Kotlin's syntactic sugar are implemented by KCP. For example, data class, @Parcelize, kotlin-android-extension, etc. Today's popular Jetpack Compose is also completed with KCP.

In theory, KCP's capabilities are a superset of KAPT, which can completely replace KAPT to improve compilation speed. However, the development cost of KCP is too high, involving the use of Gradle Plugin, Kotlin Plugin, etc. The API involves some understanding of compiler knowledge, which is difficult for general developers to master. A standard KCP architecture is shown below.

Several specific concepts are involved in the figure above:

• Plugin : Gradle plug-in is used to read Gradle configuration and pass it to KCP (Kotlin Plugin);
• Subplugin : Provide KCP with configuration information such as the address of the maven library of the custom Kotlin Plugin;
• CommandLineProcessor : Convert parameters to Kotlin Plugin recognizable parameters;
• ComponentRegistrar : Register Extension in different processes of KCP;
• Extension : realize custom Kotlin Plugin function;

KSP simplifies the entire process of KCP, developers do not need to understand the working principle of the compiler, and the cost of processing annotations has become as low as KAPT.

3. KSP and KAPT

As the name implies, KSP processes Kotlin's AST at the Symbols level, accessing elements of types, class members, functions, and related parameters. It can be compared to Kotlin AST in PSI, the structure is as shown in the figure below.

As you can see, the Kotlin AST obtained after a Kotlin source file is parsed by KSP is as follows.

KSFile
  packageName: KSName
  fileName: String
  annotations: List<KSAnnotation>  (File annotations)
  declarations: List<KSDeclaration>
    KSClassDeclaration // class, interface, object
      simpleName: KSName
      qualifiedName: KSName
      containingFile: String
      typeParameters: KSTypeParameter
      parentDeclaration: KSDeclaration
      classKind: ClassKind
      primaryConstructor: KSFunctionDeclaration
      superTypes: List<KSTypeReference>
      // contains inner classes, member functions, properties, etc.
      declarations: List<KSDeclaration>
    KSFunctionDeclaration // top level function
      simpleName: KSName
      qualifiedName: KSName
      containingFile: String
      typeParameters: KSTypeParameter
      parentDeclaration: KSDeclaration
      functionKind: FunctionKind
      extensionReceiver: KSTypeReference?
      returnType: KSTypeReference
      parameters: List<KSValueParameter>
      // contains local classes, local functions, local variables, etc.
      declarations: List<KSDeclaration>
    KSPropertyDeclaration // global variable
      simpleName: KSName
      qualifiedName: KSName
      containingFile: String
      typeParameters: KSTypeParameter
      parentDeclaration: KSDeclaration
      extensionReceiver: KSTypeReference?
      type: KSTypeReference
      getter: KSPropertyGetter
        returnType: KSTypeReference
      setter: KSPropertySetter
        parameter: KSValueParameter

Similarly, APT/KAPT is an abstraction of Java AST. We can find some correspondences. For example, Java uses Element to describe packages, classes, methods, or variables, and KSP uses Declaration.

Java/APT	Kotlin/KSP	describe
PackageElement	KSFile	A package program element that provides access to information about the package and its members
ExecuteableElement	KSFunctionDeclaration	A method, constructor, or initializer (static or instance) of a class or interface, including annotation type elements
TypeElement	KSClassDeclaration	A class or interface program element. Provides access to information about the type and its members. Note that the enumeration type is a class, and the annotation type is an interface
VariableElement	KSVariableParameter / KSPropertyDeclaration	A field, enum constant, method or constructor parameter, local variable or abnormal parameter

There is also Type information under Declaration, such as function parameters, return value types, etc. TypeMirror is used in APT to carry type information, and the detailed capabilities in KSP are implemented by KSType.

Four, KSP entrance SymbolProcessorProvider

The entrance of KSP is in SymbolProcessorProvider, the code is as follows:

interface SymbolProcessorProvider {
    fun create(environment: SymbolProcessorEnvironment): SymbolProcessor
}

SymbolProcessorEnvironment is mainly used to obtain KSP runtime dependencies and inject them into Processor:

interface SymbolProcessor {
    fun process(resolver: Resolver): List<KSAnnotated> // Let's focus on this
    fun finish() {}
    fun onError() {}
}

The process() method needs to provide a Resolver to parse the symbols on the AST, and the Resolver uses the visitor pattern to traverse the AST. As follows, the Resolver uses FindFunctionsVisitor to find the top-level functions and Class member methods in the current KSFile.

class HelloFunctionFinderProcessor : SymbolProcessor() {
    ...
    val functions = mutableListOf<String>()
    val visitor = FindFunctionsVisitor()

    override fun process(resolver: Resolver) {
        resolver.getAllFiles().map { it.accept(visitor, Unit) }
    }

    inner class FindFunctionsVisitor : KSVisitorVoid() {
        override fun visitClassDeclaration(classDeclaration: KSClassDeclaration, data: Unit) {
            classDeclaration.getDeclaredFunctions().map { it.accept(this, Unit) }
        }

        override fun visitFunctionDeclaration(function: KSFunctionDeclaration, data: Unit) {
            functions.add(function)
        }

        override fun visitFile(file: KSFile, data: Unit) {
            file.declarations.map { it.accept(this, Unit) }
        }
    }
    ...
    
    class Provider : SymbolProcessorProvider {
        override fun create(environment: SymbolProcessorEnvironment): SymbolProcessor = ...
    }
}

Five, get started quickly

5.1 Create a processor

First, create an empty gradle project.

Then, specify the version of the Kotlin plugin in the root project for use in other project modules, for example.

plugins {
    kotlin("jvm") version "1.6.0" apply false
}

buildscript {
    dependencies {
        classpath(kotlin("gradle-plugin", version = "1.6.0"))
    }
}

However, in order to unify the Kotlin version in the project, a unified configuration can be made in the gradle.properties file.

kotlin.code.style=official
kotlinVersion=1.6.0
kspVersion=1.6.0-1.0.2

Next, add a module to host the processor. And add the following steps in the build.gradle.kts file of the module.

plugins {
    kotlin("jvm")
}

repositories {
    mavenCentral()
}

dependencies {
    implementation("com.google.devtools.ksp:symbol-processing-api:1.6.0-1.0.2")
}

Next, we need to implement com.google.devtools.ksp.processing.SymbolProcessor and com.google.devtools.ksp.processing.SymbolProcessorProvider. The implementation of SymbolProcessorProvider is loaded as a service to instantiate the implemented SymbolProcessor. Pay attention to the following points when using:

• Use SymbolProcessorProvider.create() to create a SymbolProcessor. The dependencies required by the processor can be passed through the parameters provided by SymbolProcessorProvider.create().
• The main logic should be executed in the SymbolProcessor.process() method.
• Use resoler.getsymbolswithannotation() to get the content we want to process, provided that the fully qualified name of the annotation is given, such as com.example.annotation.Builder .
• A common use case of KSP is to implement a customized accessor, the interface com.google.devtools.ksp.symbol.KSVisitor , which is used to manipulate symbols.
• usage example of SymbolProcessorProvider and SymbolProcessor interface , please refer to the following files in the sample project: src/main/kotlin/BuilderProcessor.kt and src/main/kotlin/TestProcessor.kt .
• After writing your own processor, include the fully qualified name of the processor provider resources/META-INF/services/com.google.devtools.ksp.processing.SymbolProcessorProvider

5.2 Use processor

5.2.1 Using Kotlin DSL

Create another module that contains the work that the processor needs to handle. Then, add the following code in the build.gradle.kts file.

pluginManagement {
    repositories {
       gradlePluginPortal()
    }
}

In the build.gradle of the new module, we mainly complete the following things:

• Apply the com.google.devtools.ksp plugin with the specified version.
• Add ksp to the list of dependencies

for example:

plugins {
        id("com.google.devtools.ksp") version kspVersion
        kotlin("jvm") version kotlinVersion
    }

Run the ./gradlew command to build, you can find the generated code under build/generated/source/ksp The following is an example of applying the KSP plugin to the workload in build.gradle.kts.

plugins {
    id("com.google.devtools.ksp") version "1.6.0-1.0.2"
    kotlin("jvm") 
}

version = "1.0-SNAPSHOT"

repositories {
    mavenCentral()
}

dependencies {
    implementation(kotlin("stdlib-jdk8"))
    implementation(project(":test-processor"))
    ksp(project(":test-processor"))
}

5.2.2 Using Groovy

Build in your project. The Gradle file adds a plug-in block containing the KSP plug-in

plugins {
  id "com.google.devtools.ksp" version "1.5.31-1.0.0"
}

Then, add the following dependencies in dependencies.

dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
    implementation project(":test-processor")
    ksp project(":test-processor")
}

SymbolProcessorEnvironment provides processors options, which are specified in the gradle build script.

  ksp {
    arg("option1", "value1")
    arg("option2", "value2")
    ...
  }

5.3 Use IDE to generate code

By default, IntelliJ or other IDEs are not aware of the generated code, so references to these generated symbols will be marked as unresolvable. In order for IntelliJ to operate on the generated code, the following configuration needs to be added.

build/generated/ksp/main/kotlin/
build/generated/ksp/main/java/

Of course, it can also be a resource directory.

build/generated/ksp/main/resources/

When using, you also need to configure these generated directories in the KSP processor module.

kotlin {
    sourceSets.main {
        kotlin.srcDir("build/generated/ksp/main/kotlin")
    }
    sourceSets.test {
        kotlin.srcDir("build/generated/ksp/test/kotlin")
    }
}

If IntelliJ IDEA and KSP are used in the Gradle plugin, the above code snippet will give the following warning:

Execution optimizations have been disabled for task ':publishPluginJar' to ensure correctness due to the following reasons:

For this warning, we can add the following code in the module.

plugins {
    // …
    idea
}
// …
idea {
    module {
        // Not using += due to https://github.com/gradle/gradle/issues/8749
        sourceDirs = sourceDirs + file("build/generated/ksp/main/kotlin") // or tasks["kspKotlin"].destination
        testSourceDirs = testSourceDirs + file("build/generated/ksp/test/kotlin")
        generatedSourceDirs = generatedSourceDirs + file("build/generated/ksp/main/kotlin") + file("build/generated/ksp/test/kotlin")
    }
}

At present, many tripartite libraries using APT have added support for KSP, as follows.

Library	Status	Tracking issue for KSP
Room	Experimentally supported
Moshi	Officially supported
RxHttp	Officially supported
Kotshi	Officially supported
Lyricist	Officially supported
Lich SavedState	Officially supported
gRPC Dekorator	Officially supported
Auto Factory	Not yet supported	Link
Dagger	Not yet supported	Link
Hilt	Not yet supported	Link
Glide	Not yet supported	Link
DeeplinkDispatch	Supported via airbnb/DeepLinkDispatch#323