Boosting Bazel Adoption on Android With Automation

Migrating Android apps from Gradle to Bazel

In recent years, Bazel has become a compelling alternative for large Android codebases as a build tool of choice. However, due to its relative newness and non-default status on Android, the Bazel ecosystem is not as mature as Gradle’s, making the adoption process more challenging and the learning curve steeper.

There are many great resources out there that may persuade you to migrate your codebase to Bazel. In this story, I will share Turo’s experience of Bazel adoption and show you how to configure a robust automation platform once you’ve decided to give Bazel a shot.

What does Bazel migration mean?

The migration process to Bazel essentially entails creating a set of Bazel files for your project and placing them in the appropriate locations, based on the Gradle project structure. Ultimately, the effort unfolds in two key parts.

1. Determining how to build each component of the app with Bazel. This is particularly relevant to custom Gradle tasks or plugins that do not have open-source Bazel alternatives.
2. Automating incremental adoption. For each build feature that necessitates the creation of custom Bazel rules, macros, etc, an automation tool is used to populate it with every related project module.

Why automation is needed?

Automatically generating Bazel build scripts is notably more efficient than manual creation, especially when dealing with codebases that comprise hundreds or even thousands of modules. Yet, the benefits of automation go beyond merely addressing this challenge.

Bazel adoption is not an overnight process. Throughout this transition, the build configuration of your project will continuously evolve, with modules and dependencies being added or removed, etc. Using automation tooling allows you to keep Gradle as the source of truth for as long as necessary without putting undue stress on the engineering team.

Furthermore, it offers flexibility and room for experimentation. Operating two build systems side by side, at least during the migration phase, enables precise evaluation and comparison of their performance. This approach also allows for the gradual integration of Bazel into selected CI workflows, providing the option to transition incrementally while retaining the old build system as a reliable backup for the remaining workflows.

Migration tooling overview

At Turo, Bazel adoption is powered by two open-source tools, Airin and Pendant. Airin, a Gradle plugin, scans the project structure, and Pendant is then utilized to generate Starlark scripts that replicate the same project configuration with Bazel.

You can streamline Bazel adoption for your codebase using Airin as well. To setup the automation tooling the Gradle plugin is applied in the root build.gradle.kts file.

// root build.gradle.kts
plugins {
  id("io.morfly.airin.android") version "x.y.z"
}

Then it is configured using airin extension as shown in the next example.

// root build.gradle.kts
airin {
  targets += setOf(":app", ":auth-feature")

  register<AndroidLibraryModule> {
    include<JetpackComposeFeature>()
    include<HiltFeature>()
    include<ParcelizeFeature>()
    ...
  }
  register<JvmLibraryModule>()

  register<RootModule> {
    include<AndroidToolchainFeature>()
  }
}

If looking at the code snippet above, you’ll notice that the plugin configuration comprises several key components.

• targets — specifying the migration targets. These modules allow initiation of Bazel adoption not only for themselves but also for all their dependencies, including the transitive ones. A migrateToBazel Gradle task is configured for each module declared here.
• register — registering types of project modules to be migrated to Bazel.
• include — including build features these modules might optionally employ.

The primary migration target in Android projects is usually the main app module, triggering the migration for every other module in the application.

./gradlew app:migrateToBazel

However, for incremental Bazel adoption, a partial migration is possible. For instance, one could start by migrating only the authentication feature and all its dependencies.

./gradlew auth-feature:migrateToBazel

Once invoked, the task generates a set of Bazel files for a migration target, all its dependencies and the root module. Consequently, all of them can be built with Bazel.

bazelisk build //app

In many scenarios, achieving a fully automatic migration from Gradle to Bazel may not be possible without additional work. This is particularly true if your project involves custom Gradle build logic, tasks, and plugins, or relies on components lacking open-source Bazel alternatives. In such cases, additional effort becomes inevitable for a successful transition.

Nevertheless, automation tooling remains crucial even in these situations. Every time you determine how to build the missing piece with Bazel, Airin provides an API to seamlessly incorporate the related custom scripts and rule sets into the automation pipeline.

Ultimately, the automation tool is agnostic to the specific contents of generated Bazel files. Its role is to provide a framework for efficient migration, allowing for either the use of predefined file configurations or full customization for more sophisticated use cases.

Understanding the project structure

A key aspect of modern Android codebases is their modular architecture. Despite the potential for hundreds or even thousands of modules in your project, they can all be categorized into a small number of groups or module types. In Android codebases, modules typically fall into, but are not limited to, the following types.

• Android module — a feature or library module that leverages the Android SDK. It relies on com.android.library or com.android.application Gradle plugins.
• Java/Kotlin module — a library module that exclusively depends on JVM APIs. Typically, it employs Gradle plugins such as java or org.jetbrains.kotlin.jvm.
• Root module —a module that serves as the foundation for the project, encompassing project-wide configuration settings.

The categorization into module types suggests that within each type, the build configuration for modules is nearly identical. However, individual modules within a category may still employ different sets of tools and technologies, leading to minor variations in the Bazel configuration.

In other words, each module category is reinforced with a set of build features that may be optionally incorporated into each instance of the module.

• Android module
  ◦ Jetpack Compose feature
  ◦ Parcelize feature
  ◦ Hilt feature

For example, certain Android modules might utilize Jetpack Compose, while others may employ features like Parcelize or Hilt, etc. Although these features are optional, their integration into module configurations does not significantly alter the build script structure.

Automation tooling design

The aforementioned concepts form the foundation of a component-based architecture, representing module types and build features as components. Think of components as lightweight plugins that seamlessly integrate into your Gradle project configuration, offering a framework to replicate it using Bazel.

A typical configuration for an Android project can be illustrated with the following diagram.

Component-based design

To describe it more formally, the configuration of a Gradle plugin for automated migration to Bazel comprises two main component types.

• Module component — defines the Bazel files generated for each module type. It can optionally include a set of feature components.
• Feature component — contributes optional feature-related build configurations to the files generated by a related module component.

Typically, each feature component is associated with a module component. However, shared feature components have an additional role, contributing to every shared module component. We will take a closer look at shared components later.

For now, let’s examine how the migration tool operates. The Gradle to Bazel migration flow is depicted in the diagram below.

Gradle to Bazel migration flow

The migration process essentially involves 3 primary steps, performed for each module individually.

1. Pick a module component capable of migrating this specific module. Performed during the Gradle Configuration phase.
2. Pick feature component(s) for corresponding build features applied to the module. Performed during the Gradle Configuration phase.
3. Generate Bazel file(s) by utilizing the chosen components. Performed during the Gradle Execution phase.

Only one module component can be selected for the module, while multiple feature components are allowed.

Gradle task graph

During configuration, the Gradle plugin registers a series of tasks for all the modules involved in Bazel migration. Each task, at this point, is already aware of the components it should invoke. The dependency graph of these Gradle tasks is illustrated below.

Migration task graph

• migrateToBazel — is registered for migration targets. Invokes migration for all direct and transitive dependencies as well as the root module.
• migrateProjectToBazel — is registered for every module that is a direct or a transitive dependency of a migration target. Invokes migration only for this particular module.
• migrateRootToBazelFor*** — is registered for a root module and complements the migration of a specific migration target where *** is its name.

As these tasks are executed, a corresponding set of components is invoked, leading to the generation of Bazel files.

The Airin Gradle plugin offers a set of ready-to-use components encompassing commonly used module types and build features. However, if additional flexibility and customization are needed, you can create custom components, as demonstrated in the following sections.

Module components

Every module component is an abstract class that extends the ModuleComponent base class and implements 2 functions, canProcess and onInvoke.

abstract class AndroidLibraryModule : ModuleComponent {

  override fun canProcess(project: Project): Boolean {...}

  override fun ModuleContext.onInvoke(module: GradleModule) {...}
}

The former is invoked during the Gradle Configuration phase and is aimed to filter Gradle modules to which this component is applicable.

The latter is invoked during the Gradle Execution phase and contains the main logic of the component the purpose of which is to generate Bazel files for the module.

The easiest way to determine the module type is by examining its applied plugins. For example, an Android library module in Gradle typically relies upon the com.android.library plugin.

abstract class AndroidLibraryModule : ModuleComponent {

  override fun canProcess(project: Project): Boolean =
    project.plugins.hasPlugin("com.android.library")
}

Now, it’s time to generate Bazel files. Here, we utilize a Kotlin DSL that enables the creation of various types of Starlark files in a declarative fashion.

abstract class AndroidLibraryModule : ModuleComponent {

  override fun ModuleContext.onInvoke(module: GradleModule) { 
    val file = BUILD.bazel {
        ...
    }
    generate(file)
  }
}

Depending on the called builder function, a file with a corresponding name will be generated. For instance, BUILD, BUILD.bazel, WORKSPACE, WORKSPACE.bazel, *.bzl.

val build = BUILD.bazel { ... }
val workspace = WORKSPACE { ... }
val mavenDependencies = "maven_dependencies".bzl { ... }

generate(build, workspace)
generate(mavenDependencies, relativePath = "third_party")

Files are generated in the module directory or a subdirectory if a relative path is specified.

Starlark code generation

In each builder block, the contents of Bazel files are crafted using a Kotlin DSL that mirrors the syntax of a Starlark language. Essentially, you’re writing Kotlin code that looks like Starlark. The code generator is type-safe, meaning many errors would be caught by the Kotlin compiler before running the program.

override fun ModuleContext.onInvoke(module: GradleModule) {
  val file = BUILD.bazel {
    load("@io_bazel_rules_kotlin//kotlin:android.bzl", "kt_android_library")
  
    kt_android_library {
      name = module.name
      srcs = glob("src/main/**/*.kt")
      custom_package = module.androidMetadata?.packageName
      manifest = "src/main/AndroidManifest.xml"
      resource_files = glob("src/main/res/**")
    }
  }

  generate(file)
}

When executed, it produces an abstract syntax tree similar to the one created by the original Starlark interpreter. This tree is subsequently leveraged to generate formatted Starlark code.

You can find an in-depth overview of a declarative Starlark code generator in the talk at droidcon NYC 2022.

The Starlark code generator is available as an open-source library Pendant and is bundled as part of the Airin plugin.

Dependencies

A Bazel target can possess various types of dependencies, each represented by different function parameters.

# BUILD.bazel
kt_android_library(
  ...
  deps = [...],
  exports = [...],
  plugins = [...],
)

A GradleModule instance provides dependencies mapped per configuration, represented with an argument name. To designate dependencies in the generated code, the `=` function (enclosed in backticks) is used to represent an argument passed to a function.

kt_android_library {
  ...
  for ((config, deps) in module.dependencies) {
    config `=` deps.map { it.asBazelLabel().toString() }
  }
}

As a result, the following Starlark code is generated.

# generated Bazel script
kt_android_library(
  ...
  plugins = [...],
  deps = [...],
  exports = [...],
)

To properly map Gradle configurations to respective function arguments in Bazel, we make use of feature components.

Feature components

Every feature component is an abstract class that extends the FeatureComponent base class and implements 2 functions, canProcess and onInvoke.

abstract class JetpackComposeFeature : FeatureComponent() {

  override fun canProcess(project: Project): Boolean {...}

  override fun FeatureContext.onInvoke(module: GradleModule) {...}
}

The former is invoked during the Gradle Configuration phase and is aimed to filter Gradle modules to which this component is applicable.

The latter is invoked during the Gradle Execution phase and contains the main logic of the component. Its purpose is to modify Bazel files generated by a related module component as well as manage the dependencies of the module.

Overriding dependencies

When migrating Gradle modules to Bazel, an obvious task is to preserve a correct dependency graph including internal module dependencies and third-party artifacts. As it turns out, the same module might have a different set of dependencies in Gradle and Bazel.

For example, let’s consider Hilt, a dependency injection library. In Gradle, your module would need to utilize a Hilt plugin and depend on two artifacts, one of which is a Kotlin symbol processor.

// build.gradle.kts
plugins {
  id("com.google.dagger.hilt.android")
}

dependencies {
  implementation("com.google.dagger:hilt-android:x.y.z")
  ksp("com.google.dagger:hilt-android-compiler:x.y.z")
}

However, in Bazel, it has a quite different way of usage.

# BUILD.bazel
kt_android_library {
  ...
  deps = ["//:hilt-android"]
}

To address this, a feature component offers a dependency override API. Here is how it looks for the Hilt example.

// FeatureComponent.onInvoke
onDependency(MavenCoordinates("com.google.dagger", "hilt-android")) {
  overrideWith(BazelLabel(path = "", target = "hilt-android"))
}

This implies that when applied to any module, the feature component will replace occurrences of the com.google.dagger:hilt-android artifact with the //:hilt-android Bazel target.

Since Hilt requires more than one dependency in Gradle, the remaining artifacts can be disregarded in Bazel.

// FeatureComponent.onInvoke
onDependency(MavenCoordinates("com.google.dagger", "hilt-android-compiler")) {
  // ignored
}

Overriding configurations

When setting up a Gradle module, it involves not only specifying dependencies but also assigning them to specific configurations, providing instructions to Gradle on how to treat each dependency.

// build.gradle.kts
dependencies {
  implementation(...)
  api(...)
  ksp(...)
  ...
}

In Bazel, targets are declared using function calls. As an analogue to Gradle configurations, we use specific function parameters for various types of dependencies.

# BUILD.bazel
kt_android_library(
  ...
  deps = [...],
  exports = [...],
  plugins = [...],
)

Similar to dependency overrides, feature components also allow the overriding of configurations. In the example below, all implementation dependencies will be declared as deps in Bazel.

// FeatureComponent.onInvoke
onConfiguration("implementation") {
  overrideWith("deps")
}

For exporting transitive dependencies, deps and exports are used as an equivalent to Gradle’s api configuration.

// FeatureComponent.onInvoke
onConfiguration("api") {
  overrideWith("deps")
  overrideWith("exports")
}

Modifying Bazel scripts

Beyond handling dependencies, feature components can also make contributions to the Bazel files generated by module components.

Let’s revisit the code snippet from the AndroidLibraryModule component that we used in the module components section, but this time, let’s slightly update it.

// ModuleComponent.onInvoke
val file = BUILD.bazel {
  _id = "build_file"

  load("@io_bazel_rules_kotlin//kotlin:android.bzl", "kt_android_library")

  kt_android_library {
    _id = "android_library_target"
    
    name = module.name
    srcs = glob("src/main/**/*.kt")
    custom_package = module.androidMetadata?.packageName
    manifest = "src/main/AndroidManifest.xml"
    resource_files = glob("src/main/res/**")
    for ((config, deps) in module.dependencies) {
      config `=` deps.map { it.asBazelLabel().toString() }
    }
  }
}

A notable addition here is the introduction of _id fields. These can be defined within any code block enclosed by curly brackets {}. Once defined, you gain the flexibility to edit the contents of these code blocks externally.

Let’s modify the contents of the generated Bazel file using our feature component. To achieve this, within the onInvoke function, use the onContext call, specifying the type of the context to be modified, along with its identifier.

// FeatureComponent.onInvoke
onContext<BuildContext>(id = "build_file") {
  `package`(default_visibility = list["//visibility:public"])
}

onContext<KtAndroidLibraryContext>(id = "android_library_target") {
  enable_data_binding = true
}

As a result, when the AndroidLibraryModule component is invoked, it will incorporate all the modifications, including the added enable_data_binding argument, as well as the top-level package function call.

load("@io_bazel_rules_kotlin//kotlin:android.bzl", "kt_android_library")
load("@rules_jvm_external//:defs.bzl", "artifact")

kt_android_library(
    name = "my-library",
    srcs = glob(["src/main/**/*.kt"]),
    custom_package = "com.turo.mylibrary",
    manifest = "src/main/AndroidManifest.xml",
    resource_files = glob(["src/main/res/**"]),
    deps = [...],
    enable_data_binding = True, # added by a feature component 
)

# added by a feature component
package(default_visibility = ["//visibility:public"])

Customizing components with properties

In many cases, there is no need to create new custom components. Instead, it’s often easier to configure existing ones.

For example, let’s examine the AndroidToolchainFeature included in the RootModule. Its purpose is to enable Kotlin, Java, and Android rules in the Bazel workspace and configure respective toolchains.

This component already generates all the necessary Starlark code. However, it exposes certain properties that allow you to specify versions of rule sets or toolchains.

// root build.gradle.kts
airin {
  register<RootModule> {
    include<AndroidToolchainFeature> {
      rulesKotlinVersion = "1.8.1"
      rulesKotlinSha = "a630cda9fdb4f56cf2dc20a4bf873765c41cf00e9379e8d59cd07b24730f4fde"
    }
  }
}

On the other side, within the component definition, the properties are declared using a property delegate.

abstract class AndroidToolchainFeature : FeatureComponent() {
  val rulesKotlinVersion: String by property(default = "1.8.1")
  val rulesKotlinSha: String by property(default = "a630cda9fdb4f56cf2dc20a4bf873765c41cf00e9379e8d59cd07b24730f4fde")

  override fun FeatureContext.onInvoke(module: GradleModule) {

    onContext<WorkspaceContext> {
      val RULES_KOTLIN_VERSION by rulesKotlinVersion
      val RULES_KOTLIN_SHA by rulesKotlinSha
  
      http_archive(
          name = "io_bazel_rules_kotlin",
          sha256 = RULES_KOTLIN_SHA,
          urls = list["https://github.com/bazelbuild/rules_kotlin/releases/download/v%s/rules_kotlin_release.tgz" `%` RULES_KOTLIN_VERSION],
      )
      ...
    }
  }
  ...
}

Shared components

Often, invoking a feature component solely for the module it belongs to is insufficient. Take the example of Hilt. Before including it as a dependency in any module, you need to configure Dagger in your Bazel workspace by following these two steps.

• Load Dagger rules in the WORKSPACE file and register artifacts in maven_install.
• Register Hilt rules in the root BUILD.bazel file.

Check official Dagger documentation to learn more about how to configure it in Bazel projects.

In addressing such scenarios, the automation tool employs the concept of shared components. Depending on the component type, the behavior can fall into one of three categories.

• Shared module component — receives contributions from every shared feature component, even if it is not directly included in it.
• Shared feature component — contributes to every shared module component.
• Top-level feature component — does not belong to any module component specifically and contributes to all module components, even non-shared ones.

This is how they are declared in a Gradle plugin.

// root build.gradle.kts
airin {
  register<AndroidLibraryModule> {
    // shared feature component
    include<HiltFeature> { shared = true }
  }

  // shared module component
  register<RootModule> { shared = true }

  // top-level feature component
  include<AllPublicFeature>()
}

Assume the RootModule component generates WORKSPACE and BUILD.bazel files.

abstract class RootModule : ModuleComponent() {
  ...

  override fun ModuleContext.onInvoke(module: GradleModule) {
    val workspace = WORKSPACE {
      _id = "workspace_file"
    }

    val build = BUILD.bazel {
      _id = "root_build_file"
    }

    generate(build, workspace)
  }
}

With this in mind, the HiltFeature component could contribute to modules that use Hilt as well as to the root module.

abstract class HiltFeature : FeatureComponent() {

  override fun canProcess(project: Project) = 
    hasPlugin("com.google.dagger.hilt.android") ||
    project.rootProject == project

  override fun FeatureContext.onInvoke(module: GradleModule) {

    // contributing to WORKSPACE file
    onContext<WorkspaceContext>(id = "workspace_file") {
      val DAGGER_VERSION by"..."
      val DAGGER_SHA by "..."

      http_archive(
        name = "dagger",
        strip_prefix = "dagger-dagger-%s" `%` DAGGER_VERSION,
        sha256 = DAGGER_SHA,
        urls = list["https://github.com/google/dagger/archive/dagger-%s.zip" `%` DAGGER_VERSION],
      )
    }

    // contributing to root BUILD.bazel file
    onContext<BuildContext>(id = "root_build_file") {
      load("@dagger//:workspace_defs.bzl", "hilt_android_rules")
      hilt_android_rules()
    }

    // contributing to modules that depend on Hilt
    onDependency(...) { ... }
    ...
  }
}

To conclude

There can be an infinite number of unique Gradle configurations relying on custom build logic, tasks, and plugins without available open-source Bazel alternatives.

That’s why it’s crucial for the Bazel migration tooling not to attempt to cover them all at once. Of course, due to its open-source nature, Airin provides a predefined set of components for a quick head start with Android projects. However, more importantly, it provides a framework that allows full customization and ability to adjust to the codebase it is applied to.

Use these links for complete examples of module and feature components as well as a sample project.