Migrating sbt plugins to sbt 2 with sbt2-compat plugin

This post covers work done under the Sovereign Tech Fund investment umbrella: sbt 2 Stable Release and Maintenance. The work is coordinated by the Scala Center.

There’s an ongoing, community-driven effort to repopulate the sbt plugin ecosystem in preparation for the sbt 2 release. From sbt 1.x, plugin authors can cross publish against sbt 2.0 release candidates. To facilitate the plugin migration, we’ve created the sbt2-compat plugin.

The PluginCompat Pattern

sbt 2 is a major upgrade from sbt 1 and makes breaking changes on the API level. As a plugin maintainer, you want to preserve compatibility with sbt 1 when migrating - so you want to cross-publish your plugin for sbt 1 and sbt 2. Ideally, you want to have the same codebase that compiles for both sbt 1 and sbt 2. However, due to the breaking changes to the API, this is not always possible. You end up with a situation where the same concept is expressed in a different way in sbt 1 and sbt 2.

One such example is how files are represented in sbt 1 and sbt 2. In sbt 1, everything is a java.io.File, whereas in sbt 2 the types are more granular depending on the context. For example, HashedVirtualFileRef is used for classpath entries, VirtualFile - for task outputs and caching artifacts, VirtualFileRef - for path-like references. These types are defined in the xsbti package.

The current approach to handling these discrepancies is to have a single codebase that compiles for both sbt 1 and sbt 2, and use the PluginCompat pattern to provide a compatibility layer. For example, suppose you want to define a new task key that builds a JAR file and returns it.

In sbt 1, you would define it as:

lazy val assembly = taskKey[java.io.File]("Builds a deployable JAR file")

In sbt 2, you would define it as:

lazy val assembly = taskKey[xsbti.file.HashedVirtualFileRef]("Builds a deployable JAR file")

Since you want to use the same codebase for both sbt 1 and sbt 2, you would need to abstract over the file type and define it separately for each sbt version. So:

// src/main/scala-2.12/PluginCompat.scala
type FileRef = java.io.File

// src/main/scala-3/PluginCompat.scala
type FileRef = xsbti.file.HashedVirtualFileRef

You would then be able to define the task using this unified API:

lazy val assembly = taskKey[FileRef]("Builds a deployable JAR file")

This approach is a recurring pattern that you will encounter from plugin to plugin when attempting to cross-compile for sbt 1 and sbt 2. This is exactly the motivation behind the sbt2-compat plugin.

Cross-building for sbt 1 and sbt 2 using the sbt2-compat plugin

The above encoding of the differences in the file API between sbt 1 and sbt 2 is already abstracted and ready to be reused in the sbt2-compat plugin. This plugin follows a similar pattern to the sbt-compat plugin that handled cross-builds between sbt 0.x and 1.x. As a plugin maintainer, instead of manually defining the PluginCompat.scala shim as described above, you would instead add the sbt2-compat plugin to your build and use the PluginCompat pattern as before, relying on the unified implementation provided by sbt2-compat.

1. Add sbt2-compat to your plugin

Add the plugin in your plugin’s build.sbt (not project/plugins.sbt):

addSbtPlugin("com.github.sbt" % "sbt2-compat" % "<version>")

To cross-build for sbt 1 and sbt 2, see Cross building sbt plugins in the official migration guide.

ThisBuild / crossScalaVersions := Seq("3.8.1", "2.12.21")
ThisBuild / scalaVersion := crossScalaVersions.head

(pluginCrossBuild / sbtVersion) := {
  scalaBinaryVersion.value match {
    case "2.12" => "1.12.3"
    case _      => "2.0.0-RC9"
  }
}

For a concrete example, see sbt-assembly.

2. Use sbt2-compat to handle the API differences between sbt 1 and sbt 2

Let’s take a look at how sbt-assembly uses sbt2-compat to cross-build for sbt 1 and sbt 2. Here is how it is applied:

1. Task keys — Import FileRef and use it for task return types. Notice how the type comes from the sbt2-compat plugin and does not need to be defined manually. (AssemblyKeys.scala L6–10):

import sbtcompat.PluginCompat.FileRef

lazy val assembly                  = taskKey[FileRef]("Builds a deployable über JAR")
lazy val assemblyPackageScala      = taskKey[FileRef]("Produces the Scala artifact")
lazy val assemblyPackageDependency = taskKey[FileRef]("Produces the dependency artifact")

2. Shared logic — Import compat helpers and use them for classpath handling, file conversion, and module metadata (Assembly.scala L225–294):

import sbtcompat.PluginCompat.{FileRef, Out, toNioPath, toFile, toOutput, toNioPaths, toFiles, moduleIDStr, parseModuleIDStrAttribute}

// Classpath iteration
val (jars, dirs) = classpath.toVector
  .sortBy(x => toNioPath(x).toAbsolutePath().toString())
  .partition(x => ClasspathUtil.isArchive(toNioPath(x)))

// Convert classpath entry to File for JarFile
val jarFile = new JarFile(toFile(jar))

// ModuleID from metadata
val module = jar.metadata.get(moduleIDStr)
  .map(parseModuleIDStrAttribute)
  .map(m => ModuleCoordinate(m.organization, m.name, m.revision))
  .getOrElse(ModuleCoordinate("", jar.data.name.replaceAll(".jar", ""), ""))

// Return task output
toOutput(builtAssemblyJar)

3. Disabling task caching — Use Def.uncached so classpath tasks read fresh values on sbt 2 (AssemblyPlugin.scala L86–87):

import sbtcompat.PluginCompat._

assembly / fullClasspath := Def.uncached((fullClasspath or (Runtime / fullClasspath)).value),
assembly / externalDependencyClasspath := Def.uncached((externalDependencyClasspath or (Runtime / externalDependencyClasspath)).value),

Project status

sbt2-compat currently covers the core API surface needed for cross-building plugins that work with files, classpaths, and packaging:

Plugin-specific compat (e.g. custom disk caching, test API differences, Scala stdlib bridges like Streamable) stays in each plugin’s own PluginCompat.scala. sbt-assembly, for instance, still maintains local shims for its cache, test settings, and PackageOption types. The sbt2-compat README documents this design and lists known caveats.

Development model. sbt2-compat evolves iteratively by porting real-world plugins. Each port validates the existing API and may reveal missing compat methods. The idea is that as you port your plugins to sbt 2, you will discover gaps in the API that you can contribute back to sbt2-compat. For the exact development model, see the README.

Contributions are welcome! If you’re cross-building an sbt plugin for sbt 1 and sbt 2, the sbt 2 team would be happy to hear about your experiences! Share what worked, what didn’t, and what you wish sbt2-compat had. Pull requests with compat helpers that might help other plugins are welcome! You can share your experiences and participate in the discussion in the sbt 2 production-ready roadmap thread at the Scala Contributors forum.

Participation

The Scala Center has been entrusted with coordinating the commissioned Scala work for the Sovereign Tech Fund. The Scala Center is an independent, not-for-profit center sponsored by corporate members and individual backers like you to promote and facilitate Scala. If you would like to participate and/or see more of these types of efforts, please reach out to your manager to see if your company can donate engineering time or membership to the Scala Center.

See The Scala Center Fundraising Campaign for more details.