Skip to content

chore: add the scala-library projects (non-bootstrapped and bootstrapped) #23510

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/workflows/stdlib.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Compile Full Standard Library

on:
push:
branches:
- 'main'
pull_request:

permissions:
contents: read

jobs:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests CI directives e.g. [skip_ci] or [compile_nonboostrapped] are missing, currently we have already multiple jobs that would execute even though they're told to not be executed. If possible we should gradually try to unify all of GHA jobs

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer not to have them for multiple reasons:

  • This is a temporary files
  • CI will have to be redesigned anyways (subsequent PR)
  • I prefer always checking this during the transition period (until we actually proceed to the swap). I don't want to have any surprises down the road.

compile-nonbootstrapped:
runs-on: ubuntu-latest
steps:
- name: Git Checkout
uses: actions/checkout@v4

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 17
Copy link
Member Author

@hamzaremmal hamzaremmal Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are already compiling the migration of stdlib with 17, ahead of the switch to Java 17 in 3.8.0

cache: 'sbt'

- uses: sbt/setup-sbt@v1
- name: Compile `scala-library-nonbootstrapped`
run: ./project/scripts/sbt scala-library-nonbootstrapped/compile

compile-bootstrapped:
runs-on: ubuntu-latest
steps:
- name: Git Checkout
uses: actions/checkout@v4

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 17
cache: 'sbt'

- uses: sbt/setup-sbt@v1
- name: Compile `scala-library-bootstrapped`
run: ./project/scripts/sbt scala-library-bootstrapped/compile

2 changes: 2 additions & 0 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ val `scala3-bootstrapped` = Build.`scala3-bootstrapped`
val `scala3-interfaces` = Build.`scala3-interfaces`
val `scala3-compiler` = Build.`scala3-compiler`
val `scala3-compiler-bootstrapped` = Build.`scala3-compiler-bootstrapped`
val `scala-library-nonbootstrapped` = Build.`scala-library-nonbootstrapped`
val `scala-library-bootstrapped` = Build.`scala-library-bootstrapped`
val `scala3-library` = Build.`scala3-library`
val `scala3-library-bootstrapped` = Build.`scala3-library-bootstrapped`
val `scala3-library-bootstrappedJS` = Build.`scala3-library-bootstrappedJS`
Expand Down
89 changes: 89 additions & 0 deletions project/Build.scala
Original file line number Diff line number Diff line change
Expand Up @@ -1411,6 +1411,95 @@ object Build {
)
)

// ==============================================================================================
// =================================== SCALA STANDARD LIBRARY ===================================
// ==============================================================================================

/* Configuration of the org.scala-lang:scala-library:*.**.**-nonboostrapped project */
lazy val `scala-library-nonbootstrapped` = project.in(file("library"))
.settings(
name := "scala-library-nonbootstrapped",
moduleName := "scala-library",
version := dottyNonBootstrappedVersion,
versionScheme := Some("semver-spec"),
scalaVersion := referenceVersion, // nonbootstrapped artifacts are compiled with the reference compiler (already officially published)
crossPaths := false, // org.scala-lang:scala-library doesn't have a crosspath
autoScalaLibrary := false, // do not add a dependency to stdlib
// Add the source directories for the stdlib (non-boostrapped)
Compile / unmanagedSourceDirectories := Seq(baseDirectory.value / "src"),
Compile / unmanagedSourceDirectories += baseDirectory.value / "src-non-bootstrapped",
// NOTE: The only difference here is that we drop `-Werror` and semanticDB for now
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compile / scalacOptions := Seq("-deprecation", "-feature", "-unchecked", "-encoding", "UTF8", "-language:implicitConversions"),
(Compile / scalacOptions) ++= Seq(
// Needed so that the library sources are visible when `dotty.tools.dotc.core.Definitions#init` is called
"-sourcepath", (Compile / sourceDirectories).value.map(_.getCanonicalPath).distinct.mkString(File.pathSeparator),
),
// Only publish compilation artifacts, no test artifacts
Compile / publishArtifact := true,
Test / publishArtifact := false,
// Do not allow to publish this project for now
publish / skip := true,
// Project specific target folder. sbt doesn't like having two projects using the same target folder
Comment on lines +1432 to +1442
Copy link
Contributor

@WojciechMazur WojciechMazur Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the argument of explicit, duplicated, configuration, but maybe let's create a Def.settings directly above these project definitions with the common parts that are known to be always shared between the two?
Otherwise, sooner or later these 2 would diverge.

So stuff like the marked settings, crossPaths, versionScheme, moduleName can be defined in a single place

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a branch with these kind of changes but it's coming later. For now, I just need the base project to unblock some rebases like explicitly null checking the stdlib and stuff

target := target.value / "scala-library-nonbootstrapped",
)

/* Configuration of the org.scala-lang:scala-library:*.**.**-boostrapped project */
lazy val `scala-library-bootstrapped` = project.in(file("library"))
.settings(
name := "scala-library-bootstrapped",
moduleName := "scala-library",
version := dottyVersion,
versionScheme := Some("semver-spec"),
crossPaths := false, // org.scala-lang:scala-library doesn't have a crosspath
// Add the source directories for the stdlib (non-boostrapped)
Compile / unmanagedSourceDirectories := Seq(baseDirectory.value / "src"),
Compile / unmanagedSourceDirectories += baseDirectory.value / "src-bootstrapped",
// NOTE: The only difference here is that we drop `-Werror` and semanticDB for now
Compile / scalacOptions := Seq("-deprecation", "-feature", "-unchecked", "-encoding", "UTF8", "-language:implicitConversions"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: we're using here 2 ways of providing settings: using seperate strings ("-encoding", "UTF8"), and using a single argument with separator (-language:implicitConversions).
Maybe let's pick a single style and use it?
Personally I'm in favour of replacing -encoding:UTF8 change as it's easier to ready when all data is kept in single token.

Compile / scalacOptions ++= Seq(
// Needed so that the library sources are visible when `dotty.tools.dotc.core.Definitions#init` is called
"-sourcepath", (Compile / sourceDirectories).value.map(_.getCanonicalPath).distinct.mkString(File.pathSeparator),
),
// Only publish compilation artifacts, no test artifacts
Compile / publishArtifact := true,
Test / publishArtifact := false,
// Do not allow to publish this project for now
publish / skip := true,
// Project specific target folder. sbt doesn't like having two projects using the same target folder
target := target.value / "scala-library-bootstrapped",
// Configure the nonbootstrapped compiler
managedScalaInstance := false,
scalaInstance := {
val externalLibraryDeps = (`scala3-library` / Compile / externalDependencyClasspath).value.map(_.data).toSet
val externalCompilerDeps = (`scala3-compiler` / Compile / externalDependencyClasspath).value.map(_.data).toSet

// IMPORTANT: We need to use actual jars to form the ScalaInstance and not
// just directories containing classfiles because sbt maintains a cache of
// compiler instances. This cache is invalidated based on timestamps
// however this is only implemented on jars, directories are never
// invalidated.
val tastyCore = (`tasty-core` / Compile / packageBin).value
val scala3Library = (`scala3-library` / Compile / packageBin).value
val scala3Interfaces = (`scala3-interfaces` / Compile / packageBin).value
val scala3Compiler = (`scala3-compiler` / Compile / packageBin).value

val libraryJars = Array(scala3Library) ++ externalLibraryDeps
val compilerJars = Seq(tastyCore, scala3Interfaces, scala3Compiler) ++ (externalCompilerDeps -- externalLibraryDeps)

Defaults.makeScalaInstance(
scalaVersion.value,
libraryJars = libraryJars,
allCompilerJars = compilerJars,
allDocJars = Seq.empty,
state.value,
scalaInstanceTopLoader.value
)
},
scalaCompilerBridgeBinaryJar := {
Some((`scala3-sbt-bridge` / Compile / packageBin).value)
},
Comment on lines +1473 to +1500
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still uses the old compiler that uses the old stdlib, in another PR, when I add the versions of the compiler with the new stdlib as a dependency, we will proceed to the swap.

)

def dottyLibrary(implicit mode: Mode): Project = mode match {
case NonBootstrapped => `scala3-library`
case Bootstrapped => `scala3-library-bootstrapped`
Expand Down
Loading