Work with new ZIO frontend and release 4.0.0 (#1035)

* Start moving an example to new zio core * Upgrade zio version * Add temporary read_ function to minimise the client changes * Add auto derivation for Config * Move more methods to Config in Auto derivation * Implement collectAll (sequence) for ZIO core config * Run an example using autoderivation for ZIO Core config * Clean up * Rename to DeriveConfig * Introduced IndexedFlat and KeyComponent * Add comments * Implement basic typesafe config backend, and make an example work * Make sure list of primitives and list of products work * Make sure optional work and avoid cast exceptions * Make sure error message at an index is readable * Expand example for HOCON * Implement documentation * Implement prettyPrint * Fix scala3 autoderivation * Try to fix scala3 auto derivation * Make sure scala3 derivation works * Make sure to use Config.string instead of Config.succeed in autoderivations * Introduce constant and use it for derivation * Allow mapKeys in Config - Implement toKebabCase - Implement toUpperCase - Implement toSnakeCase - Implement toLowerCase * Delete functionalities from zio-config - Delete PropertyTree - Delete read module - Delete deep inspection of config for error reporting during recursive reads - Delete ConfigSource and some features - Delete ConfigSource memoization behaviour - Delete ConfigSource strictlyOnce behaviour - Delete CommandLine ConfigSource - Delete SystemEnv ConfigSource - Delete System Properties ConfigSource - Delete reporting of config - Delete configSource details from documentation - Delete writeback functionality * Delete test cases and integrate with enumeratum - Delete all property based round trip tests that existed in ZIO-CONFIG - Make sure enumeratum module works with new ZIO Config * Avoid cyclic imports * Make sure auto derivation with scala3 after removing unused classes in core * Start integrating with cats * Create applicative instance for Config and Semigroup for ConfigProvider * Remove shapeless module and support for scala211 * Fix build sbt * Remove fromISO in scalaz, and make things compile with ZIO Config * Use provider.loadAuto[A] pattern and rename to DeriveConfig * Rename scala2 magnolia descriptor to DeriveConfig * Release refined for scala3 * Delete effectful ConfigSource - Remove using ZIO to load Hocon strings - We leave it to user to define ConfigSource under an effect or not * Implement backend for yaml * Try to fix compile issues in example module * Fix all scala2 compile errors in examples module * Make sure sealed trait name and subclass name exist only if annotations are present * Delete unused methods * Revamp auto derivation for scala2 * Fix case object derivations * Remove redundant function call in scala2 autoderivation * Extend the example with custom class name, object name and field name * Fix auto derivation simple * Add more comments in nameWithLabel * Add further clean ups in scala 2 auto derivation * Fix config list * Implement ConfigProvider.at functionality * Introduce path interpolator * Clean up path * Make sure pureConfig example work with nameWithLabel annotation * Add example for manual derivation for ADT using enumerate * Make sure all yaml examples work * Make sure config docs works * Make sure all examples work * Integrate with AWS * Keep only config docs tests * Implement map * Fix typesafe test * Reformat typesafe test * Fix yaml config source * Reformat and remove tests * Reformat and remove tests * Fix Config.map * Add more * Make sure all examples work * Make sure sub config example works * Implement yaml source * Add more types in Config * Delete DerivationTest * Start using latest zio * Add sonatype resolver * Fix tests and fix ordering * Copy experimental implementation back to zio-config and revet typesafe config implementation * Fix TypesafeConfigSimpleSpec * Start fixing tests * Add a FIXME comment * Format all tests * Rename TypesafeConfigSource to TypesafeConfigProvider * Fix imports * scalafix typesafe magnolia test * Fix core module * Fix format * Delete roundtrip refined spec * Fix sub config example * Run scalafix on all and make sure examples are compiled * Make sure aws module makes use ConfigProvider.fromMap * Start fixing documentations * Start fixing documentations * Fix more test * Start fixing scala3 test * Try to fix scala3 compilation errors * Fix all tests * Auto derivation example work for scala3 * Remove markdown as many of the descriptions now don't exist * Reformat code
zio · Feb 20, 2023 · f2aa3b6 · f2aa3b6
1 parent 05137bf
commit f2aa3b6
Show file tree

Hide file tree

Showing 178 changed files with 3,342 additions and 21,059 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -49,9 +49,6 @@ jobs:
       - name: Run JS tests
         if: ${{ matrix.platform == 'JS' && !startsWith(matrix.scala, '3.') }}
         run: sbt ++${{ matrix.scala }}! testJS
-      - name: Run 2.11 JVM tests
-        if: ${{ matrix.platform == 'JVM' && startsWith(matrix.scala, '2.11') }}
-        run: sbt ++${{ matrix.scala }}! testJVM211
       - name: Run 2.12 JVM tests
         if: ${{ matrix.platform == 'JVM' && startsWith(matrix.scala, '2.12') }}
         run: sbt ++${{ matrix.scala }}! testJVM212

diff --git a/aws/shared/src/main/scala/zio/config/aws/parameterstore/ParameterStoreConfig.scala b/aws/shared/src/main/scala/zio/config/aws/parameterstore/ParameterStoreConfig.scala
diff --git a/aws/shared/src/main/scala/zio/config/aws/parameterstore/ParameterStoreConfigProvider.scala b/aws/shared/src/main/scala/zio/config/aws/parameterstore/ParameterStoreConfigProvider.scala
@@ -0,0 +1,64 @@
+package zio.config.aws.parameterstore
+
+import com.amazonaws.services.simplesystemsmanagement.model.{GetParametersByPathRequest, Parameter}
+import com.amazonaws.services.simplesystemsmanagement.{
+  AWSSimpleSystemsManagement,
+  AWSSimpleSystemsManagementClientBuilder
+}
+import zio.stream.ZStream
+import zio.{Chunk, Task, ZIO}
+
+import scala.jdk.CollectionConverters._
+
+import zio.ConfigProvider
+import zio.Config
+
+object ParameterStoreConfigProvider {
+  def from(
+    basePath: String,
+    getClient: Task[AWSSimpleSystemsManagement] = ZIO.attempt(AWSSimpleSystemsManagementClientBuilder.defaultClient())
+  ): ZIO[Any, Config.Error, ConfigProvider] =
+    getClient.flatMap { ssm =>
+      val request =
+        new GetParametersByPathRequest()
+          .withPath(basePath)
+          .withRecursive(true)
+          .withWithDecryption(true)
+
+      ZStream
+        .paginateZIO(
+          ZIO.attempt(ssm.getParametersByPath(request))
+        )(_.map { response =>
+          val currentBatchResult =
+            Chunk.fromIterable(
+              response.getParameters.asScala.toList
+            )
+          val nextToken          = response.getNextToken
+          val nextBatch          =
+            if (nextToken == null || nextToken.trim.isEmpty)
+              None
+            else
+              Some(
+                ZIO.attempt(
+                  ssm.getParametersByPath(request.withNextToken(nextToken))
+                )
+              )
+
+          (currentBatchResult, nextBatch)
+        })
+        .runCollect
+        .map { result =>
+          ConfigProvider
+            .fromMap(
+              convertParameterListToMap(result.flatten.toList, basePath),
+              pathDelim = "/"
+            )
+        }
+    }
+      .mapError(throwable => Config.Error.Unsupported(message = throwable.toString): Config.Error)
+
+  private[config] def convertParameterListToMap(list: List[Parameter], basePath: String): Map[String, String] = {
+    val str = s"$basePath/"
+    list.map(parameter => (parameter.getName.replaceFirst(str, ""), parameter.getValue)).toMap
+  }
+}
diff --git a/aws/shared/src/main/scala/zio/config/aws/parameterstore/ParameterStoreConfigSource.scala b/aws/shared/src/main/scala/zio/config/aws/parameterstore/ParameterStoreConfigSource.scala
diff --git a/aws/shared/src/main/scala/zio/config/aws/parameterstore/package.scala b/aws/shared/src/main/scala/zio/config/aws/parameterstore/package.scala
@@ -8,20 +8,12 @@ import zio._
 import zio.config._
 
 package object parameterstore {
-  implicit class FromConfigTypesafe(c: ZConfig.type) {
-    def fromParameterStore[A](
-      configDescriptor: ConfigDescriptor[A],
-      basePath: String,
-      getClient: Task[AWSSimpleSystemsManagement] = ZIO.attempt(AWSSimpleSystemsManagementClientBuilder.defaultClient())
-    )(implicit tag: Tag[A]): Layer[ReadError[String], A] =
-      ParameterStoreConfig.from(configDescriptor, basePath, getClient)
-  }
 
-  implicit class FromConfigSourceTypesafe(c: ConfigSource.type) {
+  implicit class FromConfigSourceTypesafe(c: ConfigProvider.type) {
     def fromParameterStore(
       basePath: String,
       getClient: Task[AWSSimpleSystemsManagement] = ZIO.attempt(AWSSimpleSystemsManagementClientBuilder.defaultClient())
-    ): ConfigSource =
-      ParameterStoreConfigSource.from(basePath, getClient)
+    ): ZIO[Any, Config.Error, ConfigProvider] =
+      ParameterStoreConfigProvider.from(basePath, getClient)
   }
 }
diff --git a/build.sbt b/build.sbt
@@ -22,47 +22,27 @@ inThisBuild(
   )
 )
 
-lazy val createProductBuilder = taskKey[Unit]("Generate code for ProductBuilder.scala")
-
-createProductBuilder := {
-  val productBuilderFile =
-    (zioConfigJVM / sourceDirectory).value / "main" / "scala" / "zio" / "config" / "ProductBuilder.scala"
-  val resource           = (Compile / resourceManaged).value / "scalaFmt" / "temporary"
-  val scalaFmt           = baseDirectory.value / ".scalafmt.conf"
-
-  ProductBuilderCodeGen.replaceFileSection(
-    productBuilderFile,
-    "productbuilder",
-    ProductBuilderCodeGen.productBuilderCodes :+ "",
-    resource,
-    scalaFmt
-  )
-}
-
 addCommandAlias("fmt", "; scalafmtSbt; scalafmt; test:scalafmt")
 addCommandAlias("fix", "; all compile:scalafix test:scalafix; all scalafmtSbt scalafmtAll")
 addCommandAlias("check", "; scalafmtSbtCheck; scalafmtCheckAll; compile:scalafix --check; test:scalafix --check")
 addCommandAlias(
   "checkAll",
-  "; ++2.11.12; project root2-11; check; ++2.12.13; project root2-12; check; ++2.13.5; project root2-13; check"
+  "; ++2.12.13; project root2-12; check; ++2.13.5; project root2-13; check"
 )
-addCommandAlias("compileAll", "; ++2.11.12; root2-11/compile; ++2.12.16; root2-12/compile; ++2.13.8!; root2-13/compile")
-addCommandAlias("testAll", "; ++2.11.12; root2-11/test; ++2.12.16; root2-12/test; ++2.13.8!; root2-13/test")
+addCommandAlias("compileAll", "; ++2.12.16; root2-12/compile; ++2.13.8!; root2-13/compile")
+addCommandAlias("testAll", "; ++2.12.16; root2-12/test; ++2.13.8!; root2-13/test")
 addCommandAlias(
   "testJS",
   ";zioConfigJS/test"
 )
-addCommandAlias(
-  "testJVM211",
-  ";zioConfigJVM/test;zioConfigTypesafeJVM/test;zioConfigDerivationJVM/test;zioConfigYamlJVM/test;zioConfigAwsJVM/test"
-)
+
 addCommandAlias(
   "testJVM212",
   ";zioConfigJVM/test;zioConfigTypesafeJVM/test;zioConfigDerivationJVM/test;zioConfigYamlJVM/test;examplesJVM/test;zioConfigAwsJVM/test;zioConfigZioAwsJVM/test"
 )
 addCommandAlias(
   "testJVM213",
-  ";zioConfigJVM/test;zioConfigTypesafeJVM/test;zioConfigShapelessJVM/test;zioConfigDerivationJVM/test;zioConfigYamlJVM/test;zioConfigGenJVM/test;zioConfigRefinedJVM/test;zioConfigMagnoliaJVM/test;examplesJVM/test;zioConfigTypesafeMagnoliaTestsJVM/test;zioConfigAwsJVM/test;zioConfigZioAwsJVM/test"
+  ";zioConfigJVM/test;zioConfigTypesafeJVM/test;zioConfigDerivationJVM/test;zioConfigYamlJVM/test;zioConfigRefinedJVM/test;zioConfigMagnoliaJVM/test;examplesJVM/test;zioConfigTypesafeMagnoliaTestsJVM/test;zioConfigAwsJVM/test;zioConfigZioAwsJVM/test"
 )
 addCommandAlias(
   "testJVM3x",
@@ -71,7 +51,7 @@ addCommandAlias(
 
 val awsVersion        = "1.12.360"
 val zioAwsVersion     = "5.19.8.1"
-val zioVersion        = "2.0.5"
+val zioVersion        = "2.0.9+6-e9ca719a-SNAPSHOT"
 val magnoliaVersion   = "0.17.0"
 val refinedVersion    = "0.10.1"
 val pureconfigVersion = "0.16.0"
@@ -90,7 +70,7 @@ lazy val magnoliaDependencies =
 
 lazy val refinedDependencies =
   libraryDependencies ++= {
-    if (scalaBinaryVersion.value == "2.11" || scalaVersion.value == ScalaDotty) Seq.empty // Just to make IntelliJ happy
+    if (scalaBinaryVersion.value == "2.11") Seq.empty // Just to make IntelliJ happy
     else Seq("eu.timepit" %% "refined" % refinedVersion)
   }
 
@@ -107,13 +87,11 @@ lazy val scala211projects =
     zioConfigAwsJVM,
     zioConfigNative,
     zioConfigTypesafeJVM,
-    zioConfigShapelessJVM,
     zioConfigDerivationJVM,
     zioConfigYamlJVM,
     docs
   )
-lazy val scala212projects = scala211projects ++ Seq[ProjectReference](
-  zioConfigGenJVM,
+lazy val scala212projects = Seq[ProjectReference](
   zioConfigEnumeratumJVM,
   zioConfigCatsJVM,
   zioConfigRefinedJVM,
@@ -133,6 +111,7 @@ lazy val scala3projects =
     zioConfigDerivationJVM,
     zioConfigEnumeratumJVM,
     zioConfigMagnoliaJVM,
+    zioConfigRefinedJVM,
     zioConfigScalazJVM,
     zioConfigTypesafeJVM,
     zioConfigYamlJVM,
@@ -236,6 +215,7 @@ lazy val zioConfigRefined    = crossProject(JVMPlatform)
   .in(file("refined"))
   .settings(stdSettings("zio-config-refined"))
   .settings(crossProjectSettings)
+  .settings(dottySettings)
   .settings(
     crossScalaVersions --= Seq(Scala211),
     refinedDependencies,
@@ -296,7 +276,7 @@ lazy val examples = crossProject(JVMPlatform)
         })
         .value
   )
-  .dependsOn(zioConfig, zioConfigMagnolia, zioConfigRefined, zioConfigTypesafe, zioConfigGen, zioConfigYaml)
+  .dependsOn(zioConfig, zioConfigMagnolia, zioConfigRefined, zioConfigTypesafe, zioConfigYaml)
 
 lazy val examplesJVM = examples.jvm
 
@@ -310,22 +290,6 @@ lazy val zioConfigDerivation = crossProject(JVMPlatform)
 lazy val zioConfigDerivationJVM = zioConfigDerivation.jvm
   .settings(dottySettings)
 
-lazy val zioConfigGen = crossProject(JVMPlatform)
-  .in(file("gen"))
-  .settings(stdSettings("zio-config-gen"))
-  .settings(crossProjectSettings)
-  .settings(
-    crossScalaVersions --= Seq(Scala211),
-    magnoliaDependencies,
-    libraryDependencies ++= Seq(
-      "dev.zio"       %% "zio-test-magnolia" % zioVersion,
-      "org.scalatest" %% "scalatest"         % "3.2.15" % Test
-    )
-  )
-  .dependsOn(zioConfigTypesafe, zioConfigMagnolia)
-
-lazy val zioConfigGenJVM = zioConfigGen.jvm
-
 lazy val zioConfigMagnolia    = crossProject(JVMPlatform)
   .in(file("magnolia"))
   .settings(stdSettings("zio-config-magnolia"))
@@ -351,23 +315,6 @@ lazy val zioConfigMagnolia    = crossProject(JVMPlatform)
 
 lazy val zioConfigMagnoliaJVM = zioConfigMagnolia.jvm
 
-lazy val zioConfigShapeless    = crossProject(JVMPlatform)
-  .in(file("shapeless"))
-  .settings(stdSettings("zio-config-shapeless"))
-  .settings(crossProjectSettings)
-  .settings(
-    libraryDependencies ++= Seq(
-      "dev.zio"       %% "zio-test"      % zioVersion % Test,
-      "dev.zio"       %% "zio-test-sbt"  % zioVersion % Test,
-      "org.scala-lang" % "scala-reflect" % scalaVersion.value,
-      "com.chuusai"   %% "shapeless"     % shapelessVersion
-    ),
-    testFrameworks := Seq(new TestFramework("zio.test.sbt.ZTestFramework"))
-  )
-  .dependsOn(zioConfig % "compile->compile;test->test", zioConfigDerivation)
-
-lazy val zioConfigShapelessJVM = zioConfigShapeless.jvm
-
 lazy val zioConfigTypesafe    = crossProject(JVMPlatform)
   .in(file("typesafe"))
   .settings(stdSettings("zio-config-typesafe"))
@@ -489,10 +436,8 @@ lazy val docs = project
       inProjects(
         zioConfigJVM,
         zioConfigTypesafeJVM,
-        zioConfigShapelessJVM,
         zioConfigDerivationJVM,
         zioConfigYamlJVM,
-        zioConfigGenJVM,
         zioConfigRefinedJVM,
         zioConfigMagnoliaJVM
       )
@@ -501,10 +446,8 @@ lazy val docs = project
   .dependsOn(
     zioConfigJVM,
     zioConfigTypesafeJVM,
-    zioConfigShapelessJVM,
     zioConfigDerivationJVM,
     zioConfigYamlJVM,
-    zioConfigGenJVM,
     zioConfigRefinedJVM,
     zioConfigMagnoliaJVM
   )