Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

graalvm native-image support #1716

Open
bfg opened this issue Jan 30, 2025 · 1 comment
Open

graalvm native-image support #1716

bfg opened this issue Jan 30, 2025 · 1 comment

Comments

@bfg
Copy link

bfg commented Jan 30, 2025

native-image compiled binary fails to initialize yauaa instance due to missing resources

logs:

[2025/01/30 14:33:23.382] [main] INFO  n.b.p.u.u.YauaaVersion: [] Yauaa 7.29.0 (v7.29.0 @ 2024-11-17T14:28:36Z)
[2025/01/30 14:33:23.383] [main] ERROR n.b.p.u.c.ConfigLoader: [] NO config files were found matching this expression: classpath*:UserAgents/**/*.yaml
[2025/01/30 14:33:23.383] [main] WARN  n.b.p.u.c.ConfigLoader: [] Unable to load the default resources, usually caused by classloader problems.
[2025/01/30 14:33:23.383] [main] WARN  n.b.p.u.c.ConfigLoader: [] Retrying with built in list.
[2025/01/30 14:33:23.383] [main] ERROR n.b.p.u.c.ConfigLoader: [] Cannot load the resources (usually classloading problem).
[2025/01/30 14:33:23.383] [main] ERROR n.b.p.u.c.ConfigLoader: [] - Resource   : class path resource [UserAgents/Additional-Tests.yaml]
[2025/01/30 14:33:23.383] [main] ERROR n.b.p.u.c.ConfigLoader: [] - Filename   : Additional-Tests.yaml
[2025/01/30 14:33:23.383] [main] ERROR n.b.p.u.c.ConfigLoader: [] - Description: class path resource [UserAgents/Additional-Tests.yaml]
[2025/01/30 14:33:23.383] [main] ERROR n.b.p.u.c.ConfigLoader: [] FATAL: Unable to load the specified resources for UserAgents/Additional-Tests.yaml

Workaround
I've added new file: src/main/resources/META-INF/native-image/yauaa/yauaa/resource-config.json with the following content

{
  "resources" : {
    "includes" : [
      {
        "pattern" : "UserAgents.*"
      }
    ]
  }
}

The workaround works, but it's not perfect, because yauaa fails to detect all resources on the classpath and fails back to the built-in list of resources (nice)


[2025/01/30 14:16:23.080] [main] INFO  n.b.p.u.u.YauaaVersion: [] Yauaa 7.29.0 (v7.29.0 @ 2024-11-17T14:28:36Z)
[2025/01/30 14:16:23.080] [main] ERROR n.b.p.u.c.ConfigLoader: [] NO config files were found matching this expression: classpath*:UserAgents/**/*.yaml
[2025/01/30 14:16:23.080] [main] WARN  n.b.p.u.c.ConfigLoader: [] Unable to load the default resources, usually caused by classloader problems.
[2025/01/30 14:16:23.080] [main] WARN  n.b.p.u.c.ConfigLoader: [] Retrying with built in list.
[2025/01/30 14:16:24.474] [main] INFO  n.c.b.s.b.y.YauaaBrowserInfoService: [] created yauaa user-agent analyzer in 1.394s

NOTE/WARNING: listing classpath in native-image is tricky:

  • resources-config.json patterns must look like this: your-folder.* if you want to list /your-folder/ in runtime, declaring separate files just doesn't work (tested with graalvm21)
  • classpath resources URIs in native image have URI scheme: resource:

just fyi, this is the code i'm using to list classpath directory content (took quite a lot trial and error iterations to make it work with regular classpath, uber-jars, jar-in-jars and native-image):

  @JvmStatic
  fun list(path: String): List<String> {
    if (path.isBlank()) {
      throw IllegalArgumentException("Can't list a blank path.")
    }

    val fsPath = asPath(path)
    if (!Files.isDirectory(fsPath)) {
      throw IOException("Not a directory: $path")
    }

    val origPath = path.replace("/+$".toRegex(), "")
    val res = Files.list(fsPath)
        .use { stream ->
          stream
              .map { origPath + "/" + it.fileName.toString() }
              .sorted()
              .toList()

        }

    log.trace { "listed $path [fspath=$fsPath]: $res" }
    return res
  }

  @JvmStatic
  fun asPath(path: String): Path {
    val resUri = this.javaClass.getResource(path)?.toURI()
        ?: throw FileNotFoundException("Resource not found on classpath: $path")

    val (prefix, cpPath) = splitCpPath(resUri.toString())
    log.debug { "classpath path=`$path` uri=`$resUri`, prefix=`$prefix`, path=`$cpPath`" }

    if (!(prefix.isEmpty() || prefix == "file")) {
      val fs = getFileSystem(resUri)
      log.debug { "filesystem for $resUri: $fs" }
    }

    return Paths.get(resUri)
  }

  @JvmStatic
  private fun getFileSystem(uri: URI): FileSystem {
    return try {
      FileSystems.getFileSystem(uri)
    } catch (e: FileSystemNotFoundException) {
      FileSystems.newFileSystem(uri, mapOf("create" to "true"))
    }
  }

  @JvmStatic
  private fun splitCpPath(path: String): Pair<String, String> {
    val prefixes = listOf("file:", "jar:file:", "zip:file:", "resource:")
    return prefixes.asSequence()
        .filter { path.startsWith(it) }
        .firstNotNullOfOrNull { Pair(it.substringBefore(":"), path.substringAfter(it)) }
        ?: Pair("", path)

  }
@nielsbasjes
Copy link
Owner

My experience with building a native executable is very limited at this time.
Do you have a minimal reproduction repository for me to better understand this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants