Error Reading gencode.v38.genes.json.gz File in SvAnna CLI Tool

[pragnapcu]

Hello,

I am experiencing an issue while running the SvAnna CLI tool. I receive the following error message when executing the command:
ERROR o.m.svanna.cli.cmd.PrioritizeCommand - Error: Error reading genes from /path/to/svanna-data/gencode.v38.genes.json.gz

Here is the exact command I used:
java -jar svanna-cli-1.0.4.jar prioritize -d svanna-data --vcf example.vcf --phenotype-term HP:0011890 --phenotype-term HP:0000978 --phenotype-term HP:0012147 --out-dir results --prefix example

I have verified the integrity of the gencode.v38.genes.json.gz file using checksum, and it says Ok. Additionally, the head of the file appears to be correctly formatted (JSON structure), showing gene information as expected.
[ {
"id" : {
"acc" : "ENSG00000160856.21",
"symbol" : "FCRL3",
"hgncId" : "HGNC:18506"
},
"loc" : {
"gbAccession" : "CM000663.2",
"strand" : "-",
"start" : 91255653,

SVAnna version: svanna-cli v1.0.4
java -jar --version
openjdk 11.0.13 2021-10-19
OpenJDK Runtime Environment JBR-11.0.13.7-1751.21-jcef (build 11.0.13+7-b1751.21)
OpenJDK 64-Bit Server VM JBR-11.0.13.7-1751.21-jcef (build 11.0.13+7-b1751.21, mixed mode)

I have checked file permissions and ensured that the file is readable. Despite these checks, the error persists.

Has anyone encountered this issue before, or does anyone have suggestions on how to resolve it? Any help would be greatly appreciated.

Thank you.

[ielis]

Hi @pragnapcu ,
is the line

ERROR o.m.svanna.cli.cmd.PrioritizeCommand - Error: Error reading genes from /path/to/svanna-data/gencode.v38.genes.json.gz

literally one of the lines that you see in the Terminal?

[pragnapcu]

Yes, in the output. Except /path/to/ where actual path was mentioned

[ielis]

So, I assume that you are providing a proper path via the -d option and not the svanna-data placeholder.

First, I would like to ask you to provide an absolute path, in case you already did not do that.

If absolute path does not help, I would like to ask you to increase the CLI verbosity using -vv and share the terminal output.
The verbose output should include broader error context, including stack trace that will help me troubleshoot further.

[pragnapcu]

absolute path did not help. Here is the terminal output using -vv

java -jar svanna-cli-1.0.4.jar prioritize -d /path/to/svanna-data --vcf example.vcf --phenotype-term HP:0011890 --phenotype-term HP:0000978 --phenotype-term HP:0012147 --out-dir results --prefix example -vv

---

/ / _/ | ____ ____ ____ _
_ | | / / /| | / __ / __ / __ `/
/ /| |/ / ___ |/ / / / / / / // /
// |// |// /// //_,_/

Structural Variant Annotation and Analysis
:: v1.0.4 ::

10:08:31.675 [main] INFO o.m.svanna.cli.cmd.PrioritizeCommand - Using 3 phenotype features supplied via CLI
10:08:31.682 [main] INFO o.m.svanna.cli.cmd.SvAnnaCommand - Spooling up SvAnna v1.0.4 using resources in /path/to/svanna-data
10:08:31.716 [main] DEBUG o.m.s.configuration.SvAnnaBuilder - Reading genes from /path/to/svanna-data/gencode.v38.genes.json.gz
10:08:31.980 [main] DEBUG o.m.s.i.s.SilentGenesGeneService - Assuming the file is gzipped
10:08:32.033 [main] ERROR o.m.svanna.cli.cmd.PrioritizeCommand - Error: Error reading genes from /path/to/svanna-data/gencode.v38.genes.json.gz
10:08:32.033 [main] DEBUG o.m.svanna.cli.cmd.PrioritizeCommand - Error: Error reading genes from /path/to/svanna-data/gencode.v38.genes.json.gz
org.monarchinitiative.svanna.configuration.exception.InvalidResourceException: Error reading genes from /path/to/svanna-data/gencode.v38.genes.json.gz
at org.monarchinitiative.svanna.configuration.SvAnnaBuilder.build(SvAnnaBuilder.java:105)
at org.monarchinitiative.svanna.cli.cmd.SvAnnaCommand.bootstrapSvAnna(SvAnnaCommand.java:102)
at org.monarchinitiative.svanna.cli.cmd.PrioritizeCommand.runAnalysis(PrioritizeCommand.java:229)
at org.monarchinitiative.svanna.cli.cmd.PrioritizeCommand.execute(PrioritizeCommand.java:141)
at org.monarchinitiative.svanna.cli.cmd.SvAnnaCommand.call(SvAnnaCommand.java:94)
at org.monarchinitiative.svanna.cli.cmd.SvAnnaCommand.call(SvAnnaCommand.java:29)
at picocli.CommandLine.executeUserObject(CommandLine.java:1933)
at picocli.CommandLine.access$1200(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2326)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2291)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159)
at picocli.CommandLine.execute(CommandLine.java:2058)
at org.monarchinitiative.svanna.cli.Main.main(Main.java:41)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Metadata must not be null (through reference chain: java.util.ArrayList[0])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:402)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:373)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:375)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4730)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3722)
at org.monarchinitiative.sgenes.io.json.JsonGeneParser.read(JsonGeneParser.java:68)
at org.monarchinitiative.svanna.io.service.SilentGenesGeneService.of(SilentGenesGeneService.java:40)
at org.monarchinitiative.svanna.configuration.SvAnnaBuilder.build(SvAnnaBuilder.java:103)
... 21 common frames omitted
Caused by: java.lang.NullPointerException: Metadata must not be null
at java.base/java.util.Objects.requireNonNull(Objects.java:246)
at org.monarchinitiative.sgenes.model.base.BaseTranscript.(BaseTranscript.java:26)
at org.monarchinitiative.sgenes.model.base.BaseCodingTranscript.(BaseCodingTranscript.java:21)
at org.monarchinitiative.sgenes.model.impl.CodingTranscriptDefault.(CodingTranscriptDefault.java:26)
at org.monarchinitiative.sgenes.model.impl.CodingTranscriptDefault.of(CodingTranscriptDefault.java:18)
at org.monarchinitiative.sgenes.model.Transcript.of(Transcript.java:46)
at org.monarchinitiative.sgenes.io.json.deserialize.TranscriptDeserializer.deserialize(TranscriptDeserializer.java:52)
at org.monarchinitiative.sgenes.io.json.deserialize.TranscriptDeserializer.deserialize(TranscriptDeserializer.java:19)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4706)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2879)
at com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:3343)
at org.monarchinitiative.sgenes.io.json.deserialize.GeneDeserializer.deserialize(GeneDeserializer.java:41)
at org.monarchinitiative.sgenes.io.json.deserialize.GeneDeserializer.deserialize(GeneDeserializer.java:18)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:359)
... 29 common frames omitted

 Thank you very much, Danis for your response.

[ielis]

Hi @pragnapcu
the stack trace indicates issues in one of the release files gencode.v38.genes.json.gz. However, I was not able to replicate the issue on my side.

I did a fresh download of SvAnna 1.0.4 distribution from GitHub releases and I downloaded and unpacked the 2304 database files from Downloads.

Then I downloaded the example.vcf (I believe it is the same one that you used) and ran the app with your CLI input:

java -jar svanna-cli-1.0.4.jar prioritize -d $(pwd)/data --vcf example.vcf --phenotype-term HP:0011890 --phenotype-term HP:0000978 --phenotype-term HP:0012147 --out-dir results --prefix example -vv

This is the output I got:

   _____       ___
  / ___/_   __/   |  ____  ____  ____ _
  \__ \| | / / /| | / __ \/ __ \/ __ `/
 ___/ /| |/ / ___ |/ / / / / / / /_/ /
/____/ |___/_/  |_/_/ /_/_/ /_/\__,_/

Structural Variant Annotation and Analysis
                          :: v1.0.4 ::


12:45:31.992 [main] INFO  o.m.svanna.cli.cmd.PrioritizeCommand - The output directory /home/ielis/tmp/svanna-cli-1.0.4/results does not exist. Creating the missing directories.
12:45:31.997 [main] INFO  o.m.svanna.cli.cmd.PrioritizeCommand - Using 3 phenotype features supplied via CLI
12:45:32.000 [main] INFO  o.m.svanna.cli.cmd.SvAnnaCommand - Spooling up SvAnna v1.0.4 using resources in /home/ielis/tmp/svanna-cli-1.0.4/data
12:45:32.037 [main] DEBUG o.m.s.configuration.SvAnnaBuilder - Reading genes from /home/ielis/tmp/svanna-cli-1.0.4/data/gencode.v38.genes.json.gz
12:45:32.242 [main] DEBUG o.m.s.i.s.SilentGenesGeneService - Assuming the file is gzipped
12:45:35.309 [main] DEBUG o.m.s.configuration.SvAnnaBuilder - Reading HPO file from /home/ielis/tmp/svanna-cli-1.0.4/data/hp.json
12:45:35.389 [main] DEBUG o.m.phenol.io.OntologyLoader - Read first bytes: {
  "graphs": [

12:45:35.389 [main] DEBUG o.m.phenol.io.OntologyLoader - Looks like a JSON file...
12:45:35.942 [main] DEBUG o.m.phenol.io.OntologyLoader - Finished loading ontology
12:45:35.942 [main] DEBUG o.m.phenol.io.OntologyLoader - Creating phenol ontology
12:45:35.982 [main] DEBUG o.m.p.i.o.OboGraphDocumentAdaptor - Converting graph document...
12:45:35.982 [main] DEBUG o.m.p.i.o.OboGraphDocumentAdaptor - Converting metadata...
12:45:35.986 [main] DEBUG o.m.p.i.o.OboGraphDocumentAdaptor - Converting nodes to terms...
12:45:36.112 [main] DEBUG o.m.p.i.o.OboGraphTermFactory - No meta instance exists for node: http://purl.obolibrary.org/obo/HP_0045087 Hip joint hypermobility
12:45:36.112 [main] DEBUG o.m.p.i.o.OboGraphDocumentAdaptor - Converting edges to relationships...
12:45:36.278 [main] DEBUG o.m.p.g.util.CompatibilityChecker - Checking vertices (16581) and edges (20843) for compatibility...
12:45:36.293 [main] DEBUG o.m.p.g.util.CompatibilityChecker - Vertices and edges are compatible!
12:45:36.307 [main] DEBUG o.m.p.g.util.CompatibilityChecker - Graph is simple!
12:45:36.473 [main] DEBUG o.m.phenol.io.OntologyLoader - Parsed a total of 16581 terms
12:45:37.073 [main] DEBUG o.m.s.configuration.SvAnnaBuilder - Including TAD boundaries with stability >80%
12:45:37.073 [main] DEBUG o.m.s.configuration.SvAnnaBuilder - Including VISTA enhancers
12:45:37.073 [main] DEBUG o.m.s.configuration.SvAnnaBuilder - Using `clingen` gene dosage source
12:45:41.473 [main] DEBUG o.m.s.configuration.SvAnnaBuilder - Initializing phenotype term similarity calculator RESNIK_SYMMETRIC
12:45:41.474 [main] DEBUG o.m.s.configuration.SvAnnaBuilder - Using `DATABASE` to get IC of the most informative common ancestor for HPO terms
12:45:41.476 [main] DEBUG o.m.svanna.cli.cmd.PrioritizeCommand - Validating the provided phenotype terms
12:45:41.477 [main] DEBUG o.m.svanna.cli.cmd.PrioritizeCommand - Preparing the top-level phenotype terms for the input terms
12:45:41.482 [main] INFO  o.m.svanna.cli.cmd.PrioritizeCommand - Reading variants from `example.vcf`
12:45:41.549 [main] TRACE o.m.svanna.io.parse.VcfVariantParser - Closing VCF file
12:45:41.549 [main] INFO  o.m.svanna.cli.cmd.PrioritizeCommand - Read 8 variants
12:45:41.549 [main] INFO  o.m.svanna.cli.cmd.PrioritizeCommand - Filtering out the variants with reciprocal overlap >80.0% occurring in more than 1.0% probands
12:45:41.549 [main] INFO  o.m.svanna.cli.cmd.PrioritizeCommand - Filtering out the variants where ALT allele is supported by less than 3 reads
12:45:41.552 [main] DEBUG o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `17`
12:45:41.552 [main] TRACE o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `17-31150798-43110335`
12:45:41.684 [main] DEBUG o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `2`
12:45:41.684 [main] TRACE o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `2-109923337-110405062`
12:45:41.772 [main] DEBUG o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `3`
12:45:41.772 [main] TRACE o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `3-9725702-11007013`
12:45:41.786 [main] DEBUG o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `23`
12:45:41.786 [main] TRACE o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `23-64205190-64206761`
12:45:41.818 [main] DEBUG o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `12`
12:45:41.819 [main] TRACE o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `12-6124705-6124718`
12:45:41.824 [main] DEBUG o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `13`
12:45:41.824 [main] TRACE o.m.s.c.f.PopulationFrequencyAndCoverageFilter - Filtering variants on contig `13-72835296-72835332`
12:45:41.866 [main] DEBUG o.m.s.c.p.SvPrioritizerFactory - Preparing top-level enhancer phenotype terms for the input terms
12:45:47.398 [main] INFO  o.m.svanna.cli.cmd.PrioritizeCommand - Prioritizing 8 variants on 2 threads
12:45:47.559 [main] INFO  o.m.svanna.cli.cmd.PrioritizeCommand - Prioritization finished in 0m 0s (159 ms) processing on average 50.31 items/s
12:45:47.559 [main] INFO  o.m.svanna.cli.cmd.PrioritizeCommand - Writing out the results
12:45:47.578 [main] INFO  o.m.s.c.writer.html.HtmlResultWriter - Writing HTML results to /home/ielis/tmp/svanna-cli-1.0.4/results/example.html
12:45:47.578 [main] DEBUG o.m.s.c.writer.html.HtmlResultWriter - Reporting up to 100 variants sorted by priority
12:45:48.320 [main] INFO  o.m.svanna.cli.cmd.PrioritizeCommand - We're done, bye!

So, crucially, there was no issue during deserialization of the gencode JSON file.

This is the SHA256 checksum of the JSON file:

sha256sum gencode.v38.genes.json.gz 
9d326c5fe6be97602633a1aca68275a61a0bf48b0d62c4b62c843733442eb4ed  gencode.v38.genes.json.gz

The checksum matches the content of the checksum.sha256 of the distribution ZIP.

Can you please check if you get the same checksum?

[JakeHagen]

I was having this issue but it was fixed by using 2304 version of the data zip. I was previously using the 2204 version because I was given an error (error: invalid zip file with overlapped components (possible zip bomb)) when trying to unzip 2304 (I think the checksum is duplicated or something). To get around this, I unpacked the 2304 zip on my local Mac and transferred the directory.

[ChloeL2023]

I got the same error when using 2204 version, so I tried to download the 2304 one.
However, I got the "checksum.sha256: FAILED" after "replace svanna-data/checksum.sha256? [y]es, [n]o, [A]ll, [N]one, [r]ename: y". Do you know the reason?