Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jackson CSV not taking in account the first column #208

Open
herveDarritchon opened this issue Jul 11, 2020 · 4 comments
Open

Jackson CSV not taking in account the first column #208

herveDarritchon opened this issue Jul 11, 2020 · 4 comments
Labels

Comments

@herveDarritchon
Copy link

herveDarritchon commented Jul 11, 2020

Jackson Version : 2.11.1

In Kotlin, I have a csv file (simple)

EAN;LIB ARTICLE;QUANTITE;TAXE DEEE
3571211210979;CABLE ETH DRT CAT6 3M;0;0.00836

I have some data class

data class StockElement(
    val ean: Long?,
    val label: String?,
    val quantity: Int?,
    val deeTax: Float?
)

and a simple main code

        val raw = File("src/test/resources/test-stock-file.csv").readText()

        val mapper = CsvMapper()
            .registerModule(KotlinModule())

        val schema = CsvMapper()
            .schemaFor(StockElement::class.java)
            .withHeader()
            .withColumnSeparator(';')

        val iterator: MappingIterator<StockElement> =
            mapper.readerFor(StockElement::class.java)
                .with(schema)
                .readValues(raw)

        val stockElements: List<StockElement> = iterator.readAll()

This code throws a strange error at runtime :

Cannot deserialize value of type `java.lang.Long` from String "CABLE ETH DRT CAT6 3M": not a valid Long value
 at [Source: (StringReader); line: 2, column: 15] (through reference chain: com.orange.ccmd.katalog.domain.catalog.si.stock.StockElement["ean"])
com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.lang.Long` from String "CABLE ETH DRT CAT6 3M": not a valid Long value
 at [Source: (StringReader); line: 2, column: 15] (through reference chain: com.orange.ccmd.katalog.domain.catalog.si.stock.StockElement["ean"])

It is like the first column is not processed by Jackson :(

Very strange ... I think I am doing something wrong.

Any help appreciated.

@cowtowncoder
Copy link
Member

I can have a look, but I have one question first: I think example is missing something (perhaps @JsonProperty annotations?) to link header column names to value class property names?
(if used as shown, I would expect different kind of exception, as header names and property names do not match)

@herveDarritchon
Copy link
Author

herveDarritchon commented Jul 12, 2020

@cowtowncode
Thanks for answering. Following your advice I tried something.

I have done some simplification and have added @JsonProperty to my Data Class

data class StockElement(
    @JsonProperty("EAN")
    val ean: Long?,

    @JsonProperty("LIB ARTICLE")
    val label: String?,

    @JsonProperty("QUANTITE")
    val quantity: Int?,

    @JsonProperty(value = "POIDS", required = false)
    val weight: Int?
)

and my test method

       val raw = """
            EAN;LIB ARTICLE;QUANTITE;POIDS
            3571211210979;CABLE ETH DRT CAT6 13M;0;836
            3571211210979;CABLE ETH DRT CAT6 3M;0;836
        """.trimIndent()

        val mapper = CsvMapper()
            .registerModule(KotlinModule())

        val schema = CsvMapper()
            .typedSchemaFor(StockElement::class.java)
            .withHeader()
            .withColumnSeparator(';')

        val iterator: MappingIterator<StockElement> =
            mapper.readerFor(StockElement::class.java)
                .with(schema)
                .readValues(raw)

        val stockElements: List<StockElement> = iterator.readAll()

Result is the same :

Unrecognized field "ean" (class com.orange.ccmd.katalog.domain.catalog.si.stock.StockElement), not marked as ignorable (4 known properties: "POIDS", "EAN", "LIB ARTICLE", "QUANTITE"])
 at [Source: (StringReader); line: 3, column: 1] (through reference chain: com.orange.ccmd.katalog.domain.catalog.si.stock.StockElement["ean"])
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "ean" (class com.orange.ccmd.katalog.domain.catalog.si.stock.StockElement), not marked as ignorable (4 known properties: "POIDS", "EAN", "LIB ARTICLE", "QUANTITE"])
 at [Source: (StringReader); line: 3, column: 1] (through reference chain: com.orange.ccmd.katalog.domain.catalog.si.stock.StockElement["ean"]

What I understand, it's Jackson uses JsonProperty and is searching for "EAN, POIDS, ..." when unmarshalling but don't find its. I don't understand why because there are present on the string I use as input :(

@herveDarritchon
Copy link
Author

Yeah, found a way to do it ... not sure if it is the simplest ... but it works

I only changed stuff in the Data Class, I think it is necessary because of Kotlin :)

data class StockElement(

    @get:JsonProperty("EAN")
    @param:JsonProperty("EAN")
    val ean: String?,

    @get:JsonProperty("LIB ARTICLE")
    @param:JsonProperty("LIB ARTICLE")
    val label: String?,

    @get:JsonProperty("QUANTITY")
    @param:JsonProperty("QUANTITY")
    val quantity: Int?,

    @get:JsonProperty("TAXE DEE")
    @param:JsonProperty("TAXE DEE")
    val deeeTax: String?
)

@cowtowncoder
Copy link
Member

Ok. Not sure why multiple annotations are needed (although yes, it is sort of Kotlin-specific), but I may have a clue: when creating CsvSchema, name is taken from getter (introspection used as if serializing). So getter (or field if used as getter) needs to have that information linked at least.
But then for deserialization setter (or field if used as setter) needs same information.
Usually this works fine either way, as getter/setter are linked together anyway. But maybe something in this particular case is not matching...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants