[FLINK-32817] Harnessing Jackson for Secure Serialization of YarnLocalResourceDescriptor

[YesOrNo828]

What is the purpose of the change

If you try to submit a flink jar to a yarn cluster with a filename that contains spaces, the task won't be able to parse the file path in YarnLocalResourceDescriptor. This will result in warning logs being printed by TaskExecutor repeatedly.
To avoid this issue, I plan to use Jackson for serializing and deserializing the YarnLocalResourceDescriptor instance, which will make the process safer.

Brief change log

• Using the JscksonMapperFactory to create an ObjectMapper in the YarnLocalResourceDescriptor
• To convert the YarnLocalResourceDescriptor into a JSON string, and utilize the ObjectMapper for serialization and deserialization.
• Add some UTs

Verifying this change

This change added tests and can be verified as follows:

• Modified the YarnLocalResourceDescriptionTest about the file name containing a space
• Added more malformed string tests

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: yes
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: Yarn
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? (yes / no) no
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[flinkbot]

CI report:

• b1b1dc9 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[YesOrNo828]

Hello @xintongsong , if you have a moment, could you kindly review this PR? Thank you.

[xintongsong]

@flinkbot run azure

[xintongsong]

@YesOrNo828, please take a look into the CI failures. The failures are all from the Yarn deployment, which are likely related to the changes of this PR.

[YesOrNo828]

@xintongsong Thanks for your confirmation, I'm working on that.

[YesOrNo828]

Yarn nodemanager exports the environment variables that are inconsistent with the submitted string, the double-quotes are removed after exporting environment variables, so YarnLocalResourceDescriptor couldn't parse the string which fetched from System.getenv().

e.g.:

$ export _FLINK_DIST_="{"resourceKey":"flink-dist-1.19-SNAPSHOT.jar","path":"hdfs://flink-dist-1.19-SNAPSHOT.jar","size":126733882,"modificationTime":1693301926167,"visibility":"APPLICATION","resourceType":"FILE"}"
$ echo $_FLINK_DIST_
{resourceKey:flink-dist-1.19-SNAPSHOT.jar,path:hdfs://flink-dist-1.19-SNAPSHOT.jar,size:126733882,modificationTime:1693301926167,visibility:APPLICATION,resourceType:FILE}

I found out Jackson allow to configure custom quote character to serialize Object, for example: single-quote, later I'll figure out the exactly way to serialize the string.

[YesOrNo828]

@flinkbot run azure

[YesOrNo828]

@xintongsong The CI is a success now. Do you have time to take a look?

[xintongsong]

Thanks, @YesOrNo828. LGTM. Merging.