CSHARP-3985: Support multiple SerializerRegistries #1592

papafe · 2025-01-13T17:26:15Z

sanych-sun · 2025-01-14T03:19:26Z

src/MongoDB.Bson/Serialization/IBsonSerializationDomain.cs

+    /// <summary>
+    /// //TODO
+    /// </summary>
+    public interface IBsonSerializationDomain


Why do we need such super-interface? It has everything in it: configuration, resolving and even reading/writing. Is there any use case when one need all of this together? I would prefer to use 3 separate interfaces: IBsonSerializationBuilder (or similar, to contain everything related to configuration, registration) - when one done with configuration - can call Build method and get IBsonSerializerRegistry (+ we might want to make IBsonSerializerRegistry as readonly for lookup only) and finally there is IBsonSerializer (to contain Serialize-Deserialize methods). So we will separate all use-cases: for application bootstrap/configuration - one have to use IBsonSerializationBuilder, in run-time to resolve the serializer - IBsonSerializerRegistry should be used, and finally to read/write - we have IBsonSerializer itself.

BTW, having IBsonSerializationBuilder is very well aligned with idea of having MongoClientBuilder to create MongoClients. Serialization builder could be part of the bigger MongoClientBuilder.

I think the main problem here is that we're trying to move away from the static super-class that is BsonSerializer without creating a breaking change. At the moment IBsonSerialisationDomain is an interface with everything that BsonSerialiser does, so that we can create an instance class that we can inject where we need. At a later time (possibly we can break this ticket in multiple PRs) this would allow developers to define custom serialization configuration for a certain mongo client/database/collection.
And I agree that this interface does a lot, but so did BsonSerializer and unfortunately it's difficult to separate everything from each other without breaking what's already there, given that almost everything is in the public API and we can't change the behaviour before a new major. For example, one of the issue we have is that the code to work with discriminators is not part of IBsonSerializerRegistry, but it's all over the place.
I feel that moving away from a static class would already be a positive change for the future, and that changing the way we do serialisation is definitely something we should do, but that would require changing the public API and a new major version.
What do you think?
Also @rstam tagging you here so you can follow the conversation 😄

I do understand that we are limited by not allowing to have a breaking changes, but if we decide to introduce a new interface(s), let's make them useful for initiatives that will follow soon. Let's discuss pros and cons of each idea with whole team.

src/MongoDB.Bson/Serialization/BsonSerializationDomain.cs

src/MongoDB.Bson/Serialization/BsonSerializer.cs

rstam · 2025-01-15T20:03:11Z

src/MongoDB.Bson/Serialization/IBsonSerializationDomain.cs

+        /// <summary>
+        /// //TODO
+        /// </summary>
+        IBsonCoreSerializerConfiguration SerializationConfiguration { get; }


I think this property doesn't belong in this interface.

My idea would be that the IBsonCoreSerializer interface would contain everything that can be used to do the serialization and this also includes the configuration necessary for that.

src/MongoDB.Bson/Serialization/IBsonSerializationDomain.cs

papafe · 2025-01-30T20:10:35Z

This is still a super quick and dirty proof of concept, just to verify we can actually create a custom domain and move it along down the call stack. Of course this broke down several things, and works only in certain cases, but it's a starting point.
The main idea here is:

The serialization domain (IBsonSerializationDomain) can be setup at the client/database/collection level
The serialization domain is also added to the serialization context (BsonSerializationContext/BsonDeserializationContext). This way it is passed down wherever it is needed
In order to pass the serialization domain between the collection level and the BsonSerializationContext, the serialization domain is passed down in the MessageEncoderSettings,
From MessageEncoderSettings, the domain is then added to the BsonBinaryWriterSettings in the MessageBinaryEncoderBase.
Finally, the writer settings are read in the serialization context constructor.

(This works similar for `BsonSerializationContext)

I'm not trying to say that this is the perfect way, but I just tried to come up with the "most direct" way to go from the collection to the serialization contexts, so it's mostly a proof of concept.

Also, I still did not touch the conventions, the discriminators and the class maps, that are also statics and need to be part of the serialization domain.

rstam

I didn't review every single file. I tried to pick the most relevant ones.

Overall this looks like the right direction.

But... it's going to be a pain to finish and to review.

src/MongoDB.Bson/IO/BsonReaderSettings.cs

src/MongoDB.Bson/IO/BsonWriterSettings.cs

src/MongoDB.Bson/IO/IBsonReader.cs

src/MongoDB.Bson/Serialization/Conventions/AttributeConventionPack.cs

src/MongoDB.Bson/Serialization/BsonClassMap.cs

src/MongoDB.Bson/Serialization/BsonSerializationContext.cs

src/MongoDB.Bson/Serialization/IBsonSerializationDomain.cs

… and created GetDiscriminatorsForTypeAndSubTypesInternal.

papafe · 2025-07-03T15:26:27Z

Now the PR is open for reviews. There are still a couple of open questions, but given the size of this I thought it would be better to start getting feedback the earliest possible. The next section will be both a summary and partially a guide on how to read the PR itself.

The aim of this PR is to move away from the global serialization settings we use right now (BsonSerializer, BsonClassMap, ....) and move towards a more flexible system that uses instance serialization settings.
This should allow, among the rest:

To have different serialization settings depending on the client/database/collection
To improve the testability of serialization/deserialization, as we'll move away from global state
As a first step in that direction, this PR tries to remove all the use of the global state internally, without changes for the developers using the API. We decided not to make the changes public as we would like to take this occasion for a revamp/simplification/reorganisation of the serialization API.

The main idea behind the PR is to create a serialization domain, represented by the IBsonsSerializationDomain interface, and to find a way to pass this interface around so that all the places in our code in which we refer to the global state, we use instead this instantiated domain.
IBsonSerializationDomain is an interface that contains all the global state that was contained in static classes, namely: BsonSerializer, BsonClassMap, ConventionRegistry, BsonDefaults. The methods and properties of BsonSerializer are directly represented in the interface, while for the other static classes, those are included as properties in IBsonSerializationDomain.
One thing to note is that I tried not to modify the current global serialization API, even though there are various improvements that could be done, including removing unnecessary methods, better grouping of methods by functionality (for example creating a DiscriminatorRegistry) and reducing the use of locks. I decided again doing any king of improvement as this PR is already quite broad, and having changes would have further complicated it. Nevertheless, this is something that can be done in follow up PR, mostly after we have taken a decision on the shape of the public API.

In order for the domain to be accessible where it is needed, I've added a domain property in multiple classes, among which:

MongoClientSettings/MongoDatabaseSettings/MongoClientSettings
BsonSerializationContext/BsonDeserializationContext
BsonReaderSettings/BsonWriterSettings
TranslationContext
MongoQueryProvider
MessageEncoderSettings

These new properties allow to pass down the domain where it is needed. This also required enriching interfaces with methods or properties that use the domain. For public interface, this was obtained by creating an internal interface that derives from the public one, and that contains the new method that take the domain as input, as you can see with IMemberMapConventionInternal for example. In order to keep the implementation of those interfaces hidden from the public API, those interfaces have been implemented explicitly.
Also, in order to keep compatibility with the current API, I have created the static InternalExtensions (in both the Bson and Driver package) to create internal extension methods that select the appropriate method to call depending if a class implements the internal (enriched) interface, or only the public one. For example, take a look at InternalExtensions.ApplyInternal.

Another thing to note is that I did not remove calls to the global state (mostly to BsonSerializer) everywhere I could, as I think in some cases it is the most appropriate. In those cases (for example inside the Authentication.AWS or Driver.Encryption packages ) I think BsonSerializer is used as a container for "default serialization", for example when deserializing keys for encryption. I think in the future we need to create a "default" serialization domain that can't be modified and only contains default serializers/deserializers to be used in those cases. At the moment the use of BsonSerializer is possibly risky, as we do not know how developers are modifying the global settings, even though we don't expect them to modify the most common serializers.

In order to keep the changes private and let Drivers and other packages access the internals of Bson, I had to use the InternalsVisibleToAttribute.

As a final note, I've left lots of comments in the code, in order to remember why certain decisions were taken, and a couple of questions left. Those comments have specific tags so they can be found and removed.

rstam · 2025-07-03T22:26:03Z

src/MongoDB.Bson/IO/IBsonReader.cs

+        /// <summary>
+        /// Gets the settings of the reader.
+        /// </summary>
+        BsonReaderSettings Settings { get; }


I think it was just an omission.

In any case I don't think we need this.

rstam · 2025-07-03T22:26:07Z

src/MongoDB.Bson/IO/IBsonReader.cs

@@ -62,7 +62,7 @@ public interface IBsonReader : IDisposable
        /// <summary>
        /// Pops the settings.
        /// </summary>
-        void PopSettings();
+        void PopSettings(); //TODO Why do we have push and pop methods? They are not used. We should remove them.


Something for a different PR maybe?

rstam · 2025-07-03T22:26:35Z

src/MongoDB.Bson/IO/BsonReader.cs

@@ -25,7 +25,7 @@ namespace MongoDB.Bson.IO
    /// <summary>
    /// Represents a BSON reader for some external format (see subclasses).
    /// </summary>
-    public abstract class BsonReader : IBsonReader
+    public abstract class BsonReader : IBsonReaderInternal


IBsonReaderInternal not needed.

rstam · 2025-07-03T22:27:42Z

src/MongoDB.Bson/IO/BsonReaderSettings.cs

@@ -77,6 +79,16 @@ public BsonReaderSettings FrozenCopy()
            }
        }

+        internal IBsonSerializationDomain SerializationDomain


This should not be a reader setting.

Classes in this directory are low level I/O classes that just do I/O.

Serialization is one level up and is built on TOP of these I/O classes.

rstam · 2025-07-03T22:28:11Z

src/MongoDB.Bson/IO/BsonWriterSettings.cs

+        /// <summary>
+        /// //TODO
+        /// </summary>
+        internal IBsonSerializationDomain SerializationDomain


This should not be a writer setting.

Classes in this directory are low level I/O classes that just do I/O.

Serialization is one level up and is built on TOP of these I/O classes.

rstam · 2025-07-04T00:11:18Z

src/MongoDB.Bson/Serialization/IBsonSerializationDomain.cs

@@ -0,0 +1,317 @@
+using System;


Add copyright.

rstam · 2025-07-04T00:17:08Z

src/MongoDB.Driver/Linq/Linq3Implementation/Misc/TypeExtensions.cs

-        {
-            return type.IsGenericType && type.GetGenericTypeDefinition() == typeof(Nullable<>);
-        }
+        // Commented out because there is an identical method in Bson assembly (and also in this assembly...).


See CSHARP-5632.

The duplication between Bson and Driver was because they were internal.

Be careful if you remove any of the duplication here. I don't think all the duplicate methods are 100% identical and there might be subtle reasons why (but I don't know them if there are).

rstam · 2025-07-04T00:20:33Z

src/MongoDB.Driver/Linq/Linq3Implementation/Translators/DiscriminatorAstExpression.cs

    {
-        var discriminator = discriminatorConvention.GetDiscriminator(nominalType, actualType);
+        var discriminator = discriminatorConvention.GetDiscriminatorInternal(nominalType, actualType, serializationDomain);


It's going the VERY difficult and fragile to remember NOT to call GetDiscriminator and to call GetDiscriminatorInternal (or whatever we decide to call it) instead...

rstam · 2025-07-04T00:24:20Z

src/MongoDB.Driver/Support/InternalExtensions.cs

+
+namespace MongoDB.Driver.Support
+{
+    internal static class InternalExtensions


I would prefer to see this in a file called IPipelineStageDefinitionExtensions.cs.

I also would not mix extension methods for different types in the same file (that didn't happen here though, but it did in some other file).

rstam · 2025-07-04T00:28:15Z

src/MongoDB.Driver/MongoDB.Driver.csproj

-    <Compile Include="..\MongoDB.Shared\SequenceComparer.cs" Link="Shared\SequenceComparer.cs" />
-    <Compile Include="..\MongoDB.Shared\Hasher.cs" Link="Shared\Hasher.cs" />
-  </ItemGroup>
+<!--  The followings have been removed because they are accessed directly through MongoDB.Bson, otherwise there will be a double definition   -->


Did you mean "INDIRECTLY through MongoDB.Bson"?

InternalsVisibleTo sucks.

sanych-sun reviewed Jan 14, 2025

View reviewed changes

rstam self-requested a review January 14, 2025 17:11

rstam requested changes Jan 15, 2025

View reviewed changes

papafe requested review from rstam and sanych-sun January 31, 2025 11:14

rstam requested changes Feb 7, 2025

View reviewed changes

papafe force-pushed the csharp3985 branch from a275180 to 945f07b Compare May 26, 2025 07:07

papafe marked this pull request as ready for review May 26, 2025 14:57

papafe requested a review from a team as a code owner May 26, 2025 14:57

papafe marked this pull request as draft May 26, 2025 14:58

papafe removed the request for review from a team May 26, 2025 15:04

papafe added 18 commits June 30, 2025 11:17

Rebase commit

3c99f6d

Corrected type

d087b6c

Various improvements, and added test for discriminators.

975545a

Removed comment.

ecd84a8

Small corrections.

a4d92cc

Small correction.

7280ac5

Corrected test

27ccb7b

Correction for test

e589695

Correction

5efa4da

Removed new tests untul refactoring

962861f

Added domain to MongoQueryProvider

0b92d63

Small fix

6d5fb86

Corrected freeze

d1585d1

Small correction

5e6931c

Fixed BsonClassMap.Freeze and Convention Apply

e6b8b35

Small fix.

818dc7e

Corrected test

543c396

Made conventions internal

f07be62

papafe added 15 commits June 30, 2025 11:18

Small things

4a5220a

Builders example

697784b

Removed references to certain parts of BsonDefaults.

97fd948

Various fixes

ecf6f92

Moved to internal 1

7271903

More internal

e513938

Removed final internal

4c5bdcc

Improvements

714c419

Small comments

c796201

Some comments

0f7cc7b

Added getDiscriminatorInternal

c56f972

Merged IBsonSerializationDomainInternal into IBsonSerializationDomain…

83ad951

… and created GetDiscriminatorsForTypeAndSubTypesInternal.

Missing piece.

34dce8a

Fix for getDicriminatorConvention and IBsonIdProvider

7311b35

Big serializer check

e9511c0

papafe force-pushed the csharp3985 branch from 9f2a9c2 to e9511c0 Compare June 30, 2025 09:26

papafe added 7 commits June 30, 2025 12:01

Fix for testing

a783914

Added comment

4c7d7ac

Small fix

ee4410a

Various fixes about RenderArgs plus new tests

a31265a

Small fix

ef6d8bd

Simplified tests

e7a703e

Small fixes

b86494b

papafe requested a review from rstam July 3, 2025 15:27

papafe marked this pull request as ready for review July 3, 2025 15:27

papafe requested a review from BorisDog July 3, 2025 15:27

papafe added 2 commits July 3, 2025 17:36

Removed example builder and tests

d32b3ac

Various test fixes

61ec09c

rstam requested changes Jul 4, 2025

View reviewed changes

CSHARP-3985: Support multiple SerializerRegistries #1592

Are you sure you want to change the base?

CSHARP-3985: Support multiple SerializerRegistries #1592

Conversation

papafe commented Jan 13, 2025

Uh oh!

sanych-sun Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

papafe commented Jan 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rstam left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

papafe commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sanych-sun Jan 14, 2025 •

edited

Loading

papafe commented Jan 30, 2025 •

edited

Loading

papafe commented Jul 3, 2025 •

edited

Loading