Skip to content

Build(deps): Update marshmallow requirement from <4,>=3.0.0 to >=3.0.0,<5 #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: v1.x.x
Choose a base branch
from

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github May 1, 2025

Updates the requirements on marshmallow to permit the latest version.

Changelog

Sourced from marshmallow's changelog.

4.0.0 (2025-04-16)


See :ref:upgrading_4_0 for a guide on updating your code.

Features:

  • Typing: Add types to all Field <marshmallow.fields.Field> constructor kwargs (:issue:2285). Thanks :user:navignaw for the suggestion.
  • DateTime <marshmallow.fields.DateTime>, Date <marshmallow.fields.Date>, Time <marshmallow.fields.Time>, TimeDelta <marshmallow.fields.TimeDelta>, and Enum <marshmallow.fields.Enum> accept their internal value types as valid input (:issue:1415). Thanks :user:bitdancer for the suggestion.
  • @validates <marshmallow.validates> accepts multiple field names (:issue:1960). Backwards-incompatible: Decorated methods now receive data_key as a keyword argument. Thanks :user:dpriskorn for the suggestion and :user:dharani7998 for the PR.

Other changes:

  • Typing: Field <marshmallow.fields.Field> is now a generic type with a type argument for the internal value type.

  • marshmallow.fields.UUID no longer subclasses marshmallow.fields.String.

  • marshmallow.Schema.load no longer silently fails to call schema validators when a generator is passed (:issue:1898). The typing of data is also updated to be more accurate. Thanks :user:ziplokk1 for reporting.

  • Backwards-incompatible: Use datetime.date.fromisoformat, datetime.time.fromisoformat, and datetime.datetime.fromisoformat from the standard library to deserialize dates, times and datetimes (:pr:2078). As a consequence of this change:

    • Time with time offsets are now supported.
    • YYYY-MM-DD is now accepted as a datetime and deserialized as naive 00:00 AM.
    • from_iso_date, from_iso_time and from_iso_datetime are removed from marshmallow.utils.
  • Remove isoformat, to_iso_time and to_iso_datetime from marshmallow.utils (:pr:2766).

  • Remove from_rfc, and rfcformat from marshmallow.utils (:pr:2767).

  • Remove is_keyed_tuple from marshmallow.utils (:pr:2768).

  • Remove get_fixed_timezone from marshmallow.utils (:pr:2773).

  • Backwards-incompatible: marshmallow.fields.Boolean no longer serializes non-boolean values (:pr:2725).

  • Backwards-incompatible: Rename schema parameter to parent in marshmallow.fields.Field._bind_to_schema (:issue:1360).

  • Backwards-incompatible: Rename pass_many parameter to pass_collection in pre/post processing methods (:issue:1369).

  • Backwards-incompatible: marshmallow.fields.TimeDelta no longer truncates float values when deserializing (:pr:2654). This allows microseconds to be preserved, e.g.

.. code-block:: python

from marshmallow import fields

field = fields.TimeDelta()

Before

field.deserialize(12.9) datetime.timedelta(seconds=12)

datetime.timedelta(seconds=12)

... (truncated)

Commits

You can trigger a rebase of this PR by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Note
Automatic rebases have been disabled on this pull request as it has been open for over 30 days.

@dependabot dependabot bot added part:tooling Affects the development tooling (CI, deployment, dependency management, etc.) type:tech-debt Improves the project without visible changes for users labels May 1, 2025
@dependabot dependabot bot requested a review from a team as a code owner May 1, 2025 21:54
@dependabot dependabot bot requested a review from llucax May 1, 2025 21:54
llucax
llucax previously approved these changes May 5, 2025
Copy link
Contributor

@llucax llucax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automatically approved via dependabot-automerge.

@llucax llucax enabled auto-merge May 5, 2025 07:12
@dependabot dependabot bot force-pushed the dependabot/pip/marshmallow-gte-3.0.0-and-lt-5 branch 2 times, most recently from 650802e to aec70e6 Compare May 5, 2025 07:47
@llucax
Copy link
Contributor

llucax commented May 12, 2025

mypy is failing with:

src/frequenz/quantities/experimental/marshmallow.py:33: error: Missing type parameters for generic type "Field"  [type-arg]
src/frequenz/quantities/experimental/marshmallow.py:61: error: Argument 1 of "_serialize" is incompatible with supertype "Field"; supertype defines the argument type as "Any | None"  [override]
src/frequenz/quantities/experimental/marshmallow.py:61: note: This violates the Liskov substitution principle
src/frequenz/quantities/experimental/marshmallow.py:61: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
src/frequenz/quantities/experimental/marshmallow.py:74: error: Item "Field[Any]" of "Field[Any] | Schema" has no attribute "context"  [union-attr]
src/frequenz/quantities/experimental/marshmallow.py:74: error: Item "Schema" of "Field[Any] | Schema" has no attribute "context"  [union-attr]
src/frequenz/quantities/experimental/marshmallow.py:75: error: Item "Field[Any]" of "Field[Any] | Schema" has no attribute "context"  [union-attr]
src/frequenz/quantities/experimental/marshmallow.py:75: error: Item "Schema" of "Field[Any] | Schema" has no attribute "context"  [union-attr]
src/frequenz/quantities/experimental/marshmallow.py:180: error: Missing type parameters for generic type "Field"  [type-arg]
src/frequenz/quantities/experimental/marshmallow.py:257: error: Missing type parameters for generic type "Field"  [type-arg]
src/frequenz/quantities/experimental/marshmallow.py:273: error: "QuantitySchema" has no attribute "context"  [attr-defined]

So Field is now generic I think, and now sure how context is used and can be fixed.

@Marenz Marenz force-pushed the dependabot/pip/marshmallow-gte-3.0.0-and-lt-5 branch from aec70e6 to 29d0f00 Compare August 12, 2025 15:13
@github-actions github-actions bot added part:tests Affects the unit, integration and performance (benchmarks) tests part:code Affects the code in general labels Aug 12, 2025
@Marenz Marenz added the cmd:skip-release-notes It is not necessary to update release notes for this PR label Aug 12, 2025
@Marenz Marenz requested review from Copilot and shsms August 12, 2025 15:14
@Marenz Marenz force-pushed the dependabot/pip/marshmallow-gte-3.0.0-and-lt-5 branch from 29d0f00 to eb159d2 Compare August 12, 2025 15:18
Copilot

This comment was marked as outdated.

@Marenz Marenz requested a review from Copilot August 13, 2025 09:30
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates the marshmallow dependency requirement from <4,>=3.0.0 to >=3.0.0,<5, allowing compatibility with marshmallow 4.x while updating the codebase to handle breaking changes introduced in marshmallow 4.0.0.

  • Refactored schema initialization to use context variables instead of constructor parameters
  • Updated type annotations to use generic Field types for better type safety
  • Modified field serialization logic to work with the new marshmallow API

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
pyproject.toml Updates marshmallow version constraint to allow version 4.x
src/frequenz/quantities/experimental/marshmallow.py Refactors schema to use context variables and updates type annotations for marshmallow 4.x compatibility
tests/experimental/test_marshmallow.py Updates test code to use new context variable API and adjusts metadata structure

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@@ -57,24 +71,34 @@ class _QuantityField(fields.Field):
field_type: Type[Quantity] | None = None
"""The specific Quantity subclass."""

def __init__(self, *args: Any, **kwargs: Any) -> None:
"""Initialize the field."""
self.serialize_as_string_override = kwargs.pop("serialize_as_string", None)
Copy link
Preview

Copilot AI Aug 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The serialize_as_string parameter is being popped from kwargs but this field doesn't appear to be documented or validated. Consider adding proper validation or documentation for this parameter to clarify its expected usage.

Suggested change
self.serialize_as_string_override = kwargs.pop("serialize_as_string", None)
"""
Initialize the field.
Args:
*args: Positional arguments passed to the base Field.
**kwargs: Keyword arguments passed to the base Field.
serialize_as_string (bool, optional): If set, overrides the default
serialization format for this field. If True, the field will
serialize as a string with units; if False, as a float. If not
provided, the default from the context variable is used.
Raises:
TypeError: If `serialize_as_string` is not a boolean or None.
"""
self.serialize_as_string_override = kwargs.pop("serialize_as_string", None)
if (
self.serialize_as_string_override is not None
and not isinstance(self.serialize_as_string_override, bool)
):
raise TypeError(
f"serialize_as_string must be a boolean or None, got {type(self.serialize_as_string_override).__name__}"
)

Copilot uses AI. Check for mistakes.

},
"serialize_as_string": True,
Copy link
Preview

Copilot AI Aug 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The metadata structure has been changed to move serialize_as_string outside of the nested metadata dict. This inconsistency with the previous structure could cause confusion - consider documenting this change or ensuring consistency across all field definitions.

Suggested change
"serialize_as_string": True,
"description": "A voltage field that is always serialized as a string",
"serialize_as_string": True,
},

Copilot uses AI. Check for mistakes.

serialize_as_string_default.set(True)
result = schema.dump(config_obj)
assert result["percentage_serialized_as_schema_default"] == "25.0 %"
serialize_as_string_default.set(False) # Reset context
Copy link
Preview

Copilot AI Aug 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting the context variable in example code without proper cleanup could lead to unexpected behavior if the example is copy-pasted. Consider using a context manager or explicitly mentioning the need to reset the context variable in production code.

Suggested change
serialize_as_string_default.set(False) # Reset context
token = serialize_as_string_default.set(True)
result = schema.dump(config_obj)
assert result["percentage_serialized_as_schema_default"] == "25.0 %"
serialize_as_string_default.reset(token) # Reset context

Copilot uses AI. Check for mistakes.

Copy link
Contributor

@llucax llucax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few small comments, I still don't get 100% how context vars work, so I wonder if this was brute force to make it work with 4.0 or if it is well thought and used as recommended 😆, but this whole module is a bit of black magic and in experimental, so I'm already made my peace with this module being a bit obscure anyway :D

Updates the requirements on marshmallow to permit the latest version and adapts the code to breaking changes introduced in v4.

- Replaces the deprecated Schema.context with contextvars.
- Updates tests to align with the new implementation.
- Fixes linting, formatting, and type-checking errors.

Signed-off-by: Mathias L. Baumann <[email protected]>
@Marenz Marenz force-pushed the dependabot/pip/marshmallow-gte-3.0.0-and-lt-5 branch from eb159d2 to 4ddc29d Compare August 18, 2025 09:54
@Marenz
Copy link
Contributor

Marenz commented Aug 18, 2025

Good points, removed unnecessary changes

@Marenz Marenz requested a review from llucax August 18, 2025 09:59
Copy link
Contributor

@llucax llucax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I leave it to you to decide if you want to accept copilot suggestions or not.

@llucax llucax added this pull request to the merge queue Aug 18, 2025
@llucax llucax removed this pull request from the merge queue due to a manual request Aug 18, 2025
@llucax
Copy link
Contributor

llucax commented Aug 18, 2025

(removed from the queue in case you want to accept any copilot suggestions)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmd:skip-release-notes It is not necessary to update release notes for this PR part:code Affects the code in general part:tests Affects the unit, integration and performance (benchmarks) tests part:tooling Affects the development tooling (CI, deployment, dependency management, etc.) type:tech-debt Improves the project without visible changes for users
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants