Skip to content

Commit

Permalink
Set up export/import action mixin
Browse files Browse the repository at this point in the history
  • Loading branch information
TheSuperiorStanislav committed Jan 21, 2025
1 parent 89dc78a commit 037dba3
Show file tree
Hide file tree
Showing 24 changed files with 868 additions and 323 deletions.
6 changes: 6 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,9 @@ repos:
pass_filenames: false
types: [python]
stages: [pre-push]
- id: doc_build_verify
name: verify that docs could be build
entry: inv docs.build
language: system
pass_filenames: false
stages: [pre-push]
11 changes: 8 additions & 3 deletions HISTORY.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,12 @@ UNRELEASED
------------------

* Add explicit `created_by` argument to `CeleryResourceMixin` and pass it in
`ExportJobSerializer` validation
`ExportJobSerializer` validation
* Add export/import action mixins `api.mixins.ExportStartActionMixin`
and `api.mixins.ImportStartActionMixin`
* Add `api.views.BaseExportJobViewSet`, `BaseExportJobForUsersViewSet`,
`api.views.BaseImportJobViewSet` and `BaseImportJobForUsersViewSet` for
job management

1.3.1 (2025-01-13)
------------------
Expand All @@ -21,8 +26,8 @@ UNRELEASED
* Small actions definition refactor in `ExportJobViewSet/ExportJobViewSet` to allow easier overriding.
* Add support for ordering in `export`
* Add settings for DjangoFilterBackend and OrderingFilter in export api.
`DRF_EXPORT_DJANGO_FILTERS_BACKEND` with default `django_filters.rest_framework.DjangoFilterBackend` and
`DRF_EXPORT_ORDERING_BACKEND` with default `rest_framework.filters.OrderingFilter`.
`DRF_EXPORT_DJANGO_FILTERS_BACKEND` with default `django_filters.rest_framework.DjangoFilterBackend` and
`DRF_EXPORT_ORDERING_BACKEND` with default `rest_framework.filters.OrderingFilter`.

1.2.0 (2024-12-26)
------------------
Expand Down
Binary file added docs/_static/images/action-bands-openapi.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 18 additions & 0 deletions docs/api_drf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,27 @@ API (Rest Framework)
.. autoclass:: import_export_extensions.api.ExportJobForUserViewSet
:members:

.. autoclass:: import_export_extensions.api.BaseImportJobViewSet
:members:

.. autoclass:: import_export_extensions.api.BaseExportJobViewSet
:members:

.. autoclass:: import_export_extensions.api.BaseImportJobForUserViewSet
:members:

.. autoclass:: import_export_extensions.api.BaseExportJobForUserViewSet
:members:

.. autoclass:: import_export_extensions.api.LimitQuerySetToCurrentUserMixin
:members:

.. autoclass:: import_export_extensions.api.ImportStartActionMixin
:members:

.. autoclass:: import_export_extensions.api.ExportStartActionMixin
:members:

.. autoclass:: import_export_extensions.api.CreateExportJob
:members: create, validate

Expand Down
20 changes: 19 additions & 1 deletion docs/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ a ``status page``, where you can monitor the progress of the import/export proce
A screenshot of Django Admin export status page

Import/Export API
-----------------
----------------------------------------------------------------

The ``api.views.ExportJobViewSet`` and ``api.views.ImportJobViewSet`` are provided to create
the corresponding viewsets for the resource.
Expand Down Expand Up @@ -169,3 +169,21 @@ the OpenAPI specification will be available.
.. figure:: _static/images/bands-openapi.png

A screenshot of the generated OpenAPI specification


Import/Export API actions mixins
----------------------------------------------------------------

Alternatively you can use ``api.mixins.ExportStartActionMixin`` and ``api.mixins.ImportStartActionMixin``
to add to your current viewsets ability to create import/export jobs.
You would also need to use ``api.views.BaseExportJobViewSet`` or ``BaseExportJobForUsersViewSet``
and ``api.views.BaseImportJobViewSet`` or ``BaseImportJobForUsersViewSet`` to setup endpoints to be able to:

* ``list`` - Returns a list of jobs for the ``resource_class`` set in ViewSet.
* ``retrieve`` - Returns details of a job based on the provided ID.
* ``cancel`` - Stops the import/export process and sets the job's status to ``CANCELLED``.
* ``confirm`` - Confirms the import after the parse stage. This action is available only in import jobs.

.. figure:: _static/images/action-bands-openapi.png

A screenshot of the generated OpenAPI specification
14 changes: 13 additions & 1 deletion docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,25 @@ By default, it uses the `mimetypes.types_map <https://docs.python.org/3/library/
from Python's mimetypes module.

``STATUS_UPDATE_ROW_COUNT``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Defines the number of rows after import/export of which the task status is
updated. This helps to increase the speed of import/export. The default value
is 100. This parameter can be specified separately for each resource by adding
``status_update_row_count`` to its ``Meta``.

``DRF_EXPORT_DJANGO_FILTERS_BACKEND``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Specifies filter backend class for ``django-filters`` in export action.
Default: ``django_filters.rest_framework.DjangoFilterBackend``

``DRF_EXPORT_ORDERING_BACKEND``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Specifies filter backend class for ``ordering`` in export action.
Default: ``rest_framework.filters.OrderingFilter``

Settings from django-import-export
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Additionally, the package supports settings from the original django-import-export package.
Expand Down
2 changes: 1 addition & 1 deletion docs/migrate_from_original_import_export.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ install the package by following the :ref:`the installation guide<Installation a
Then, all you need to do is update the base classes for your resource and admin models.

Migrate resources
-----------------
----------------------------------------------------------------

To enable import/export via Celery, simply replace the base resource classes from the original package
with ``CeleryResource`` or ``CeleryModelResource`` from ``django-import-export-extensions``:
Expand Down
10 changes: 9 additions & 1 deletion import_export_extensions/api/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
from .mixins import LimitQuerySetToCurrentUserMixin
from .mixins import (
ExportStartActionMixin,
ImportStartActionMixin,
LimitQuerySetToCurrentUserMixin,
)
from .serializers import (
CreateExportJob,
CreateImportJob,
Expand All @@ -8,6 +12,10 @@
ProgressSerializer,
)
from .views import (
BaseExportJobForUserViewSet,
BaseExportJobViewSet,
BaseImportJobForUserViewSet,
BaseImportJobViewSet,
ExportJobForUserViewSet,
ExportJobViewSet,
ImportJobForUserViewSet,
Expand Down
3 changes: 3 additions & 0 deletions import_export_extensions/api/mixins/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from .common import LimitQuerySetToCurrentUserMixin
from .export_mixins import ExportStartActionMixin
from .import_mixins import ImportStartActionMixin
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@ class LimitQuerySetToCurrentUserMixin:

def get_queryset(self):
"""Return user's jobs."""
if self.action == "start":
if self.action in (
getattr(self, "import_action", ""),
getattr(self, "export_action", ""),
):
# To make it consistent and for better support of drf-spectacular
return super().get_queryset() # pragma: no cover
return (
Expand Down
141 changes: 141 additions & 0 deletions import_export_extensions/api/mixins/export_mixins.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
import collections.abc
import contextlib
import typing

from django.conf import settings
from django.utils import module_loading

from rest_framework import (
decorators,
request,
response,
status,
)

from ... import resources
from .. import serializers


class ExportStartActionMixin:
"""Mixin which adds start export action."""

resource_class: type[resources.CeleryModelResource]
export_action = "start_export_action"
export_action_name = "export"
export_action_url = "export"
export_detail_serializer_class = serializers.ExportJobSerializer
export_ordering: collections.abc.Sequence[str] = ()
export_ordering_fields: collections.abc.Sequence[str] = ()
export_open_api_description = (
"This endpoint creates export job and starts it. "
"To monitor progress use detail endpoint for jobs to fetch state of "
"job. Once it's status is `EXPORTED`, you can download file."
)

def __init_subclass__(cls) -> None:
super().__init_subclass__()
# Skip if it is has no resource_class specified
if not hasattr(cls, "resource_class"):
return
filter_backends = [
module_loading.import_string(
settings.DRF_EXPORT_DJANGO_FILTERS_BACKEND,
),
]
if cls.export_ordering_fields:
filter_backends.append(
module_loading.import_string(
settings.DRF_EXPORT_ORDERING_BACKEND,
),
)

def start_export_action(
self: "ExportStartActionMixin",
request: request.Request,
) -> response.Response:
return self.start_export(request)

setattr(cls, cls.export_action, start_export_action)
decorators.action(
methods=["POST"],
url_name=cls.export_action_name,
url_path=cls.export_action_url,
detail=False,
queryset=cls.resource_class.get_model_queryset(),
serializer_class=cls().get_export_create_serializer_class(),
filterset_class=getattr(
cls.resource_class,
"filterset_class",
None,
),
filter_backends=filter_backends,
ordering=cls.export_ordering,
ordering_fields=cls.export_ordering_fields,
)(getattr(cls, cls.export_action))
# Correct specs of drf-spectacular if it is installed
with contextlib.suppress(ImportError):
from drf_spectacular import utils

utils.extend_schema_view(
**{
cls.export_action: utils.extend_schema(
description=cls.export_open_api_description,
filters=True,
responses={
status.HTTP_201_CREATED: cls().get_export_detail_serializer_class(), # noqa: E501
},
),
},
)(cls)

def get_queryset(self):
"""Return export model queryset on export action.
For better openapi support and consistency.
"""
if self.action == self.export_action:
return self.resource_class.get_model_queryset() # pragma: no cover
return super().get_queryset()

def get_export_detail_serializer_class(self):
"""Get serializer which will be used show details of export job."""
return self.export_detail_serializer_class

def get_export_create_serializer_class(self):
"""Get serializer which will be used to start export job."""
return serializers.get_create_export_job_serializer(
self.resource_class,
)

def get_export_resource_kwargs(self) -> dict[str, typing.Any]:
"""Provide extra arguments to resource class."""
return {}

def get_serializer(self, *args, **kwargs):
"""Provide resource kwargs to serializer class."""
if self.action == self.export_action:
kwargs.setdefault(
"resource_kwargs",
self.get_export_resource_kwargs(),
)
return super().get_serializer(*args, **kwargs)

def start_export(self, request: request.Request) -> response.Response:
"""Validate request data and start ExportJob."""
ordering = request.query_params.get("ordering", "")
if ordering:
ordering = ordering.split(",")
serializer = self.get_serializer(
data=request.data,
ordering=ordering,
filter_kwargs=request.query_params,
)
serializer.is_valid(raise_exception=True)
export_job = serializer.save()
return response.Response(
data=self.get_export_detail_serializer_class()(
instance=export_job,
).data,
status=status.HTTP_201_CREATED,
)
Loading

0 comments on commit 037dba3

Please sign in to comment.