Skip to content

Update Object Storage tutorial to match learning path tone#49170

Merged
kaxil merged 1 commit intoapache:mainfrom
cmarteepants:tutorial-object-storage
Apr 14, 2025
Merged

Update Object Storage tutorial to match learning path tone#49170
kaxil merged 1 commit intoapache:mainfrom
cmarteepants:tutorial-object-storage

Conversation

@cmarteepants
Copy link
Collaborator

@cmarteepants cmarteepants commented Apr 12, 2025

This PR updates the Object Storage API tutorial to:

  • Use a friendlier, more instructive tone consistent with the other tutorials
  • Highlight the “why” behind using ObjectStoragePath across cloud providers
  • Walk users through a real-world example: fetching API data, storing to S3, analyzing with DuckDB
  • Clarify how to use Airflow connections and URL-based paths
  • Preserve all code snippets exactly as-is

Related: #45649

@cmarteepants cmarteepants changed the title Tutorial object storage Update Object Storage tutorial to match learning path tone Apr 13, 2025
@cmarteepants cmarteepants marked this pull request as ready for review April 13, 2025 00:36
@cmarteepants cmarteepants requested a review from kaxil April 13, 2025 00:36
Copy link
Member

@kaxil kaxil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment about relative links; rest lgtm

- Rewrote and restructured the TaskFlow tutorial to improve tone and readability
- Added sections: How XComs Work, Running Your DAG, Task Parameterization, What’s Next
- Clarified DAG definition, task declaration, and dependency wiring
- Added old vs new code comparison to highlight TaskFlow improvements
- Moved advanced features (e.g. Docker, Kubernetes, sensors) to a dedicated section
- Added explanatory intros before each advanced example
- Updated internal links and headings for consistency
@kaxil kaxil merged commit 9534c06 into apache:main Apr 14, 2025
49 checks passed
@cmarteepants cmarteepants deleted the tutorial-object-storage branch April 14, 2025 15:54
kaxil added a commit to astronomer/airflow that referenced this pull request Apr 14, 2025
Was removed by mistake in apache#49170 and caused failures on main due to cross references

```

============================== apache-airflow-providers-standard ==============================
------------------------------ Error   1 --------------------
 WARNING: undefined label: 'taskflow/accessing_context_variables'

File path: /opt/airflow/providers/standard/docs/operators/bash.rst:85

  80 |             :language: python
  81 |             :dedent: 4
  82 |             :start-after: [START howto_operator_bash_template]
  83 |             :end-before: [END howto_operator_bash_template]
  84 |
> 85 | Using the ``@task.bash`` TaskFlow decorator allows you to return a formatted string and take advantage of
  86 | having all :ref:`execution context variables directly accessible to decorated tasks <taskflow/accessing_context_variables>`.
  87 |
  88 | .. exampleinclude:: /../../../airflow-core/src/airflow/example_dags/example_bash_decorator.py
  89 |     :language: python
  90 |     :dedent: 4
```
kaxil added a commit to astronomer/airflow that referenced this pull request Apr 14, 2025
Was removed by mistake in apache#49170 and caused failures on main due to cross references

```

============================== apache-airflow-providers-standard ==============================
------------------------------ Error   1 --------------------
 WARNING: undefined label: 'taskflow/accessing_context_variables'

File path: /opt/airflow/providers/standard/docs/operators/bash.rst:85

  80 |             :language: python
  81 |             :dedent: 4
  82 |             :start-after: [START howto_operator_bash_template]
  83 |             :end-before: [END howto_operator_bash_template]
  84 |
> 85 | Using the ``@task.bash`` TaskFlow decorator allows you to return a formatted string and take advantage of
  86 | having all :ref:`execution context variables directly accessible to decorated tasks <taskflow/accessing_context_variables>`.
  87 |
  88 | .. exampleinclude:: /../../../airflow-core/src/airflow/example_dags/example_bash_decorator.py
  89 |     :language: python
  90 |     :dedent: 4
```
kaxil added a commit that referenced this pull request Apr 14, 2025
Was removed by mistake in #49170 and caused failures on main due to cross references

```

============================== apache-airflow-providers-standard ==============================
------------------------------ Error   1 --------------------
 WARNING: undefined label: 'taskflow/accessing_context_variables'

File path: /opt/airflow/providers/standard/docs/operators/bash.rst:85

  80 |             :language: python
  81 |             :dedent: 4
  82 |             :start-after: [START howto_operator_bash_template]
  83 |             :end-before: [END howto_operator_bash_template]
  84 |
> 85 | Using the ``@task.bash`` TaskFlow decorator allows you to return a formatted string and take advantage of
  86 | having all :ref:`execution context variables directly accessible to decorated tasks <taskflow/accessing_context_variables>`.
  87 |
  88 | .. exampleinclude:: /../../../airflow-core/src/airflow/example_dags/example_bash_decorator.py
  89 |     :language: python
  90 |     :dedent: 4
```
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request May 29, 2025
Was removed by mistake in apache/airflow#49170 and caused failures on main due to cross references

```

============================== apache-airflow-providers-standard ==============================
------------------------------ Error   1 --------------------
 WARNING: undefined label: 'taskflow/accessing_context_variables'

File path: /opt/airflow/providers/standard/docs/operators/bash.rst:85

  80 |             :language: python
  81 |             :dedent: 4
  82 |             :start-after: [START howto_operator_bash_template]
  83 |             :end-before: [END howto_operator_bash_template]
  84 |
> 85 | Using the ``@task.bash`` TaskFlow decorator allows you to return a formatted string and take advantage of
  86 | having all :ref:`execution context variables directly accessible to decorated tasks <taskflow/accessing_context_variables>`.
  87 |
  88 | .. exampleinclude:: /../../../airflow-core/src/airflow/example_dags/example_bash_decorator.py
  89 |     :language: python
  90 |     :dedent: 4
```

GitOrigin-RevId: f8c3bbcfa67594197a6efe461c856797f24687f4
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 24, 2025
Was removed by mistake in apache/airflow#49170 and caused failures on main due to cross references

```

============================== apache-airflow-providers-standard ==============================
------------------------------ Error   1 --------------------
 WARNING: undefined label: 'taskflow/accessing_context_variables'

File path: /opt/airflow/providers/standard/docs/operators/bash.rst:85

  80 |             :language: python
  81 |             :dedent: 4
  82 |             :start-after: [START howto_operator_bash_template]
  83 |             :end-before: [END howto_operator_bash_template]
  84 |
> 85 | Using the ``@task.bash`` TaskFlow decorator allows you to return a formatted string and take advantage of
  86 | having all :ref:`execution context variables directly accessible to decorated tasks <taskflow/accessing_context_variables>`.
  87 |
  88 | .. exampleinclude:: /../../../airflow-core/src/airflow/example_dags/example_bash_decorator.py
  89 |     :language: python
  90 |     :dedent: 4
```

GitOrigin-RevId: f8c3bbcfa67594197a6efe461c856797f24687f4
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Oct 22, 2025
Was removed by mistake in apache/airflow#49170 and caused failures on main due to cross references

```

============================== apache-airflow-providers-standard ==============================
------------------------------ Error   1 --------------------
 WARNING: undefined label: 'taskflow/accessing_context_variables'

File path: /opt/airflow/providers/standard/docs/operators/bash.rst:85

  80 |             :language: python
  81 |             :dedent: 4
  82 |             :start-after: [START howto_operator_bash_template]
  83 |             :end-before: [END howto_operator_bash_template]
  84 |
> 85 | Using the ``@task.bash`` TaskFlow decorator allows you to return a formatted string and take advantage of
  86 | having all :ref:`execution context variables directly accessible to decorated tasks <taskflow/accessing_context_variables>`.
  87 |
  88 | .. exampleinclude:: /../../../airflow-core/src/airflow/example_dags/example_bash_decorator.py
  89 |     :language: python
  90 |     :dedent: 4
```

GitOrigin-RevId: f8c3bbcfa67594197a6efe461c856797f24687f4
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Feb 28, 2026
Was removed by mistake in apache/airflow#49170 and caused failures on main due to cross references

```

============================== apache-airflow-providers-standard ==============================
------------------------------ Error   1 --------------------
 WARNING: undefined label: 'taskflow/accessing_context_variables'

File path: /opt/airflow/providers/standard/docs/operators/bash.rst:85

  80 |             :language: python
  81 |             :dedent: 4
  82 |             :start-after: [START howto_operator_bash_template]
  83 |             :end-before: [END howto_operator_bash_template]
  84 |
> 85 | Using the ``@task.bash`` TaskFlow decorator allows you to return a formatted string and take advantage of
  86 | having all :ref:`execution context variables directly accessible to decorated tasks <taskflow/accessing_context_variables>`.
  87 |
  88 | .. exampleinclude:: /../../../airflow-core/src/airflow/example_dags/example_bash_decorator.py
  89 |     :language: python
  90 |     :dedent: 4
```

GitOrigin-RevId: f8c3bbcfa67594197a6efe461c856797f24687f4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants