Refactor installation pages (apache#18282)

This PR splits ad improves the installation-related documentation
for Airflow. The "installation" page had become overloaded
wiht everything-but-the-kitchen-sink and it became rather
difficult to navigate and link to relevant sections.

Also there was not a single page where one could have an overview
on different installation methods possible, cases when
each instalation works best as well as understanding what
is involved in following each installation method in terms of
maintenance, and expectations that users should have when it
comes to what Apache Airflow Community provides.

The PR leaves the installation page as basically a summary of
all installation methods with all above explained and links
to detailed pages explaining prerequisites, dependencies,
database setup and supported versions.

dev/README_RELEASE_AIRFLOW.md

@@ -728,14 +728,9 @@ Dear Airflow community,
 
 I'm happy to announce that Airflow ${VERSION} was just released.
 
-The source release, as well as the binary "sdist" release, are available
-here:
+The released sources and packages can be downloaded via https://airflow.apache.org/installation/installing-from-sources.html
 
-https://dist.apache.org/repos/dist/release/airflow/${VERSION}/
-
-We also made this version available on PyPI for convenience (`pip install apache-airflow`):
-
-https://pypi.python.org/pypi/apache-airflow
+Other installation methods are described in https://airflow.apache.org/installation/
 
 The documentation is available on:
 https://airflow.apache.org/

dev/README_RELEASE_HELM_CHART.md

@@ -569,9 +569,9 @@ Dear Airflow community,
 
 I am pleased to announce that we have released Apache Airflow Helm chart $VERSION 🎉 🎊
 
-The source release, as well as the "binary" Helm Chart release, is available here https://downloads.apache.org/airflow/helm-chart/$VERSION/
+The source release, as well as the "binary" Helm Chart release, are available:
 
-📦   Official Source: https://downloads.apache.org/airflow/helm-chart/$VERSION
+📦   Official Sources: https://airflow.apache.org/helm-chart/installing-helm-chart-from-sources.html
 📦   ArtifactHub: https://artifacthub.io/packages/helm/apache-airflow/airflow
 📚   Docs: https://airflow.apache.org/docs/helm-chart/$VERSION/
 

dev/README_RELEASE_PROVIDER_PACKAGES.md

@@ -805,15 +805,11 @@ I'm happy to announce that new versions of Airflow Providers packages were just
 
 The source release, as well as the binary releases, are available here:
 
-https://dist.apache.org/repos/dist/release/airflow/providers/
+https://airflow.apache.org/docs/apache-airflow-providers/installing-from-sources
 
-We also made those versions available on PyPi for convenience ('pip install apache-airflow-providers-*'):
+You can install the providers via PyPI  https://airflow.apache.org/apache-airflow-providers/installing-from-pypi
 
-https://pypi.org/search/?q=apache-airflow-providers
-
-The documentation is available at https://airflow.apache.org/docs/ and linked from the PyPI packages:
-
-<PASTE TWINE UPLOAD LINKS HERE. SORT THEM BEFORE!>
+The documentation is available at https://airflow.apache.org/docs/ and linked from the PyPI packages.
 
 Cheers,
 <your name>

docs/README.rst

@@ -122,7 +122,7 @@ Role ``:class:`` works well with references between packages. If you want to use
 
 .. code-block:: rst
 
-    :doc:`apache-airflow:installation`
+    :doc:`apache-airflow:installation/index`
     :ref:`apache-airflow-providers-google:write-logs-stackdriver`
 
 If you still feel confused then you can view more possible roles for our documentation:

docs/apache-airflow-providers-amazon/operators/_partials/prerequisite_tasks.rst

@@ -26,7 +26,7 @@ To use these operators, you must do a few things:
 
       pip install 'apache-airflow[amazon]'
 
-    Detailed information is available :doc:`apache-airflow:installation`
+    Detailed information is available :doc:`apache-airflow:installation/index`
 
   * :doc:`Setup Connection </connections/aws>`.
 

docs/apache-airflow-providers-google/operators/_partials/prerequisite_tasks.rst

@@ -28,6 +28,6 @@ To use these operators, you must do a few things:
 
       pip install 'apache-airflow[google]'
 
-    Detailed information is available for :doc:`Installation <apache-airflow:installation>`.
+    Detailed information is available for :doc:`Installation <apache-airflow:installation/index>`.
 
   * :doc:`Setup a Google Cloud Connection </connections/gcp>`.

docs/apache-airflow-providers-microsoft-azure/operators/_partials/prerequisite_tasks.rst

@@ -26,7 +26,7 @@ To use these operators, you must do a few things:
 
       pip install 'apache-airflow[azure]'
 
-    Detailed information is available :doc:`apache-airflow:installation`
+    Detailed information is available :doc:`apache-airflow:installation/index`
 
   * :doc:`Setup Connection </connections/azure>`.
 

docs/apache-airflow-providers/index.rst

@@ -143,7 +143,7 @@ package.
 
 Each community provider has corresponding extra which can be used when installing airflow to install the
 provider together with ``Apache Airflow`` - for example you can install airflow with those extras:
-``apache-airflow[google,amazon]`` (with correct constraints -see :doc:`apache-airflow:installation`) and you
+``apache-airflow[google,amazon]`` (with correct constraints -see :doc:`apache-airflow:installation/index`) and you
 will install the appropriate versions of the ``apache-airflow-providers-amazon`` and
 ``apache-airflow-providers-google`` packages together with ``Apache Airflow``.
 
@@ -417,10 +417,5 @@ specific Airflow versions.
     Operators and hooks <operators-and-hooks-ref/index>
     Core Extensions <core-extensions/index>
     Update community providers <howto/create-update-providers>
-
-.. toctree::
-    :maxdepth: 2
-    :hidden:
-    :caption: Resources
-
     Installing from sources <installing-from-sources>
+    Installing from PyPI <installing-from-pypi>

docs/apache-airflow-providers/installing-from-pypi.rst

@@ -0,0 +1,45 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+ ..   http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+Installation from PyPI
+----------------------
+
+.. contents:: :local:
+
+
+This page describes installations using the ``apache-airflow-providers`` package `published in
+PyPI <https://pypi.org/search/?q=apache-airflow-providers>`__.
+
+Installation tools
+''''''''''''''''''
+
+Only ``pip`` installation is currently officially supported.
+
+While there are some successes with using other tools like `poetry <https://python-poetry.org/>`_ or
+`pip-tools <https://pypi.org/project/pip-tools/>`_, they do not share the same workflow as
+``pip`` - especially when it comes to constraint vs. requirements management.
+Installing via ``Poetry`` or ``pip-tools`` is not currently supported. If you wish to install airflow
+using those tools you should use the constraints and convert them to appropriate
+format and workflow that your tool requires.
+
+Typical command to install airflow from PyPI looks like below (you need to use the right Airflow version and Python version):
+
+.. code-block::
+
+    pip install "apache-airflow-providers-celery" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.1.4/constraints-3.6.txt"
+
+This is an example, see :doc:`apache-airflow:installation/installing-from-pypi` for more examples, including how to upgrade the providers.

docs/apache-airflow-providers/installing-from-sources.rst

@@ -36,6 +36,11 @@ Released packages
          </ul>
 
 
+    You can also install ``Apache Airflow Providers`` - as most Python packages - via :doc:`PyPI <installing-from-pypi>`.
+    You can choose different version of Airflow by selecting different version from the drop-down at
+    the top-left of the page.
+
+
 Release integrity
 '''''''''''''''''
 

docs/apache-airflow/extra-packages-ref.rst

@@ -18,7 +18,7 @@
 Reference for package extras
 ''''''''''''''''''''''''''''
 
-Here's the list of all the :ref:`extra dependencies <installation:airflow_extra_dependencies>`.
+Here's the list of all the extra dependencies of Apache Airflow.
 
 The entries with ``*`` in the ``Preinstalled`` column indicate that those extras (providers) are always
 pre-installed when Airflow is installed.

docs/apache-airflow/index.rst

@@ -77,9 +77,8 @@ unit of work and continuity.
     project
     license
     start/index
-    installation
-    upgrading-to-2
-    upgrade-check
+    installation/index
+    upgrading-from-1-10/index
     tutorial
     tutorial_taskflow_api
     howto/index
@@ -117,9 +116,3 @@ unit of work and continuity.
     Configurations <configurations-ref>
     Extra packages <extra-packages-ref>
     Database Migrations <migrations-ref>
-
-.. toctree::
-    :hidden:
-    :caption: Resources
-
-    Installing from sources <installing-from-sources>