Merge shade and os-client-config into the tree

This sucks in the git history for both projects, then moves their files
in place. It should not introduce any behavior changes to any of the
existing openstacksdk code, nor to openstack.config and openstack.cloud
- other than the name change.

TODO(shade) comments have been left indicating places where further
integration work should be done.

It should not be assumed that these are the final places for either to
live. This is just about getting them in-tree so we can work with them.

The enforcer code for reasons surpassing understanding does not work
with python setup.py build_sphinx but it does work with sphinx-build
(what?) For now turn it off. We can turn it back on once the build
sphinx job is migrated to the new PTI.

Change-Id: I9523e4e281285360c61e9e0456a8e07b7ac1243c

Author: emonty

.gitignore

@@ -29,6 +29,7 @@ cover/*
 .tox
 nosetests.xml
 .testrepository
+.stestr
 
 # Translations
 *.mo

.mailmap

@@ -1,3 +1,6 @@
 # Format is:
 # <preferred e-mail> <other e-mail 1>
-# <preferred e-mail> <other e-mail 2>
+# <preferred e-mail> <other e-mail 2>
+<[email protected]> <[email protected]>
+<[email protected]> <[email protected]>
+<[email protected]> <[email protected]>

.stestr.conf

@@ -0,0 +1,3 @@
+[DEFAULT]
+test_path=./openstack/tests/unit
+top_dir=./

.testr.conf

@@ -1,16 +1,45 @@
-If you would like to contribute to the development of OpenStack,
-you must follow the steps in this page:
+.. _contributing:
 
-   https://docs.openstack.org/infra/manual/developers.html
+===================================
+Contributing to python-openstacksdk
+===================================
 
-Once those steps have been completed, changes to OpenStack
-should be submitted for review via the Gerrit tool, following
-the workflow documented at:
+If you're interested in contributing to the python-openstacksdk project,
+the following will help get you started.
 
-   https://docs.openstack.org/infra/manual/developers.html#development-workflow
+Contributor License Agreement
+-----------------------------
 
+.. index::
+   single: license; agreement
+
+In order to contribute to the python-openstacksdk project, you need to have
+signed OpenStack's contributor's agreement.
+
+Please read `DeveloperWorkflow`_ before sending your first patch for review.
 Pull requests submitted through GitHub will be ignored.
 
-Bugs should be filed on Launchpad, not GitHub:
+.. seealso::
+
+   * http://wiki.openstack.org/HowToContribute
+   * http://wiki.openstack.org/CLA
+
+.. _DeveloperWorkflow: http://docs.openstack.org/infra/manual/developers.html#development-workflow
+
+Project Hosting Details
+-------------------------
+
+Project Documentation
+    http://docs.openstack.org/sdks/python/openstacksdk/
+
+Bug tracker
+    https://bugs.launchpad.net/python-openstacksdk
+
+Mailing list (prefix subjects with ``[sdk]`` for faster responses)
+    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
+
+Code Hosting
+    https://git.openstack.org/cgit/openstack/python-openstacksdk
 
-   https://bugs.launchpad.net/python-openstacksdk
+Code Review
+    https://review.openstack.org/#/q/status:open+project:openstack/python-openstacksdk,n,z

CONTRIBUTING.rst

@@ -1,4 +1,49 @@
-python-openstacksdk Style Commandments
-======================================
+openstacksdk Style Commandments
+===============================
 
-Read the OpenStack Style Commandments https://docs.openstack.org/hacking/latest/
+Read the OpenStack Style Commandments
+http://docs.openstack.org/developer/hacking/
+
+Indentation
+-----------
+
+PEP-8 allows for 'visual' indentation. Do not use it. Visual indentation looks
+like this:
+
+.. code-block:: python
+
+  return_value = self.some_method(arg1, arg1,
+                                  arg3, arg4)
+
+Visual indentation makes refactoring the code base unneccesarily hard.
+
+Instead of visual indentation, use this:
+
+.. code-block:: python
+
+  return_value = self.some_method(
+      arg1, arg1, arg3, arg4)
+
+That way, if some_method ever needs to be renamed, the only line that needs
+to be touched is the line with some_method. Additionaly, if you need to
+line break at the top of a block, please indent the continuation line
+an additional 4 spaces, like this:
+
+.. code-block:: python
+
+  for val in self.some_method(
+          arg1, arg1, arg3, arg4):
+      self.do_something_awesome()
+
+Neither of these are 'mandated' by PEP-8. However, they are prevailing styles
+within this code base.
+
+Unit Tests
+----------
+
+Unit tests should be virtually instant. If a unit test takes more than 1 second
+to run, it is a bad unit test. Honestly, 1 second is too slow.
+
+All unit test classes should subclass `openstack.tests.unit.base.BaseTestCase`. The
+base TestCase class takes care of properly creating `OpenStackCloud` objects
+in a way that protects against local environment.

HACKING.rst

@@ -1,36 +1,119 @@
-OpenStack Python SDK
-====================
+openstacksdk
+============
 
-The ``python-openstacksdk`` is a collection of libraries for building
-applications to work with OpenStack clouds. The project aims to provide
-a consistent and complete set of interactions with OpenStack's many
-services, along with complete documentation, examples, and tools.
+openstacksdk is a client library for for building applications to work
+with OpenStack clouds. The project aims to provide a consistent and
+complete set of interactions with OpenStack's many services, along with
+complete documentation, examples, and tools.
 
-This SDK is under active development, and in the interests of providing
-a high-quality interface, the APIs provided in this release may differ
-from those provided in future release.
+It also contains a simple interface layer. Clouds can do many things, but
+there are probably only about 10 of them that most people care about with any
+regularity. If you want to do complicated things, the per-service oriented
+portions of the SDK are for you. However, if what you want is to be able to
+write an application that talks to clouds no matter what crazy choices the
+deployer has made in an attempt to be more hipster than their self-entitled
+narcissist peers, then the ``openstack.cloud`` layer is for you.
 
-Usage
------
+A Brief History
+---------------
 
-The following example simply connects to an OpenStack cloud and lists
-the containers in the Object Store service.::
+openstacksdk started its life as three different libraries: shade,
+os-client-config and python-openstacksdk.
 
-   from openstack import connection
-   conn = connection.Connection(auth_url="http://openstack:5000/v3",
-                                project_name="big_project",
-                                username="SDK_user",
-                                password="Super5ecretPassw0rd")
-   for container in conn.object_store.containers():
-      print(container.name)
+``shade`` started its life as some code inside of OpenStack Infra's nodepool
+project, and as some code inside of Ansible. Ansible had a bunch of different
+OpenStack related modules, and there was a ton of duplicated code. Eventually,
+between refactoring that duplication into an internal library, and adding logic
+and features that the OpenStack Infra team had developed to run client
+applications at scale, it turned out that we'd written nine-tenths of what we'd
+need to have a standalone library.
 
-Documentation
--------------
+``os-client-config`` was a library for collecting client configuration for
+using an OpenStack cloud in a consistent and comprehensive manner.
+In parallel, the python-openstacksdk team was working on a library to expose
+the OpenStack APIs to developers in a consistent and predictable manner. After
+a while it became clear that there was value in both a high-level layer that
+contains business logic, a lower-level SDK that exposes services and their
+resources as Python objects, and also to be able to make direct REST calls
+when needed with a properly configured Session or Adapter from python-requests.
+This led to the merger of the three projects.
 
-Documentation is available at
-https://developer.openstack.org/sdks/python/openstacksdk/
+The contents of the shade library have been moved into ``openstack.cloud``
+and os-client-config has been moved in to ``openstack.config``. The next
+release of shade will be a thin compatibility layer that subclasses the objects
+from ``openstack.cloud`` and provides different argument defaults where needed
+for compat. Similarly the next release of os-client-config will be a compat
+layer shim around ``openstack.config``.
 
-License
--------
+openstack.config
+================
 
-Apache 2.0
+``openstack.config`` will find cloud configuration for as few as 1 clouds and
+as many as you want to put in a config file. It will read environment variables
+and config files, and it also contains some vendor specific default values so
+that you don't have to know extra info to use OpenStack
+
+* If you have a config file, you will get the clouds listed in it
+* If you have environment variables, you will get a cloud named `envvars`
+* If you have neither, you will get a cloud named `defaults` with base defaults
+
+Sometimes an example is nice.
+
+Create a ``clouds.yaml`` file:
+
+.. code-block:: yaml
+
+     clouds:
+      mordred:
+        region_name: Dallas
+        auth:
+          username: 'mordred'
+          password: XXXXXXX
+          project_name: 'shade'
+          auth_url: 'https://identity.example.com'
+
+Please note: ``openstack.config`` will look for a file called ``clouds.yaml``
+in the following locations:
+
+* Current Directory
+* ``~/.config/openstack``
+* ``/etc/openstack``
+
+More information at https://developer.openstack.org/sdks/python/openstacksdk/users/config
+
+openstack.cloud
+===============
+
+Create a server using objects configured with the ``clouds.yaml`` file:
+
+.. code-block:: python
+
+    import openstack.cloud
+
+    # Initialize and turn on debug logging
+    openstack.cloud.simple_logging(debug=True)
+
+    # Initialize cloud
+    # Cloud configs are read with openstack.config
+    cloud = openstack.openstack_cloud(cloud='mordred')
+
+    # Upload an image to the cloud
+    image = cloud.create_image(
+        'ubuntu-trusty', filename='ubuntu-trusty.qcow2', wait=True)
+
+    # Find a flavor with at least 512M of RAM
+    flavor = cloud.get_flavor_by_ram(512)
+
+    # Boot a server, wait for it to boot, and then do whatever is needed
+    # to get a public ip for it.
+    cloud.create_server(
+        'my-server', image=image, flavor=flavor, wait=True, auto_ip=True)
+
+Links
+=====
+
+* `Issue Tracker <https://storyboard.openstack.org/#!/project/760>`_
+* `Code Review <https://review.openstack.org/#/q/status:open+project:openstack/python-openstacksdk,n,z>`_
+* `Documentation <https://developer.openstack.org/sdks/python/openstacksdk/>`_
+* `PyPI <https://pypi.python.org/pypi/python-openstacksdk/>`_
+* `Mailing list <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>`_

README.rst

@@ -0,0 +1,8 @@
+# This is a cross-platform list tracking distribution packages needed by tests;
+# see http://docs.openstack.org/infra/bindep/ for additional information.
+
+build-essential [platform:dpkg]
+python-dev [platform:dpkg]
+python-devel [platform:rpm]
+libffi-dev [platform:dpkg]
+libffi-devel [platform:rpm]

bindep.txt

@@ -0,0 +1,54 @@
+# Install and configure **openstacksdk** library in devstack
+#
+# To enable openstacksdk in devstack add an entry to local.conf that looks like
+#
+# [[local|localrc]]
+# enable_plugin openstacksdk git://git.openstack.org/openstack/python-openstacksdk
+
+function preinstall_openstacksdk {
+    :
+}
+
+function install_openstacksdk {
+    if use_library_from_git "python-openstacksdk"; then
+        # don't clone, it'll be done by the plugin install
+        setup_dev_lib "python-openstacksdk"
+    else
+        pip_install "python-openstacksdk"
+    fi
+}
+
+function configure_openstacksdk {
+    :
+}
+
+function initialize_openstacksdk {
+    :
+}
+
+function unstack_openstacksdk {
+    :
+}
+
+function clean_openstacksdk {
+    :
+}
+
+# This is the main for plugin.sh
+if [[ "$1" == "stack" && "$2" == "pre-install" ]]; then
+    preinstall_openstacksdk
+elif [[ "$1" == "stack" && "$2" == "install" ]]; then
+    install_openstacksdk
+elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
+    configure_openstacksdk
+elif [[ "$1" == "stack" && "$2" == "extra" ]]; then
+    initialize_openstacksdk
+fi
+
+if [[ "$1" == "unstack" ]]; then
+    unstack_openstacksdk
+fi
+
+if [[ "$1" == "clean" ]]; then
+    clean_openstacksdk
+fi