Azure cloud solutions are built on virtual machines (emulation of physical computer hardware), which enable agile packaging of software deployments and better resource consolidation than physical hardware. Docker containers and the docker ecosystem has dramatically expanded the ways you can develop, ship and manage distributed software. Application code in a container is isolated from the host VM and other containers on the same VM. This isolation gives you more development and deployment agility.
Azure offers the following Docker values:
- Many different ways to create Docker hosts for containers to suit your situation
- The Azure Container Service creates clusters of container hosts using orchestrators such as marathon and swarm.
- Azure Resource Manager and resource group templates to simplify deploying and updating complex distributed applications
- integration with a large array of both proprietary and open-source configuration management tools
And because you can programmatically create VMs and Linux containers on Azure, you can also use VM and container orchestration tools to create groups of Virtual Machines (VMs) and to deploy applications inside both Linux containers and now Windows Containers.
This article not only discusses these concepts at a high level, it also contains tons of links to more information, tutorials, and products related to container and cluster usage on Azure. If you know all this, and just want the links, they're right here at tools for working with containers.
Virtual machines run inside an isolated hardware virtualization environment provided by a hypervisor. In Azure, the Virtual Machines service handles all that for you: You create Virtual Machines by choosing the operating system and configuring —or by uploading a custom VM image. Virtual Machines are a time-tested, "battle-hardened" technology, and there are many tools available to manage the OS and apps they contain. Apps in a VM are hidden from the host OS. From the point of view of an application or user on a VM, the VM appears to be an autonomous physical computer.
Linux containers and those created and hosted using docker tools, do not use a hypervisor to provide isolation. With containers, the container host uses process and file system isolation features of the Linux kernel to expose to the container, its apps, certain kernel features and its own isolated file system. From the point of view of an app running inside a container, the container appears to be a unique OS instance. A contained app cannot see processes or any other resources outside of its container.
Far fewer resources are used in a Docker container than are used in a VM. Docker containers employ an application isolation and execution model which does not share the kernel of the Docker host. The container has much lower disk footprint as it doesn’t include the entire OS. Start-up time and required disk space are significantly lower than in a VM. Windows Containers provide the same advantages as Linux containers for apps that run on Windows. Windows Containers support the Docker image format and Docker API, but they can also be managed using PowerShell. Two container runtimes are available with Windows Containers, Windows Server Containers and Hyper-V Containers. Hyper-V Containers provide an additional layer of isolation by hosting each container in a super-optimized VM. To learn more about Windows Containers see About Windows Containers. To get started with Windows Containers in Azure, learn how to deploy an Azure Container Service cluster.
Containers can improve:
- The speed application code can be developed and shared widely
- The speed and confidence an app can be tested
- The speed and confidence an app can be deployed
Containers execute on a container host—an operating system, and in Azure that means an Azure Virtual Machine. Even if you already love the idea of containers, you're still going to need a VM infrastructure hosting the containers, but the benefits are that containers do not care on which VM they are running (although whether the container wants a Linux or Windows execution environment will be important, for example).
They're great for many things, but they encourage—as do Azure Cloud Services and Azure Service Fabric—the creation of single-service, microservice-oriented distributed applications, in which application design is based on more small, composable parts rather than on larger, more strongly coupled components.
This is especially true in public cloud environments like Azure, in which you rent VMs when and where you want them. Not only do you get isolation and rapid deployment and orchestration tools, but you can make more efficient application infrastructure decisions.
For example, you might currently have a deployment consisting of 9 Azure VMs of a large size for a highly-available, distributed application. If the components of this application can be deployed in containers, you might be able to use only 4 VMs and deploy your application components inside 20 containers for redundancy and load balancing.
This is just an example, of course, but if you can do this in your scenario, you can adjust to usage spikes with more containers rather than more Azure VMs, and use the remaining overall CPU load much more efficiently than before.
In addition, there are many scenarios that do not lend themselves to a microservices approach; you will know best whether microservices and containers will help you.
In general, it's easy to see that container technology is a step forward, but there are more specific benefits as well. Let's take the example of Docker containers. This topic will not dive deeply into Docker right now (read What is Docker? for that story, or wikipedia), but Docker and its ecosystem offer tremendous benefits to both developers and IT professionals.
Developers take to Docker containers quickly, because above all it makes using Linux and Windows containers easy:
- They can use simple, incremental commands to create a fixed image that is easy to deploy and can automate building those images using a dockerfile
- They can share those images easily using simple, git-style push and pull commands to public or private docker registries
- They can think of isolated application components instead of computers
- They can use a large number of tools that understand docker containers and different base images
IT and operations professionals also benefit from the combination of containers and virtual machines.
- contained services are isolated from VM host execution environment
- contained code is verifiably identical
- contained services can be started, stopped, and moved quickly between development, test, and production environments
Features like these—and there are more—excite established businesses, where professional information technology organizations have the job of fitting resources—including pure processing power—to the tasks required to not only stay in business, but increase customer satisfaction and reach. Small businesses, ISVs, and startups have exactly the same requirement, but they might describe it differently.
Virtual machines provide the backbone of cloud computing, and that doesn't change. If virtual machines start more slowly, have a larger disk footprint, and do not map directly to a microservices architecture, they do have very important benefits:
- By default, they have much more robust default security protections for host computer
- They support any major OS and application configurations
- They have longstanding tool ecosystems for command and control
- They provide the execution environment to host containers
The last item is important, because a contained application still requires a specific operating system and CPU type, depending upon the calls the application will make. It's important to remember that you install containers on VMs because they contain the applications you want to deploy; containers are not replacements for VMs or operating systems.
The following table describes at a very high level the kind of feature differences that—without much extra work—exist between VMs and Linux containers. Note that some features maybe more or less desirable depending upon your own application needs, and that as with all software, extra work provides increased feature support, especially in the area of security.
| Feature | VMs | Containers |
|---|---|---|
| "Default" security support | to a greater degree | to a slightly lesser degree |
| Memory on disk required | Complete OS plus apps | App requirements only |
| Time taken to start up | Substantially Longer: Boot of OS plus app loading | Substantially shorter: Only apps need to start because kernel is already running |
| Portability | Portable With Proper Preparation | Portable within image format; typically smaller |
| Image Automation | Varies widely depending on OS and apps | Docker registry; others |
At this point, any architect, developer, or IT operations specialist might be thinking, "I can automate ALL of this; this really IS Data-Center-As-A-Service!".
You're right, it can be, and there are any number of systems, many of which you may already use, that can either manage groups of Azure VMs and inject custom code using scripts, often with the CustomScriptingExtension for Windows or the CustomScriptingExtension for Linux. You can—and perhaps already have—automated your Azure deployments using PowerShell or Azure CLI scripts.
These abilities are often then migrated to tools like Puppet and Chef to automate the creation of and configuration for VMs at scale. (Here are some links to using these tools with Azure.)
More recently, Azure released the Azure resource management REST API, and updated PowerShell and Azure CLI tools to use it easily. You can deploy, modify, or redeploy entire application topologies using Azure Resource Manager templates with the Azure resource management API using:
- the Azure portal using templates—hint, use the "DeployToAzure" button
- the Azure CLI
- the Azure PowerShell modules
There are several popular systems that can deploy entire groups of VMs and install Docker (or other Linux container host systems) on them as an automatable group. For direct links, see the containers and tools section, below. There are several systems that do this to a greater or lesser extent, and this list is not exhaustive. Depending upon your skill set and scenarios, they may or may not be useful.
Docker has its own set of VM-creation tools (docker-machine) and a load-balancing, docker-container cluster management tool (swarm). In addition, the Azure Docker VM Extension comes with default support for docker-compose, which can deploy configured application containers across multiple containers.
In addition, you can try out Mesosphere's Data Center Operating System (DCOS). DCOS is based on the open-source mesos "distributed systems kernel" that enables you to treat your datacenter as one addressable service. DCOS has built-in packages for several important systems such as Spark and Kafka (and others) as well as built-in services such as Marathon (a container control system) and Chronos (a distributed scheduler). Mesos was derived from lessons learned at Twitter, AirBnb, and other web-scale businesses. You can also use swarm as the orchestration engine.
Also, kubernetes is an open-source system for VM and container group management derived from lessons learned at Google. You can even use kubernetes with weave to provide networking support.
Deis is an open source "Platform-as-a-Service" (PaaS) that makes it easy to deploy and manage applications on your own servers. Deis builds upon Docker and CoreOS to provide a lightweight PaaS with a Heroku-inspired workflow. You can easily create a 3-Node Azure VM group and install Deis on Azure and then install a Hello World Go application.
CoreOS, a Linux distribution with an optimized footprint, Docker support, and their own container system called rkt, also has a container group management tool called fleet.
Ubuntu, another very popular Linux distribution, supports Docker very well, but also supports Linux (LXC-style) clusters.
Working with containers and Azure VMs uses tools. This section provides a list of only some of the most useful or important concepts and tools about containers, groups, and the larger configuration and orchestration tools used with them.
Note
This area is changing amazingly rapidly, and while we will do our best to keep this topic and its links up to date, it might well be an impossible task. Make sure you search on interesting subjects to keep up to date!
Some Linux container technologies:
Windows Container links:
Visual Studio Docker links:
Docker tools:
Docker on Microsoft Azure:
- Docker VM Extension for Linux on Azure
- Azure Docker VM Extension User Guide
- Using the Docker VM Extension from the Azure Command-line Interface (Azure CLI)
- Using the Docker VM Extension from the Azure portal
- How to use docker-machine on Azure
- How to use docker with swarm on Azure
- Get Started with Docker and Compose on Azure
- Using an Azure resource group template to create a Docker host on Azure quickly
- The built-in support for
composefor contained applications - Implement a Docker private registry on Azure
Linux distributions and Azure examples:
Configuration, cluster management, and container orchestration:
-
Deis
-
Kubernetes
-
Powershell DSC for Linux
Check out Docker and Windows Containers.