OpenStack - the beauty of the Beast - part 1
Introduction
Almost two years ago I embarked on a challenge which led me through the depths of OpenStack and the hardship of managing a multi-region OpenStack deployment. OpenStack is a beast, really, it can get enormous even with just "basic" cloud features. For example, if I want to launch a Virtual Machine (VM) in the cloud and have it display a static webpage via HTTP it will, at least, need the following from the cloud provider:
- a way to identify a cloud user
- a way to store images to boot the VM
- hypervisor nodes to launch the VM in
- some kind of scheduler to select to which hypervisor the VM should be allocated to
- network virtualization on top of the physical network layer, so our user network is separated from other users
- some way to access the VM from the exterior
- a way for all of the services to communicate with each other
- a place to store the state of the cloud at any given moment
That said, there is a lot of service interaction, and they are amazingly independent services that can do their part on their own, and in the end it works (mostly) as expected and it is somewhat straightforward to use.
We can use OpenStack for managing VMs (nova), containers (magnum) or bare metal (ironic) hosts. In this first part of presenting my OpenStack experience we will focusing on VMs and will be covering:
- OpenStack overall architecture
- Just enough service dependencies to get a working OpenStack cluster
- Common difficulties (especially when coming from interacting with older releases)
- Basic usage (user perspective)
In a future second (or even third) part we will cover other topics like operating the cloud, dealing with migrations, planning for high availability, and debugging networking issues (insert ominous sounds here).
Architecture
Let us start from a user-centric perspective. As we can see in Figure 1 the user interacts with each service independently. However, in order to do that there needs no be some kind of authentication, and it should also exist some kind of service discovery. We can depict a possible interaction as follows:
- Cloud User communicates with the Identity service, authenticating against it and receiving a session token after a successful authentication.
- Cloud User communicates with the Identity service using the previously received token and asks for all of the known public service endpoints.
- Cloud User communicates with the desired service through its public endpoint, using the same token it received previously.
OpenStack service interaction from the user perspective
So we can see that the Identity service is a keystone for OpenStack's workflow, and is no surprise that it is called Keystone ;-)
Although there are obvious dependencies for using each service, the beauty of OpenStack is that a failure of one service does not mean a complete failure of another service. We may, however, lack some functionality of the service, for example, if your identity plane crashes spectacularly you may not be able to launch new VMs but your VMs' will still be up and running, and the network is also reachable.
This way of interacting is the usual CLI interaction path, you can, however, not see any of this and use the Dashboard service, also known as Horizon, so that you can see in a web application all of the available services and all of the available features from those services, acting as a proxy for the previous interaction.
Services
In this section we will see every service, one by one, in more detail.
Identity
Keystone is an OpenStack service that provides API client authentication, service discovery, and distributed multi-tenant authorization by implementing OpenStack’s Identity API. [1]
The central piece of every OpenStack cloud, it is responsible for keeping records of users, generating authentication tokens and other access mechanisms like API keys, maintaining a registry of available services for the cloud, and ensure individual service authentication and access control for different users to different projects (aka tenants).
Image
The Image service (glance) project provides a service where users can upload and discover data assets that are meant to be used with other services. This currently includes images and metadata definitions. [2]
This service is responsible for maintaining images that will be used to launch VM instances. They can be public, usually made available by administrators, private or shared, so one tenant can upload an image and share it with other tenants.
Block Storage
Cinder is the OpenStack Block Storage service for providing volumes to Nova virtual machines, Ironic bare metal hosts, containers and more. [3]
This services is responsible to store volumes created from cinder images so we can boot our VMs with persistent root storage, persistent volumes to attach to instances for additional storage, or even create encrypted volumes using another OpenStack service responsible for secrets management we will not cover here (barbican).
DNS
Designate is a multi-tenant DNSaaS service for OpenStack. It provides a REST API with integrated Keystone authentication. It can be configured to auto-generate records based on Nova and Neutron actions. Designate supports a variety of DNS servers including Bind9 and PowerDNS 4. [4]
With DNSaaS incorporated in an OpenStack cloud deployment you can have every instance you launch to have valid DNS records for their interfaces, and records for floating IPs you create, making it simple to manage DNS for your projects.
Network
Neutron is an OpenStack project to provide "network connectivity as a service" between interface devices (e.g., vNICs) managed by other OpenStack services (e.g., nova). It implements the OpenStack Networking API. [5]
This service is what gives network connectivity to and from your VM instances. It enables users to create Routers, making it possible to create multiple networks within your projects, connect your routers to the outside network, assigning public IPs to your VM instances, announcing Border Gateway Protocol (BGP) routes among other network operations, like creating firewall rules outside Operating System (OS) space, vital to your services operations.
Compute
Nova is the OpenStack project that provides a way to provision compute instances (aka virtual servers). Nova supports creating virtual machines, baremetal servers (through the use of ironic), and has limited support for system containers. Nova runs as a set of daemons on top of existing Linux servers to provide that service. [6]
This is the service responsible for allocating VMs to hypervisors, and it depends on keystone, glance and neutron components to operate on a basic level.
Dashboard
Horizon is the canonical implementation of OpenStack’s Dashboard, which provides a web based user interface to *OpenStack services including Nova, Swift, Keystone, etc. [7]
It is basically a way for users to interact with the OpenStack cloud in a simple graphical way, avoiding to perform complex operations through the OpenStack API or the OpenStack CLI. It is also an easy way to look at resources available to individual projects, and resource details all in one place.
Service Communication
All of the above services have to communicate with each other, or even between different agents for the same service. This is achieved through message queueing and at this time it supports the following backends:
- RabbitMQ
- Qpid
- ZeroMQ
OpenStack does not support message-level confidence, such as message signing. Consequently, you must secure and authenticate the message transport itself. For high-availability (HA) configurations, you must perform queue-to-queue authentication and encryption. [8]
Service State
State for the above services has to be kept in an atomic, consistent, isolated and durable way. So we should use a database backend which is compliant to this model. OpenStack considers using either PostgreSQL or MySQL as the database backend, I personally like MySQL (or MariaDB) as it provides high availability options out of the box.
The choice of database server is an important consideration in the security of an OpenStack deployment. Multiple factors should be considered when deciding on a database server. [9]
Openstack Landscape
After this fast run-down of services we can look at another representation of an OpenStack cloud, as we can see in Figure 2.
OpenStack service architecture
Here we can see three layers:
- External Layer
- Service Layer
- Physical Layer
In the External Layer we have our cloud users, which interact with the OpenStack cloud through either the Horizon Dashboard or through its REST API, either directly or through the openstackclient wrapper. These interactions usually start with asking the Keystone service for a token, and using that token to interact with the other cloud services. The service discovery is also made via the Keystone service, as every service must register with the Keystone endpoint.
We could consider the Dashboard to be part of the service layer, although I like to see it more like the wrapper to the REST API that it is, a web front-end, so we will leave it in a limbo between the External and the Service layer
In the Service Layer we have several distinct Planes where specific operations occur.
In the Control Plane we have all of our services' controllers (neutron, designate, nova) or standalone services (keystone, cinder, glance). These are responsible for, respectively, either delegate actions to other service agents (i.e: delegate VM creation to specific compute nodes), or take direct actions themselves (i.e: upload a cloud image to the storage backend).
In the Compute Plane we have the compute nodes, which receive VM allocation requests in the nova agents, and create those VMs using the appropriate hypervisor.
The Network Plane is probably the most complex, and is responsible for plugging and unplugging ports, create networks or subnets, and provide IP addressing. These plug-ins and agents differ depending on the vendor and technologies used in the particular cloud. OpenStack Networking ships with plug-ins and agents for Cisco virtual and physical switches, NEC OpenFlow products, Open vSwitch, Linux bridging, and the VMware NSX product [10] , via neutron agents and plug-ins. It can also be responsible for announcing BGP routes (via a specific neutron agent), and manage DNS zones and records (via a designate agent).
The Storage Plane is where cloud images (glance) and volumes (cinder) are effectively stored. Openstack supports a wide variety or storage backends, although in their documentation they say the most common are Ceph RBD, NFS or Local Disk (LVM).
The Control, Compute and Network Planes interact through two channels. A messaging queue (usually RabbitMQ) is used for services to publish and collect messages in specific service queues, where operations to perform and their results are sent to. A database (usually MySQL or PostgreSQL) is used to store the state of each service, describing the state of the whole OpenStack cloud state. Obviously these channels are very important for a smooth operation, sometimes being an operational bottleneck, and so they should be separate services and ideally clustered.
Finally, the Physical Layer is where the services actual run be it VMs, bare metal machines or containers. In Figure 2 we can see a correspondence between the Service Layer and the Physical Layer through their colours (sorry for not choosing colorblind safe colours):
- Control Plane runs in Controller Nodes
- Compute Plane runs in Compute Nodes
- Network Plane runs in Network Nodes
- Storage Plane runs in Ceph Cluster
- State Storage runs in MariaDB Cluster
- Message Queue runs in RabbitMQ Cluster
And with all of these services configured properly we have a minimal working OpenStack Cloud, neat!
For a broader view of what OpenStack can provide, we can look at Figure 3 for a list of all of the projects that exist until the day of this post.
Map of OpenStack projects [11]
Conclusion
In this first post we took a look at different OpenStack components, namely the bare minimum to have a functional cloud:
- manage users, projects and their access tokens (Identity)
- manage images to boot VM instances from
- manage volumes to store arbitrary data, attaching and detaching from VM instances
- manage network components such as routers, networks, subnets, DNS
- launch VM instances from an available image or volume
This article and it referred resources should give you everything you need to be able to deploy a minimal and functional OpenStack cluster.
Feel free to leave comments below, any improvement to this and other articles is always welcome.
Thanks for reading!
References
| [1] | --, Keystone, the OpenStack Identity Service - OpenStack Docs [link] |
| [2] | --, Welcome to Glance's documentation! - OpenStack Docs [link] |
| [3] | --, OpenStack Block Storage (Cinder) documentation - OpenStack Docs [link] |
| [4] | --, Designate, a DNSaaS component for OpenStack - OpenStack Docs [link] |
| [5] | --, Welcome to Neutron’s documentation! - OpenStack Docs [link] |
| [6] | --, OpenStack Compute (nova) - OpenStack Docs [link] |
| [7] | --, Horizon: The OpenStack Dashboard Project - OpenStack Docs [link] |
| [8] | --, Message queuing - OpenStack Docs [link] |
| [9] | --, Databases - OpenStack Docs [link] |
| [10] | --, Networking service overview - Openstack Docs [link] |
| [11] | --, Introduction, Openstack Docs [link] |
Tags: linux devops cloud computing inspec compliance integration testing
Related Content
- Integration and Compliance with InSpec
- Linux Networking Introduction
- Hacking diskimage-builder for Fun and Profit
- Virtualenvwrapper Installation and Usage
- Creating Custom Resources for Your Cookbooks