Tag Archives: Keystone

Installing OpenStack Keystone on Fedora

I have been playing a bit with cloud services, in particular Amazon Web Services but I recently wanted to install OpenStack to see what all the hype was about and to better understand the underlying components and technologies.  It’s possible to do a full OpenStack install on a single server or virtual machine running Fedora using the RDO instructions, however I wanted to do the build by hand.

I started with a minimal install of Fedora using the standard file system layout, I ran yum update and rebooted.  Once the system was back up I installed the RDO relase RPM as per the RDO quickstart instructions:

sudo yum install http://rdo.fedorapeople.org/openstack-havana/rdo-release-havana.rpm

This gives access to pre-built RPMs for all of the OpenStack Havana components.  RDO makes use of a number of other components to provide a DevOps style approach to installation.  I didn’t want this extra functionality so I edited /etc/yum.repos.d/foreman.repo and /etc/yum.repos.d/puppetlabs.repo to disable both of those repositories.

OpenStack supports a variety of database backends, but the simplest and best documented seems to be MySQL.  Fedora has switched to using the MariaDB fork of MySQL so that’s what I installed, along with the MySQL module for Python:

sudo yum install mariadb mariadb-server MySQL-python
sudo systemctl start mariadb.service
sudo systemctl enable mysqld.service
sudo mysql_secure_installation

Note that only the database server needs mariadb-server package.  Next I installed the OpenStack utils package:

sudo yum install openstack-utils

As well as the database the other piece of infrastructure that OpenStack needs is a messaging service that provides AMQP.  The two main implementations of this are RabbitMQ or Qpid.  I’ve chosen to use Qpid:

sudo yum install qpid-cpp-server memcached

For simplicity I turned off authentication in Qpid by adding auth=no to /etc/qpidd.conf, you probably wouldn’t do this in a production deployment!  Start and enable qpidd:

sudo systemctl start qpidd.service
sudo systemctl enable qpidd.service

Keystone is the identity component of OpenStack, similar to IAM in AWS terms.  Install the Keystone packages:

yum install openstack-keystone python-keystoneclient

Keystone needs to be configured to use the database we installed.  The openstack-config command allows use to set values in various config files without firing up vi.

sudo openstack-config --set /etc/keystone/keystone.conf sql connection mysql://keystone:keystone_db_pass@controller/keystone

The arguments to this command are the --set option indicating you want to set a value; the file that contains the value we want to set; the section within the file (if you edit the file with vi you can search for [sql]); the parameter we want to set: connection; and the value for the parameter.  In this case we’re setting the SQL Alchemy connection string which is an RFC1738 URL.

Now that the database connection is configured it can be initialised.  Note that you need to pass the same password value (in my case “keystone_db_pass“) as you configured in the SQL Alchemy connection string/URL:

sudo openstack-db --init --service keystone --password keystone_db_pass

You’ll be prompted for the database root password you set when you ran the mysql_secure_installation command.

Setup the main admin user password, the first command creates a random password and stores it in a shell environment variable so you can use it in subsequent commands:

ADMIN_TOKEN=$(openssl rand -hex 10)
sudo openstack-config --set /etc/keystone/keystone.conf DEFAULT admin_token $ADMIN_TOKEN
sudo keystone-manage pki_setup --keystone-user keystone --keystone-group keystone

The second command initialises the certificates that Keystone uses to create the cryptographically strong authentication tokens that we will use later when accessing the service via the command line or API.  There’s fuller discussion in the OpenStack Keystone documentation.

Now we can start the service:

sudo chown -R keystone:keystone /etc/keystone/ /var/log/keystone/keystone.log
sudo systemctl start openstack-keystone.service
sudo systemctl enable openstack-keystone.service

We need to set up a couple of environment variables so that we can use the command line tools.  OS_SERVICE_TOKEN is the password we created with the previous openssl command.  OS_SERVICE_ENDPOINT is the URL for the Keystone API, I’m using the IP address 10.0.0.29, but you should use the appropriate hostname or IP address for your environment:

export OS_SERVICE_TOKEN="763237339bc02dd92bfb"
export OS_SERVICE_ENDPOINT="http://10.0.0.29:35357/v2.0"

With all of that done we can now start using the keystone command to actual create tenants, users, and services:

keystone tenant-create --name=admin --description="Admin Tenant"
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
| description |           Admin Tenant           |
|   enabled   |               True               |
|      id     | 4b7e1355bb4d4afb960da724a9dfa0fc |
|     name    |              admin               |
+-------------+----------------------------------+
keystone tenant-create --name=service --description="Service Tenant"
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
| description |          Service Tenant          |
|   enabled   |               True               |
|      id     | c2e553ac9d164c74aff6d1a130f0f099 |
|     name    |             service              |
+-------------+----------------------------------+

These two commands create our first two tenants.  In OpenStack tenants can be thought of as groups that hold users and other resources that clouds provide.  For example, in a public cloud a tenant might represent a customer of that cloud service or in a private cloud a department or business line.  The admin tenant will hold the admin users for the cloud and the service tenant will hold the services that the cloud provides.  The names aren’t special you could call them anything.

We can also use the keystone command to list the tenants:

keystone tenant-list
+----------------------------------+---------+---------+
|                id                |   name  | enabled |
+----------------------------------+---------+---------+
| 4b7e1355bb4d4afb960da724a9dfa0fc |  admin  |   True  |
| c2e553ac9d164c74aff6d1a130f0f099 | service |   True  |
+----------------------------------+---------+---------+

The next step is to create an admin user, you should give this user a better password than I’ve chosen here!

keystone user-create --name=admin --pass=admin --email=admin@example.org
+----------+----------------------------------+
| Property |              Value               |
+----------+----------------------------------+
|  email   |        admin@example.org         |
| enabled  |               True               |
|    id    | 22f1020799b7425cabbf22837934d510 |
|   name   |              admin               |
+----------+----------------------------------+

Privileges in OpenStack are assigned to users through roles, the privileges are associated with the role and the role is then associated with a user.  We’ve got the admin user so the next step is to create the admin role.  In this case the role name is important as it needs to match the role name in the policy.json file that controls rights and access.

keystone role-create --name=admin
+----------+----------------------------------+
| Property |              Value               |
+----------+----------------------------------+
|    id    | 676f70baed8e430799138acf75a3f8b3 |
|   name   |              admin               |
+----------+----------------------------------+

The final step is to tie the tenant, user, and role together:

keystone user-role-add --user=admin --tenant=admin --role=admin

To summarise, creating a user consists of four steps:

  1. If necessary create a new tenant – keystone tenant-create.
  2. Create the new user – keystone user-create.
  3. If necessary create the new role (remember the role name must match that in the policy.json file) – keystone role-create.
  4. Tie the tenant, role, and user together – keystone user-role-add.

In OpenStack all of the cloud resources are presented as Services, this includes Keystone itself.  Our next step is to create the Keystone service and then make it available:

keystone service-create --name=keystone --type=identity --description="Keystone Identity Service"
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
| description |    Keystone Identity Service     |
|      id     | dbb075345d404db5a64e33918a8e96f4 |
|     name    |             keystone             |
|     type    |             identity             |
+-------------+----------------------------------+

Having created the service we need to create endpoints for consumers to access the service.  Note that there are three different endpoints, this is to support the common deployment scenario where the server hosting Keystone has three network interface cards – one for public access (i.e. users of the cloud), one for internal access (i.e. other services within the cloud), and one for admin access.  In this test deployment they’re all on the same interface card.  The --service-id parameter is the UUID that was returned as the id parameter in the keystone service-create command above.

keystone endpoint-create  --service-id=dbb075345d404db5a64e33918a8e96f4 --publicurl=http://10.0.0.29:5000/v2.0 --internalurl=http://10.0.0.29:5000/v2.0 --adminurl=http://10.0.0.29:35357/v2.0
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
|   adminurl  |   http://10.0.0.29:35357/v2.0    |
|      id     | 35bfb42f44194228a66ec8a70b44493e |
| internalurl |    http://10.0.0.29:5000/v2.0    |
|  publicurl  |    http://10.0.0.29:5000/v2.0    |
|    region   |            regionOne             |
|  service_id | dbb075345d404db5a64e33918a8e96f4 |
+-------------+----------------------------------+

We can now verify that the tenant, user, and service we’ve created are all working.  To do this we first need to clear the credentials and service endpoint we’ve been using so far:

unset OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT

We can now use the keystone command with the username, password, and service endpoint that we just created:

keystone --os-username=admin  --os-password=admin --os-auth-url=http://10.0.0.29:35357/v2.0 token-get

We can do the same thing, but additionally specifying the tenant name:

keystone --os-username=admin  --os-password=admin --os-tenant-name=admin --os-auth-url=http://10.0.0.29:35357/v2.0 token-get

OpenStack authentication works on the principle that you supply valid credentials to a service endpoint and in return you get a token which you present to the service when you make subsequent requests.  The previous two commands are using the token-get parameter to request a token.

It can get tedious to have to type in the username, password, tenant name, and endpoint parameters for each command so OpenStack allows you to set these are environment variables:

export OS_USERNAME=admin
export OS_PASSWORD=admin
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://10.0.0.29:35357/v2.0

Which then allows you to shorten commands:

keystone user-list
+----------------------------------+-------+---------+-------------------+
|                id                |  name | enabled |       email       |
+----------------------------------+-------+---------+-------------------+
| 22f1020799b7425cabbf22837934d510 | admin |   True  | admin@example.org |
+----------------------------------+-------+---------+-------------------+

Note that putting passwords, especially admin ones, into environment variables probably isn’t best practice!

Now that command line access is working we can do exactly the same things using the Keystone API.  In the following example we make an HTTP POST request to the tokens URL passing our credentials as a JSON document in the request payload.

In response we get a token that we can use in further API calls, a service catalog detailing the service endpoints, and information about our user, role, and tenant.

curl -k -X 'POST' -v http://10.0.0.29:35357/v2.0/tokens -d '{"auth":{"passwordCredentials":{"username": "admin", "password":"admin"}, "tenantId":"4b7e1355bb4d4afb960da724a9dfa0fc"}}' -H 'Content-type: application/json'
* About to connect() to 10.0.0.29 port 35357 (#0)
*   Trying 10.0.0.29...
* Connected to 10.0.0.29 (10.0.0.29) port 35357 (#0)
> POST /v2.0/tokens HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.0.0.29:35357
> Accept: */*
> Content-type: application/json
> Content-Length: 121
> 
* upload completely sent off: 121 out of 121 bytes
< HTTP/1.1 200 OK
< Vary: X-Auth-Token
< Content-Type: application/json
< Content-Length: 2347
< Date: Sun, 03 Nov 2013 17:22:39 GMT
< 
{
  "access": {
    "token": {
      "issued_at": "2013-11-03T17:22:39.311048", 
      "expires": "2013-11-04T17:22:39Z", 
      "id": "MIIErwYJKoZIhvcNAQcCoIIEoDCCBJwCAQExCTAHBgUrDgMCGjCCAwUGCSqGSIb3DQEHAaCCAvYEggLyeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0xMS0wM1QxNzoyMjozOS4zMTEwNDgiLCAiZXhwaXJlcyI6ICIyMDEzLTExLTA0VDE3OjIyOjM5WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIkFkbWluIFRlbmFudCIsICJlbmFibGVkIjogdHJ1ZSwgImlkIjogIjRiN2UxMzU1YmI0ZDRhZmI5NjBkYTcyNGE5ZGZhMGZjIiwgIm5hbWUiOiAiYWRtaW4ifX0sICJzZXJ2aWNlQ2F0YWxvZyI6IFt7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMC4wLjAuMjk6MzUzNTcvdjIuMCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4wLjAuMjk6NTAwMC92Mi4wIiwgImlkIjogIjIxMjhiOWExMDc0OTQ3ZDU4NDI0YWQwOTJmNTM3MTdhIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTAuMC4wLjI5OjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidXNlcm5hbWUiOiAiYWRtaW4iLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjIyZjEwMjA3OTliNzQyNWNhYmJmMjI4Mzc5MzRkNTEwIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAiYWRtaW4ifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAsICJyb2xlcyI6IFsiNjc2ZjcwYmFlZDhlNDMwNzk5MTM4YWNmNzVhM2Y4YjMiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBATAHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQCUcTFJU550veZlBYtXQos0Q24BJVbw2acBSZ2p42Ifw2itZxHRa6RpYKyPhltTE93v8zbLbNLVS+KI-+U-SP3zsTzWrrFxS2Bt7AWh2qPhPossGqmxmv3DnFZPk5bOXk3fMWMRnYydsH5hFknmhilbPX4EwJNV6qLyZvDjpg4szIc8YBVludPiy-6aGrv7eWNZUhMi7zz3b7SSYJ0gTTB7brTzmtcH946ayY33a0lx8fSlcfUWV22Ey7BWPFHzVQxzF+2Ho46uIqPDs3ohV9q5I-XSOwTvA+lWvI35VbFHnBKnhjpYGrGAjexhQyTD7InCGYejKCu6H1yedr2c0aci", 
      "tenant": {
        "description": "Admin Tenant", 
        "enabled": true, 
        "id": "4b7e1355bb4d4afb960da724a9dfa0fc", 
        "name": "admin"
      }
    }, 
    "serviceCatalog": [{
      "endpoints": [{
        "adminURL": "http://10.0.0.29:35357/v2.0",
        "region": "regionOne",
        "internalURL": "http://10.0.0.29:5000/v2.0",
        "id": "2128b9a1074947d58424ad092f53717a",
        "publicURL": "http://10.0.* Connection #0 to host 10.0.0.29 left intact
0.29:5000/v2.0"
      }],
      "endpoints_links": [],
      "type": "identity",
      "name": "keystone"
    }],
    "user": {
      "username": "admin",
      "roles_links": [],
      "id": "22f1020799b7425cabbf22837934d510",
      "roles": [{
        "name": "admin"
      }],
      "name": "admin"
    },
    "metadata": {
      "is_admin": 0,
      "roles": ["676f70baed8e430799138acf75a3f8b3"]
    }
  }
}

The next example uses the authentication toke we’ve just received to make a API call listing the extensions that are available in this OpenStack instance, note that this is an HTTP GET request so there’s no payload this time:

curl -k -D - -H "X-Auth-Token: MIIErwYJKoZIhvcNAQcCoIIEoDCCBJwCAQExCTAHBgUrDgMCGjCCAwUGCSqGSIb3DQEHAaCCAvYEggLyeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0xMS0wM1QxNzoyMjozOS4zMTEwNDgiLCAiZXhwaXJlcyI6ICIyMDEzLTExLTA0VDE3OjIyOjM5WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIkFkbWluIFRlbmFudCIsICJlbmFibGVkIjogdHJ1ZSwgImlkIjogIjRiN2UxMzU1YmI0ZDRhZmI5NjBkYTcyNGE5ZGZhMGZjIiwgIm5hbWUiOiAiYWRtaW4ifX0sICJzZXJ2aWNlQ2F0YWxvZyI6IFt7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMC4wLjAuMjk6MzUzNTcvdjIuMCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4wLjAuMjk6NTAwMC92Mi4wIiwgImlkIjogIjIxMjhiOWExMDc0OTQ3ZDU4NDI0YWQwOTJmNTM3MTdhIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTAuMC4wLjI5OjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidXNlcm5hbWUiOiAiYWRtaW4iLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjIyZjEwMjA3OTliNzQyNWNhYmJmMjI4Mzc5MzRkNTEwIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAiYWRtaW4ifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAsICJyb2xlcyI6IFsiNjc2ZjcwYmFlZDhlNDMwNzk5MTM4YWNmNzVhM2Y4YjMiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBATAHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQCUcTFJU550veZlBYtXQos0Q24BJVbw2acBSZ2p42Ifw2itZxHRa6RpYKyPhltTE93v8zbLbNLVS+KI-+U-SP3zsTzWrrFxS2Bt7AWh2qPhPossGqmxmv3DnFZPk5bOXk3fMWMRnYydsH5hFknmhilbPX4EwJNV6qLyZvDjpg4szIc8YBVludPiy-6aGrv7eWNZUhMi7zz3b7SSYJ0gTTB7brTzmtcH946ayY33a0lx8fSlcfUWV22Ey7BWPFHzVQxzF+2Ho46uIqPDs3ohV9q5I-XSOwTvA+lWvI35VbFHnBKnhjpYGrGAjexhQyTD7InCGYejKCu6H1yedr2c0aci" -X 'GET' -v http://10.0.0.29:35357/v2.0/extensions  -H 'Content-type: application/json'

In response we get a HTTP 200 OK from the server and a JSON document that lists the available extensions:

{
  "extensions": {
    "values": [{
      "updated": "2013-07-07T12:00:0-00:00", 
      "name": "OpenStack S3 API", 
      "links": [{
        "href": "https://github.com/openstack/identity-api", 
        "type": "text/html", 
        "rel": "described by"
      }], 
      "namespace": "http://docs.openstack.org/identity/api/ext/s3tokens/v1.0", 
      "alias": "s3tokens", 
      "description": "OpenStack S3 API."
    }, {
      "updated": "2013-07-11T17:14:00-00:00", 
      "name": "OpenStack Keystone Admin", 
      "links": [{
        "href": "https://github.com/openstack/identity-api", 
        "type": "text/html", 
        "rel": "described by"
      }], 
      "namespace": "http://docs.openstack.org/identity/api/ext/OS-KSADM/v1.0", 
      "alias": "OS-KSADM", 
      "description": "OpenStack extensions to Keystone v2.0 API enabling Administrative Operations."
    }, {
      "updated": "2013-07-07T12:00:0-00:00", 
      "name": "OpenStack EC2 API", 
      "links": [{
        "href": "https://github.com/openstack/identity-api",
        "type": "text/html",
        "rel": "described by"
      }],
      "namespace": "http://docs.openstack.org/identity/api/ext/OS-EC2/v1.0", 
      "alias": "OS-EC2",
      "description": "OpenStack EC2 Credentials ba* Connection #0 to host 10.0.0.29 left intact
ckend."
    }, {
      "updated": "2013-07-23T12:00:0-00:00",
      "name": "Openstack Keystone Endpoint Filter API",
      "links": [{
        "href": "https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3-os-ep-filter-ext.md",
        "type": "text/html",
        "rel": "described by"
      }],
      "namespace": "http://docs.openstack.org/identity/api/ext/OS-EP-FILTER/v1.0",
      "alias": "OS-EP-FILTER",
      "description": "Openstack Keystone Endpoint Filter API."
    }]
  }
}

At this point we’ve got Keystone up and running and demonstrated that we can use the service both through the command line tools and the API.  The next step is to add additional OpenStack services that will make use of Keystone.

One thing I found confusing the first time I went through a Keystone deployment was the variety of users and passwords that I needed to create, so here’s a summary:

  1. The root or admin user for the database your are using.  In my case this was the MySQL root user password set when I ran the mysql_secure_installation command.
  2. The password that Keystone will use when access it’s own database, set when configuring and initialising the Keystone database.
  3. The Keystone admin user password.  This is effectively the root password for Keystone and is stored in the /etc/keystone/keystone.conf file unencrypted.  Only to be used during initial Keystone deployment and configuration.
  4. Finally the Keystone service admin user that you should create as soon as the service is up and running.  This is the account that you’ll use to perform all ongoing admin tasks.  Best practise would be to create individual accounts for all users that will need admin privileges and assign them to the admin role.