Using OpenStack Database (Trove): Replication and Clustering; Implementation details.

[Editor's Note: This post is part 2 in a 3 part series about using Trove. You can read part 1 here.]

The user is able to issue the various replication related commands using the trove client (python-troveclient). In particular these commands are detach_replication, and extensions to the create and show commands. These commands and their outputs were described in the previous blog post.

The python-trove client depends on some new and modified API’s.

The creation of the master in a replicated pair is just as it is today. The command to create a slave extends on the current create command as shown below. The request to create a slave identifies the master instance. In the request below, the reference to the master is provided in the slaveOf attribute (highlighted).

POST /instances
{
  "instance": {
               "name": "products-s1",
               "datastore": {
                             "type": "mysql",
                             "version": "5.5"
                            },
               "slaveOf": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
               "configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
               "flavorRef": "7",
               "volume": {
                          "size": 1
                         }
              }
}

The response to this is also an extension of the current response to the create API call. As shown below, the response identifies the master and the slave.

POST /instances
{
 "instance": {
               "status": "BUILD",
               "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
               "name": "products-s1",
               "created": "...",
               "updated": "...",
               "links": [{...}],
               "datastore": {
                             "type": "mysql",
                             "version": "5.5"
                            },
               "slaveOf": {
                           "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
                           "links":[{..}],
                          }
               "configuration": {
                                 "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
                                 "links": [{...}],
                                },
               "flavor": { 
                          "id": "7",
                          "links": [{...}],
                         },
               "volume": {
                          "size": 1
                         }
              }

}

Once a replicated pair is created, the client command to detach_replication results in an API call as shown below.

POST /instances/{id}/action
{
 "detach_replication": {}
}

Observe that unlike the CREATE API calls which are POST’s to the endpoint of /instances, the detach_replication call is posted to the specific instance endpoint /instances/{id}/action

As shown above, this change requires that the Trove Taskmanager implement these API calls. First, there is the implementation of the detach_replication() API call, and then there is the change to the create_instance() API call to handle the slaveOf argument.

We now describe the changes to the GuestAgent in detail. Replication in Trove is based on snapshots. We have designed this in such a way that it is a feature that is easily extensible to other data stores. The GuestAgent API will have four new methods:

• get_replication_snapshot()

• attach_replication_slave()

• detach_replication_slave()

• demote_replication_master()

It will be up to the guest agent for each data store to implement these methods. In this way, the contents of the snapshot are entirely shielded from the taskmanager and higher-level components and the guest agent is free to store all the information appropriate to that data store in the snapshot.

The get_replication_snapshot() API call will cause a snapshot to be created. The MySQL guest agent will use xtrabackup to create a snapshot and will store into that snapshot the binlog position and any network information about the master that will be required to setup replication.

{
 "master": {
            "host": "192.168.0.1",
            "port": 3306
           },
 "dataset": {
             "datastore": "mysql",
             "datastore_version": "mysql-5.5",
             "dataset_size": 2,
             "snapshot_href": "http://..."
            },
 "binlog_position": <binlog position>
}

The attach_replication_slave() API call will cause the master’s information to be inserted into the selected site and this will cause replication updates to be received from the master site.

The detach_replication_slave() API call will cause a slave instance to stop replicating from a master. Once this is done, no further updates from a master will be received by this slave and the master will no longer contain any reference to the detached slave.

{
 "topology": {
              "members": [{
                           "id": "{master-id}",
                           "name": "master"
                          },
                          {
                           "id": "{slave2-id}",
                           "name": "slave2",
                           "mysql": {
                                     "slave_of": [{"id": "{master-id}"}],
                                     "read_only": true
                                    }
                          }]
             }
}

Finally, the demote_replication_master() API call will cause the master to return to its pre replication state. This will cause bin logging to be turned off and any other configuration changes or log files created for the purpose of logging will be removed.