Thursday, January 27, 2011

Growing a REST API, Part I of 2

This REST API that I had a hand in (more like both hands in) for the Vyatta router (also known as the Vyatta Remote Access API) was a complete rewrite of a more traditional XML state-aware API (and which replaced an even earlier version).

In retrospect, given the progression of versions it was with REST that the simplest HTTP based API was designed. I'd even say that the work was well focused due to REST constraints applied to the design of the interface.



A number of REST APIs were looked at, but for the application at hand (a router) there wasn't really anything to directly compare to. With one notable exception being the Sun cloud initiative--by the way I can't say what the status of this project is after Sun was consumed by Oracle (although I fear the worse), but the scope of the project was larger than this implementation.

I'm not going to reach down too deep into the philosophy of what makes an API RESTful. If you want to get down to the dirt read the original dissertation by Roy Fielding.

However, some basic REST design tenants are:

  • No statefulness (i.e. no state maintained on the server between requests).
  • No body with the request.
  • Use transport (i.e. http header) parameters whenever possible or whenever this makes sense.

This work exposed the rich set of Vyatta commands (6000+ in configuration mode alone) used to control the Vyatta Router appliance thingy. Given that the API should live up to the richness of the already implemented command set, a reasonable goal was to design the API to be dynamic in nature. Meaning that as commands are added/modified or dropped from the Vyatta router (even at runtime), the corresponding changes are automatically reflected in this API. The API should even allow some kind of introspection to see what commands are available on the device.

The API must be secure, robust and predictable. A well thought out implementation opens up the system to new creative remote uses. Areas such as management/configuration within a cloud environment, mobile access/control, multiple systems interacting in a coordinated fashion, etc.

To give you an concrete example of what I'm talking about below is one of the API calls (which just says to enter configuration mode):

POST /rest/conf HTTP/1.1
User-Agent: curl/7.21.0 (i486-pc-linux-gnu) 
  libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
Host: 127.0.0.1
Accept: */*
authorization: Basic dnlhdHRhOnZ5YXR0YQ==
content-length:0

and returns a token in the location header parameter:

HTTP/1.1 201 Created
Content-Type: application/json
Location: rest/conf/EA9C0A51D0BE6997
Vyatta-Specification-Version: 0.2
Cache-Control: no-cache
Transfer-Encoding: chunked
Date: Fri, 14 Jan 2011 18:40:00 GMT
Server: lighttpd/1.4.28-devel-485M

The response is using the https code 201 saying a resource was created, and that the Location header parameter contains the new location where this new resource can be found. In fact then all interaction with this new resource will be through this location path, i.e. "rest/conf/EA9C0A51D0BE6997".

Before going into more detail with the API I should briefly describe the current command layout as it exists on the appliance. These commands are accessed via a shell, i.e. CLI (command line interface). There are two main systems that a user of a Vyatta router interacts with: The configuration of the system, and status commands (what are operational data) of the system. The distinction between the two really are:

  • Configuration: Changes the state of the system in a permanent meaningful way.
  • Operational: Provides useful system runtime data, but does not change the state of the system. Operational mode commands may run forever.

Why I'm digressing is that with these two branches there are basic differences that affect the design of the API. For example, one can expect that an operational command response may contain a potentially large set of data or take a long time to complete (maybe even never complete). This is in comparison to configuration data commands that tend to contain no/minimal response bodies, and are expected to complete execution in a short period of time. These command differences require completely different methods of execution, and therefore different means to interact with these systems.

Finally, a third branch was created outside of the configuration and operational behavior: the application (App) branch. This branch is unique to the API and has no equivalent within the traditional shell access. This third branch allows us to construct behaviors outside of the defined command sets already set up. Or if you will, a Vyatta REST custom or specialized interface. The inherent difference with App mode is that each App behavior requires some code installation on the server (for each App command).

The three command branches are:

  • Op: status about the system
  • Conf: change state of the system
  • App: custom API only commands

And that's that--at least for a quick, broadbrush overview.

Common features to all API commands

All commands require basic authentication. Since these are stateless commands, sessions are not maintained or tracked server side. Therefore, each command needs to carry an authentication parameter:

authorization: Basic dnlhdHRhOnZ5YXR0YQ==

Which is a base64 encoded username and password. What's keeping this secure is that commands are only accepted over https. Therefore, Hypertext Transfer Protocol with the SSL/TLS protocol are provided a secure transport for the username and password. And it's probably not a good idea to keep this stored in a browser cookie (see browser session management).

The command mode branches (read root URI) are distinguished by the following paths:

operational = rest/op/command
configuration = rest/conf/command
application = rest/app/command

Commands within the specific command modes follow as additional URI switches on these statements.

Configuration mode:

Configuration mode commands change the state of the system. BEFORE you are allowed to play though a configuration mode token must be obtained from the server.

That command to do that is:
POST /rest/conf

with corresponding header response:

HTTP/1.1 201 Created
Content-Type: application/json
Location: rest/conf/EA9C0A51D0BE6997
Vyatta-Specification-Version: 0.2
Cache-Control: no-cache
Transfer-Encoding: chunked
Date: Fri, 14 Jan 2011 18:40:00 GMT
Server: lighttpd/1.4.28-devel-485M

Which is basically saying that a configuration token was created (hence the http 201 status code), and the token is found on this path: rest/conf/EA9C0A51D0BE6997. This path you will use to interact with the remainder of your configuration commands.

And OK--so I lied a bit here. The creation of a token (and I was trying hard not to say it but couldn't) means that a bit of state was in fact set up on the server within the configuration tree of the appliance. This is something that couldn't be avoided in order to keep from diverting resources into a complete redesign of the current appliance command system. The configuration token does provide a great deal of functionality that couldn't exactly be supported through a pure stateless system. So, another case of reality (implementation) versus vision (true REST) bites the dust. I think you will find this to be true on most functional REST APIs.

Getting down to the nuts though. The breakdown of the sub-commands underneath configuration mode are (italics are optional):

#create a configuration token
POST   rest/conf/DESCRIPTION
#perform an action on a configuration token
POST   rest/conf/[REST-TOKEN|DESCRIPTION]/ACTION

#set/delete a value in the configuration
PUT    rest/conf/[REST-TOKEN|DESCRIPTION]/ACTION/COMMAND

#get the state of configuration value
GET    rest/conf/[REST-TOKEN|DESCRIPTION]/COMMAND
#get information on all your configuration tokens
GET    rest/conf

#delete a configuration token you created
DELETE rest/conf/[REST-TOKEN|DESCRIPTION]

No commands require a body with the request. All commands or values with url-unsafe characters need to have them encoded accordingly.

Finally, one last note. The PUT command is used to set or delete a node rather than relying on the HTTP command DELETE, PUT--the decision was to use the action in the path-to-the-node so as not to overload the meaning of delete within the scope of this API. So, to "set" a node you PUT a "set/path/to/my/node", or to delete you PUT a "delete/path/to/my/node"--and at any rate this delete isn't really a delete until the command is committed.

Any http 2xx response means that the system is happy with your command.

An example of a typical configuration sequence using the API is illustrated below. This involves creation of the configuration token, setting a value, committing your changes, and finally deleting the configuration token.

Figure 1: Typical configuration session


The equivalent sequence is below, showing the explicit request/response pairs.


1. Request and receive the command token.
POST /rest/conf HTTP/1.1
User-Agent: curl/7.21.0 (i486-pc-linux-gnu) libcurl/7.21.0 
  OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
Host: 127.0.0.1
Accept: */*
authorization: Basic dnlhdHRhOnZ5YXR0YQ==
content-length:0
 
HTTP/1.1 201 Created
Content-Type: application/json
Location: rest/conf/4D9B6632866B97DF
Vyatta-Specification-Version: 0.2
Cache-Control: no-cache
Transfer-Encoding: chunked
Date: Wed, 19 Jan 2011 00:52:26 GMT
Server: lighttpd/1.4.28

2. Put our action on the server with this command token. In this case we are enabling the telnet daemon on the router.
PUT /rest/conf/1D58D0A8E0594514/set/service/telnet HTTP/1.1
User-Agent: curl/7.21.0 (i486-pc-linux-gnu) libcurl/7.21.0 
  OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
Host: 127.0.0.1
Accept: */*
authorization: Basic dnlhdHRhOnZ5YXR0YQ==
content-length:0
 
HTTP/1.1 200 OK
Content-Type: application/json
Vyatta-Specification-Version: 0.2
Cache-Control: no-cache
Transfer-Encoding: chunked
Date: Wed, 19 Jan 2011 00:53:23 GMT
Server: lighttpd/1.4.28

3. Now commit (or activate) our command.
POST /rest/conf/1D58D0A8E0594514/commit HTTP/1.1
User-Agent: curl/7.21.0 (i486-pc-linux-gnu) libcurl/7.21.0 
  OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
Host: 127.0.0.1
Accept: */*
authorization: Basic dnlhdHRhOnZ5YXR0YQ==
content-length:0
 
HTTP/1.1 200 OK
Content-Type: application/json
Vyatta-Specification-Version: 0.2
Cache-Control: no-cache
Content-Length: 58
Date: Wed, 19 Jan 2011 00:54:05 GMT
Server: lighttpd/1.4.28

{
  "message": " "
}


And finally deletion of the configuration session (not shown).

The last command you will notice returns a response body, any of these commands can return this. In this case the response body is empty, but in the case of an error, this would usually contain some useful error details (you will know this is an error as the http response will be a 4xx or 5xx type variety).

If you are curious about the state of your session, you can perform the following query:

GET /rest/conf HTTP/1.1
User-Agent: curl/7.21.0 (i486-pc-linux-gnu) libcurl/7.21.0 
  OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
Host: 127.0.0.1
Accept: */*
authorization: Basic dnlhdHRhOnZ5YXR0YQ==
content-length:0
 
HTTP/1.1 200 OK
Content-Type: application/json
Vyatta-Specification-Version: 0.2
Cache-Control: no-cache
Content-Length: 598
Date: Wed, 19 Jan 2011 01:04:59 GMT
Server: lighttpd/1.4.28

{
  "session": [
    {
      "id": "1D58D0A8E0594514",
      "username": "barney",
      "description": "",
      "started": "1295398267",
      "modified": "false",
      "updated": "1295398267"
    }
  ],
  "message": " "
}

Which says that you have one configuration token on this system associated to user "barney", and there are no outstanding changes associated with this token. These tokens can be used, deleted or just left in an open state. Obviously too many open tokens can at some point has the potential to stave the system of resources.

Finally, let's play with one more configuration REST command. This one returns information about specific commands supported from the system (a kind of introspection):

GET /rest/conf/1D58D0A8E0594514/service/telnet HTTP/1.1
User-Agent: curl/7.21.0 (i486-pc-linux-gnu) libcurl/7.21.0 
  OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
Host: 127.0.0.1
Accept: */*
authorization: Basic dnlhdHRhOnZ5YXR0YQ==
content-length:0
 
HTTP/1.1 200 OK
Content-Type: application/json
Vyatta-Specification-Version: 0.2
Cache-Control: no-cache
Content-Length: 498
Date: Wed, 19 Jan 2011 01:10:56 GMT
Server: lighttpd/1.4.28

{
  "children": [
    {
      "name": "allow-root",
      "state": "none",
      "deactivate_state": "enable"
    },
    {
      "name": "listen-address",
      "state": "none",
      "deactivate_state": "enable"
    },
    {
      "name": "port",
      "state": "none",
      "deactivate_state": "enable"
    }
  ],
  "type": [
    "none"
  ],
  "deactivate_state": "enable",
  "help": " Enable/disable Network Virtual Terminal Protocol (TELNET) protocol",
  "name": "telnet",
  "state": "none"
}

As you can see the body of the response is more interesting here. Basically, the command is exposing everything it knows about the state and implementation of the specific configuration element.

That's it for an overview of the Configuration side of things via the REST interface.

Next up in part 2, Operational and Application mode...

No comments:

Post a Comment