Adventures with Services and Key Based Authentication (UPDATED)

(UPDATE #2) This article pertains to Services 2.x and won’t work with Services 3.x Look for an updated article for Services 3.x in the future.

(UPDATE) I’ve added some new information regarding the hash value generation. Take a look.

I love open source. I really do. But one of the big complaints people have with it is documentation. Most of the time documents are written in such a way that you have to already know how to use the software to know what the docs are saying. They’re more reference manuals than training or support documents. And a lot of the time this can be frustrating. With Services and making XML-RPC calls using the key authentication in Drupal, this was made more apparent than ever.

Now I don’t know how many of you have used the Services module in Drupal but it is a pretty nifty little module. Essentially it allows you to make calls to functions within Drupal from remote sites. So say you have a personal blog site, and you’d like to publish a note on another site saying a new post was made on your blog. You can have your site send a request over HTTP to the other site to post a node story saying “New blog post! Come on over and read it.”. Now I realize there are probably better ways to accomplish this but this is just one example. You can potentially create users remotely, add content, get views data, remotely authenticate, whatever you want.

The module itself is at the time of writing at v2.2 with some new API changes and lots of supplementary modules. But the Services module has been around for a long time. I mean a LONG time. It spent years in 2.x-dev. In that time a lot of people started using it and subsequently started writing documentation which is great! The only issue now is that a lot of this documentation is now months or even years old and can lead to hours of frustration trying to figure out what still holds true, what’s changed and well what’s missing altogether.

Enter this morning. I’ve been trying to get make service calls using XML-RPC to another Drupal site using the API Key Authentication module which comes with Services.

Now enter my frustrations. There are literally half a dozen posts out there saying “this is how you do this” and others saying “no no, that guys wrong, here’s the REAL way to do it” and a few saying “I don’t know what they’re talking about but for some reason this code works for me!”. Sufficed to say, I was confused.

So here I am, tossing in my two cents and making this the definitive post on getting Services with API Key authentication working.

Step 1

The first thing you want to do is install and configure a Drupal site with the Services XML-RPC server enabled. Download the latest copy from http://drupal.org/project/services (from the 2.x branch, haven’t tried 3.x yet), install and enable the Services module, the Key Authentication and the XML-RPC Server.

Now if you’ve got a module already that has services we can connect to (say another contrib module or a custom one) you should be able to use that to test with. For anyone else, go ahead and enable the “node service” module. You’ll also want to enable the “System service” module. More on that later.

[[{“type”:”media”,”view_mode”:”media_original”,”fid”:”8”,”attributes”:{“alt”:”“,”class”:”media-image”,”typeof”:”foaf:Image”}}]]

Step 2

As of Services 2.1, there is now an access argument and access callback defined for each service call. This means if you want to expose certain data or functions, you’ll have to ensure the right user role has permission to use or see that data. Since we’ll be using the node service, we just have to ensure that “Anonymous” can “access content”. Go to Administer > User Management > Permissions and set your permissions appropriately.

(There are ways to have external sites authenticate with a user/pass and thus giving the potential for them to have a different role than anonymous. For simplicity, I’m going to omit that here.)

Step 3

The next step is to generate an API key which we’ll use to connect to our exposed services.

Go to Administer > Site Building > Services. Then click on Settings. Under the authentication fieldset, select “Key Authentication”.

[[{“type”:”media”,”view_mode”:”media_original”,”fid”:”9”,”attributes”:{“alt”:”“,”class”:”media-image”,”typeof”:”foaf:Image”}}]]

For now, you can leave the token lifetime and Session ID fields at default (30 and off). We’ll come back to that later.

Step 4

Now we’ll create a key for our other site to connect from. Click on the “Keys” tab and click “Create Key”. Fill in the required fields. For our example, be sure to check off access to “node.view”.

[[{“type”:”media”,”view_mode”:”media_original”,”fid”:”10”,”attributes”:{“alt”:”“,”class”:”media-image”,”typeof”:”foaf:Image”}}]]

As far as I can tell, it doesn’t really matter if the value for “domain” is the actual domain you’ll be connecting from. It would make things easier to keep track of what calls what once you’ve got a bunch of services though. For now, you can just use “localhost”.

Step 5

Let’s test out what we’ve got so far. Click on the “Browse” tab. You should see the list of available services. Click on “node.view”

You should see something along these lines.

[[{“type”:”media”,”view_mode”:”media_original”,”fid”:”11”,”attributes”:{“alt”:”“,”class”:”media-image”,”typeof”:”foaf:Image”}}]]

Ensure the domain is set to “localhost” (it may not say that by default). Next, enter the node ID of some content on your site (if you don’t have any nodes created, go head and make a few then come back here). Click submit.

Once the page finished reloading, you should see at the bottom the fruits of your labors. It should have loaded the node you specified. If you get errors along the lines of “Access Denied” make sure the domain is set properly and that your permissions are set properly in the Permissions page.

Step 6

So far we’ve just been setting up and testing our environment. Now we get to try out some code!

You have two options here:

  1. Use the “devel” module and go to /devel/php and run the code manually there.
  2. Write a quick module to test with

Either way should work. Personally I’d prefer the module approach since it could be used as a reference in the future. Just create a dummy module with one function that looks like this

function yourmodulename_service_call_test($nid) {
  // @todo fill in the function with stuff
}

If you’ve got your module setup or are running from devel/php you can move on to the next step. The module or devel call can be made from the same Drupal instance as where your Services server resides. But this should work from just about anywhere.

Step 7

This is where all that ranting earlier about bad documentation comes into play. Depending on where you go and what you read, the order in which you add and send the data from the server.

When building the function call using the xmlrpc() function, the order of the parameters is now different from those in the current docs. Here’s the proper order

xmlrpc($host, $remote_procedure, $hash, $domain, $timestamp, $nonce, $parameter_1, $parameter_2, etc...);

Let’s go over each item

$host
URL of the server you’re connecting to. Ex: http://example.com/services/xmlrpc $remote_procedure
The name of the function you’re calling. In our example it is node.view $hash
A hash value generated using the API key and a few other values. More on this below. $timestamp
The current time as a timestamp. Ensure you typecast it using (string)$timestamp to avoid wierdness $nonce
Random salt value used while hashing. using user_password() to generate this value is pretty standard $parameter_1
Lastly include your specific parameters for the function you’re calling.

Best way to know the order to send stuff in is to look back at Step 5, on that page, the order the fields are rendered. That’s the order to send stuff in.

Now about this $hash value. This is generated using sha256 with a mix of the API key, remote procedure name, timestamp and nonce value. Essentially just do this

$hash_array = array(
  'timestamp' => (string)time(),  
  'domain' => 'localhost',
  'nonce' => user_password(),
  'rpc' => 'node.view',
);
  $hash = hash_hmac('sha256', implode(';', $hash_array), $api_key);

Note the order of the array elements. This is the exact order that the service_keyauth module builds the hash in. You’re taking the timestamp, domain, nonce and remote procedure name, turning them into a semicolon delimited string and hashing it with your API key. Since we’re going to be turning this into a string, the order of the array elements will affect the generated string, and therefore affect the generated hash.

Now that that is settled, here’s what your entire connection string should look like

/**
 * Tests a connection to a remote service.
 *
 * @return
 * Returns the data returned from the service call, FALSE otherwise
 */
function yourmodulename_service_call_test($nid) {
  // Check if the nid passed is a number
  if (is_int($nid) && $nid > 0) {
    $api_key = 'd371f3862693abb97121ba9cb1844234';
    $host = 'http://localhost/services/xmlrpc';
    $hash_array = array(
      'timestamp' => (string)time(),      
      'domain' => 'localhost',
      'nonce' => user_password(),
      'rpc' => 'node.view',
    );
    $hash = hash_hmac('sha256', implode(';', $hash_array), $api_key);
    $result = xmlrpc($host, $hash_array['rpc'], $hash, $hash_array['domain'], $hash_array['timestamp'], $hash_array['nonce'], $nid);
    if ($error = xmlrpc_error()) {
      if ($error->code <= 0) {
        $error->message = t('Outgoing HTTP request failed. Possibly because the socket could not be opened');
      }
      drupal_set_message(t('Could not get data because the remote site returned an error: %message (@code).', array('%message' => $error->message, '@code' => $error->code)));
    }
    else {
      return $result;
    }
  }
  return FALSE;
}

Step 8

Last but not least, trigger the function call and see what happens! I like devel for this. Go to /devel/php and put in the following

dpm(yourmodulename_service_call_test(5));

Be sure to change the node id to match your node instead. If you’ve been using devel/php directly, then you probably know you can just paste the code in from above, change your values and then see the results.

If all went well, you should see the output of dsm printing your node view.

Notes

Now some of you may have noticed that we’re missing a few things. We aren’t getting a session ID from the remote server before making our call. This is why we enabled the “System service” earlier. You would call “system.connect” first, get a session ID and then include it in the $hash_array and only pass it as parameter when making the xmlrpc() call. This is used to verify the user attempting to access that function and can trigger the proper access controls in Drupal (i.e. the ‘#access arguments’ for your service passed through user_access() get triggered for the user with the given session id). I’ll leave that as an exercise for the class. But if you have problems or better yet, an example solution for that, feel free to drop it into the comments.

Hopefully this helps clear things up with respect to Services and the Key Auth stuff and save your the headache I had this morning…

Till next time!