xmlrpcserver - a simple and complete XML-RPC server module for python

This document describes xmlrpcserver 0.99.2, the latest version, which was released on 2005/08/16. Download

Overview

xmlrpcserver is a small and simple but fairly complete XML-RPC server module for Python written by Julien Oster, implemented on top of the standard module
xmlrpclib. This module may, for example, be used in CGIs, inside application servers or within an application, or even standalone as an HTTP server waiting for XML-RPC requests. xmlrpcserver is completely written in python and has no dependencies other than python standard modules.

xmlrpcserver includes compliance to and announcement of the following de-facto standards:

Here is a completely operational example for usage as CGI:

#!/usr/bin/env python

import sys
from xmlrpcserver import XmlRpcServer

def testmethod(meta, argument1, argument2):
    print (meta, argument1, argument2)
    return [1, 2, 3, 4]

server = XmlRpcServer()
server.register('test.test', testmethod)

# read the request, perform the call and send the response
sys.stdout.write("Content-type: text/xml\n\n")
server.execute()
Methods are always called with a metadata dictionary as first argument, which is filled by the XmlRpcServer and may be filled with additional information at execute()-time, and then with exactly the arguments as specified in the originating XML-RPC request. The XML-RPC response is the return value of the called method or a Fault if a Fault or Exception is thrown.

In addition to the possibility of registering single methods you may also register whole classes.

xmlrpcserver also includes a handy standalone HTTP server, XmlRpcHTTPServer, derived from both BaseHTTPServer.HTTPServer and XmlRpcServer. Additional connection data is provided with the metadata dictionary.

Table of Contents

About XML-RPC and this module

Quoted from
xmlrpc.com:
XML-RPC is a Remote Procedure Calling protocol that works over the Internet.

An XML-RPC message is an HTTP-POST request. The body of the request is in XML. A procedure executes on the server and the value it returns is also formatted in XML.

Procedure parameters can be scalars, numbers, strings, dates, etc.; and can also be complex record and list structures.

The whole XML-RPC Specification is also available at xmlrpc.com.

The Python standard distribution already includes a handy module named xmlrpclib which implements the general parts of the XML-RPC specification and an XML-RPC client. Part of those general parts are convenience functions and classes to marshall and unmarshall XML-RPC requests and responses which may also be used to implement an XML-RPC server. Doing so is pretty easy for small tasks, however, if you want to do it right, you'll quickly find yourself reimplementing things like method dispatching, fault and exception handling and even standard methods over and over again.

While implementing a small XML-RPC server using xmlrpclib for another project, that small server quickly evolved into a more powerful and simple to use generic XML-RPC server which, in the meantime also being used in other projects, allowed me to instantiate a new server in just one line, register methods to it in no time and serving requests as CGI or in Web Applications without noticeable code overhead.

So, I eventually decided that it may be useful for others, as it reduces implementation of new custom XML-RPC servers to virtually no time and started to clean up the code, document it, package it and release it online. Well, here's the result, and I hope you find it useful and like working with it. Even if you don't, you can still contact me according to the Contact section of this document for bug reports, improvement and feature requests, patches, flames or pie. I especially like the "pie" part.

Prerequisites

xmlrpcserver is a Python module and thus needs a Python interpreter. It has been tested with Python versions 2.3 and 2.4. It may work with earlier implementations, but this has not been tested.

xmlrpcserver is completely written in Python, with no additional code in C or any other language. It currently uses the Python modules sys, xmlrpclib, StringIO and BaseHTTPServer, all of which are Python standard modules and thus available with the Python standard distribution.

No additional modules or software is needed.

Installaton

Since xmlrpcserver.py contains of just one plain Python module and no additional code which would need to be compiled or similar, simply copy the xmlrpcserver.py file to somwhere to your Python module path or include it in your own project, where you can import it with
import xmlrpcserver
or another similar line, depending on where you packaged it in your project.

Usage

Usage of xmlrpcserver is intended to be as simple and straightforward as possible, while still retaining the possibility to do more complex things and tweaks.

See the Overview section of this document for an already operational example of a simple CGI.

obtaining an XmlRpcServer instance

Your first step is to create an instance of the XmlRpcServer class.

XmlRpcServer is the class containing the actual XML-RPC server, which reads a request from a stream, unmarshalls the request using xmlrpclib and dispatches the call to a method priorly registered to this server. After the call finished, it sends out the marshalled response consisting of either the return value of the called method if it succeeded or a XML-RPC Fault if the method raised a Fault or any exception.

The constructor of XmlRpcServer is called with the following arguments:

    XmlRpcServer(rpcmethods={}, rpcclasses={})
The parameters rpcmethods and rpcclasses can be used to preinitialize methods and classes which are to be served, see the register() and register_classes() methods below for details of key and value. If you decide to register your methods and classes later, you can just do:
    from xmlrpcserver import XmlRpcServer

    server = XmlRpcServer()

registering methods

Now that you have a server instance, you theoretically have everything you need to receive XML-RPC requests from a stream and get methods called. However, you probably want to register any methods so there's actually anything to get called via XML-RPC (apart from the preregistered internal methods, which we will talk about later). You can do this using the register() method of your server, which takes the following arguments:
    register(methodname, method)
methodname is the name of the XML-RPC method as it appears in the request. Note that when registering a method with register(), you have to specify the whole method name as it appears in the request, including any possible class names and dots. Of course, the method name in the XML-RPC request may be completely different from the real (Python) method or function that is actually called.

method is the Python method to be called by the server if a request with the specified method name comes in. method can be an actual method of a class instance or just a plain python function. Specifically, method can be anything in python which may be called, including functions, methods, classes which implement the __call__() operator or whatever else there may be.

Here's an example of a function being registered using register():

    def myfunction (meta, argument1, argument2):
        print (meta, argument1, argument2)
        return 'success!'

    server.register('myclass.mymember', myfunction)
Again, as you can see, in the case of using register(), the XML-RPC member name doesn't have to resemble any existing object at all.

If you have a lot of methods to register which all have the same class in their name (myclass in the example above), you may choose to register a whole class instead of just a method, using register_class(). In that case, you specify only the class name as it appears in the XML-RPC request and the class (or class instance) where the call should be dispatched to. The method which should be called is derived from the method name in the XML-RPC request.

A quick example:

    class myexampleclass:
        def method1 (self, meta, oneargument, anotherone):
            print (meta, oneargument, anotherone)
            return 0

        def method2 (self, meta, *args)
            sys.exit(0)

    myinstance = myexampleclass()
    server.register_class('myclass', myinstance)

Now, if an XML-RPC request may request the call of myclass.method1 or myclass.method2, both requests will succeed and result in the call of the appropriate method.

Although the XML-RPC class name may still be completely different from the real class/instance name, the XML-RPC method name is now directly the name of the python method being called in the registered class or instance.

However, when an XML-RPC request resolves to a registered class, methods starting with '_' (underscore) are never called. This is to protect Python internal attributes (starting with '__') from being maliciously called via XML-RPC and to allow you to add 'internal' methods to your classes which can only be called from inside your server code.

WARNING: Be VERY careful when using register_class(), as this may open numerous security risks. As soon as a whole class is registered to XML-RPC server, the client may not only ask ANY method within that class to be called, whether you intended it to be available via XML-RPC or not (well, if it's something you registered, you should have intended it), but also anything else there which is callable, as long as its name doesn't start with '_'. If you are not entirely sure that the class you registered is safe of 'unwanted' methods in any aspect, use register() instead to dedicately register every intended method of that class or instance!

In addition to registering methods and classes via register() and register_class(), you may also register them at server instantiation time by passing appropriate dictionaries as rpcmethods and rpcclasses argument to the constructor. Those consist of, respectively, the method name as key and the method as value or the class name as key and the class or instance as value.

Registered methods or classes may be unregistered at any time using unregister(methodname) or unregister_class(classname).

processing a request

Processing an incoming request is done by executing the execute() method of your XmlRpcServer instance. execute() is taking the request from a stream (possibly waiting on a blocking read) and processes it.

If no exception (other than an xmlrpclib.Fault) occured, execute() returns a tuple of the form (rpccall, params, result, meta) which consists of the complete method name (including class and all dots) of the request, its parameters, the result (which may also be a xmlrpclib.Fault) and the supplied meta data. This is useful for logging purposes.

execute() arguments

  execute (request=sys.stdin, response=sys.stdout, meta=None)
request and response are both streams. For a stream, any file-like object can be used, like a file, a socket or something else which implements the relevant methods. Currently, only the read() and readline() methods for request and only the write() method for response need to be implemented.

If you want to use strings for request and response, you can do so by using the StringIO module which wraps strings in streams.

request is the stream where the incoming XML request document is read from. No seek is performed, prior to reading, so the current position of the stream must point to the beginning of the document (this is already fine for CGIs, their stdin should only contain that document).

response is the stream where every response gets written to, again as an appropriate XML document. It may be the same stream as request, but doesn't have to.

meta is a dictionary consisting of additional metadata you might want to provide. It can be left empty or 'None', server metadata will still be filled in. The whole subject of metadata is subscribed later.

execute() returns after exactly one XML-RPC request has been processed, specifically: after processing one incoming request from the 'request' stream, calling the method if possible and sending out the response to the 'response' stream. If you want to serve several requests consecutively, you have to loop yourself.

execute() return value

if successful, execute() returns a tuple consisting of, in order: the full method name as it appeared in the XML-RPC request, the return value of the called method and a tuple containing the arguments of the XML-RPC request.

example: CGI usage

The default arguments for request and response are sys.stdin and sys.stdout. This is especially suitable for CGIs, where you are ready to go without noteworthy overhead. The request XML document will be available as the only thing on stdin and you are almost ready to send your response back to the client to stdout.

Almost, because prior to sending content, the webserver requires you to at least send a "Content-type" header, possibly along with other headers you might want to send.

However, once you called execute() the only possible output following from that point on can and will be an XML document. So you can hardwire the "Content-type" header by doing this prior to execute():

      sys.stdout.write("Content-type: text/xml\n\n")
Thus, a completely operational example of an XML-RPC server CGI looks like this:
      #!/usr/bin/env python

      import sys
      from xmlrpc import XmlRpcServer

      server = XmlRpcServer()

      # you might want to register methods here

      sys.stdout.write("Content-type: text/xml\n\n")
      server.execute()

method arguments

Using xmlrpcserver, XML-RPC methods or, more specifically, its resulting callable object, gets called with the following arguments:
So, suppose a client issues a request representing the call of myclass.mymethod(1234, 'hello server!', [9,8,7]) and the python function myfunction is registered as name myclass.mymethod, then the server will perform equivalently to the following manual call:
    myfunction({...}, 'hello server!', [9, 8, 7])
As you can see, pretty much everything is preserved and the only significant difference is the addition of a metadata dictionary (which will be described later).

This also means that your callable which you map your XML-RPC method to should be defined to accept the right number of arguments. Failure to do so will result in Python throwing a TypeError exception which is catched by XmlRpcServer, reported to the calling client as a XML-RPC Fault (-32500, application error) and then rethrowed to your application (see the section about exceptions and faults for details).

However, if you'd like your method to be called with a variable number of arguments, you can of course just use default arguments or python's syntax for arbitrary argument lists to get them wrapped up in a tuple. Since xmlrpcserver translates XML-RPCs to ordinary calls, everything is just like you would do it without XML-RPC:

    def withdefaultargs (meta, mandatoryarg, optionalarg=1234):
        ...
    def completelyvariable (meta, *variableargs):
        ...

the metadata dictionary

The metadata allows you to provide any data you want to the function or other callable which gets called as first argument. It is a dictionary and may contain everything which is allowed in a dictionary (which is, pretty much, everything).

The metadata dictionary is filled when you call the server instance's execute() method, where you provide it as the "meta" parameter.

The server itself adds the following metadata key/value pairs. Note that any already existing pair with one of those keys is silently replaced. Thus, to prevent confusion in future versions, every server generated metadata key is prefixed with '_' (underscore) and the caller should not insert any metadata string keys starting with '_' by itself.

method return value

If no Fault or exception occurs, XmlRpcServer wraps up the return value of your function in an XML-RPC method response and sends it back through the output stream. Any type supported by
xmlrpclib is supported here (which is quite a lot). As with method arguments, the return value is preserved "as is", only being marshalled to an XML-RPC response using xmlrpclib. xmlrpclib's allow_none flag is enabled, so you may even return None or not specify any return value at all. If you don't want None values for compliance reasons, just don't return any.

Simple examples:

    return 42
    return 'hello, XML!'
    return [12, 'foo', None, (2, 3)]
    return
    return {'key': 12, 'anotherkey': [1,2,3]}
If your method raises a Fault or an exception, an XML-RPC Fault is generated instead of a return value. This is explained more thoroughly in the very next section.

Fault and exception handling

When your function or other callable is being processed, the following things may happen:

1. Your function (or whatever, let's just call it 'function' for simplicity) processes correctly and returns a value, or no value.

This is the most simple case. The return value (or None) is marshalled into an XML-RPC response and sent back to the client.

2. Your function explicitly generates an XML-RPC Fault, by raising an xmlrpclib.Fault exception with appropriate error code and description.

In this case, your function didn't return any value, because it left its execution flow by raising an exception instead of returning. However, no return value is needed, because the generated Fault is being taken and passed to the client as XML-RPC Fault response. Your Fault code and description are not modified in any way.

After the response is sent into the stream, the whole call is considered "completed" and no further exception or processing is being performed, exactly the same state that would result from a normal function return.

XmlRpcServer itself only generates codes specified in the Specification for Fault Code Interoperability, but your code may generate any other code and description and it will be passed to the client untouched. In fact this is the preferred method, since the interoperability Fault codes are mostly specified for server errors. However, in some cases it may perfectly make sense to, for example, creating a Fault with code -32602 which means "invalid method parameters".

3. Your function or anything within it raises some other exception unrelated to xmlrpclib.Fault. xmlrpcserver doesn't want to mask out possibly fatal exceptions while at the same time it would like to automatically inform the client that the call resulted in an uncatched exception. However, xmlrpcserver has no means to differentiate between fatal or non-fatal exceptions. So, this is what happens:

XmlRpcServer catches the exception. It then immediately proceeds to send out an XML-RPC Fault with code -32500 (meaning 'application error'). After the response it sent, XmlRpcServer does NOT continue like it would with a normal call return or a catched XML-RPC Fault.

Instead, it reraises the exception to the originating caller of the XmlRpcServer.execute() method. The reraised exception is now free to be catched by your application where you may still choose to ignore it (for example, because it's a TypeError which very often just means that the client called a method with an incorrect number of arguments, thus not being your problem).

internal methods

Internal methods are built-in server methods which are preregistered into the server, providing some basic methods applicable to every XML-RPC server.

Internal method may be unregistered just like every other method, in case you don't want them being handled. Simply use the unregister() method (they are always registered as method, never as class).

The following internal methods are available:

system.getCapabilities()

system.getCapabilities() takes no arguments (-32602, 'invalid method parameters' is reported to the client otherwise) and returns a list of implemented server capabilities, according to
http://groups.yahoo.com/group/xml-rpc/message/2897

Supported and reported capabilities are listed in the Overview of this document.

system.listMethods()

system.listMethods() takes no arguments (-32602, 'invalid method parameters' is reported to the client otherwise) and returns a list of callable methods available on this server instance.

system.methodHelp(methodname)

system.methodHelp() takes one argument, the full name of an available method (-32602, 'invalid method parameters' is reported to the client otherwise). It returns the Python doc-String of the callable mapped to the supplied method name or an empty string if no doc-String is available.

Fault -32601 ('non-existing method') is raised in case that the supplied method name isn't registered.

further tweaking

Each instance contains the following publically relevant preinitialized attributes, which may be read or changed:
self.systemerrors
A dictionary consisting of an error number as key and an error description as value. It is used when the server constructs an XML-RPC Fault itself and NOT used when it passes on a fault generated by the called function. You may change it if you don't like xmlrpcserver's standard descriptions for server errors.
self.capabilities
A dictionary consisting of a capability name as key and a tuple as value. The tuple's first element is the specification's URL of the named capability and its second element a version number as int. See
http://groups.yahoo.com/group/xml-rpc/message/2897 for specification of this.

You may change it for adding new capabilities or removing any capabilities you feel the server shouldn't announce. (If you want to disable the whole system.getCapabilities() method, just unregister() it)

XmlRpcHTTPServer

This is a small but fully functional HTTP Server which serves XML-RPC. It's derived from both XmlRpcServer and the HTTPServer of the BaseHTTPServer module. It's usage is pretty straightforward, see the end of xmlrpcserver.py itself for an example. The following metadata is additionally provided with each call: _client_address, _path, _request_version, _headers

Acknowledgements

Thanks to Frederik Lundh and Secret Labs AB for writing the very good xmlrpclib and thus enormously accelerating development time for xmlrpcserver.

Thanks to Guido van Rossum and everyone involved in the development of Python for making my most favourite programming language, by far.

Thanks to my friends who supported me during development by suggestions or testing.

Contact the author

The author, Julien Oster, can be reached at julien-xmlrpc@julien-oster.de, his homepage is available at http://www.julien-oster.de. Julien Oster is not paranoid about receiving mails and may be reached for pretty much anything which comes into your mind, even things which may have nothing to do with this project. Except for spam, of which he has already received and accumulated a big enough supply in his mailbox, enough to survive the next winter. However, please only expect Julien to answer on reasonable subjects.

Download

You can always get the latest version of xmlrpcserver for Python here:

http://www.julien-oster.de/projects/xmlrpcserver/dist

License

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

Changelog

0.99.1: (2005/08/14) initial release

0.99.2: (2005/08/16)