Userspace Driver Market

Introduction

The Userspace Driver Market does for userspace drivers what IOKit matching does for kernel drivers. It is a forum in which drivers can haggle for the devices they want to own. It is a central arena where the system brings newly discovered devices, so that potential drivers can see them.

As a bonus, the Market makes it easy to publish services provided by drivers through Rendezvous. In fact it can even publish services of kernel drivers that wish to participate, if the driver must be a kernel driver. There is even a Preference Pane where the user can select which devices ought to be shared with the local network.

The Market is a missing piece of Mac OS X. When developing a driver for a USB or FireWire device, the developer is faced with a choice of two paths: the IOKit kernel driver or the IOKitLib userspace driver. Kernel drivers are difficult to debug, and errors can be catastrophic. But IOKit has an arbitration system to choose the best driver for a device, and new devices are automatically matched with drivers when they appear. It is also easy to provide services to applications through a device-class protocol (e.g. standard WebCam protocol for any web cam) using IOKit's 'user client' API.

In contrast IOKitLib ties the driver for a device to the application that uses it. This means multiple drivers cannot compete for the device to find the best driver. It means that when a device is plugged in, the application must be running to do anything about it (or a daemon must be running all the time whose only purpose is to check for this). And most importantly, it means that the device is bound to that specific application - if another WebCam application wants to use the device, then it must implement a driver for the device itself - and two WebCam applications cannot access the camera at the same time. These are just some of the undesirable consequences of binding up two things which are logically (and easily!) separable.

The Market brings together all the benefits of writing a kernel driver (matching, discovery, publishing) with those of writing a userspace driver (debugging, safety, admin access not required). There are of course still reasons to go to the kernel (network card driver) or straight into an application (hardware dongle checker), but for many drivers the Market is the perfect environment. And most existing in-application drivers should really consider putting themselves onto the Market to get the free sharing via Rendezvous that it provides.

Sometimes there is a well-defined protocol for a device-class over a particular transport, that would suggest a generic driver in an application would be fine. However, usually applications do not care about the protocol - for example, iMovie did not support USB 2.0 DV cameras for a long time since it only implemented a FireWire driver, and there was no sane way for anyone to write a driver for a new device. Also, hardware devices are notorious for not sticking to specifications, are extending them in device-specific ways. Separating the application from the driver/transport-specific-protocol allows these idiosyncrasies to be accommodated.

How it works

Drivers bundles are put into the folder "Library/Market Drivers", either by the administrator into the root directory, or by any user into their personal Library directory. The user drivers are available only when that user is logged in. The root directory drivers are available all the time, even when no user is logged in.

Drivers have keys in their Info.plist that specify what kind of devices they can drive. These keys are very similar to the matching keys used in kernel IOKit drivers.

The process 'udmarket' runs all the time as background daemon. It reads the Info.plist files of any installed drivers. At startup, and whenever a new USB or FireWire device is plugged in, the daemon receives a notification and attempts to find a driver for the device. All the drivers compete, and the one that wins is matched to the device, and then owns it until it gives it up, the device goes away, or a better driver comes along (IOKit cannot do that last part in fact).

What the driver bundle does when it gets control of the device is up to it. It can just initialise the device then sit there doing nothing. It can create a new thread (of udmarket) to drive the device. It can create a new process to drive the device. And independent of any of those, it can register a service (or multiple services) with udmarket for publishing to the world.

A service is an Objective-C object that applications can use to access the functionality of the device. A service is always of some well-defined type, e.g. a WebCam service, a MMInput service, a RemoteControlReceiver service. A service type implies a protocol that defines the calls and arguments that may be made by its users. Shared memory mappings may also be specified as part of the protocol.

Services are published with Rendezvous, so that any apps on the local machine or on the network can see and use the service. Of course, it is up to the user whether or not the services provided by the drivers for their devices should be shared with the rest of the network. Rendezvous also automatically notifies everyone when a service is withdrawn (e.g. device unplugged).

When a new service is published, udmarket can optionally launch an application (or daemon) that has been identified by the user as the default app to launch for that kind of service. The driver can give a hint about what app it would prefer too. e.g. open a TV app when a TV tuner is plugged in.

Applications search for services with the normal Rendezvous methods. Rendezvous can easily restrict its searches to a local machine if the user has indicated to the application that it is only interested in devices on the local machine. There is a small helper library to extract the service object once it has been located, so the methods in its protocol may be called naturally. This library takes care of setting up shared memory mappings between processes on the same machine, or emulating them when they are on different machines.

Services may be derived from IOKit kernel drivers by creating a 'driver' for their user client. The amount of processing that happens locally in the service object in the application and the amount that happens remotely in the driver is up to the backer of the service. In the case of a service derived from an IOKit kernel driver, the local service object would talk directly to the user client if it is on the same machine - so there is no unnecessary overhead.


Remote Control Receiver Family and Daemon

The Remote Control Receiver Family defines a protocol that can be used to get button presses from remote control devices, as received by a remote control receiver. Applications can use RCR services directly if they want to get information directly from a RCR. But practically all applications will only need the events that the daemon sends.

The Remote Control Receiver Daemon listens for all RCR services all the time. By default it only listens on the local machine, but through the preference pane you can select devices on other machines in the network. When it finds a RCR service it opens it and registers for events. When it gets events, it translates them into keypresses or mouse movements. And it sends a special 'remote control button' event through the HID event system too.

The configuration of which remote button->key mapping to use is done in the preference pane. There are a whole lot of buttons that are usually the same on all remotes. These are easily mapped. There are almost always also device-specific buttons. To make it worse, not all RCRs can receive the full button identifier information, sometimes only the last few bits are available.

The Preference Pane allows the user to match up (multiple) predefined button layouts with particular RCRs. It also allows the user to specify what action should be taken when the button is pressed, first on an application-specific level, then if there is no application-specific setting, on a system-wide level. OK there's going to be quite a few scrolling lists here! The actions can be generating key presses or mouse movement events, running applescripts, or running shell scripts. Some common operations are provided through applescripts, including launching/switching to an application, quitting an application, switching windows, changing the volume up or down, and probably some more I haven't thought of yet.


IP over DVB linkage

This is done with a daemon + preference pane, or an application. Probably go for a daemon so it can run when no-one is logged in.

There isn't much point looking for remote devices on the network (may as well be done on that machine), so it just looks on the local machine. When the user configures a device to be used for IP over DVB, the userspace app takes care of obtaining the device (sorry, service) and tuning it to the appropriate frequency. (Other apps looking at the device will still be able to access stuff on the same multiplex.)

For getting the actual data packets into the kernel network stack, there is a kext which registers itself as a IPoverDVB linker. The daemon finds this user client and tells it the memory mapping of the driver's data buffer. This memory was quite likely originally from a PCI card driver in the kernel, but it's made a trip through a userspace application now, so that we can work with Market drivers too. So it looks like my wonderful in-kernel MMInput API will never be used after all, boo hoo :(. Then the kext pulls the packets straight out the buffer and puts the IP datagrams into the network stack.


Startup and shutdown daemon

It sucks that everyone has to make their own one of these. Still it doesn't really fit in the udm and trying to add it there would only pollute it and diminish its appeal. So it looks like iTele will have its very own daemon for this too. Only necessary when you have made scheduled recordings and ticked the 'turn off my computer' or 'put my computer to sleep' when it's doing nothing box.


Service arbitration

Issue remaining to be integrated: sharing services. Pretty much up to the service vendor. First in best dressed works for most devices (e.g. scanners). Some can queue up requests as they are not time-sensitive (e.g. printers). Some can provide a lower level of service when opened for the second time (e.g. tv tuners - can't change freq but can still select pids to get added to buffer). And others can provide full service no matter how many have opened them (RCRs, (most) web cams).

If the first opener of a tuner goes away, then the second one wants to inherit full control. So there will have to be some unsolicited notification about this. Incredibly fortunately, distributed Objective-C provides exactly this! And it can be 'oneway' so we needn't worry about blocking there and getting our state stuffed up.

Note that the local user can always turn off sharing of the device if they want control of it themselves.


OK that's it! First version 'till early on 13th of April 2005.