OpenSolaris

HAL reverse engineered

Artem Kachitchkine <artem.kachitchkin@sun.com>
Last updated: 4/3/2006

Table of Contents

1. 1. Introduction
2. 2. Architecture overview
3. 3. Properties
4. 4. Installed components
5. 5. HAL interfaces
6. 6. Source structure
7. 7. Hotplug and coldplug
8. 8. Sources of device information
   1. 8.1 sysfs
   2. 8.2 Probing
9. 9. Addons and callouts

1. Introduction

In the beginning I spent a fair amount of time reverse engineering HAL, as many of the implementation details are not in the spec (nor should they be), but as an engineer I wanted to know more. I captured my initial findings here, and though a bit dated, it can save time to the HAL newbies out there. This document can be used as a supplement to the HAL specification. There's a bit of Solaris bias in the article. Feel free to send be updates or requests for enhancements.

2. Architecture overview

Removable media managers are essentially specialized automounters. Vold plays this role in Solaris, ready to pass the reign to gnome-volume-manager and friends. It is important to understand that HAL itself is not a media manager: it does not mount anything, it only provides an abstration that allows to write the actual managers in a portable way.

HAL specification includes this short and rather precise summary:

HAL (Hardware Abstraction Layer) provides a view of the various hardware attached to a system [updated dynamically as hardware configuration changes, via hotplug or other mechanisms]. HAL represents a piece of hardware as a device object. A device object is identified by a unique identifer and carries a set of key/value pairs referred to as device properties. Some properties are derived from the actual hardware, some are merged from device information files (.fdi files) and some are related to the actual device configuration.

HAL provides an easy-to-use API through D-BUS which is an IPC framework that, among other things, provides a system-wide message-bus that allows applications to talk to one another. Specifically, D-BUS provides asynchronous notification such that HAL can notify other peers on the message-bus when devices are added and removed as well as when properties on a device are changing.

The most important goal of HAL is to provide plug-and-play facilities for UNIX-like desktops with focus on providing a rich and extensible description of device characteristics and features. HAL has no other major dependencies apart from D-BUS which, given sufficient infrastructure, allows it to be implemented on many UNIX-like systems. The major focus, initially, is systems running the Linux 2.6 series kernels.

D-BUS is a simple IPC (interprocess communication) protocol that allows asynchronous messages to be dispatched either through a daemon process (hub model), or directly between applications (peer-to-peer model). HAL exports two D-BUS interfaces: "org.freedesktop.Hal.Manager" for locating device objects and "org.freedesktop.Hal.Device" for individual device object services.

See also a high-level diagram of the HAL architecture.

Some important observations:

• Mounting/unmounting of filesystems is not part of HAL. Even though it might have the mounting code, it only invoked by an external application.
• HAL tries not to enforce any policies on any class of devices. External agents must define and enforce such policies, building on HAL foundation.
• HAL is largely a "one-way" abstraction: it presents devices in a generic way, but does not provide access to these devices. Applications simply extract the device path from the respective property and use OS-specific API to access. Some work is being done, however, to add access methods in the future.

Properties are structured into namespaces, separated by dot (.). E.g. storage.cdrom.read_speed property is part of the storage.cdrom namespace, which in turn is part of the storage namespace. Below are a few examples of properties from various namespaces (full list in the spec):

Here are two examples of HAL objects, taken with lshal command on Ubuntu Linux with HAL 0.5.2. Here's how a UHCI controller looks like:

udi = '/org/freedesktop/Hal/devices/usb_device_0_0_0000_00_1d_1'
  info.udi = '/org/freedesktop/Hal/devices/usb_device_0_0_0000_00_1d_1'  (string)
  linux.subsystem = 'usb'  (string)
  linux.hotplug_type = 1  (0x1)  (int)
  usb_device.bus_number = 2  (0x2)  (int)
  usb_device.can_wake_up = false  (bool)
  usb_device.is_self_powered = true  (bool)
  usb_device.version_bcd = 272  (0x110)  (int)
  usb_device.speed_bcd = 4608  (0x1200)  (int)
  usb_device.serial = '0000:00:1d.1'  (string)
  usb_device.linux.device_number = 1  (0x1)  (int)
  usb_device.num_ports = 2  (0x2)  (int)
  usb_device.max_power = 0  (0x0)  (int)
  usb_device.device_revision_bcd = 518  (0x206)  (int)
  info.product = 'Intel Corp. 82801CA/CAM USB (Hub #2)'  (string)
  usb_device.product = 'Intel Corp. 82801CA/CAM USB (Hub #2)'  (string)
  info.vendor = 'Linux 2.6.10-5-386 uhci_hcd'  (string)
  usb_device.vendor = 'Linux 2.6.10-5-386 uhci_hcd'  (string)
  usb_device.product_id = 0  (0x0)  (int)
  usb_device.vendor_id = 0  (0x0)  (int)
  usb_device.device_protocol = 0  (0x0)  (int)
  usb_device.device_subclass = 0  (0x0)  (int)
  usb_device.device_class = 9  (0x9)  (int)
  usb_device.num_interfaces = 1  (0x1)  (int)
  usb_device.num_configurations = 1  (0x1)  (int)
  usb_device.configuration_value = 1  (0x1)  (int)
  usb_device.linux.sysfs_path = '/sys/devices/pci0000:00/0000:00:1d.1/usb2'  (string)
  info.linux.driver = 'usb'  (string)
  info.bus = 'usb_device'  (string)
  info.parent = '/org/freedesktop/Hal/devices/pci_8086_2484'  (string)
  linux.sysfs_path_device = '/sys/devices/pci0000:00/0000:00:1d.1/usb2'  (string)
  linux.sysfs_path = '/sys/devices/pci0000:00/0000:00:1d.1/usb2'  (string)

This is the IDE hard drive:

udi = '/org/freedesktop/Hal/devices/storage_model_MSC_U02'
  info.addons = {'hald-addon-storage'} (string list)
  storage.policy.should_mount = true  (bool)
  block.storage_device = '/org/freedesktop/Hal/devices/storage_model_MSC_U02'  (string)
  info.udi = '/org/freedesktop/Hal/devices/storage_model_MSC_U02'  (string)
  storage.requires_eject = false  (bool)
  storage.hotpluggable = true  (bool)
  info.capabilities = {'storage', 'block'} (string list)
  info.category = 'storage'  (string)
  info.product = 'MSC-U02'  (string)
  info.vendor = 'Sony'  (string)
  storage.removable = true  (bool)
  storage.physical_device = '/org/freedesktop/Hal/devices/usb_device_54c_56_noserial_if0'  (string)
  storage.lun = 0  (0x0)  (int)
  storage.drive_type = 'disk'  (string)
  storage.vendor = 'Sony'  (string)
  storage.model = 'MSC-U02'  (string)
  storage.automount_enabled_hint = true  (bool)
  storage.media_check_enabled = true  (bool)
  storage.no_partitions_hint = false  (bool)
  storage.bus = 'usb'  (string)
  block.is_volume = false  (bool)
  block.minor = 0  (0x0)  (int)
  block.major = 8  (0x8)  (int)
  block.device = '/dev/sda'  (string)
  linux.hotplug_type = 3  (0x3)  (int)
  info.parent = '/org/freedesktop/Hal/devices/usb_device_54c_56_noserial_if0_scsi_host_scsi_device_lun0'  (string)
  linux.sysfs_path_device = '/sys/block/sda'  (string)
  linux.sysfs_path = '/sys/block/sda'  (string)

HAL is usually installed either in / or /usr/local. Following is the contents of HAL 0.5.2:

And these are programs that depend on libhal.so in the Ubuntu distro:

• gnome-cd
• gnome-video-thumbnailer
• gnome-volume-manager
• gnome-volume-properties
• nautilus-cd-burner
• sound-juicer
• totem
• totem-video-thumbnailer
• update-notifier

It is easy to get confused about HAL interfaces by looking at what interfaces HAL clients are using, for instance:

$ objdump -T gnome-volume-manager | grep -i hal
00000000      DF *UND*  00000092              hal_device_property_watch_all
00000000      DF *UND*  000001b8              hal_device_property_exists
00000000      DF *UND*  00000026              hal_free_string
00000000      DF *UND*  00000205              hal_find_device_by_capability
00000000      DF *UND*  00000206              hal_get_all_devices
00000000      DF *UND*  0000006d              hal_device_query_capability
00000000      DF *UND*  00000239              hal_device_get_property_string
00000000      DF *UND*  0000020f              hal_initialize
00000000      DF *UND*  000000a6              hal_shutdown
00000000      DF *UND*  00000050              hal_free_string_array
00000000      DF *UND*  00000252              hal_device_get_property_bool

These functions are part of libhal. However, this C API is only a collection of helper routines. The real HAL API is defined in terms of D-BUS methods, signals, interfaces and properties. For each D-BUS method and signal there is a corresponding libhal function, e.g. libhal_get_all_devices() corresponds to the "org.freedesktop.Hal.Manager"/"GetAllDevices" method.

There are other D-BUS bindings than C: Python, Java, Perl, etc. Any of these can be used to talk to the HAL daemon.

It is important to note that before HAL reaches 1.0 it does not guarantee any backwards compatibility whatsoever. In fact, HAL interfaces have changed dramatically in the past few months. Current rev is 0.5.6.

But C is still the dominant language for system programming in Solaris, and used by most volume managers, so I'm including a short summary of libhal.h and libhal-storage.h interfaces below.

udev(8) facility maintains the /dev namespace in Linux similarly to devfsadm(1M) in Solaris.

Linux kernel subsystems that support hotplugging trigger /sbin/hotplug helper with a bunch of env variables set: ACTION (add or remove), SUBSYSTEM (usb, pci), DEVPATH and a few others. Hotplug helper then invokes programs located in /etc/hotplug.d. udev installs a handler in /etc/hotplug.d, so /dev is dynamically updated as devices come and go.

As udev creates or removes /dev entries, it runs programs located in /etc/dev.d. HAL installs a handler in /etc/dev.d, hal.hotplug, which sends a message to hald daemon over datagram socket. See hald/linux2/osspec.c:osspec_init() for socket initialization and hald/linux2/osspec.c:hald_helper_data() for how the data is processed.

When HAL starts up, it has to discover devices that already exist. It does so by getting a list of devices from udev and synthesizing a hotplug event for each of them. See hald/linux2/coldplug.c.

8. Sources of device information

HAL collects device information from two main sources: sysfs and probing.

8.1 sysfs

HAL creates many device properties from sysfs - a filesystem that presents device information as a filesystem. E.g. UHCI controller, whose properties were shown in section 3 above, has linux.sysfs_device property set to /sys/devices/pci0000:00/0000:00:1d.1/usb2:

$ cd /sys/devices/pci0000:00/0000:00:1d.1/usb2
$ ls -l
total 0
drwxr-xr-x  3 root root    0 2005-07-10 22:51 2-0:1.0
-r--r--r--  1 root root 4096 2005-07-10 18:16 bcdDevice
-rw-r--r--  1 root root 4096 2005-07-10 22:51 bConfigurationValue
-r--r--r--  1 root root 4096 2005-07-10 18:16 bDeviceClass
-r--r--r--  1 root root 4096 2005-07-10 22:51 bDeviceProtocol
-r--r--r--  1 root root 4096 2005-07-10 22:51 bDeviceSubClass
-r--r--r--  1 root root 4096 2005-07-10 22:51 bmAttributes
-r--r--r--  1 root root 4096 2005-07-10 22:51 bMaxPower
-r--r--r--  1 root root 4096 2005-07-10 18:16 bNumConfigurations
-r--r--r--  1 root root 4096 2005-07-10 22:51 bNumInterfaces
-r--r--r--  1 root root 4096 2005-07-10 22:51 configuration
-rw-r--r--  1 root root 4096 2005-07-10 22:51 detach_state
-r--r--r--  1 root root 4096 2005-07-10 18:16 devnum
lrwxrwxrwx  1 root root    0 2005-07-10 18:16 driver -> ../../../../bus/usb/drivers/usb
-r--r--r--  1 root root 4096 2005-07-10 18:16 idProduct
-r--r--r--  1 root root 4096 2005-07-10 18:16 idVendor
-r--r--r--  1 root root 4096 2005-07-10 22:51 manufacturer
-r--r--r--  1 root root 4096 2005-07-10 22:51 maxchild
drwxr-xr-x  2 root root    0 2005-07-10 22:51 power
-r--r--r--  1 root root 4096 2005-07-10 22:51 product
-r--r--r--  1 root root 4096 2005-07-10 22:51 serial
-r--r--r--  1 root root 4096 2005-07-10 22:51 speed
-r--r--r--  1 root root 4096 2005-07-10 22:51 version
$ cat bDeviceClass bDeviceProtocol
09
00
$ cd "2-0:1.0"
$ ls -l
total 0
-r--r--r--  1 root root 4096 2005-07-10 22:51 bAlternateSetting
-r--r--r--  1 root root 4096 2005-07-10 18:16 bInterfaceClass
-r--r--r--  1 root root 4096 2005-07-10 22:51 bInterfaceNumber
-r--r--r--  1 root root 4096 2005-07-10 18:16 bInterfaceProtocol
-r--r--r--  1 root root 4096 2005-07-10 18:16 bInterfaceSubClass
-r--r--r--  1 root root 4096 2005-07-10 18:16 bNumEndpoints
-rw-r--r--  1 root root 4096 2005-07-10 22:51 detach_state
lrwxrwxrwx  1 root root    0 2005-07-10 18:16 driver -> ../../../../../bus/usb/drivers/hub
drwxr-xr-x  2 root root    0 2005-07-10 22:51 power

Solaris has libdevinfo(3LIB) for this purpose. Sun Ray has its own APIs.

8.2 Probing

Probes are special programs used to probe a device and fill out its properties. HAL forks a process for each device probe. Arguments are passed through environment variables; probes call libhal functions to set properties; and probe's exit status has a special meaning. HAL passes at least UDI and device path to the probe. Probe usually opens the device and issues OS-specific ioctls to collect information about the device. Examples of Linux HAL probes:

• probe-storage does drive_id for IDE and SCSI, but not for USB, because INQUIRY can hang many USB drives. It determines CD/DVD burning capabilities. It also does volume_id, but only to see if there is an fs present. Actual volume properties will be set in probe-volume.
• probe-volume calls volume_id and fills out properties such as fs type, partition info and CD content type (audio, data, blank).

Addons are used for various purposes, e.g.:

• addon-storage polls storage devices for media changes and also triggers device reprobes. It is only there because there are no kernel events for media changes, so HAL has to poll instead. This addon is used only for devices for devices with storage.media_check_enabled = true (grep fdi files to see where this is set) and there is one addon-storage process per such device.
• addon-acpi listens to ACPI events and asks HAL to reprobe a device when its state changes

Callouts are programs that are invoked when:

• devices are added/removed from GDL; these callouts are located in etc/hal/device.d
• device capabilities added/removed; located in etc/hal/capability.d
• properties added/removed/modified; located in etc/hal/property.d