A device register is not an ordinary memory variable

A device register uses an address, but accessing it can perform an operation. A write may start work, a read may consume an item, and a returned store instruction may leave the write waiting in an interconnect. Treating that address like an ordinary C variable can therefore lose data or mistake command delivery for completion.

This article is for software developers beginning Linux driver work. It assumes basic pointers and binary masks. We will separate access width, compiler behavior, bus delivery and device protocol, then execute a fictional device model. The Python program does not map hardware, issue MMIO transactions or measure a real bus. Its state is intentionally visible so you can inspect distinctions that a real driver must establish through documented interfaces.

Start with the register contract

Suppose a fictional device exposes three aligned, little-endian, 32-bit registers. CMD at offset 0 accepts a START write. STATUS at offset 4 reports BUSY in bit zero and DONE in bit one. RX at offset 8 returns the next received byte in its low eight bits, with the remaining bits zero, and consumes that byte. STATUS has no read-to-clear action and does not consume received data. A STATUS read can nevertheless deliver a pending command; that command delivery clears an old DONE flag, rather than the read intrinsically clearing it.

Those are separate contracts even though all three registers occupy four bytes. Reading RX twice obtains two queue items, not two observations of one stored variable. Writing a byte to CMD is not automatically equivalent to writing the specified 32-bit word. Splitting an access can change which transaction the device sees, and an unsupported access width can be invalid even when the numeric value looks correct.

Linux’s bus-independent device-access guide defines readb, readw, readl and readq for 8-, 16-, 32- and 64-bit quantities. Here, the historical name readl means a 32-bit device access; it does not follow the size of C long on the machine compiling the driver. Device byte order is another choice: the guide distinguishes generic accessors for little-endian peripherals from the big-endian accessor family and warns about architecture-specific exceptions.

Do not assume that a 64-bit register can always be read atomically on a 32-bit processor. The same guide provides low/high and high/low helpers for devices that permit split operations, with the order determined by the device. A register specification must establish whether splitting is allowed and whether a snapshot or latch protocol is required. Choosing a wider C integer does not create those hardware guarantees.

View full-size image ↗

The offsets and semantics are invented for the lesson. A real driver must obtain them from its device documentation and resource-discovery path.

Use an I/O mapping and its accessor API

A discovered device address is not an ordinary userspace pointer. In portable Linux kernel code, an MMIO mapping is represented by an __iomem pointer token, commonly obtained through the appropriate mapping helper. The token should be passed to I/O accessors. On some architectures it represents more than a directly dereferenceable virtual address, which is one reason a cast followed by a plain load is not a portable replacement.

Resource discovery, ownership, mapping attributes and cleanup also belong to the driver lifecycle. The introductory ioremap example in the device-access guide requires releasing the mapping with iounmap; resource-managed helpers can provide a different cleanup arrangement. This article does not provide a complete driver probe or permission to map an arbitrary address. The lesson starts after a valid mapping has been established according to the subsystem’s rules.

Likewise, ordinary memcpy and memset are not interchangeable with the kernel’s I/O-specific copy and fill helpers. An optimizing memory routine can choose access sizes or ordering that do not satisfy device semantics. Even an I/O copy helper is not a universal substitute for repeatedly accessing a FIFO register: whether addresses advance and whether each read consumes data must match the particular register interface.

Volatile does not establish every needed order

A volatile-qualified access can constrain compiler optimization under the implementation’s rules. It does not turn a shared value into a synchronization primitive or prove that another kind of memory access has become visible. GCC’s volatile-access documentation explicitly states that non-volatile accesses are not ordered with respect to volatile accesses. A normal-memory write followed by a volatile flag write is not, by that fact alone, a publication protocol.

The same documentation warns that volatile bit-fields can introduce implicit reads or partial accesses. That is dangerous when a device interprets reading as an operation, or when only certain access widths are supported. Use the specified full-width access and explicit masks where the register contract requires them; do not infer the transaction sequence from how conveniently a C structure represents the bits.

For Linux, Jonathan Corbet’s explanation of volatile in kernel code directs MMIO accesses through the accessor functions, which already handle the relevant compiler constraints. Locks and the appropriate synchronization APIs address concurrent software access. The article does not say that a bare-metal implementation can never use volatile internally; it explains why adding it to a driver pointer is not a substitute for the kernel interface.

Keep compiler ordering, CPU/interconnect ordering and device completion separate. A compiler barrier is not automatically a hardware barrier. A hardware ordering operation is not automatically a command-completion event. DMA visibility introduces another protocol, so writing a command that refers to a buffer requires the appropriate DMA mapping and ownership rules as well as correct MMIO.

Delivery and completion are different events

The Linux device-access guide explains that PCI writes can be posted: software can continue before the write has reached the device. In the documented situations, a read from the same device can ensure the earlier write has arrived. Choosing that read requires care. The register must be safe to read, the device must be accessible, and reset or bus-specific behavior may require a different documented mechanism.

Receiving a START write only tells us that the device can begin work. It does not establish that the work has finished. A device might expose a completion flag, a queue entry or an interrupt with its own acknowledgement rules. Those semantics determine completion. Reading a safe register to flush a posted write is not a replacement for them.

Our model makes the distinction explicit. write_start queues one command. deliver accepts it, sets BUSY and clears the previous DONE indication. Each successful register read first delivers a pending command. Only step_engine completes an accepted command. We choose one abstract engine step for simplicity, with no implied duration. Explicit delivery is also available; real interconnects need not wait for a software read to deliver writes.

The model permits only one command in flight and rejects a second submission until the accepted command completes. This single-command limitation is a teaching input restriction, not a claim that hardware or a real driver can inspect a posted-write Boolean. snapshot exposes simulator state only. Received input must be a finite sequence of integer bytes; Boolean and out-of-range values are rejected. Reading an empty receive queue is also rejected rather than inventing a device-specific empty-read result.

class Device:
    def __init__(self, received=(0x41, 0x42)):
        values = tuple(received)
        if any(type(v) is not int or not 0 <= v <= 255 for v in values):
            raise ValueError('received values must be bytes')
        self.received = list(values)
        self.posted = self.busy = self.done = False

    def write_start(self):
        if self.posted or self.busy:
            raise ValueError('this model permits only one command in flight')
        self.posted = True

    def deliver(self):
        if self.posted:
            self.posted = False
            self.busy, self.done = True, False

    def read_status(self):
        self.deliver()
        return int(self.busy) | (int(self.done) << 1)

    def read_rx(self):
        if not self.received:
            raise ValueError('this model rejects an empty receive read')
        self.deliver()
        return self.received.pop(0)

    def step_engine(self):
        if self.busy:
            self.busy, self.done = False, True

    def snapshot(self):
        return self.posted, self.busy, self.done, tuple(self.received)

safe = Device()
safe.write_start()
print('after write:', safe.snapshot())
safe.step_engine()
assert safe.snapshot() == (True, False, False, (65, 66))
print('status after delivery:', safe.read_status())
assert safe.snapshot() == (False, True, False, (65, 66))
safe.step_engine()
print('status after engine step:', safe.read_status())
assert safe.read_status() == 2 and safe.received == [65, 66]

consuming = Device()
consuming.write_start()
print('receive read:', consuming.read_rx())
assert consuming.snapshot() == (False, True, False, (66,))
print('next receive read:', consuming.read_rx())
assert consuming.received == []

after write: (True, False, False, (65, 66))
status after delivery: 1
status after engine step: 2
receive read: 65
next receive read: 66

After the initial write, the command is posted but the engine is idle. Advancing the engine at that point does nothing because no command has arrived. The STATUS read then delivers the command and returns one, meaning BUSY. Only after the engine step does STATUS return two, meaning DONE. Repeated STATUS reads preserve the receive queue under this fictional register contract.

View full-size image ↗

The second instance uses RX as the flushing read. It also delivers the command, but consumes byte 65. The next read returns byte 66. A debug print or a seemingly harmless read-back can therefore alter the state you meant to inspect. The model’s two register reads have the same delivery effect and different protocol effects.

Review each guarantee at its own boundary

The generic Linux accessors provide documented ordering guarantees, while relaxed variants intentionally provide a narrower contract. Do not select a relaxed accessor merely because it sounds faster. Identify what the particular access must order against, including DMA where relevant, and explain why the weaker operation is sufficient. The guide also warns that posted writes are not strictly ordered against a spinlock; software mutual exclusion and device receipt remain different questions.

For a driver review, identify the exact mapping, width, byte order and register side effects first. Then identify the compiler and architecture guarantees supplied by the chosen API, the point where a required write is known to have arrived, and the separate evidence for operation completion. If the last two rely on the same read, the device specification must establish both meanings. Neither one follows merely from the presence of a read instruction in source code.

Extend the model by completing a command through explicit deliver followed by an engine step before any STATUS read. Verify that the later read reports DONE without consuming receive data. Then try repeated START writes, an empty RX queue and a new command after a previous completion. Check that the new command’s delivery clears old DONE. These exercises expose assumptions about delivery, freshness and side effects before those assumptions become difficult hardware failures.