[lowrisc-dev] Re: [sw-dev] Is a de facto standard memory map helpful or harmful?

Arnd Bergmann arnd at arndb.de
Thu Jul 28 15:38:55 BST 2016

On Friday, July 22, 2016 2:14:57 PM CEST Wesley Terpstra wrote:
> Since no one else responded, I end up again defending the config
> string. Here we go...
> On Wed, Jul 20, 2016 at 2:32 AM, Arnd Bergmann <arnd at arndb.de> wrote:
> > My reading of the config string interface is that it assumes that a
> > type of information can always be expressed as a string or a single
> > integer number
> That's how the convenience methods I wrote work, but actually, the
> config string allows an arbitrary number of integers or strings
> following a key. I was going to add an additional 'nth' parameter to
> those methods.


> > In a system-wide view, having global numbers or names as identifiers
> > for resources does not scale, because you can have additional
> > providers for things like irqs, gpios, dma, leds, ... on external buses,
> The config string is 'XML-like' in that you have recursive
> descriptions, so the complete path need be globally unique.

What I mean is that you may need to represent a device that has
multiple interrupts that are handled by different controllers,
so each one needs to be described by

- name of the irq as seen from the slave ("rx data", "tx data",
- path to the controller
- identifier as interpreted by that irqchip
- additional data needed by the irqchip

In devicetree we went through several additions here until we
could describe all hardware. Open Firmware started out assuming
there would be only one irqchip in the system, possibly with
just a single number for the irq. Then the "#interrupt-cells"
property was added to describe complex irq-chips, then
"interrupt-parent" was added to allow each device have its
interrupts get directed to a particular irqchip, and "interrupt-map"
was added to allow devices to have interrupts on multiple buses.
This was all part of ieee-1275, and after we introduced the flattened
format for Linux, we extended it futher with "interrupt-names"
to allow compatibility with Linux drivers that refer to each
interrupt line by a string (e.g. "rxdata") instead of picking
the first or second interrupt from an array, and finally we added
"interrupts-extended" because the syntax of the old "interrupt-map"
property was considered just too impractical.

For things that came later (dma, clocks, iommu), we tried to avoid
those mistakes and made something that always works like
"interrupts-extended", but that still requires the separate
"-names" property.

I'd suggest that if you require a reference to another node,
you follow a similar model to what we ended up with dt, with
a set of "local name", "provider", "identifier" and "extra
attributes", to learn from the mistakes we made in the past.

> > - an irqchip needing to pass local hwirq number plus flags (polarity,
> >   type of interrupt) that the consumer driver doesn't know.
> > - dmaengine requiring request number plus dma master identifier
> > - gpio passing some settings (bank-nr, polarity, drive strength, ...)
> > - pwm period and flags
> I don't see a problem with most of these. You can have hierarchical
> information at any given path in the config string, and drivers attach
> at the point in the hierarchy where they assume responsibility.


> > Note that you should always be able to describe a full system
> > with a flattened device tree, even if it started out using some
> > other format (atags, fex, bios, ...), since the DT is basically
> > a superset of information that you can have elsewhere.
> Right. Except when you translate between formats things might end up
> either missing or in an unexpected location.

I meant a hand-written dts file translated into a dtb here, not
an automatic conversion.

> > There is already an ongoing unification with the "device properties"
> > interface that allows you to query generic properties from a
> > 'struct device' in Linux, regardless of whether it was instantiated
> > from architecture code (no legacy machines), using devicetree
> > (on most modern embedded systems) or from ACPI (on embedded x86),
> > since all of them have a way to attach numbers or strings to named
> > keys.
> If we stick with the config string, I will definitely explore adding
> the config-string meta-data to this interface.


> > The same thing can of course be used for querying config strings
> > that are embedded in a RISC-V SoC, but it quickly becomes awkward
> > when you need to have cross-references to other nodes, as those
> > cannot easily be generalised.
> This is one of my main concerns. It's true that device tree has a lot
> of cruft, but it's also a widely used spec, so some of what looks
> overly complex actually solves a problem you didn't know existed.
> Case-and-point: cross references. This is something the config-string
> does not have, although I am sure its advocates would say: "Nothing
> prevents you from including a string that refers to another path in
> the hierarchy". However, unless that cross-reference format has been
> specified somewhere, you won't get the sort of reuse that makes a
> general-purpose parser and convenience functions available.

Right. In DT, the cross-references are what makes things work, since
originally the assumption was that a system could be represented
as a tree. This was correct 32 years ago, but no longer suffices
to describe any modern machine. A lot of nodes are slaves to multiple
buses (i2c, mmio, ...), take references from nodes elsewhere in 
the tree (gpio, irq, dmarq, ...) and are masters on other buses
that may be further up in the hierarchy.

> > The device tree is used for several things beyond that today, each
> > of which needs a solution:
> >
> > - passing configuration from the bootloader, including amount of
> >   installed memory, root console settings and the kernel command
> >   line. These are often handled in some architecture specific
> >   way and I assume there is something in place already
> Currently you get this information from a combination of SBI
> (essentially the BIOS) calls and the config string. We were planning
> to reduce the number of SBI calls to the bare minimum needed to hand
> off the config string, and then put all the information you need in
> there.

Hmm, that sounds dangerous, as it repeats what has been criticized
a lot for DT, that it mixes up runtime configuration with
hardware description too much. If you continue down that path,
you may end up with a similar convolut of data that is in DT,
just incompatible.

I think ideally a new hardware description format would be done
in a way that it only describes what is within the SoC, and
then you generate that data along with the HDL in a way that makes
it impossible to diverge from what is actually in present in
the hardware.

The way Xilinx handles it for their FPGAs (if I understand
it correctly) is that they generate a .dtsi file along with
the hardware model and have all the data in that file synchronized
with the hardware that is used, and a board design would include
that .dtsi file in its .dts file to get a full system description.

> >   If the config strings can be embedded in each on-chip device
> >   in a way that they are always accessible without the need
> >   for a boot loader interface in the way that PCI is discoverable,
> >   that means we don't need DT for this and can just have a
> >   single DT node for the SoC itself, and make that present itself
> >   as a combined irqchip/gpiochip/pwm/led/clk/reset/mmio/...
> >   provider.
> Currently, the idea was to bake the config string for the SoC into the
> boot ROM for that SoC. That means one ROM for the whole SoC, not one
> ROM per device.

Ok, that should work just as well, and makes it easier to reuse
existing IP blocks that don't follow a particular scheme of
where to store the data.

> > - describing off-chip non-discoverable buses, there really
> >   isn't much of an alternative to DT, as these can be rather
> >   complex, each device can both be a consumer and a provider
> >   of resources (e.g. gpios or regulators controlled over i2c),
> >   and we don't want to describe individual machines in source
> >   code any more. embedded x86 machines with ACPI can get around
> >   some of these problems by sticking data that is compatible
> >   with the DT format into ACPI tables.
> This is a definite gap in the current scheme. Probably the best answer
> with the current system is that the boot loader for that board would
> inject additional data describing the i2c components on the board into
> the config string. However, DT already has a scheme for overlaying
> data like this, and IMO it seems rather foolish to reinvent the wheel
> with the config string.

Right, and it's related to what I think is the biggest question:

How much do you actually want to describe in the config strings?

I can immediately see the use for describing on-chip components,
and that would be manageable, you just need a couple of subsystem
specific handlers for describing clocks, regulators, irqchips, dma,
mmio etc. Inside of the chip you can probably make some
simplifications by assuming there is only one type of each of
those, or by enforcing some structure within the subsystem that
is not valid across other architectures.

Doing system-level configuration requires more stuff, but it's
mostly architecture specific.

Describing off-chip devices however would need another huge set
of subsystem specific changes in pinmux, gpio, led, pwm, chipselect,
i2c, spi, pci, usb, ...

Once you try to do all of the above, you will have the same
amount of complexity that we have in DT, and have annoyed all
subsystem maintainers that don't care the least about the
architecture but now have to worry about yet another incompatible
representation. Doing the initial DT support for each subsystem
met significant resistence from subsystem maintainers because
of this, for embedded ACPI it got worse, and adding a third way
to describe the same things will be even harder, no matter whether
that method is actually nicer or not.


More information about the lowrisc-dev mailing list