[lowrisc-dev] Linux booting help

Francesco Viggiano francesco.vgg at gmail.com
Tue Jul 5 23:12:29 BST 2016

Hello Wei,

Thank you for your help, you seem the only who actually care about this
project! I have made great progress with your help, but  I still need your
help in the Linux booting process.

I managed to boot Linux on your lowrisc chip, everything went fine. So I am
now trying to boot it on my own design, using the very same vmlinux image,
bbl code, and driver code (the only difference here is the DRAM, UART and
SPI base addresses). The problem is that when Linux boots, it get stuck at
"random" points during the boot. I see the initial info printks and then it
get stuck. If I check the PC, it is always different, so I cannot figure
out a common pattern in order to identify the problem. I have also ran
several bare metal application to check the integrity of my design, and all
run fine.

In brief, my system has your Rocket Top chisel code, except for the
TL/NASTI converters, which I have replaced with my own converters. I am
also using the same UART and SPI Xilinx IPs but with different base
addresses. The DRAM is placed at 0x0 instead of 0x4000_0000, the I/O base
address is the same 0x8000_0000.  I don't need the first stage boot loader
and the BRAM, since I can load the bbl directly in DRAM. I also checked
that the load process of vmlinux in memory is correct.

To reflect this base address changing, I have modified the base addresses
in uart.h, spi.h and device_map.h in the driver folder, and then the
InitMemBase in Configs.scala. *1) Is there something else that I have to
take into account?* Maybe some hardcoded constants such as clock frequency,
addresses, etc...

*2) Do you have any idea what could be the problem  or have any suggestion
how to find it?*

Furthermore, *3) is there any accurate and low-level documentation about
the TileLink protocol?*

Thank you so much for your help,


On Sat, Jun 4, 2016 at 7:54 AM Wei Song <ws327 at cam.ac.uk> wrote:

> Not really. But if vmlinux runs to load root.bin, it is already nearly
> success.
> vmlinux is the Linux kernel.
> When Linux kernel needs support from the lower level, such as console and
> IO devices, bbl will do it for the kernel using an interface called sbi.
> So when an actual lower level request is needed, Linux kernel produce an
> sbi call (similar to a syscall), then the bbl will act like a hypervisor to
> serve this sbi call.
> sbi calls are defined in bbl/sbi*, while the sbi service is defined in
> bbl/mtrap*
> For the root.bin, it is a disk image which is drived by a htif-blk device
> in kernel.
> When loading the disk, the htif-blk device will generate
> sbi_device_request, which ultimately is served by mcall_dev_req in mtrap.
> In mcall_dev_req, you will then see how FatFS is used to access the actual
> root.bin in SD.
> -Wei
> On 04/06/16 08:51, Francesco Viggiano wrote:
> So,  everything that happens after the "eret"  is due to vmlinux running,
> am I right? I see that it manages to open the root.bin file,  then nothing
> else so far. I keep digging into the code..
> Francesco
> On Fri, Jun 3, 2016, 18:03 Wei Song <ws327 at cam.ac.uk> wrote:
>> Hello Francesco,
>> Glad to know you managed to load the vmlinux.
>> The load_elf() in bbl loads vmlinux to virtual memory space rather than
>> physical memory space.
>> So the actual address depends on the virtual address in vmlinux and the
>> page table initialize by vm.c supervisor_vm_init()
>> Generally speaking, the initial page table statically maps the DDR space
>> to the page table structure.
>> As for eret, I suppose the eret in run_loaded_program()?
>> Processor leaves the current privilege level and jump back to the
>> supposedly caller's address stored in epc.
>> Here the mepc CSR is the target address.
>> mepc CSR is the enrtry point of vmlinux, which is stored in the current
>> data structure during load_elf().
>> Hopefully you can figure it out,
>> Wei
>> On 03/06/16 21:11, Francesco Viggiano wrote:
>> Thanks for the clarification Wei.  Now I can read and load the vmlinux
>> file from SD, but I'm stacked somewhere else in the code I think.  I'm
>> trying to debug the bbl inserting some printk here and there in the code.
>> Could you explain me please,  if you know it,  1)where the vmlinux is
>> loaded in memory,  2) what happen after the call to "eret" assembly
>> instruction
>> Thank you for you help!
>> Francesco.
>> On Fri, Jun 3, 2016, 06:36 Wei Song <ws327 at cam.ac.uk> wrote:
>>> Hello Francesco,
>>> On 02/06/2016 21:25, Francesco Viggiano wrote:
>>> > Hi all,
>>> >
>>> > I'm trying to boot Linux on my own system architecture. As for now, I
>>> can
>>> > successfully run the SD card test example provided with lowRisc
>>> project.
>>> > I'd like more information on how to boot Linux, specifically:
>>> >
>>> > 1) My 1Gb DDR3 starts from address 0x0. I have a way to directly put a
>>> > program in Main memory. So I don't need the first stage bootloader,
>>> but I
>>> > can directly load the revised bbl in DDR. I have just modified the DDR,
>>> > uart and spi addresses on both bbl/ and  driver/ folders. Is this the
>>> right
>>> > way to proceed?
>>> Yes, I believe you can directly use bbl if you can put it into DDR at
>>> address 0x0 (but make sure the L1/L2 cache is clean?).
>>> I am not what is needed to be changed in bbl. I was thinking you only
>>> need to change the SPI_BASE and UART_BASE in driver.
>>> >
>>> > 2) On my SD card I have put the pre-compiled linux image following this
>>> > guide: www.lowrisc.org/docs/untether-v0.2/fpga-demo/#boot. So on my
>>> SD card
>>> > there are vmlinux and root.bin files.
>>> >
>>> > 3) When I run the bbl, It reaches the SD card, but it states: "fail to
>>> open
>>> > file vmlinux"
>>> >
>>> > My questions are:
>>> > 1) Do I need to compile my own linux image (if so, what do I have to
>>> > change?), or I can easily run yours?
>>> No, you should be able to use mine.
>>> >
>>> > 2) Is the FPU in the rocket core mandatory in order to run Linux?
>>> No, in theory. Somebody has managed to run with a soft FPU, although I
>>> am not what has been changed to do it other then disabling the hard FPU.
>>> >
>>> > 3) Why does it fail to load the vmlinux file?
>>> I cannot tell right now.
>>> I believe it is the bbl/elf.c load_elf() complaining fail to load
>>> vmlinux?
>>> Basically this means the bbl cannot read the vmlinux file from SD.
>>> The SD card should be format in FAT32 which is by default.
>>> It looks like you have done the sdcard test? example/sdcard
>>> If that test works OK but bbl fails to load vmlinux, you might want to
>>> dig deeper to see what is the error in file_open(), which is defined in
>>> bbl/file().
>>> Both the sdcard test and the bbl use the FatFS to support SD FAT32.
>>> FatFS is located in driver/ff*
>>> FatFS will be able to give you more detailed error code, then we might
>>> be able to tell what went wrong.
>>> Best regards,
>>> Wei
>>> >
>>> > Thank you for your time and support,
>>> >
>>> > Francesco Viggiano
>>> >
>>> >
>>> --
>> Francesco
>> --
> Francesco
> --
| Francesco Viggiano,  Columbia University, Staff associate researcher at
Computer Science Building |
| Phone: +1 646-9829-535, Skype ID: francesco.vgg |
| Department of Computer Science 1214 Amsterdam Avenue |
| Columbia University Mail Code 0401 | New York, NY 10027-7003 |

More information about the lowrisc-dev mailing list