[lowrisc-dev] Linux booting help

Wei Song ws327 at cam.ac.uk
Wed Jul 6 10:04:34 BST 2016

Hello Francesco,

> Thank you for your help, you seem the only who actually care about
> this project! I have made great progress with your help, but  I still
> need your help in the Linux booting process.

Probably it is because no body has a clue why kernel stuck at random
places without further hint, including me.

> I managed to boot Linux on your lowrisc chip, everything went fine. So
> I am now trying to boot it on my own design, using the very same
> vmlinux image, bbl code, and driver code (the only difference here is
> the DRAM, UART and SPI base addresses). The problem is that when Linux
> boots, it get stuck at "random" points during the boot. I see the
> initial info printks and then it get stuck. If I check the PC, it is
> always different, so I cannot figure out a common pattern in order to
> identify the problem. I have also ran several bare metal application
> to check the integrity of my design, and all run fine.
> In brief, my system has your Rocket Top chisel code, except for the
> TL/NASTI converters, which I have replaced with my own converters. I
> am also using the same UART and SPI Xilinx IPs but with different base
> addresses. The DRAM is placed at 0x0 instead of 0x4000_0000, the I/O
> base address is the same 0x8000_0000.  I don't need the first stage
> boot loader and the BRAM, since I can load the bbl directly in DRAM. I
> also checked that the load process of vmlinux in memory is correct.

Go directly with bbl is OK. I believe it is doable although I have not
tried it myself.
For booting Linux on the untethered lowRISC, there is no change inside
the Linux Kernel. All changes are in the bbl. So my feeling still goes
to something wrong in bbl affects the kernel.
If there is anything I particularly suspect, it would the page table
perhaps. Make sure the page tables are constructed correctly.

Good luck,

> To reflect this base address changing, I have modified the base
> addresses in uart.h, spi.h and device_map.h in the driver folder, and
> then the InitMemBase in Configs.scala. /*1) Is there something else
> that I have to take into account?*/ Maybe some hardcoded constants
> such as clock frequency, addresses, etc...
> */2) Do you have any idea what could be the problem  or have any
> suggestion how to find it?/*
> Furthermore, */3) is there any accurate and low-level documentation
> about the TileLink protocol?/*
> Thank you so much for your help,
> Francesco
> On Sat, Jun 4, 2016 at 7:54 AM Wei Song <ws327 at cam.ac.uk
> <mailto:ws327 at cam.ac.uk>> wrote:
>     Not really. But if vmlinux runs to load root.bin, it is already
>     nearly success.
>     vmlinux is the Linux kernel.
>     When Linux kernel needs support from the lower level, such as
>     console and IO devices, bbl will do it for the kernel using an
>     interface called sbi.
>     So when an actual lower level request is needed, Linux kernel
>     produce an sbi call (similar to a syscall), then the bbl will act
>     like a hypervisor to serve this sbi call.
>     sbi calls are defined in bbl/sbi*, while the sbi service is
>     defined in bbl/mtrap*
>     For the root.bin, it is a disk image which is drived by a htif-blk
>     device in kernel.
>     When loading the disk, the htif-blk device will generate
>     sbi_device_request, which ultimately is served by mcall_dev_req in
>     mtrap.
>     In mcall_dev_req, you will then see how FatFS is used to access
>     the actual root.bin in SD.
>     -Wei
>     On 04/06/16 08:51, Francesco Viggiano wrote:
>>     So,  everything that happens after the "eret"  is due to vmlinux
>>     running,  am I right? I see that it manages to open the root.bin
>>     file,  then nothing else so far. I keep digging into the code..
>>     Francesco
>>     On Fri, Jun 3, 2016, 18:03 Wei Song <ws327 at cam.ac.uk
>>     <mailto:ws327 at cam.ac.uk>> wrote:
>>         Hello Francesco,
>>         Glad to know you managed to load the vmlinux.
>>         The load_elf() in bbl loads vmlinux to virtual memory space
>>         rather than physical memory space.
>>         So the actual address depends on the virtual address in
>>         vmlinux and the page table initialize by vm.c
>>         supervisor_vm_init()
>>         Generally speaking, the initial page table statically maps
>>         the DDR space to the page table structure.
>>         As for eret, I suppose the eret in run_loaded_program()?
>>         Processor leaves the current privilege level and jump back to
>>         the supposedly caller's address stored in epc.
>>         Here the mepc CSR is the target address.
>>         mepc CSR is the enrtry point of vmlinux, which is stored in
>>         the current data structure during load_elf().
>>         Hopefully you can figure it out,
>>         Wei
>>         On 03/06/16 21:11, Francesco Viggiano wrote:
>>>         Thanks for the clarification Wei.  Now I can read and load
>>>         the vmlinux file from SD, but I'm stacked somewhere else in
>>>         the code I think.  I'm trying to debug the bbl inserting
>>>         some printk here and there in the code.
>>>         Could you explain me please,  if you know it,  1)where the
>>>         vmlinux is loaded in memory,  2) what happen after the call
>>>         to "eret" assembly instruction
>>>         Thank you for you help!
>>>         Francesco.
>>>         On Fri, Jun 3, 2016, 06:36 Wei Song <ws327 at cam.ac.uk
>>>         <mailto:ws327 at cam.ac.uk>> wrote:
>>>             Hello Francesco,
>>>             On 02/06/2016 21:25, Francesco Viggiano wrote:
>>>             > Hi all,
>>>             >
>>>             > I'm trying to boot Linux on my own system
>>>             architecture. As for now, I can
>>>             > successfully run the SD card test example provided
>>>             with lowRisc project.
>>>             > I'd like more information on how to boot Linux,
>>>             specifically:
>>>             >
>>>             > 1) My 1Gb DDR3 starts from address 0x0. I have a way
>>>             to directly put a
>>>             > program in Main memory. So I don't need the first
>>>             stage bootloader, but I
>>>             > can directly load the revised bbl in DDR. I have just
>>>             modified the DDR,
>>>             > uart and spi addresses on both bbl/ and  driver/
>>>             folders. Is this the right
>>>             > way to proceed?
>>>             Yes, I believe you can directly use bbl if you can put
>>>             it into DDR at
>>>             address 0x0 (but make sure the L1/L2 cache is clean?).
>>>             I am not what is needed to be changed in bbl. I was
>>>             thinking you only
>>>             need to change the SPI_BASE and UART_BASE in driver.
>>>             >
>>>             > 2) On my SD card I have put the pre-compiled linux
>>>             image following this
>>>             > guide:
>>>             www.lowrisc.org/docs/untether-v0.2/fpga-demo/#boot
>>>             <http://www.lowrisc.org/docs/untether-v0.2/fpga-demo/#boot>.
>>>             So on my SD card
>>>             > there are vmlinux and root.bin files.
>>>             >
>>>             > 3) When I run the bbl, It reaches the SD card, but it
>>>             states: "fail to open
>>>             > file vmlinux"
>>>             >
>>>             > My questions are:
>>>             > 1) Do I need to compile my own linux image (if so,
>>>             what do I have to
>>>             > change?), or I can easily run yours?
>>>             No, you should be able to use mine.
>>>             >
>>>             > 2) Is the FPU in the rocket core mandatory in order to
>>>             run Linux?
>>>             No, in theory. Somebody has managed to run with a soft
>>>             FPU, although I
>>>             am not what has been changed to do it other then
>>>             disabling the hard FPU.
>>>             >
>>>             > 3) Why does it fail to load the vmlinux file?
>>>             I cannot tell right now.
>>>             I believe it is the bbl/elf.c load_elf() complaining
>>>             fail to load vmlinux?
>>>             Basically this means the bbl cannot read the vmlinux
>>>             file from SD.
>>>             The SD card should be format in FAT32 which is by default.
>>>             It looks like you have done the sdcard test? example/sdcard
>>>             If that test works OK but bbl fails to load vmlinux, you
>>>             might want to
>>>             dig deeper to see what is the error in file_open(),
>>>             which is defined in
>>>             bbl/file().
>>>             Both the sdcard test and the bbl use the FatFS to
>>>             support SD FAT32.
>>>             FatFS is located in driver/ff*
>>>             FatFS will be able to give you more detailed error code,
>>>             then we might
>>>             be able to tell what went wrong.
>>>             Best regards,
>>>             Wei
>>>             >
>>>             > Thank you for your time and support,
>>>             >
>>>             > Francesco Viggiano
>>>             >
>>>             >
>>>         -- 
>>>         Francesco
>>     -- 
>>     Francesco
> -- 
> | Francesco Viggiano,  Columbia University, Staff associate researcher
> at Computer Science Building |
> | Phone: +1 646-9829-535, Skype ID: francesco.vgg |
> | Department of Computer Science 1214 Amsterdam Avenue |
> | Columbia University Mail Code 0401 | New York, NY 10027-7003 |

More information about the lowrisc-dev mailing list