Hello Francesco, On 02/06/2016 21:25, Francesco Viggiano wrote: > Hi all, > > I'm trying to boot Linux on my own system architecture. As for now, I can > successfully run the SD card test example provided with lowRisc project. > I'd like more information on how to boot Linux, specifically: > > 1) My 1Gb DDR3 starts from address 0x0. I have a way to directly put a > program in Main memory. So I don't need the first stage bootloader, but I > can directly load the revised bbl in DDR. I have just modified the DDR, > uart and spi addresses on both bbl/ and driver/ folders. Is this the right > way to proceed? Yes, I believe you can directly use bbl if you can put it into DDR at address 0x0 (but make sure the L1/L2 cache is clean?). I am not what is needed to be changed in bbl. I was thinking you only need to change the SPI_BASE and UART_BASE in driver. > > 2) On my SD card I have put the pre-compiled linux image following this > guide: www.lowrisc.org/docs/untether-v0.2/fpga-demo/#boot. So on my SD card > there are vmlinux and root.bin files. > > 3) When I run the bbl, It reaches the SD card, but it states: "fail to open > file vmlinux" > > My questions are: > 1) Do I need to compile my own linux image (if so, what do I have to > change?), or I can easily run yours? No, you should be able to use mine. > > 2) Is the FPU in the rocket core mandatory in order to run Linux? No, in theory. Somebody has managed to run with a soft FPU, although I am not what has been changed to do it other then disabling the hard FPU. > > 3) Why does it fail to load the vmlinux file? I cannot tell right now. I believe it is the bbl/elf.c load_elf() complaining fail to load vmlinux? Basically this means the bbl cannot read the vmlinux file from SD. The SD card should be format in FAT32 which is by default. It looks like you have done the sdcard test? example/sdcard If that test works OK but bbl fails to load vmlinux, you might want to dig deeper to see what is the error in file_open(), which is defined in bbl/file(). Both the sdcard test and the bbl use the FatFS to support SD FAT32. FatFS is located in driver/ff* FatFS will be able to give you more detailed error code, then we might be able to tell what went wrong. Best regards, Wei > > Thank you for your time and support, > > Francesco Viggiano > > --

Hello Francesco, Glad to know you managed to load the vmlinux. The load_elf() in bbl loads vmlinux to virtual memory space rather than physical memory space. So the actual address depends on the virtual address in vmlinux and the page table initialize by vm.c supervisor_vm_init() Generally speaking, the initial page table statically maps the DDR space to the page table structure. As for eret, I suppose the eret in run_loaded_program()? Processor leaves the current privilege level and jump back to the supposedly caller's address stored in epc. Here the mepc CSR is the target address. mepc CSR is the enrtry point of vmlinux, which is stored in the current data structure during load_elf(). Hopefully you can figure it out, Wei On 03/06/16 21:11, Francesco Viggiano wrote: Thanks for the clarification Wei. Now I can read and load the vmlinux file from SD, but I'm stacked somewhere else in the code I think. I'm trying to debug the bbl inserting some printk here and there in the code. Could you explain me please, if you know it, 1)where the vmlinux is loaded in memory, 2) what happen after the call to "eret" assembly instruction Thank you for you help! Francesco. On Fri, Jun 3, 2016, 06:36 Wei Song <ws327(a)cam.ac.uk&gt; wrote: > Hello Francesco, > > On 02/06/2016 21:25, Francesco Viggiano wrote: > > Hi all, > > > > I'm trying to boot Linux on my own system architecture. As for now, I > can > > successfully run the SD card test example provided with lowRisc project. > > I'd like more information on how to boot Linux, specifically: > > > > 1) My 1Gb DDR3 starts from address 0x0. I have a way to directly put a > > program in Main memory. So I don't need the first stage bootloader, but > I > > can directly load the revised bbl in DDR. I have just modified the DDR, > > uart and spi addresses on both bbl/ and driver/ folders. Is this the > right > > way to proceed? > Yes, I believe you can directly use bbl if you can put it into DDR at > address 0x0 (but make sure the L1/L2 cache is clean?). > I am not what is needed to be changed in bbl. I was thinking you only > need to change the SPI_BASE and UART_BASE in driver. > > > > > 2) On my SD card I have put the pre-compiled linux image following this > > guide: www.lowrisc.org/docs/untether-v0.2/fpga-demo/#boot. So on my SD > card > > there are vmlinux and root.bin files. > > > > 3) When I run the bbl, It reaches the SD card, but it states: "fail to > open > > file vmlinux" > > > > My questions are: > > 1) Do I need to compile my own linux image (if so, what do I have to > > change?), or I can easily run yours? > No, you should be able to use mine. > > > > > 2) Is the FPU in the rocket core mandatory in order to run Linux? > No, in theory. Somebody has managed to run with a soft FPU, although I > am not what has been changed to do it other then disabling the hard FPU. > > > > > 3) Why does it fail to load the vmlinux file? > I cannot tell right now. > I believe it is the bbl/elf.c load_elf() complaining fail to load vmlinux? > Basically this means the bbl cannot read the vmlinux file from SD. > The SD card should be format in FAT32 which is by default. > It looks like you have done the sdcard test? example/sdcard > If that test works OK but bbl fails to load vmlinux, you might want to > dig deeper to see what is the error in file_open(), which is defined in > bbl/file(). > Both the sdcard test and the bbl use the FatFS to support SD FAT32. > FatFS is located in driver/ff* > FatFS will be able to give you more detailed error code, then we might > be able to tell what went wrong. > > Best regards, > Wei > > > > > Thank you for your time and support, > > > > Francesco Viggiano > > > > > > -- Francesco --

Not really. But if vmlinux runs to load root.bin, it is already nearly success. vmlinux is the Linux kernel. When Linux kernel needs support from the lower level, such as console and IO devices, bbl will do it for the kernel using an interface called sbi. So when an actual lower level request is needed, Linux kernel produce an sbi call (similar to a syscall), then the bbl will act like a hypervisor to serve this sbi call. sbi calls are defined in bbl/sbi*, while the sbi service is defined in bbl/mtrap* For the root.bin, it is a disk image which is drived by a htif-blk device in kernel. When loading the disk, the htif-blk device will generate sbi_device_request, which ultimately is served by mcall_dev_req in mtrap. In mcall_dev_req, you will then see how FatFS is used to access the actual root.bin in SD. -Wei On 04/06/16 08:51, Francesco Viggiano wrote: So, everything that happens after the "eret" is due to vmlinux running, am I right? I see that it manages to open the root.bin file, then nothing else so far. I keep digging into the code.. Francesco On Fri, Jun 3, 2016, 18:03 Wei Song <ws327(a)cam.ac.uk&gt; wrote: > Hello Francesco, > > Glad to know you managed to load the vmlinux. > The load_elf() in bbl loads vmlinux to virtual memory space rather than > physical memory space. > So the actual address depends on the virtual address in vmlinux and the > page table initialize by vm.c supervisor_vm_init() > Generally speaking, the initial page table statically maps the DDR space > to the page table structure. > > As for eret, I suppose the eret in run_loaded_program()? > Processor leaves the current privilege level and jump back to the > supposedly caller's address stored in epc. > Here the mepc CSR is the target address. > mepc CSR is the enrtry point of vmlinux, which is stored in the current > data structure during load_elf(). > > Hopefully you can figure it out, > Wei > > On 03/06/16 21:11, Francesco Viggiano wrote: > > Thanks for the clarification Wei. Now I can read and load the vmlinux > file from SD, but I'm stacked somewhere else in the code I think. I'm > trying to debug the bbl inserting some printk here and there in the code. > > Could you explain me please, if you know it, 1)where the vmlinux is > loaded in memory, 2) what happen after the call to "eret" assembly > instruction > > Thank you for you help! > > Francesco. > > On Fri, Jun 3, 2016, 06:36 Wei Song <ws327(a)cam.ac.uk&gt; wrote: > >> Hello Francesco, >> >> On 02/06/2016 21:25, Francesco Viggiano wrote: >> > Hi all, >> > >> > I'm trying to boot Linux on my own system architecture. As for now, I >> can >> > successfully run the SD card test example provided with lowRisc >> project. >> > I'd like more information on how to boot Linux, specifically: >> > >> > 1) My 1Gb DDR3 starts from address 0x0. I have a way to directly put a >> > program in Main memory. So I don't need the first stage bootloader, >> but I >> > can directly load the revised bbl in DDR. I have just modified the DDR, >> > uart and spi addresses on both bbl/ and driver/ folders. Is this the >> right >> > way to proceed? >> Yes, I believe you can directly use bbl if you can put it into DDR at >> address 0x0 (but make sure the L1/L2 cache is clean?). >> I am not what is needed to be changed in bbl. I was thinking you only >> need to change the SPI_BASE and UART_BASE in driver. >> >> > >> > 2) On my SD card I have put the pre-compiled linux image following this >> > guide: www.lowrisc.org/docs/untether-v0.2/fpga-demo/#boot. So on my >> SD card >> > there are vmlinux and root.bin files. >> > >> > 3) When I run the bbl, It reaches the SD card, but it states: "fail to >> open >> > file vmlinux" >> > >> > My questions are: >> > 1) Do I need to compile my own linux image (if so, what do I have to >> > change?), or I can easily run yours? >> No, you should be able to use mine. >> >> > >> > 2) Is the FPU in the rocket core mandatory in order to run Linux? >> No, in theory. Somebody has managed to run with a soft FPU, although I >> am not what has been changed to do it other then disabling the hard FPU. >> >> > >> > 3) Why does it fail to load the vmlinux file? >> I cannot tell right now. >> I believe it is the bbl/elf.c load_elf() complaining fail to load >> vmlinux? >> Basically this means the bbl cannot read the vmlinux file from SD. >> The SD card should be format in FAT32 which is by default. >> It looks like you have done the sdcard test? example/sdcard >> If that test works OK but bbl fails to load vmlinux, you might want to >> dig deeper to see what is the error in file_open(), which is defined in >> bbl/file(). >> Both the sdcard test and the bbl use the FatFS to support SD FAT32. >> FatFS is located in driver/ff* >> FatFS will be able to give you more detailed error code, then we might >> be able to tell what went wrong. >> >> Best regards, >> Wei >> >> > >> > Thank you for your time and support, >> > >> > Francesco Viggiano >> > >> > >> >> -- > > Francesco > > > -- Francesco --

Thank you for your help, you seem the only who actually care about this project! I have made great progress with your help, but I still need your help in the Linux booting process.

I managed to boot Linux on your lowrisc chip, everything went fine. So I am now trying to boot it on my own design, using the very same vmlinux image, bbl code, and driver code (the only difference here is the DRAM, UART and SPI base addresses). The problem is that when Linux boots, it get stuck at "random" points during the boot. I see the initial info printks and then it get stuck. If I check the PC, it is always different, so I cannot figure out a common pattern in order to identify the problem. I have also ran several bare metal application to check the integrity of my design, and all run fine. In brief, my system has your Rocket Top chisel code, except for the TL/NASTI converters, which I have replaced with my own converters. I am also using the same UART and SPI Xilinx IPs but with different base addresses. The DRAM is placed at 0x0 instead of 0x4000_0000, the I/O base address is the same 0x8000_0000. I don't need the first stage boot loader and the BRAM, since I can load the bbl directly in DRAM. I also checked that the load process of vmlinux in memory is correct.

To reflect this base address changing, I have modified the base addresses in uart.h, spi.h and device_map.h in the driver folder, and then the InitMemBase in Configs.scala. /*1) Is there something else that I have to take into account?*/ Maybe some hardcoded constants such as clock frequency, addresses, etc... */2) Do you have any idea what could be the problem or have any suggestion how to find it?/* Furthermore, */3) is there any accurate and low-level documentation about the TileLink protocol?/* Thank you so much for your help, Francesco On Sat, Jun 4, 2016 at 7:54 AM Wei Song <ws327(a)cam.ac.uk <mailto:ws327@cam.ac.uk>> wrote: Not really. But if vmlinux runs to load root.bin, it is already nearly success. vmlinux is the Linux kernel. When Linux kernel needs support from the lower level, such as console and IO devices, bbl will do it for the kernel using an interface called sbi. So when an actual lower level request is needed, Linux kernel produce an sbi call (similar to a syscall), then the bbl will act like a hypervisor to serve this sbi call. sbi calls are defined in bbl/sbi*, while the sbi service is defined in bbl/mtrap* For the root.bin, it is a disk image which is drived by a htif-blk device in kernel. When loading the disk, the htif-blk device will generate sbi_device_request, which ultimately is served by mcall_dev_req in mtrap. In mcall_dev_req, you will then see how FatFS is used to access the actual root.bin in SD. -Wei On 04/06/16 08:51, Francesco Viggiano wrote: > > So, everything that happens after the "eret" is due to vmlinux > running, am I right? I see that it manages to open the root.bin > file, then nothing else so far. I keep digging into the code.. > > Francesco > > > On Fri, Jun 3, 2016, 18:03 Wei Song <ws327(a)cam.ac.uk > <mailto:ws327@cam.ac.uk>> wrote: > > Hello Francesco, > > Glad to know you managed to load the vmlinux. > The load_elf() in bbl loads vmlinux to virtual memory space > rather than physical memory space. > So the actual address depends on the virtual address in > vmlinux and the page table initialize by vm.c > supervisor_vm_init() > Generally speaking, the initial page table statically maps > the DDR space to the page table structure. > > As for eret, I suppose the eret in run_loaded_program()? > Processor leaves the current privilege level and jump back to > the supposedly caller's address stored in epc. > Here the mepc CSR is the target address. > mepc CSR is the enrtry point of vmlinux, which is stored in > the current data structure during load_elf(). > > Hopefully you can figure it out, > Wei > > On 03/06/16 21:11, Francesco Viggiano wrote: >> >> Thanks for the clarification Wei. Now I can read and load >> the vmlinux file from SD, but I'm stacked somewhere else in >> the code I think. I'm trying to debug the bbl inserting >> some printk here and there in the code. >> >> Could you explain me please, if you know it, 1)where the >> vmlinux is loaded in memory, 2) what happen after the call >> to "eret" assembly instruction >> >> Thank you for you help! >> >> Francesco. >> >> >> On Fri, Jun 3, 2016, 06:36 Wei Song <ws327(a)cam.ac.uk >> <mailto:ws327@cam.ac.uk>> wrote: >> >> Hello Francesco, >> >> On 02/06/2016 21:25, Francesco Viggiano wrote: >> > Hi all, >> > >> > I'm trying to boot Linux on my own system >> architecture. As for now, I can >> > successfully run the SD card test example provided >> with lowRisc project. >> > I'd like more information on how to boot Linux, >> specifically: >> > >> > 1) My 1Gb DDR3 starts from address 0x0. I have a way >> to directly put a >> > program in Main memory. So I don't need the first >> stage bootloader, but I >> > can directly load the revised bbl in DDR. I have just >> modified the DDR, >> > uart and spi addresses on both bbl/ and driver/ >> folders. Is this the right >> > way to proceed? >> Yes, I believe you can directly use bbl if you can put >> it into DDR at >> address 0x0 (but make sure the L1/L2 cache is clean?). >> I am not what is needed to be changed in bbl. I was >> thinking you only >> need to change the SPI_BASE and UART_BASE in driver. >> >> > >> > 2) On my SD card I have put the pre-compiled linux >> image following this >> > guide: >> www.lowrisc.org/docs/untether-v0.2/fpga-demo/#boot >> <http://www.lowrisc.org/docs/untether-v0.2/fpga-demo/#boot>;. >> So on my SD card >> > there are vmlinux and root.bin files. >> > >> > 3) When I run the bbl, It reaches the SD card, but it >> states: "fail to open >> > file vmlinux" >> > >> > My questions are: >> > 1) Do I need to compile my own linux image (if so, >> what do I have to >> > change?), or I can easily run yours? >> No, you should be able to use mine. >> >> > >> > 2) Is the FPU in the rocket core mandatory in order to >> run Linux? >> No, in theory. Somebody has managed to run with a soft >> FPU, although I >> am not what has been changed to do it other then >> disabling the hard FPU. >> >> > >> > 3) Why does it fail to load the vmlinux file? >> I cannot tell right now. >> I believe it is the bbl/elf.c load_elf() complaining >> fail to load vmlinux? >> Basically this means the bbl cannot read the vmlinux >> file from SD. >> The SD card should be format in FAT32 which is by default. >> It looks like you have done the sdcard test? example/sdcard >> If that test works OK but bbl fails to load vmlinux, you >> might want to >> dig deeper to see what is the error in file_open(), >> which is defined in >> bbl/file(). >> Both the sdcard test and the bbl use the FatFS to >> support SD FAT32. >> FatFS is located in driver/ff* >> FatFS will be able to give you more detailed error code, >> then we might >> be able to tell what went wrong. >> >> Best regards, >> Wei >> >> > >> > Thank you for your time and support, >> > >> > Francesco Viggiano >> > >> > >> >> -- >> >> Francesco >> > > -- > > Francesco > -- | Francesco Viggiano, Columbia University, Staff associate researcher at Computer Science Building | | Phone: +1 646-9829-535, Skype ID: francesco.vgg | | Department of Computer Science 1214 Amsterdam Avenue | | Columbia University Mail Code 0401 | New York, NY 10027-7003 |