10:34 p.m.
Hello lowRISC developers,
currently the SPI bus on the Nexys 4 DDR board is exposed through an
explicit controller block, that software can talk (Q)SPI over.
I think that lowRISC could do the following instead:
- After reset, the SPI flash (whose primary purpose is to store a
bitstream) is mapped into the physical address space. Reads from this
area are translated to reads from the flash, writes are either
discarded or cause a trap.
- The existing SPI controller block is made available after a special
control register is written, potentially only after the memory-mapped
flash controller has been disabled. As I understand it, it would be
risky to access the SPI bus through two different controllers more or
less at the same time, because the commands and responses might mix
up.
- An alternate boot ROM could simply jump into the SPI flash, without
performing any hardware initialization.
The flash part on the Nexys 4 DDR is 16MiB big, of which about 4 are
used for the bitstream, leaving plenty of space for boot firmware and,
for example, a kernel and an initrd.
Separating the boot firmware from the FPGA bitstream has the advantage
that the bitstream doesn't need to be recompiled during boot firmware
development, even if early initialization code is changed. A
disadvantage of using the SPI flash is that it isn't as easily
accessible as the µSD card slot. I also haven't tested whether Vivado
can program the high 12MiB of the flash.
What do you think about memory-mapping the flash?
Best regards,
Jonathan Neuschäfer
3:14 a.m.
On 19.07.2016 04:34, Jonathan Neuschäfer wrote:
What do you think about memory-mapping the flash?
Hi Jonathan,
I generally agree with you about using the flash. I proposed to use it
slightly differently here: https://github.com/lowRISC/lowrisc-chip/issues/15
Summary: We need a bootloader that is static for all (three) usage
scenarios. The NAND flash can be one source of booting.
You are right, we can also load the bootloader into the NAND flash, but
I don't see that much of an advantage over updating the bitstream with a
newer bootloader. It should rarely be necessary and it's not overly slow.
Cheers,
Stefan
7:16 a.m.
Hello Jonathan,
Sorry if I misunderstand something here.
Although SD card and Flash both use SPI interfaces, they are not
requested to use the same SPI interface.
To support Flash, a second Quad SPI interface can be added. Then there
should be no issue in using them at the same time.
As for choosing where the firmware and the Kernel is loaded from, they
can be loaded from flash as well if that is with benefits for some reason.
However, the future plan is to use CoreBoot, which I think providing
more flexibility.
Best regards,
Wei
On 19/07/2016 03:34, Jonathan Neuschäfer wrote:
Hello lowRISC developers,
currently the SPI bus on the Nexys 4 DDR board is exposed through an
explicit controller block, that software can talk (Q)SPI over.
I think that lowRISC could do the following instead:
- After reset, the SPI flash (whose primary purpose is to store a
bitstream) is mapped into the physical address space. Reads from this
area are translated to reads from the flash, writes are either
discarded or cause a trap.
- The existing SPI controller block is made available after a special
control register is written, potentially only after the memory-mapped
flash controller has been disabled. As I understand it, it would be
risky to access the SPI bus through two different controllers more or
less at the same time, because the commands and responses might mix
up.
- An alternate boot ROM could simply jump into the SPI flash, without
performing any hardware initialization.
The flash part on the Nexys 4 DDR is 16MiB big, of which about 4 are
used for the bitstream, leaving plenty of space for boot firmware and,
for example, a kernel and an initrd.
Separating the boot firmware from the FPGA bitstream has the advantage
that the bitstream doesn't need to be recompiled during boot firmware
development, even if early initialization code is changed. A
disadvantage of using the SPI flash is that it isn't as easily
accessible as the µSD card slot. I also haven't tested whether Vivado
can program the high 12MiB of the flash.
What do you think about memory-mapping the flash?
Best regards,
Jonathan Neuschäfer
12:12 p.m.
Hello Wei,
On Tue, Jul 19, 2016 at 12:16:09PM +0100, Wei Song wrote:
Hello Jonathan,
Sorry if I misunderstand something here.
Although SD card and Flash both use SPI interfaces, they are not
requested to use the same SPI interface.
The misunderstanding was on my part: I thought that the SPI controller
was connected to the flash.
To support Flash, a second Quad SPI interface can be added. Then
there
should be no issue in using them at the same time.
As for choosing where the firmware and the Kernel is loaded from, they
can be loaded from flash as well if that is with benefits for some reason.
However, the future plan is to use CoreBoot, which I think providing
more flexibility.
My GSoC project is more or less to bring coreboot to RISC-V hardware, so
I'll try to give some background information:
Generally, the job of coreboot is to:
- Initialize the RAM, if it hasn't happened already in hardware
- Initialize other resources that the OS can't initialize (on x86, the
PCI busses need to be enumerated)
- Load and run a "payload", which can be a kernel, a graphical boot
loader or something else. Payloads always reside in a structure
called CBFS (coreboot file system) on the boot medium (which is a
flash in most cases).
If the OS is loaded from external media, the payload needs to do that.
Some payloads supported by coreboot are: uboot, Grub, SeaBIOS (an
implmentation of the old x86 BIOS calls and MBR boot mechanism),
TianoCore (a UEFI implementation). None of those runs on RISC-V yet.
- On RISC-V, to implement the SBI, and to secure it against privilege
escalation of S-mode code into M-mode, if possible.
My idea was that porting coreboot could be easier if the contents of the
flash were memory-mapped, so that the first stage boot program could
simply jump into the flash. This would be similar to x86 systems where
the BIOS (be it coreboot or not) resides in the BIOS flash.
I think any (software) code that goes into the bitstream (and later,
mask ROM) should be kept simple, to avoid bugs: A bitstream can be
recompiled, but an ASIC can't.
Instead of memory-mapping the flash, it would also be possible to
provide an explicit SPI controller, and, in the on-chip bootloader, read
the first stage of coreboot into a buffer and run it there. I would
prefer a memory-mapped flash though, because it's easier to handle,
software wise. (Reading all of coreboot into DRAM would work on the
Nexys 4, because RAM initialization is done it hardware, but it would
likely be more difficult on a lowRISC ASIC.)
Thanks for your attention,
Jonathan Neuschäfer
12:42 p.m.
Hello Jonathan,
This is actually rather interesting.
May I ask which GSoC organization you are working right now and who is
your mentor?
Which hardware/software platform will you port coreboot to?
Any chance for porting on a lowRISC platform?
I think I can see the benefit of letting coreboot sitting in a flash,
but there are several questions:
1. Does coreboot need heap and stack? I suppose yes. Then will heap and
stack be mapped to Flash as well? Would not this be potentially harmful
for flash?
2. Does coreboot use cache or must be uncached? Whether this flash space
is still observable in OS or later stage bootloaders (I suppose Yes as
well).
3. What does x86 BIOS do? The BISO is actually running on flash?
And there is question for me: Even if I would like to, how can I support
memory-mapped flash using a quad-spi (which is hard-wired on the
Nexys4-DDR).
Stefan, do you know how to do it? It seems like a hardware controller to
translate memory accesses to SPI transactions.
Best regards,
Wei
On 19/07/2016 17:12, Jonathan Neuschäfer wrote:
Hello Wei,
On Tue, Jul 19, 2016 at 12:16:09PM +0100, Wei Song wrote:
> Hello Jonathan,
>
> Sorry if I misunderstand something here.
> Although SD card and Flash both use SPI interfaces, they are not
> requested to use the same SPI interface.
The misunderstanding was on my part: I thought that the SPI controller
was connected to the flash.
> To support Flash, a second Quad SPI interface can be added. Then there
> should be no issue in using them at the same time.
> As for choosing where the firmware and the Kernel is loaded from, they
> can be loaded from flash as well if that is with benefits for some reason.
> However, the future plan is to use CoreBoot, which I think providing
> more flexibility.
My GSoC project is more or less to bring coreboot to RISC-V hardware, so
I'll try to give some background information:
Generally, the job of coreboot is to:
- Initialize the RAM, if it hasn't happened already in hardware
- Initialize other resources that the OS can't initialize (on x86, the
PCI busses need to be enumerated)
- Load and run a "payload", which can be a kernel, a graphical boot
loader or something else. Payloads always reside in a structure
called CBFS (coreboot file system) on the boot medium (which is a
flash in most cases).
If the OS is loaded from external media, the payload needs to do that.
Some payloads supported by coreboot are: uboot, Grub, SeaBIOS (an
implmentation of the old x86 BIOS calls and MBR boot mechanism),
TianoCore (a UEFI implementation). None of those runs on RISC-V yet.
- On RISC-V, to implement the SBI, and to secure it against privilege
escalation of S-mode code into M-mode, if possible.
My idea was that porting coreboot could be easier if the contents of the
flash were memory-mapped, so that the first stage boot program could
simply jump into the flash. This would be similar to x86 systems where
the BIOS (be it coreboot or not) resides in the BIOS flash.
I think any (software) code that goes into the bitstream (and later,
mask ROM) should be kept simple, to avoid bugs: A bitstream can be
recompiled, but an ASIC can't.
Instead of memory-mapping the flash, it would also be possible to
provide an explicit SPI controller, and, in the on-chip bootloader, read
the first stage of coreboot into a buffer and run it there. I would
prefer a memory-mapped flash though, because it's easier to handle,
software wise. (Reading all of coreboot into DRAM would work on the
Nexys 4, because RAM initialization is done it hardware, but it would
likely be more difficult on a lowRISC ASIC.)
Thanks for your attention,
Jonathan Neuschäfer
1:34 p.m.
On Tue, Jul 19, 2016 at 05:42:24PM +0100, Wei Song wrote:
Hello Jonathan,
This is actually rather interesting.
May I ask which GSoC organization you are working right now and who is
your mentor?
I'm working with Ron Minnich and Furquan Shaikh (CC'd) in the coreboot
organization.
Which hardware/software platform will you port coreboot to?
SPIKE, and probably lowRISC on the Nexys 4 DDR.
Any chance for porting on a lowRISC platform?
Yes, that's my intention.
I think I can see the benefit of letting coreboot sitting in a
flash,
but there are several questions:
1. Does coreboot need heap and stack? I suppose yes. Then will heap and
stack be mapped to Flash as well? Would not this be potentially harmful
for flash?
Yes, coreboot needs a stack (and a heap), which requires writable
memory. I wouldn't map a flash writable, as writing means erasing a
whole page (which is usually 512 bytes, if I am not mistaken) and
rewriting it.
For things like a stack it's useful to have a small amount of RAM inside
the CPU, which can be used immediately after reset. DRAM can't usually
be used directly after reset, because the DRAM controller has to
configured to use the correct timings etc., so it should be SRAM.
LowRISC on nexys4 already has 64KiB of Block RAM at 0x40000000.
2. Does coreboot use cache or must be uncached? Whether this flash
space
is still observable in OS or later stage bootloaders (I suppose Yes as
well).
I think it doesn't matter whether the flash is cached or not, because it
won't be preprogrammed during boot. The OS doesn't need to be able to
access the flash, but it probably doesn't hurt, either.
3. What does x86 BIOS do? The BISO is actually running on flash?
Yes. Coreboot roughly performs the following steps on x86 (and other
BIOSes probably do it similarly):
1. It starts executing in flash and switches into 32-bit mode
2. It sets up the CPU's cache as RAM, so it can have a stack and run C
code
3. It detects the installed RAM modules and initializes the DRAM
controller
4. It loads the so-called ramstage into RAM, and starts running in RAM
5. It loads the payload (bootloader or OS) into RAM and runs it
And there is question for me: Even if I would like to, how can I
support
memory-mapped flash using a quad-spi (which is hard-wired on the
Nexys4-DDR).
Stefan, do you know how to do it? It seems like a hardware controller to
translate memory accesses to SPI transactions.
Yes, that would be roughly how it works. Unfortunately I can't help much
with hardware implementation questions, because of limited experience.
Thanks,
Jonathan
6:27 a.m.
Hello Jonathan,
Thanks for the explanation. It seems pretty clear why a memory mapped
flash is needed now.
The good news is, I just find out that modern SPI Flash chips do support
an XIP (execution in place) mode which is dedicated for this purpose and
the Xilinx Quad SPI IP already support the XIP mode.
So I think I can add support to this memory mapped flash to lowRISC
early next month, hopefully you still get time to try?
Best regards,
Wei
On 19/07/2016 18:34, Jonathan Neuschäfer wrote:
On Tue, Jul 19, 2016 at 05:42:24PM +0100, Wei Song wrote:
> Hello Jonathan,
>
> This is actually rather interesting.
> May I ask which GSoC organization you are working right now and who is
> your mentor?
I'm working with Ron Minnich and Furquan Shaikh (CC'd) in the coreboot
organization.
> Which hardware/software platform will you port coreboot to?
SPIKE, and probably lowRISC on the Nexys 4 DDR.
> Any chance for porting on a lowRISC platform?
Yes, that's my intention.
> I think I can see the benefit of letting coreboot sitting in a flash,
> but there are several questions:
> 1. Does coreboot need heap and stack? I suppose yes. Then will heap and
> stack be mapped to Flash as well? Would not this be potentially harmful
> for flash?
Yes, coreboot needs a stack (and a heap), which requires writable
memory. I wouldn't map a flash writable, as writing means erasing a
whole page (which is usually 512 bytes, if I am not mistaken) and
rewriting it.
For things like a stack it's useful to have a small amount of RAM inside
the CPU, which can be used immediately after reset. DRAM can't usually
be used directly after reset, because the DRAM controller has to
configured to use the correct timings etc., so it should be SRAM.
LowRISC on nexys4 already has 64KiB of Block RAM at 0x40000000.
> 2. Does coreboot use cache or must be uncached? Whether this flash space
> is still observable in OS or later stage bootloaders (I suppose Yes as
> well).
I think it doesn't matter whether the flash is cached or not, because it
won't be preprogrammed during boot. The OS doesn't need to be able to
access the flash, but it probably doesn't hurt, either.
> 3. What does x86 BIOS do? The BISO is actually running on flash?
Yes. Coreboot roughly performs the following steps on x86 (and other
BIOSes probably do it similarly):
1. It starts executing in flash and switches into 32-bit mode
2. It sets up the CPU's cache as RAM, so it can have a stack and run C
code
3. It detects the installed RAM modules and initializes the DRAM
controller
4. It loads the so-called ramstage into RAM, and starts running in RAM
5. It loads the payload (bootloader or OS) into RAM and runs it
> And there is question for me: Even if I would like to, how can I support
> memory-mapped flash using a quad-spi (which is hard-wired on the
> Nexys4-DDR).
> Stefan, do you know how to do it? It seems like a hardware controller to
> translate memory accesses to SPI transactions.
Yes, that would be roughly how it works. Unfortunately I can't help much
with hardware implementation questions, because of limited experience.
Thanks,
Jonathan
10:58 a.m.
On Wed, Jul 20, 2016 at 11:27:11AM +0100, Wei Song wrote:
Hello Jonathan,
Thanks for the explanation. It seems pretty clear why a memory mapped
flash is needed now.
The good news is, I just find out that modern SPI Flash chips do support
an XIP (execution in place) mode which is dedicated for this purpose and
the Xilinx Quad SPI IP already support the XIP mode.
Good
So I think I can add support to this memory mapped flash to lowRISC
early next month, hopefully you still get time to try?
Yes. The GSoC coding period ends mid-August, and I'll probably keep
working on RISC-V/coreboot after the GSoC.
Jonathan
11:02 a.m.
The memory mapped SPI is great news! Thanks Wei!
ron
On Wed, Jul 20, 2016 at 7:58 AM Jonathan Neuschäfer <j.neuschaefer(a)gmx.net>
wrote:
On Wed, Jul 20, 2016 at 11:27:11AM +0100, Wei Song wrote:
> Hello Jonathan,
>
> Thanks for the explanation. It seems pretty clear why a memory mapped
> flash is needed now.
> The good news is, I just find out that modern SPI Flash chips do support
> an XIP (execution in place) mode which is dedicated for this purpose and
> the Xilinx Quad SPI IP already support the XIP mode.
Good
> So I think I can add support to this memory mapped flash to lowRISC
> early next month, hopefully you still get time to try?
Yes. The GSoC coding period ends mid-August, and I'll probably keep
working on RISC-V/coreboot after the GSoC.
Jonathan
9:26 a.m.
Hi Wei,
Yes this is quite neat.
I should try to pull your work and try it out.
Does this mean we can persist the first stage and cold boot without downloading the
bitstream when the device is power cycled? I tried this but my skills are not there at
this level. I read something about the frequency of the Flash/EEPROM needing to be at a
particular setting for it to load the bust ram from Flash/EEPROM.
I would like to try the lowrisc-chip when you have another release...
Thanks,
Michael
Sent from my iPhone
On 21/07/2016, at 3:02 AM, ron minnich <rminnich(a)gmail.com>
wrote:
The memory mapped SPI is great news! Thanks Wei!
ron
On Wed, Jul 20, 2016 at 7:58 AM Jonathan Neuschäfer <j.neuschaefer(a)gmx.net>
wrote:
>> On Wed, Jul 20, 2016 at 11:27:11AM +0100, Wei Song wrote:
>> Hello Jonathan,
>>
>> Thanks for the explanation. It seems pretty clear why a memory mapped
>> flash is needed now.
>> The good news is, I just find out that modern SPI Flash chips do support
>> an XIP (execution in place) mode which is dedicated for this purpose and
>> the Xilinx Quad SPI IP already support the XIP mode.
>
> Good
>
>> So I think I can add support to this memory mapped flash to lowRISC
>> early next month, hopefully you still get time to try?
>
> Yes. The GSoC coding period ends mid-August, and I'll probably keep
> working on RISC-V/coreboot after the GSoC.
>
>
> Jonathan
>
9:54 a.m.
Hello Michael,
It is possible to configure an FPGA using a bitstream in Flash. For this
purpose, there is no need to change any hardware design.
What I understand, the trick is all on the Vivado software.
As long as the bitstream is correctly put into the FLASH using Vivado,
with correct jumpers on board, FPGA will be configured from Flash.
See Nexys4DDR demos for more information.
I have not done this before and probably will not support it as a way to
run lowRISC demo, unless there are benefits to do so.
The changes I have made to current lowRISC is to allow reusing the spare
Flash space as an execution in place (XIP) ROM.
Therefore a bootloader, such as coreboot, can be put there.
Also with some benefit that the bootloader does not need to be
re-downloaded when hardware changed.
Next big release probably will be December or January next year with
tagged memory and minions.
Why not now?
Best regards,
Wei
On 28/07/2016 14:26, Michael Clark wrote:
Hi Wei,
Yes this is quite neat.
I should try to pull your work and try it out.
Does this mean we can persist the first stage and cold boot without downloading the
bitstream when the device is power cycled? I tried this but my skills are not there at
this level. I read something about the frequency of the Flash/EEPROM needing to be at a
particular setting for it to load the bust ram from Flash/EEPROM.
I would like to try the lowrisc-chip when you have another release...
Thanks,
Michael
Sent from my iPhone
> On 21/07/2016, at 3:02 AM, ron minnich <rminnich(a)gmail.com> wrote:
>
> The memory mapped SPI is great news! Thanks Wei!
>
> ron
>
> On Wed, Jul 20, 2016 at 7:58 AM Jonathan Neuschäfer <j.neuschaefer(a)gmx.net>
> wrote:
>
>>> On Wed, Jul 20, 2016 at 11:27:11AM +0100, Wei Song wrote:
>>> Hello Jonathan,
>>>
>>> Thanks for the explanation. It seems pretty clear why a memory mapped
>>> flash is needed now.
>>> The good news is, I just find out that modern SPI Flash chips do support
>>> an XIP (execution in place) mode which is dedicated for this purpose and
>>> the Xilinx Quad SPI IP already support the XIP mode.
>> Good
>>
>>> So I think I can add support to this memory mapped flash to lowRISC
>>> early next month, hopefully you still get time to try?
>> Yes. The GSoC coding period ends mid-August, and I'll probably keep
>> working on RISC-V/coreboot after the GSoC.
>>
>>
>> Jonathan
>>
5:43 p.m.
Hi Wei,
I could give it a try.
I don't really have much time to take risk so I'm focusing my effort on things
I'm more certain about and that's the software side.
If I had the time I would spend time learning Chisel and Verilog.
What about adding this cache control to BOOM and using BOOM in lowRISC? I guess both
Rocket and BOOM need it. Are you considering BOOM?
~mc
Sent from my iPhone
On 29/07/2016, at 1:54 AM, Wei Song <ws327(a)cam.ac.uk> wrote:
Hello Michael,
It is possible to configure an FPGA using a bitstream in Flash. For this
purpose, there is no need to change any hardware design.
What I understand, the trick is all on the Vivado software.
As long as the bitstream is correctly put into the FLASH using Vivado,
with correct jumpers on board, FPGA will be configured from Flash.
See Nexys4DDR demos for more information.
I have not done this before and probably will not support it as a way to
run lowRISC demo, unless there are benefits to do so.
The changes I have made to current lowRISC is to allow reusing the spare
Flash space as an execution in place (XIP) ROM.
Therefore a bootloader, such as coreboot, can be put there.
Also with some benefit that the bootloader does not need to be
re-downloaded when hardware changed.
Next big release probably will be December or January next year with
tagged memory and minions.
Why not now?
Best regards,
Wei
> On 28/07/2016 14:26, Michael Clark wrote:
> Hi Wei,
>
> Yes this is quite neat.
>
> I should try to pull your work and try it out.
>
> Does this mean we can persist the first stage and cold boot without downloading the
bitstream when the device is power cycled? I tried this but my skills are not there at
this level. I read something about the frequency of the Flash/EEPROM needing to be at a
particular setting for it to load the bust ram from Flash/EEPROM.
>
> I would like to try the lowrisc-chip when you have another release...
>
> Thanks,
> Michael
>
> Sent from my iPhone
>
>> On 21/07/2016, at 3:02 AM, ron minnich <rminnich(a)gmail.com> wrote:
>>
>> The memory mapped SPI is great news! Thanks Wei!
>>
>> ron
>>
>> On Wed, Jul 20, 2016 at 7:58 AM Jonathan Neuschäfer
<j.neuschaefer(a)gmx.net>
>> wrote:
>>
>>>> On Wed, Jul 20, 2016 at 11:27:11AM +0100, Wei Song wrote:
>>>> Hello Jonathan,
>>>>
>>>> Thanks for the explanation. It seems pretty clear why a memory mapped
>>>> flash is needed now.
>>>> The good news is, I just find out that modern SPI Flash chips do support
>>>> an XIP (execution in place) mode which is dedicated for this purpose and
>>>> the Xilinx Quad SPI IP already support the XIP mode.
>>> Good
>>>
>>>> So I think I can add support to this memory mapped flash to lowRISC
>>>> early next month, hopefully you still get time to try?
>>> Yes. The GSoC coding period ends mid-August, and I'll probably keep
>>> working on RISC-V/coreboot after the GSoC.
>>>
>>>
>>> Jonathan
9:40 a.m.
Supporting write-back control in cache should be independent from which
core is used.
But using BOOM is in the future plan.
We hope to have both Rocket and BOOM cores in the lowRISC test chip.
-Wei
On 28/07/2016 22:43, Michael Clark wrote:
Hi Wei,
I could give it a try.
I don't really have much time to take risk so I'm focusing my effort on things
I'm more certain about and that's the software side.
If I had the time I would spend time learning Chisel and Verilog.
What about adding this cache control to BOOM and using BOOM in lowRISC? I guess both
Rocket and BOOM need it. Are you considering BOOM?
~mc
Sent from my iPhone
> On 29/07/2016, at 1:54 AM, Wei Song <ws327(a)cam.ac.uk> wrote:
>
> Hello Michael,
>
> It is possible to configure an FPGA using a bitstream in Flash. For this
> purpose, there is no need to change any hardware design.
> What I understand, the trick is all on the Vivado software.
> As long as the bitstream is correctly put into the FLASH using Vivado,
> with correct jumpers on board, FPGA will be configured from Flash.
> See Nexys4DDR demos for more information.
>
> I have not done this before and probably will not support it as a way to
> run lowRISC demo, unless there are benefits to do so.
>
> The changes I have made to current lowRISC is to allow reusing the spare
> Flash space as an execution in place (XIP) ROM.
> Therefore a bootloader, such as coreboot, can be put there.
> Also with some benefit that the bootloader does not need to be
> re-downloaded when hardware changed.
>
> Next big release probably will be December or January next year with
> tagged memory and minions.
> Why not now?
>
> Best regards,
> Wei
>
>> On 28/07/2016 14:26, Michael Clark wrote:
>> Hi Wei,
>>
>> Yes this is quite neat.
>>
>> I should try to pull your work and try it out.
>>
>> Does this mean we can persist the first stage and cold boot without downloading
the bitstream when the device is power cycled? I tried this but my skills are not there at
this level. I read something about the frequency of the Flash/EEPROM needing to be at a
particular setting for it to load the bust ram from Flash/EEPROM.
>>
>> I would like to try the lowrisc-chip when you have another release...
>>
>> Thanks,
>> Michael
>>
>> Sent from my iPhone
>>
>>> On 21/07/2016, at 3:02 AM, ron minnich <rminnich(a)gmail.com> wrote:
>>>
>>> The memory mapped SPI is great news! Thanks Wei!
>>>
>>> ron
>>>
>>> On Wed, Jul 20, 2016 at 7:58 AM Jonathan Neuschäfer
<j.neuschaefer(a)gmx.net>
>>> wrote:
>>>
>>>>> On Wed, Jul 20, 2016 at 11:27:11AM +0100, Wei Song wrote:
>>>>> Hello Jonathan,
>>>>>
>>>>> Thanks for the explanation. It seems pretty clear why a memory
mapped
>>>>> flash is needed now.
>>>>> The good news is, I just find out that modern SPI Flash chips do
support
>>>>> an XIP (execution in place) mode which is dedicated for this purpose
and
>>>>> the Xilinx Quad SPI IP already support the XIP mode.
>>>> Good
>>>>
>>>>> So I think I can add support to this memory mapped flash to lowRISC
>>>>> early next month, hopefully you still get time to try?
>>>> Yes. The GSoC coding period ends mid-August, and I'll probably keep
>>>> working on RISC-V/coreboot after the GSoC.
>>>>
>>>>
>>>> Jonathan
7:52 a.m.
Hello Jonathan,
I have added a memory mapped flash to lowrisc-chip, currently in branch
update.
A very simple test program fpga/bare_metal/examples/flash.c is provided.
It just reads and prints the content of the flash to console.
Since the flash has a pre-built image from Digilient, I can see
something and it seems OK.
To run this test, go to fpga/board/nexys4_ddr
make flash
make program-updated
Then depending on which CONFIG is used in Makefile, you either need the
trace debugger to open a terminal or normal UART terminal.
It should constantly printing the flash content.
For more information about how to run it, see the tutorial for debug-v0.3 at
http://www.lowrisc.org/docs/debug-v0.3/
Is it possible that you can have try and tell me it is OK or something
else is needed or actually it has some bugs?
It can be buggy as I have just implemented an AXI bus narrower to make
it work.
Best regards,
Wei
On 20/07/2016 15:58, Jonathan Neuschäfer wrote:
On Wed, Jul 20, 2016 at 11:27:11AM +0100, Wei Song wrote:
> Hello Jonathan,
>
> Thanks for the explanation. It seems pretty clear why a memory mapped
> flash is needed now.
> The good news is, I just find out that modern SPI Flash chips do support
> an XIP (execution in place) mode which is dedicated for this purpose and
> the Xilinx Quad SPI IP already support the XIP mode.
Good
> So I think I can add support to this memory mapped flash to lowRISC
> early next month, hopefully you still get time to try?
Yes. The GSoC coding period ends mid-August, and I'll probably keep
working on RISC-V/coreboot after the GSoC.
Jonathan
12:42 p.m.
On Thu, Jul 28, 2016 at 12:52:29PM +0100, Wei Song wrote:
Hello Jonathan,
I have added a memory mapped flash to lowrisc-chip, currently in branch
update.
A very simple test program fpga/bare_metal/examples/flash.c is provided.
It just reads and prints the content of the flash to console.
Since the flash has a pre-built image from Digilient, I can see
something and it seems OK.
That's great. I'll be travelling for a week, but when I return, I'll
test it.
Jonathan
To run this test, go to fpga/board/nexys4_ddr
make flash
make program-updated
Then depending on which CONFIG is used in Makefile, you either need the
trace debugger to open a terminal or normal UART terminal.
It should constantly printing the flash content.
For more information about how to run it, see the tutorial for debug-v0.3 at
http://www.lowrisc.org/docs/debug-v0.3/
Is it possible that you can have try and tell me it is OK or something
else is needed or actually it has some bugs?
It can be buggy as I have just implemented an AXI bus narrower to make
it work.
Best regards,
Wei
9:17 a.m.
Can you run out of the L1/L2 cache with write back disabled until you have configured
DRAM? Is the cache big enough working space before DRAM has been configured to load a
larger stage2 that can enumerate the hardware.
stage1 in ROM just needs to be big enough to turn on RAM and get stage 2 from
EEPROM/Flash?
If you are aware of the cache associativity you can treat cache as a small piece of linear
RAM assuming it defaults to up-to-date (doesn't try to fetch) and has write-back
disabled through some cache control CSRs.
This way you don't waste SRAM (space) between cache and /some special boot memory/
This wisdom may be wrong however we do need to separate /technique/ from /format/ e.g.
stage 3 being ELF with the RISC-V hardware descriptor (UTF-8 config string which is
scalable to 128-bit or compiled binary derivative vs a binary table format that is limited
to 32 and then later extended to 64-bit) passed via say the ELF auxiliary vector (a la
bbl) vs say loading a complex and large UEFI COFF EXE (COFF is the precursor to ELF and is
the basis of Windows EXE). This is at complete odds with System-V evolution from COFF to
ELF which all the UNIX vendors, Linux and BSD derivatives settled upon. The reason being
standardised shared library support.
The reason I'm thinking about ELF for the stage3 boot loader or kernel payload is it
is the way bbl loads vmlinux as ELF (no silly 512 byte boot sector stuff) and it would be
symmetrical with virtualised booting from of Linux or other OS kernels from a Linux or BSD
Hypervisors such as KVM, Bhyve or Xen.
The symmetry is nice. kexecv essentially is execv in S mode instead of U mode with a
different attribute in the auxiliary vector. AT_RISCV_CONFIG. It's a well known
protocol.
SBI and config string and boot command line (argv) from EEPROM, Flash or NVRAM may be all
that is needed with DRAM set up as Linux or the boot loader in flash can enumerate
hardware. _start or main.
For nested virtualisation having a process/VM with access to sptbr and S mode CSRs. Google
for project DUNE from Stanford. They put Linux processes into S mode instead of U mode
with the main Linux running in root mode (H mode).
BIOS/UEFI etc were all designed pre-x86 virtualisation.
Food for thought.
~mc
PS. Sorry if this is a bit of a stream of consciousness, because it is.
Sent from my iPhone
On 20/07/2016, at 5:34 AM, Jonathan Neuschäfer
<j.neuschaefer(a)gmx.net> wrote:
> On Tue, Jul 19, 2016 at 05:42:24PM +0100, Wei Song wrote:
> Hello Jonathan,
>
> This is actually rather interesting.
> May I ask which GSoC organization you are working right now and who is
> your mentor?
I'm working with Ron Minnich and Furquan Shaikh (CC'd) in the coreboot
organization.
> Which hardware/software platform will you port coreboot to?
SPIKE, and probably lowRISC on the Nexys 4 DDR.
> Any chance for porting on a lowRISC platform?
Yes, that's my intention.
> I think I can see the benefit of letting coreboot sitting in a flash,
> but there are several questions:
> 1. Does coreboot need heap and stack? I suppose yes. Then will heap and
> stack be mapped to Flash as well? Would not this be potentially harmful
> for flash?
Yes, coreboot needs a stack (and a heap), which requires writable
memory. I wouldn't map a flash writable, as writing means erasing a
whole page (which is usually 512 bytes, if I am not mistaken) and
rewriting it.
For things like a stack it's useful to have a small amount of RAM inside
the CPU, which can be used immediately after reset. DRAM can't usually
be used directly after reset, because the DRAM controller has to
configured to use the correct timings etc., so it should be SRAM.
LowRISC on nexys4 already has 64KiB of Block RAM at 0x40000000.
> 2. Does coreboot use cache or must be uncached? Whether this flash space
> is still observable in OS or later stage bootloaders (I suppose Yes as
> well).
I think it doesn't matter whether the flash is cached or not, because it
won't be preprogrammed during boot. The OS doesn't need to be able to
access the flash, but it probably doesn't hurt, either.
> 3. What does x86 BIOS do? The BISO is actually running on flash?
Yes. Coreboot roughly performs the following steps on x86 (and other
BIOSes probably do it similarly):
1. It starts executing in flash and switches into 32-bit mode
2. It sets up the CPU's cache as RAM, so it can have a stack and run C
code
3. It detects the installed RAM modules and initializes the DRAM
controller
4. It loads the so-called ramstage into RAM, and starts running in RAM
5. It loads the payload (bootloader or OS) into RAM and runs it
> And there is question for me: Even if I would like to, how can I support
> memory-mapped flash using a quad-spi (which is hard-wired on the
> Nexys4-DDR).
> Stefan, do you know how to do it? It seems like a hardware controller to
> translate memory accesses to SPI transactions.
Yes, that would be roughly how it works. Unfortunately I can't help much
with hardware implementation questions, because of limited experience.
Thanks,
Jonathan
9:41 a.m.
Precursor:
I work for lowRISC and have limited knowledge of how coreboot works.
I am trying to describe what is currently possible and what is not
possible yet but can be done in the future.
Can you run out of the L1/L2 cache with write back disabled until you
have configured DRAM? Is the cache big enough working space before DRAM has been
configured to load a larger stage2 that can enumerate the hardware.
This is not
possible yet in current Rocket-chip.
There is no method to turn off write back and, even worse, cache line is
always fetched from memory when there is a miss.
Need investigation on how to do it and whether it is urgent to have it
soon. Developing capability is limited right now.
stage1 in ROM just needs to be big enough to turn on RAM and get
stage 2 from EEPROM/Flash?
There is a boot ram on-chip which can be used as the 1
stage bootloader,
which is 64KB in current configuration.
For FPGA, there is no need to driver DDR as it is statically configured.
Another story for ASIC.
So this 64KB is enough to load an image from flash or SD to DDR. Also it
can hand over control to Flash.
If run inside flash, this 1st bootloader space can be used as RAM,
considering L2 cache cannot be used as RAM yet.
-Wei
On 28/07/2016 14:17, Michael Clark wrote:
Can you run out of the L1/L2 cache with write back disabled until you
have configured DRAM? Is the cache big enough working space before DRAM has been
configured to load a larger stage2 that can enumerate the hardware.
>
stage1 in ROM just needs to be big enough to turn on RAM and get
stage 2 from EEPROM/Flash?
>
> If you are aware of the cache associativity you can treat cache as a small piece of
linear RAM assuming it defaults to up-to-date (doesn't try to fetch) and has
write-back disabled through some cache control CSRs.
>
> This way you don't waste SRAM (space) between cache and /some special boot
memory/
>
> This wisdom may be wrong however we do need to separate /technique/ from /format/
e.g. stage 3 being ELF with the RISC-V hardware descriptor (UTF-8 config string which is
scalable to 128-bit or compiled binary derivative vs a binary table format that is limited
to 32 and then later extended to 64-bit) passed via say the ELF auxiliary vector (a la
bbl) vs say loading a complex and large UEFI COFF EXE (COFF is the precursor to ELF and is
the basis of Windows EXE). This is at complete odds with System-V evolution from COFF to
ELF which all the UNIX vendors, Linux and BSD derivatives settled upon. The reason being
standardised shared library support.
>
> The reason I'm thinking about ELF for the stage3 boot loader or kernel payload is
it is the way bbl loads vmlinux as ELF (no silly 512 byte boot sector stuff) and it would
be symmetrical with virtualised booting from of Linux or other OS kernels from a Linux or
BSD Hypervisors such as KVM, Bhyve or Xen.
>
> The symmetry is nice. kexecv essentially is execv in S mode instead of U mode with a
different attribute in the auxiliary vector. AT_RISCV_CONFIG. It's a well known
protocol.
>
> SBI and config string and boot command line (argv) from EEPROM, Flash or NVRAM may be
all that is needed with DRAM set up as Linux or the boot loader in flash can enumerate
hardware. _start or main.
>
> For nested virtualisation having a process/VM with access to sptbr and S mode CSRs.
Google for project DUNE from Stanford. They put Linux processes into S mode instead of U
mode with the main Linux running in root mode (H mode).
>
> BIOS/UEFI etc were all designed pre-x86 virtualisation.
>
> Food for thought.
>
> ~mc
>
> PS. Sorry if this is a bit of a stream of consciousness, because it is.
>
> Sent from my iPhone
>
>> On 20/07/2016, at 5:34 AM, Jonathan Neuschäfer <j.neuschaefer(a)gmx.net>
wrote:
>>
>>> On Tue, Jul 19, 2016 at 05:42:24PM +0100, Wei Song wrote:
>>> Hello Jonathan,
>>>
>>> This is actually rather interesting.
>>> May I ask which GSoC organization you are working right now and who is
>>> your mentor?
>> I'm working with Ron Minnich and Furquan Shaikh (CC'd) in the coreboot
>> organization.
>>
>>> Which hardware/software platform will you port coreboot to?
>> SPIKE, and probably lowRISC on the Nexys 4 DDR.
>>
>>> Any chance for porting on a lowRISC platform?
>> Yes, that's my intention.
>>
>>> I think I can see the benefit of letting coreboot sitting in a flash,
>>> but there are several questions:
>>> 1. Does coreboot need heap and stack? I suppose yes. Then will heap and
>>> stack be mapped to Flash as well? Would not this be potentially harmful
>>> for flash?
>> Yes, coreboot needs a stack (and a heap), which requires writable
>> memory. I wouldn't map a flash writable, as writing means erasing a
>> whole page (which is usually 512 bytes, if I am not mistaken) and
>> rewriting it.
>>
>> For things like a stack it's useful to have a small amount of RAM inside
>> the CPU, which can be used immediately after reset. DRAM can't usually
>> be used directly after reset, because the DRAM controller has to
>> configured to use the correct timings etc., so it should be SRAM.
>> LowRISC on nexys4 already has 64KiB of Block RAM at 0x40000000.
>>
>>> 2. Does coreboot use cache or must be uncached? Whether this flash space
>>> is still observable in OS or later stage bootloaders (I suppose Yes as
>>> well).
>> I think it doesn't matter whether the flash is cached or not, because it
>> won't be preprogrammed during boot. The OS doesn't need to be able to
>> access the flash, but it probably doesn't hurt, either.
>>
>>> 3. What does x86 BIOS do? The BISO is actually running on flash?
>> Yes. Coreboot roughly performs the following steps on x86 (and other
>> BIOSes probably do it similarly):
>>
>> 1. It starts executing in flash and switches into 32-bit mode
>> 2. It sets up the CPU's cache as RAM, so it can have a stack and run C
>> code
>> 3. It detects the installed RAM modules and initializes the DRAM
>> controller
>> 4. It loads the so-called ramstage into RAM, and starts running in RAM
>> 5. It loads the payload (bootloader or OS) into RAM and runs it
>>
>>> And there is question for me: Even if I would like to, how can I support
>>> memory-mapped flash using a quad-spi (which is hard-wired on the
>>> Nexys4-DDR).
>>> Stefan, do you know how to do it? It seems like a hardware controller to
>>> translate memory accesses to SPI transactions.
>> Yes, that would be roughly how it works. Unfortunately I can't help much
>> with hardware implementation questions, because of limited experience.
>>
>>
>> Thanks,
>> Jonathan
5:23 a.m.
On 28 July 2016 at 14:17, Michael Clark <michaeljclark(a)mac.com> wrote:
Can you run out of the L1/L2 cache with write back disabled until you
have configured DRAM? Is the cache big enough working space before DRAM has been
configured to load a larger stage2 that can enumerate the hardware.
stage1 in ROM just needs to be big enough to turn on RAM and get stage 2 from
EEPROM/Flash?
If you are aware of the cache associativity you can treat cache as a small piece of
linear RAM assuming it defaults to up-to-date (doesn't try to fetch) and has
write-back disabled through some cache control CSRs.
We discussed this idea with Ron Minnich some while back, and I really
like it. For now it's not a high priority, and as Wei says we have to
be very selective about where we put our development effort - it's
definitely something I'd like to see though, and we'd be happy to
advise anyone who wanted to have a go at implementing it.
Best,
Alex
2496
days inactive
2509
days old
17 comments
6 participants
participants (6)
-
Alex Bradbury -
Jonathan Neuschäfer -
Michael Clark -
ron minnich -
Stefan Wallentowitz -
Wei Song