zybo_z7: added PS IOP region connected to S_AXI_GP0

[mkuhn99]

I added the IO Region of the PS as a new memory Region and connected it to the second slave gp axi port of the ps.
Also I renamed the Memory Region refering to the PS DDR to main_ram and changed the Base-Address accordingly.

To do this I used the add_mapped function from this pull request.

[trabucayre]

I have tried to reproduce that using my arty_z7 (I haven't zybo) but can't see anything using PS's uart.
Could you provides the way to have access to the PS's uart?
Thanks.

[mkuhn99]

I made a new branch for litex and litex-boards where I am currently implementing uart.
It is still WiP and the uart read is a bit laggy

[JoyBed]

I made a new branch for litex and litex-boards where I am currently implementing uart. It is still WiP and the uart read is a bit laggy

Did you got it working? Did the initial tests in the Litex BIOS worked for you? I tried it but after few hours i only got failed memtest and memspeed gets reads at 59kb/s and write never completed.

[mkuhn99]

I made a new branch for litex and litex-boards where I am currently implementing uart. It is still WiP and the uart read is a bit laggy

Did you got it working? Did the initial tests in the Litex BIOS worked for you? I tried it but after few hours i only got failed memtest and memspeed gets reads at 59kb/s and write never completed.

I just updated the linked litex-boards/ps_software branch and moved the ps_io_region to the IO-Region of the litex-SoC so that it isn't cached anymore and I don't have to flush the cache all the time. Now it is working without a problem (not lagging) on my zybo device.

Regarding your memory problems: make sure that you are initializing the ps correctly:

xsct% connect 
xsct% targets 1
xsct% source build/digilent_zybo_z7/gateware/digilent_zybo_z7.gen/sources_1/ip/ps/ps7_init.tcl 
xsct% rst
xsct% ps7_init 
xsct% ps7_post_config
xsct% fpga build/digilent_zybo_z7/gateware/digilent_zybo_z7.bit

I get sometimes weird behavior if I skip the rst, ps7_init and ps7_post_config step.

[JoyBed]

I made a new branch for litex and litex-boards where I am currently implementing uart. It is still WiP and the uart read is a bit laggy

Did you got it working? Did the initial tests in the Litex BIOS worked for you? I tried it but after few hours i only got failed memtest and memspeed gets reads at 59kb/s and write never completed.

I just updated the linked litex-boards/ps_software branch and moved the ps_io_region to the IO-Region of the litex-SoC so that it isn't cached anymore and I don't have to flush the cache all the time. Now it is working without a problem (not lagging) on my zybo device.

Regarding your memory problems: make sure that you are initializing the ps correctly:

xsct% connect 

xsct% targets 1

xsct% source build/digilent_zybo_z7/gateware/digilent_zybo_z7.gen/sources_1/ip/ps/ps7_init.tcl 

xsct% rst

xsct% ps7_init 

xsct% ps7_post_config

xsct% fpga build/digilent_zybo_z7/gateware/digilent_zybo_z7.bit

I get sometimes weird behavior if I skip the rst, ps7_init and ps7_post_config step.

I thought these steps are made by the Litex scripts automatically.

[mkuhn99]

I thought these steps are made by the Litex scripts automatically.

As far as I know litex only loads the bitstream to the fpga/programmable logic, but does not initialize the processing system.
But I could be mistaken. But it should work now using these xsct commands.

[JoyBed]

As far as I know litex only loads the bitstream to the fpga/programmable logic, but does not initialize the processing system.

But I could be mistaken. But it should work now using these xsct commands.

I was suspecting that its the ps7 not initialised, but did not found any initialisation of the ps7 in the whole Litex so I was trying to find a way to add the initialisation to Litex. I will try the steps recommended by you. If it would work and there would be a way to add it to the Litex than the Zynq biard would be able to run the Linux on VexRiscV Litex project. That would be awesome thing! It was my intention from the start. I firstly wanted to just bring back the original Zybo and then hot better understanding of Litex and then I prioretized the Linux on it. Hence why I started adding more and more thing for the Zybo boards. I am preparing my new pull request with quite a few new things!

[mkuhn99]

I was suspecting that its the ps7 not initialised, but did not found any initialisation of the ps7 in the whole Litex so I was trying to find a way to add the initialisation to Litex. I will try the steps recommended by you. If it would work and there would be a way to add it to the Litex than the Zynq biard would be able to run the Linux on VexRiscV Litex project. That would be awesome thing! It was my intention from the start. I firstly wanted to just bring back the original Zybo and then hot better understanding of Litex and then I prioretized the Linux on it. Hence why I started adding more and more thing for the Zybo boards. I am preparing my new pull request with quite a few new things!

In theory there is also a 'ps7_init.c' being generated by vivado which could be somewhat included into the BIOS, but the vivado scripts are called after the compilation of the BIOS so this approach would probably need some bigger changes in the build process of Litex, so including the xsct commands in a skript is probably the more convenient approach.

I actually got linux-on-litex running on my zybo-board using the ddr-ram and UART of the PS, by just building a bitstream for vexriscv-smp and uploading the prebuilt Linux images via UART.

[JoyBed]

In theory there is also a 'ps7_init.c' being generated by vivado which could be somewhat included into the BIOS, but the vivado scripts are called after the compilation of the BIOS so this approach would probably need some bigger changes in the build process of Litex, so including the xsct commands in a skript is probably the more convenient approach.

I actually got linux-on-litex running on my zybo-board using the ddr-ram and UART of the PS, by just building a bitstream for vexriscv-smp and uploading the prebuilt Linux images via UART.

I tried the initialisation and now it kinda works.

Can you add your changes to newest fork of litex-board? I cant properly implement your newest changes to latest fork of litex-boards.

[JoyBed]

I tried your branches you linked here, it doesnt compile for me.

[mkuhn99]

I tried your branches you linked here, it doesnt compile for me.

Is there a "xil_cache.h" and "xil_cache.c" file in the include directory?

[JoyBed]

Yes they are there.

[JoyBed]

I just commented out that part for now. I cant get it to boot linux tho. And i was still wondering why I am getting errors in mem_test. I checked the main_ram content. Each read yields different content.

[mkuhn99]

I tried your branches you linked here, it doesnt compile for me.

I just fixed that on the ps_software branch. I actually just forgot to push the compile rule :)

I just commented out that part for now. I cant get it to boot linux tho. And i was still wondering why I am getting errors in mem_test. I checked the main_ram content. Each read yields different content.

I just tested my two linked branches for litex and litex-boards and I have no issues with the DDR ram.

I build with python3 litex-boards/litex_boards/targets/xilinx_zybo_z7.py --cpu-type=vexriscv --integrated-sram-size=0x10000 --with-ps7 --variant=z7-20 --build

I only can guess what your problem is:

• you are using the wrong zybo variant: z7-10 and not z7-20

• there is another process writing to your ddr:

  1. something is running on the PS-CPU?(stop it with 'xsct stop')
  2. some IO-Device is writing to the DDR, disconnect all non needed IO-Cables
  

If you would commend the command you use to build the bitstream, I could try to reproduce your errors.

[JoyBed]

I tried your branches you linked here, it doesnt compile for me.

I just fixed that on the ps_software branch. I actually just forgot to push the compile rule :)

I just commented out that part for now. I cant get it to boot linux tho. And i was still wondering why I am getting errors in mem_test. I checked the main_ram content. Each read yields different content.

I just tested my two linked branches for litex and litex-boards and I have no issues with the DDR ram.

I build with python3 litex-boards/litex_boards/targets/xilinx_zybo_z7.py --cpu-type=vexriscv --integrated-sram-size=0x10000 --with-ps7 --variant=z7-20 --build

I only can guess what your problem is:

* you are using the wrong zybo variant: z7-10 and not z7-20

* there is another process writing to your ddr:
  ```
  1. something is running on the PS-CPU?(stop it with 'xsct stop')
  2. some IO-Device is writing to the DDR, disconnect all non needed IO-Cables
  ```

If you would commend the command you use to build the bitstream, I could try to reproduce your errors.

I have the original Zybo not the Z7 hence why i added back the support for it. In meantime I fixed the DDR and now with your Makefile fix I will try to do a compilation with PS uart peripheral instead of the UART board on the PMOD. I also noticed you have another branch with enet support. Why it isnt together in one branch?
EDIT: Check out my forks now.

[JoyBed]

After some time of using it I noticed 2 things. First is that even tho Litex BIOS can use the UART of the PS7, the Linux images cant, I needed to still use PMOD-UART. Second thing is that with this type of SDRAM implementation the video framebuffer isnt able to compile as it is trying to connect to crossover thats not existant in this implementation. A small sidenote is that performance of the SDRAM is dependant on the core clock. I cant use any other clock source other than the core clock. I also tried using the slave HP ports to gain better ram performance but they are failing too. I dont understand why tho as the HP ports should be directly connected to the DDR controller and use same AXI addresses as acording to this from the datasheets.

Any thoughts?

[mkuhn99]

After some time of using it I noticed 2 things. First is that even tho Litex BIOS can use the UART of the PS7, the Linux images cant, I needed to still use PMOD-UART.

This is true. My other comment was misleading. I only tested the ram of the PS and not the PS-UART. I'm very sorry for the misstatement. If one wants to use the PS-UART with the linux they have to recompile the linux/buildroot Image and link the new UART functions somehow. I just looked into it a bit, but I don't understand yet how this is done.

Second thing is that with this type of SDRAM implementation the video framebuffer isnt able to compile as it is trying to connect to crossover thats not existant in this implementation.

I never looked into the video framebuffer so I can't help you here.

A small sidenote is that performance of the SDRAM is dependant on the core clock. I cant use any other clock source other than the core clock.

When I open the "Customize-IP GUI" of the ps I can only use the DDR PLL as a clock source for the DDR , which seems reasonable to me, so I don't know exactly why one would want to change that.

I also tried using the slave HP ports to gain better ram performance but they are failing too. I dont understand why tho as the HP ports should be directly connected to the DDR controller and use same AXI addresses as acording to this from the datasheets.

Using the HP for communication with the DDR sounds like a good idea to me, but i haven't looked into it yet. According to the Zynq manual (UG585, p. 786 & p. 132) the AXI-HP have some registers you might want to look into for debugging if you haven't done yet. According to UG585 (p. 140) the AXI-HP also uses additional signals. Is there a compatible AXI-HP interface and/or AXI-HP to AXI-full interface for litex?

[JoyBed]

After some time of using it I noticed 2 things. First is that even tho Litex BIOS can use the UART of the PS7, the Linux images cant, I needed to still use PMOD-UART.

This is true. My other comment was misleading. I only tested the ram of the PS and not the PS-UART. I'm very sorry for the misstatement. If one wants to use the PS-UART with the linux they have to recompile the linux/buildroot Image and link the new UART functions somehow. I just looked into it a bit, but I don't understand yet how this is done.

Second thing is that with this type of SDRAM implementation the video framebuffer isnt able to compile as it is trying to connect to crossover thats not existant in this implementation.

I never looked into the video framebuffer so I can't help you here.

20 minutes before you replied to me I got something together that might work for the framebuffer, right now its synthesizing so I will let you know the results. If it will work I will add that to my forks.

A small sidenote is that performance of the SDRAM is dependant on the core clock. I cant use any other clock source other than the core clock.

When I open the "Customize-IP GUI" of the ps I can only use the DDR PLL as a clock source for the DDR , which seems reasonable to me, so I don't know exactly why one would want to change that.

I didnt meant the clock for the DDR chips but the AXI bus clock going from SoC to AXI GP slave port.

I also tried using the slave HP ports to gain better ram performance but they are failing too. I dont understand why tho as the HP ports should be directly connected to the DDR controller and use same AXI addresses as acording to this from the datasheets.

Using the HP for communication with the DDR sounds like a good idea to me, but i haven't looked into it yet. According to the Zynq manual (UG585, p. 786 & p. 132) the AXI-HP have some registers you might want to look into for debugging if you haven't done yet. According to UG585 (p. 140) the AXI-HP also uses additional signals. Is there a compatible AXI-HP interface and/or AXI-HP to AXI-full interface for litex?

Yes actually Litex has the whole AXI_HP_Slave function and in the brackets its just as GP slave. I tried it but it doesnt work... I will look into the registers youre talking about. Will let you know.

[mkuhn99]

I managed to get the PS-UART running using Interrupts from the PS (litex, litex-boards). Using the interrupts is not essential to use the UART, but it might be a good example on how to connect and handle interrupts from PS-IO devices.
I could add these features to this pull request or make a new one after this one got merged. I am also not sure if I added the c code appropriately.

[JoyBed]

I managed to get the PS-UART running using Interrupts from the PS (litex, litex-boards). Using the interrupts is not essential to use the UART, but it might be a good example on how to connect and handle interrupts from PS-IO devices.

I could add these features to this pull request or make a new one after this one got merged. I am also not sure if I added the c code appropriately.

Well if I use your changes will the linux be working? Or will it generate the dts properly?

[mkuhn99]

I managed to get the PS-UART running using Interrupts from the PS (litex, litex-boards). Using the interrupts is not essential to use the UART, but it might be a good example on how to connect and handle interrupts from PS-IO devices.
I could add these features to this pull request or make a new one after this one got merged. I am also not sure if I added the c code appropriately.

Well if I use your changes will the linux be working? Or will it generate the dts properly?

No. It was just a remark on this pull request.