Embedded Linux

Neso, Microblaze And Linux: How To Boot Linux On Neso Artix 7 FPGA Module From SPI Flash

1523 views May 18, 2016 rohitsingh 5

In the previous article we saw how to build Linux Kernel and run it on Neso Artix 7 FPGA Module using Xilinx Platform Cable USB and XMD. Though this is the easier method, there are a couple of drawbacks associated to it. You will need a Xilinx Platform Cable USB which is slightly expensive and the images are programmed to SRAM/DDR instead of SPI flash. It is possible to program the onboard SPI flash with the images which would retain contents even after power cycle and this can be done without buying expensive JTAG cables.

To program the onboard SPI flash with, we will need to make a single binary image that will contain the following individual images.

  • FPGA bit file for Microblaze
  • Linux kernel
  • A bootloader

A bootloader is necessary here to copy the Linux kernel from SPI flash to DDR and execute it. The primary requirements for the bootloader is that it should be capable of reading data from the SPI flash and should fit within the BRAM cache size.

First, we need to modify the previous Vivado project.

Step 1:

We have already created a block diagram for previous article, we need to add SPI in the block diagram to be able to use flash tool, for that open the block diagram in vivado, click on “add IP” and add “AXI Quad SPI”. Double click on AXI Quad SPI block and change its settings as shown in the figure. Make sure to check the “Enable STARTUPE2 Primitive” option.

 Click on “Run Block Automation” on the top left corner of the window to complete the design. Click OK for Vivado to automatically configure the blocks for you. Once Block Automation is complete, run “Connection Automation” so Vivado can connect the blocks together to make a complete system.

Final diagram should look like this:

Step 2: Now “Create HDL Wrapper” from the popup menu. Click OK on the window that appears to finish generating wrapper. Then click on the “Run Implementation”. Vivado will now synthesize and implement the design. Status of implementation process will be shown in the log window. Once implementation is complete, click the “Generate Bitstream” button next to the implementation button to generate the bitstream for the design. Now that the design is implemented successfully, we need to export the bitstream and other related files to be used with SDK. On the file menu, select Export > Export Hardware.

 

Generating Device Tree

Having completed all the steps before, now download the device-tree repository from Xilinx’s github account: https://github.com/Xilinx/device-tree-xlnx.git . If you downloaded the zip, extract it in the same location as your SDK workspace. In case of the author, the directory structure was “D:\VIVADO_PROJECTS\LINUX_NESO\LINUX_NESO.sdk\device-tree-xlnx-master\”.

Step 1: In Vivado, select File -> Launch SDK. This opens the Xilinx SDK

Step 2: In SDK, go to Xilinx Tools -> Repositories.

Neso_Linux_3_repositories

Step 3: In the Dialog window, we have a section named “Local Repositories”.  Select “New” and select the device tree folder which you had downloaded and extracted to SDK workspace. Final result should look like in the below image.

Neso_Linux_4_repositories_new

Step 4: Go to File -> New -> Board Support Package. In the Dialog window, select “device_tree” under “Board Support Package OS” and press “Finish” and then “OK” in another dialog that opens.

Neso_Linux_5_device_tree_bsp

This should generate a system.dts file along with a .dtsi file. We need both of them.

Neso_Linux_6_device_tree_files

Step 5: We have to merge the ‘.dtsi’ and ‘.dts’ files. So open both of them in some text editor (Notepad++ etc) and copy the contents of system.dts and paste them in the .dtsi file just after the line ” model = “Xilinx MicroBlaze” .  Don’t forget to add “/dts-v1/;” at the top of .dtsi file also. The final content should look like this:

Neso_Linux_7_device_tree_merged-1

Now, save this modified file as “nesoartix7.dts”. Now build linux image , for this follow the process same as in the previous article, but use the .dts file that we just ctreated, as it conytains the additional needed information about SPI flash.

To build a bootloader, launch SDK from Vivado and create a new application project. Let’s call this project “neso_linux_bootloader”. When selecting the application template, select “Hello World”. Once the project is created, open the “neso_linux_bootloader” project in the “Project Explorer” on the left. Double click on the linker script lscript.ld and make sure that all sections are mapped to BRAM (should look something like microblaze_0_local_memory_ilmb_bram_if_cntlr_microblaze_0_local_memory_dlmb_bram_if_cntlr). If not, map all sections to BRAM manually and save the linker script. After saving linker script, build the project and make sure that no errors are reported.

 

Open helloworld.c in the project and replace the entire code with the code below. Please note, the code below is a bare-bones bootloader designed to be simpler than being robust.

#include <stdio.h>
#include "platform.h"
#include "xparameters.h"
#include "xstatus.h"
#include "xspi.h" 
#include "xil_cache.h"
 
void print(char *str);
 
//Set the offset and size of image in SPI flash
#define FLASH_IMAGE_START_ADDRESS 0x500000
#define FLASH_IMAGE_SIZE 0x700000
 
//Set the address where image will be loaded. This will usually point to
//DDR or SRAM depending on the board architecture. Remember to build
//your application/Linux kernel with this address as base address
#define IMAGE_LOAD_ADDRESS XPAR_MIG_7SERIES_0_BASEADDR
 
//Define ID of the SPI peripheral that is connected to the SPI flash
#define SPI_DEVICE_ID XPAR_SPI_0_DEVICE_ID
 
void (*imageEntry)();
XSpi Spi;
 
u8 spiReadData()
{
    while(!(XSpi_ReadReg(Spi.BaseAddr, XSP_SR_OFFSET) & 0x02));
    return XSpi_ReadReg(Spi.BaseAddr,XSP_DRR_OFFSET);
}
 
//This function writes one byte to the SPI peripheral
void spiWriteData(u8 data)
{
    while(XSpi_GetStatusReg(&Spi) & 0x08);
    XSpi_WriteReg(Spi.BaseAddr, XSP_DTR_OFFSET, data);
}
 
int loadAppImage()
{
    XSpi_Config *cfgPtr;
    u8 recBuffer[4];
    u32 i = 0, index = 0, ddrPtr = 0;
 
    print("Initializing Numato Lab Neso SPI Linux Image Loader...\n\r");
    print("*** http://numato.com ***\n\r");
    print("\n\r");
 
    //Lookup SPI peripheral configuration details
    cfgPtr = XSpi_LookupConfig(SPI_DEVICE_ID);
    if (cfgPtr == NULL)
    {
        return XST_DEVICE_NOT_FOUND;
    }
 
    if(XSpi_CfgInitialize(&Spi, cfgPtr, cfgPtr->BaseAddress) != XST_SUCCESS)
    {
        return XST_FAILURE;
    }
 
    //Beyond this point we will use only low level APIs in favor of smaller
    //and simpler code.
 
    //Set up SPI controller. Master, manual slave select. The SPI peripheral
    //is configured with no FIFO
    XSpi_SetControlReg(&Spi, 0x86);
 
    //Disable interrupts
    XSpi_IntrGlobalDisable(&Spi);
 
    //Cycle CS to reset the flash to known state
    XSpi_WriteReg(Spi.BaseAddr, XSP_SSR_OFFSET, 0x00);
    XSpi_WriteReg(Spi.BaseAddr, XSP_SSR_OFFSET, 0x01);
    XSpi_WriteReg(Spi.BaseAddr, XSP_SSR_OFFSET, 0x00);
 
    //Write command 0x0b (fast read) to SPI flash and do a dummy read
    spiWriteData(0x0b);
    spiReadData();
 
    //Send the address from where the image needs to be loaded.
    //Dummy read after every write as usual
    spiWriteData((FLASH_IMAGE_START_ADDRESS >> 16) & 0xff);
    spiReadData();
    spiWriteData((FLASH_IMAGE_START_ADDRESS >> 8) & 0xff);
    spiReadData();
    spiWriteData((FLASH_IMAGE_START_ADDRESS) & 0xff);
    spiReadData();
 
    //A dummy write/read as per W25Q128FV datasheet
    spiWriteData(0x00);
    spiReadData();
 
    print("Loading application image...\n\r");
 
    for(i=0; i<=FLASH_IMAGE_SIZE; i++)
    {
        //Do a dummy write
        spiWriteData(0x00);
 
        //Read data back
        recBuffer[index] = spiReadData();
        index++;
 
        //Write the data to DDR/SRAM four bytes at a time
        if(index >= 4)
        {
            *((u32*)(ddrPtr + IMAGE_LOAD_ADDRESS)) = *((u32*)(&recBuffer));
            ddrPtr += 4;
            index = 0;
        }
    }
 
    print("Executing application image...\n\r");

    //Invalidate instruction cache to clean up all existing entries
    Xil_ICacheInvalidate();

    //Execute the loaded image
    imageEntry = (void (*)())IMAGE_LOAD_ADDRESS;
    (*imageEntry)();
 
    //We shouldn't be here
    return 0;
}
 
 
 
int main()
{
    init_platform();
    loadAppImage();
    cleanup_platform();
    return 0;
}

Rebuild the project with newly added code and make sure that no errors reported. If any errors pop up, please fix them before continuing.

The above code will load the kernel image from SPI flash memory and write to DDR and jump to kernel entry point. This code assumes that the kernel image is sitting at a specific address in the SPI flash (0x500000 in this case) and has a specific size (0x700000) in this case. 0x700000 is little bigger than the actual kernel image but that shouldn’t cause any problems. The area starting from 0x000000 to 0x4FFFFF is reserved for FPGA configuration data (enough to hold configuration data for XC7A100T devices whose bitstreams’ size is 3825788 bytes, or 0x3A607C bytes in hex). So in essence, we are targeting the following memory map for the SPI flash binary image we are going to create.

  1. 0x000000 – 0x4FFFFF -> FPGA configuration data
  2. 0x500000 – 0xBFFFFF -> Kernel image
  3. 0xC00000 upwards -> User data (not used in our case)

At this point, we have the bit file for FPGA configuration, Linux kernel image and bootloader image we just built. Before we can proceed with packing these images in to a single binary file, there is one crucial step we need to do. You may remember that fact the the kernel image we built (simpleImage.nesoartix7) is in ELF format. This is helpful when downloading kernel using XMD but the metadata (in addition to executable code) is going to be a problem since our bootloader does not understand ELF format. One option is to add ELF loader code to bootloader but it is going to be time consuming and simply an overkill for such a simple implementation like this. Another option would be to strip the metadata and generate a pure executable binary file.  This can be easily done with objcopy tool (objcopy is part of gnu binutils). But the tricky thing here is that you can not use any objcopy (eg: objcopy built for host machine). You will need to use objcopy that is built for Microblaze. Fortunately, buildroot already built cross objcopy that runs on host machine but can work on Microblaze executables. This happened when we built the Linux kernel earlier (buildroot builds all cross tools needed for building the kernel). All that we need to do is to find where these cross compiled tools are placed by buildroot. With the specific buildroot version we used, the objcopy executable we need was placed in the directory “output/host/usr/microblazeel-buildroot-linux-gnu/bin” under the buildroot root directory.

Switch to output/images directory under buildroot root directory where Linux kernel image is available in ELF format and execute the following command.

cmd>../host/usr/microblazeel-buildroot-linux-gnu/bin/objcopy -O binary simpleImage.nesoartix7 neso_linux.bin

If the command was successful, you will see a new file created with the name neso_linux.bin. Copy this file over to the Windows machine.

On the Windows machine, place the following files in the same directory.

  1. The Linux image in binary format (neso_linux.bin we just created)
  2. The bit file(.bit) for the Microblaze system and the Memory Map file (.mmi)  (Both files can be found in your SDK’s Project Explorer under project named “neso_linux_mb_wrapper_hw_platform_0” or similar. Please see the image below)Neso_Bootloader_1_mmi_bit_files
  3. The bootloader executable (This file can be found in the directory neso_linux_bootloader\Debug under the SDK bootloader project folder and has .elf extension)

 

In my case, I have the following files in my directory as shown in the image. Your file names may be different depending on the project names etc…

Neso_Bootloader_2_all_files

With all necessary images available now, let us move forward with creating the final binary image. We will do the following steps to create a single binary image.

  1. Merge the FPGA configuration (.bit file) and the Bootloader executable image (.elf). This is because the contents of the executable needs to go in to the FPGA BRAM when FPGA initialize after power up.
  2. Concatenate the combined FPGA configuration file (generated in previous step) with Linux kernel image.

To merge Bootloader executable image with bit file, run the following command.

cmd> C:\Xilinx\Vivado\2015.2\bin\vivado.bat -mode tcl
cmd> updatemem -force -meminfo "neso_linux_mb_wrapper.mmi" -bit "neso_linux_mb_wrapper.bit" -data "neso_linux_bootloader.elf" -proc neso_linux_mb_i/microblaze_0 -out "download.bit"

Neso_Bootloader_3_updatemem

If the command ran successfully, you will see a new file created in the directory with name “download.bit”. This file now has both the FPGA configuration stream and Bootloader code.

Now we can concatenate the FPGA configuration file we just generated (download.bit) with Linux kernel binary image (neso_linux.bin) to create a single binary image that can be downloaded to the SPI flash. Run the following commands to generate single binary image.

cmd>exit
cmd>C:\Xilinx\14.7\ISE_DS\settings64.bat
cmd>promgen -w -p bin -c FF -o download -s 16384 -u 0 download.bit -data_file up 500000 neso_linux.bin -spi

First command exits the Vivado tcl shell. Second command loads the ISE command line environment. And finally the third one actually does the work of merging two files into a final download.bin file.

Neso_Bootloader_4_final_image

If this command succeeded, you will see a few new files in the directory and one of them would be “download.bin”. This is the final binary image that we need to download to Neso’s SPI flash. The following image shows all files in my directory after executing all above mentioned steps.

Neso_Bootloader_5_all_files

The final image is approximately 11.10MB in size and this leaves with approximately 4.9MB of available space in the SPI flash for custom user data.

Now download the latest  Neso Artix 7 FPGA Module configuration tool from the product page and use that to download the final binary image “download.bin” to the SPI flash (see image below).

Neso_Bootloader_10

It may take a couple of minutes for the download to complete. While download is in progress, go ahead and start your favorite serial terminal emulation program (Hyperterminal, PUTTY, TeraTerm etc..) and open the COM port corresponding to Neso’s FT2232 channel B and set baud rate to 115200 and set handshaking to off/none (You should have configured channel B of Neso’s FT2232H as serial port. If not, please follow this tutorial to do so).

Once the binary image is completed downloading, the tool will reboot the FPGA and let it load the configuration data from newly downloaded image. If everything went well, the bootloader will start copying Linux kernel image to DDR.

Neso_Bootloader_6_bootloader_init

It may take a few seconds for the bootloader to copy the Linux kernel. Once copying is completed, bootloader will start the kernel and you will see the kernel booting as shown below.

Neso_Bootloader_7_linux_booting

You can login to Linux as root with password as “neso”. Image below shows root directory structure, cpuinfo and “uname -a” outputs:

Neso_Bootloader_8_commands-1

Was this helpful?

5 Comments
  • Tony says:

    Hello,
    I followed this procedure to the best of my ability and i have a few things that i’d like to ask.
    1) The code that you have above where we are supposed to c&p into the helloworld.c file. I noticed that I get the following 3 errors when it came to build
    a) ‘XPAR_MIG_7SERIES_0_BASEADDR’ undeclared (first use in this function)
    b) conflicting types for ‘print’
    c) make: *** [src/helloworld.o] Error 1
    What i did to overcome this was to make the following adjustments to the code
    WAS:
    #define IMAGE_LOAD_ADDRESS XPAR_MIG_7SERIES_0_BASEADDR
    IS:
    #define IMAGE_LOAD_ADDRESS XPAR_MIG7SERIES_0_BASEADDR
    WAS:
    void print(char *str);
    IS:
    //void print(char *str);

    once I made edits to “IS” i then re-built and this seemed to solve the problems. However, I dont know if there are other issues that now have been invoked by doing so.
    2) When I first used the updatemem command I got another that stated the following
    ERROR: [Updatemem 57-85] Invalid processor specification of: msys_i/sys_microblaze. The known processors are: msys_i/sys_microblaze

    To solve this issue I went into the *.mmi file and did the following edit:
    WAS:

    IS:

    Again like in 1) I dont know if this causes other issues down the road (I dont think so) but it seemed to have to solve that issue for now

    3) Now when I seemed to overcame 2) I got another error that stated the following (this one has stumped for the time being)
    CRITICAL WARNING: [Updatemem 57-154] Matching address space for code segment 1 n
    ot found. Code segment occupies [0x80000000:0x800033AB]
    ERROR: [Updatemem 57-153] Failed to update the BRAM INIT strings for neso_linux_bootloader.elf and msys_wrapper.mmi.
    ERROR: [Common 17-39] ‘update_mem’ failed due to earlier errors.

    Any thoughts? I am thinking that perhaps it has something to do with the *.ld file but I could be wrong.

    4) I decided to put this version aside and start a new one using my co workers SPI Boot sources and using your procedure. With this version I have overcame the problem with the updatemem command (I dont know how! but ok) but I have gotten stumped again with the promgen command from the following error
    Release 14.7 – Promgen P.20131013 (nt64)
    Copyright (c) 1995-2013 Xilinx, Inc. All rights reserved.
    0x947a5c (9730652) bytes loaded up from 0x0
    ERROR:Bitstream:32 – 0x62cfb4 bytes loaded up from 0x500000 overlaps load at 0x0

    Any thoughts? If i come across something ill come back and post something for future people following this

    March 16, 2017 at 5:30 pm
    • Tony says:

      Sorry I noticed that my WAS and IS in 2) was deleted for some reason. here they are

      WAS
      Processor Endianness=”Little” InstPath=”msys_i/sys_microblaze”

      IS
      Processor Endianness=”Little” InstPath=”neso_linux_mb_i/microblaze_0″

      March 16, 2017 at 5:33 pm
      • Tony says:

        Ok, so I was able to solve 3). Since the error it produced said that “Code segment occupies [0x80000000:0x800033AB]” I went to the lscript.ld file and opened it up in Xilnx SDK with through the use of the text editor tool. I then made the following edit:

        WAS
        axi_bram_ctrl_0_S_AXI_BASEADDR : ORIGIN = 0xC0000000, LENGTH = 0x2000

        IS
        axi_bram_ctrl_0_S_AXI_BASEADDR : ORIGIN = 0xC0000000, LENGTH = 0x4000

        Since the error said it took 33AB worth of code segment, I increased the size to 4000 and it seems to take it.

        I then rebuilt and was able to produce the download.bit file from the updatemem command

        I now am stuck at 4) on both versions.

        March 16, 2017 at 9:02 pm
    • Rohit Singh says:

      Hi Tony,

      For the Error 4) This is the log you are getting:
      0x947a5c (9730652) bytes loaded up from 0x0
      ERROR:Bitstream:32 – 0x62cfb4 bytes loaded up from 0x500000 overlaps load at 0x0

      ^ There is clearly something wrong going on here. Command is loading 0x947A5C bytes from offset 0x00, which will clearly overlap with 0x500000 offset from where the linux image is going to be loaded.

      Can you check the size of your bitstream file? It should be around 2-3MBs and not more, definitely should not be 0x947A5C bytes.

      For Error 1) Xilinx might have changed the #defines in newer Vivado versions. If that’s the case, please update it. And you need to overwrite the complete helloworld.c file created by Xilinx with the code in this article. Also would also need to update the linker script to make sure entire code is mapped to Block RAM and not DDR.

      For 4) you can also use Vivado’s command with this format: write_cfgmem -force -format BIN -size 16 -interface SPIx1 -loadbit “up 0x00 download.bit” -loaddata “up 0x500000 neso_linux.bin” download.bin

      March 17, 2017 at 5:07 am
  • Tony says:

    Hi Rohit.. ahh.. yes. So I have my own microblaze design using an Artix 7 board but yes my bitstream file is about 9.5MB. I was just reading right now on how to use the promgen command. I tinkered around with it and did the following

    WAS

    promgen -w -p bin -c FF -o download -s 16384 -u 0 download.bit -data_file up 500000 neso_linux.bin -spi

    IS

    promgen -w -p bin -c FF -o download -s 16384 -u 0 download.bit -data_file up 950000 neso_linux.bin -spi

    that produced this output

    C:Xilinx14.7ISE_DS>promgen -w -p bin -c FF -o download -s 16384 -u 0 download.bit -data_file up 950000 neso_linux.bin -spi
    Release 14.7 – Promgen P.20131013 (nt64)
    Copyright (c) 1995-2013 Xilinx, Inc. All rights reserved.
    0x947a5c (9730652) bytes loaded up from 0x0
    0x62cfb4 (6475700) bytes loaded up from 0x950000
    Using user-specified prom size of 16384K
    Writing file “download.bin”.
    Writing file “download.prm”.
    Writing file “download.cfi”.

    but i have a feeling that i dont think this is all i have to do i have to go back and do this whole process again because i have to change perhaps the following as well

    #define FLASH_IMAGE_START_ADDRESS 0x500000

    to

    #define FLASH_IMAGE_START_ADDRESS 0x950000

    in the helloworld.c file correct? i think? it seems like its something i need to do. anyways im going to tinker around some more

    March 17, 2017 at 4:54 pm
Leave A Comment
*
*