Build Video Application

Developing Video Applications on Spartan-3A DSP FPGAs

Video Starter Kit Tutorial – Lab 3

Create a Simulink Edge Detector

This Page Intentionally Left Blank

VSK Tutorial Lab D Overview

In this part of the tutorial you will learn how a high-level Simulink executable specification can be used with model-based design techniques to create an FPGA hardware design. A series of design will be created that demonstrate how a frame based Simulink design is serialized and turned into a streaming design suitable for implementation in an FPGA.

The objective of this lab is to provide an introduction to the Xilinx Simulink to FPGA design methodology. For more detailed training on System and the Xilinx DSP Blockset please visit the Xilinx e-learning web site and check out the free System Generator e-learning training class at:

http://www.xilinx.com/support/training/free-courses.htm

Objectives

·  Introduce users to the model based design flow for video applications targeting Xilinx FPGAs

·  Show how frame based executable specifications can be used for serial streaming hardware designs

·  Introduce users to Xilinx System Generator for DSP

Part 1: Create a Simulink Simplified Executable Specification

Procedure

  1. Copy the file “edge_detection_of_gamma_image.mdl” from the Lab2 folder into the Lab3 folder and rename this file “edge_detect_hw_arch_def.mdl”.
  2. Launch Simulink and set the working directory to the Lab3 folder and open the “edge_detect_hw_arch.mdl” file. Re-simulate the design to make sure it still functions correctly.

Note: In this part of the lab we need to convert the image frame of data into a serial pixel stream that enters the hardware and then take the serial stream output from the hardware and build back up a frame of data that can be used by the abstract Simulink Imaging and Video Blockset

  1. Add the following blocks to the Simulink diagram

-  Signal Processing Blockset/Math Functions/Matrices and Linear Algebra/Matrix Operations/Transpose

-  Signal Processing Blockset/Signal Management/Signal Attributes/Convert 2D to 1D

-  Signal Processing Blockset/Signal Management/Signal Attributes/Frame Conversion

-  Signal Processing Blockset/Signal Management/Buffers/Unbuffer

  1. Once added connect after the “Color Space Converter” subsystem as shown below:

Next we will convert the serial data stream back to a frame. Then we can verify the logic is working correctly.

  1. Add the following blocks to the Simulink diagram

-  Signal Processing Blockset/Math Functions/Matrices and Linear Algebra/Matrix Operations/Transpose

-  Signal Processing Blockset/Signal Management/Signal Attributes/Convert 1D to 2D

-  Signal Processing Blockset/Math Functions/Matrices and Linear Algebra/Matrix Operations/Sub Matrix

-  Signal Processing Blockset/Signal Management/Buffers/Buffer

-  Simulink/Signal Attributes/Rate Transition

-  Simulink/Commonly Used Blocks/Data Type Conversion

-  Simulink/Video and Image Blockset/Video Viewer – Wrong directory and block

And connect the blocks as shown below.

  1. Push the four frames to serial blocks into a subsystem and rename the subsystem “Frame to Serial”. Then push the four serial to frame blocks into a subsystem called “Serial to Frame”. Again this will help keep the diagram manageable.
  1. Change the output datatype in the “From Multimedia File” block to “uint8”

We now need to setup some parameters on the second subsystem to specify the image size. It makes sense to set these parameters as variables in the model properties “call backs” because they will be used in several places.

  1. Execute the pulldown menu “File -> Model Properties” and set the following 3 variables in the “InitFcn” field of the model callbacks. These variables will be set to the workspace and we can reference them in the properties fields of the Simulink blocks. Our frame size is 240 rows by 360 columns executes at 15 frames per second. Once done OK the form.
  1. Push down into the second subsystem that re-creates the image and double click on the “Rate Transition” block and set the “Output Port Sample Time” to 1/(row*col*fps).

Note: Inclusion of the “Rate Transition” block is technically not necessary for this exercise as the output sample times will be correctly calculated by the 4 remaining blocks. However this block is needed later for a lab that involves hardware co-simulation. It doesn’t hurt to put this in now and then we have a serial-to-frame conversion subsystem that will support all our future development efforts.

  1. Double click on the “buffer” block and set the following parameter. The buffer block needs to know how big a frame is. Once done OK for form.

  1. Double-click on the Convert from 1D to 2D block and set the following parameters. Simulink needs to know the size of the matrix you wish to create. Once done “OK” the form.

  1. Once done simulate the design – you should see a black and white version of image because we only have the red video stream connected. We have demonstrated that we have successfully converted the image into a streaming format then converted the streaming image back into the original frame.

We are now ready to begin architecting a Sobel Edge Detector on streaming data!

Part 3: Define a hardware Architecture using Simulink

In this part of the exercise we are going to create a streaming model of our color space converter and edge detector that represents the hardware architecture we wish to use.

The “Y” output of an RGB to Y’CrCb color space converter can be implemented using the following equation:

Y = 0.299R + 0.587G + 0.114B

The block diagram below shows the hardware architecture for a modified Sobel edge detector that does an approximation on the X and Y gradient using rows of data but not columns. The filter coefficients are displayed above each instance of the 3-tap FIR blocks.

For thresholding we’ll compare the summation to the value 100 and if it is greater set the output to 255 (white) or if is smaller set the output to 0 (black). This will give us clear edges for the gradients.

Procedure

  1. Add the following blocks to the diagram

Block / Library / Qty
Embedded MATLAB Function / Simulink/User-Defined Functions / 1
Integer Delay / Simulink/Discrete / 2
Digital Filter / Signal Processing Blockset/Filtering/Filter Designs / 3
add / Simulink/Math Operations / 2
abs / Simulink/Math Operations / 1
Constant / Simulink/Sources / 2
Switch / Simulink/Signal Routing / 1
Data Type Conversion / Simulink/Commonly Used Blocks / 1
  1. Double click on the Embedded MATLAB function and modify the MATLAB function as shown below: The embedded MATLAB function block provides a simple way to implement the color space converter that we need for this design.
  1. Connect the blocks as shown below:
  1. Double-click on the Integer Delay block and set the delay = “col”. Remember we set this as a Model Properties Callback InitFcn variable. We will use this simple delay to model a line buffer
  2. Double-click on each of the 3 Digital Filters and set the coefficients to the corresponding values shown in the block diagram at the beginning of Part 3.
  3. Set the constant values to “255” and “0”
  4. Set the Switch “Threshold” field = 90.
  5. Set the “Convert” block to output “double” datatypes. This Simulink digital filter requires a double as input.
  6. Once done push all the logic for this edge detector into a subsystem – this will help keep the top-level diagram manageable


The Edge Detector subsystem should be connected at the top level as shown below.

  1. Set the simulation period to “1/fps” or “1/15”. The video source executes at 15 “frames per second” and for each Simulink simulation cycle will process 15 frames of data. For hardware development purposes we can limit this to one frame for now. Since we set the “fps” variable in the callbacks we can reference it here.
  1. Once complete Simulate the design – You will notice some slight differences between the Video and Imaging Blockset Sobel Edge Detector and the hardware accurate model we created from Simulink – this is primarily due the specification of a lower threshold value. It is set for the lighting in the building rather then the outdoor lighting of the video. The basic results should look acceptable. If you wish you can adjust the threshold value specified in the switch block on the Simulink edge detector to get a more exact image.


Part 4: Define a hardware Architecture using System Generator

In this part of the exercise we are going to design the exact same Edge Detector using System Generator and the Xilinx DSP Blockset. This should be a straightforward process as System Generator has blocks that correspond to the ones used in the Simulink model

Procedure

  1. Add the following blocks to the Diagram

Block / Library / Qty
System Generator / Xilinx Blockset/Index / 1
Gateway In / Xilinx Blockset/Index / 1
Gateway Out / Xilinx Blockset/Index / 1
FIR Compiler 4.0 / Xilinx Blockset/Index / 3
AddSub / Xilinx Blockset/Index / 5
Constant / Xilinx Blockset/Index / 3
Relational / Xilinx Blockset/Index / 1
Mux / Xilinx Blockset/Index / 1
Delay / Xilinx Blockset/Index / 3
Cmult / Xilinx Blockset/Index / 3
Convert / XilinxBlockset/Index / 1
  1. Open the library called “lab3_lib.mdl” and add two of the ABS blocks. The ABS block doesn’t exist in the Xilinx Blockset. An ABS block has been saved to a library to save time. This ABS block is actually provided in one of the other VSK reference designs – it is in the “DVI_Pass_Through_Demo”. If you wish you can get it from there as well.
  2. First we will create the color space converter Wire up the design as shown in the diagram below.
  1. Double-click on the “Convert” block and set the output datatype to Unsigned [8 0]
  1. Set the constant values of the Cmult blocks to .299 for the R input, .587 for the “G” input and .114 for the “B” input. Also set the latency equal to 1. You don’t need to modify the fixed point values but you may if you wish – the default setting is provides more bits than are necessary. The Spartan-3A DSP supports arithmetic operations up to 18 bits with a hard DSP48A slice.
  1. Once done with the CSC – push the logic into a subsystem (cntrl <g>) and connect the edge detector as shown below
  1. Set the parameters on the AddSub and Delay blocks to have a latency of 1. This will force a pipeline register into the block to improve performance.
  2. Double-click on the “Gateway In” blocks and set the following parameters:

Note: System Generator requires that the actual sample rate be specified on the Gateway In and System Generator blocks. In this model a frame of data is being input to the system with a sample rate of 1/fps. Because this “Frame” is decomposed into a vector of data that gets passed into System Generator than the system generator design must run at a much higher rate – it is “row*col” faster to be exact. This is accomplished by setting the “Sample Period” field = 1/(row*col*fps). This is an important concept to understand when using System Generator with the Video and Imaging Blockset

  1. Set the Coefficients on each of the FIR Compiler 4.0 blocks to the same values as the Simulink Digital Filter blocks. There 3 different coefficients. Below is an example of one of them..
  1. On the FIR Compiler Implementation Tab select the “Transpose_Multiply_Acculate” architecture.
  1. Set the constant blocks to the values of “0”, “255” and “90”
  2. Set the System Generator Block Simulink system period to 1/(row*col*fps) as shown below:
  1. Once the design is wired together and the block parameters are set push the System Generator blocks into a subsystem as you did in the previous lab. Your diagram should look something like this:
  1. Once done Simulate the design. If you get an error about “continues states” double check all the sample rate settings. This means that two sample rates have come together somewhere in the Simulink diagram where they are not supposed to. You should see results that are similar as before.

End of Lab 3

Developoing Video Applications on Xilinx FPGAs Page 13 of 13 April 23rd, 2008