Skip to main content

Video Decoding

This section shows how to configure the TouchGFX Generator to generate a TouchGFX HAL that has video decoding capabilities.

Before reading this scenario, please read the documentation on using MJPEG Video in TouchGFX and the documentation on the Video widget itself.

The following scenario details how to enable TouchGFX HAL to support video decoding through either software (LibJPEG) or hardware (JPEG). It covers both STM32F7 and STM32H7 because the STM32CubeMX configuration to support Hardware (JPEG) decoding is slightly different from the STM32F7.

Be sure to read all sections in this article before configuring the MCU for your specific project.

Generally, the LibJPEG and JPEG configurations can be found in STM32CubeMX in the Middleware and Multimedia catagories:

JPEG and LibJPEG setting in STM32CubeMX

  • The latest version of the STM32H750-DK, STM32F769-DISCO and STM32F746G-DISCO TouchGFX Board Setups (TBS) support both Software and JPEG hardware decoding through their STM32CubeMX configurations.
  • Use the STM32CubeMX configurations found in these example projects as inspiration to get your specific MCU to support MJPEG Video Decoding.
  • RTOS Support

    The Generator User Guide mentions that Single- and Double buffer decoding strategies require a CMSIS compliant RTOS, such as FreeRTOS. TouchGFX Generator generates an entry point function videoTaskFunc(), that must be associated with a Video decoding task. STM32CubeMX can generate this configuration by defining the task and entry point function in the Tasks and queues tab of the FreeRTOS Middleware configuration.

    The video task stack size (defined in words for CMSIS V2) and RTOS heap size are two important factors.

    For software decoding the stack size must be carefully set because LibJPEG uses dynamic memory allocation. For hardware decoding the stack size can be substantially lower because there is no software stack which dynamically allocates memory.

    The FreeRTOS Heap should be large enough for your general application + 0xA000.

    FreeRTOS Task Configuration

    FreeRTOS Heap Size Configuration

    Based on the above configuration, STM32CubeMX generates the following code for the video task:

    /* Definitions for VideoTask */
    osThreadId_t VideoTaskHandle;
    const osThreadAttr_t VideoTask_attributes = {
    .name = "VideoTask",
    .stack_size = 1000 * 4,
    .priority = (osPriority_t) osPriorityLow,
    void main()
    /* creation of VideoTask */
    VideoTaskHandle = osThreadNew(videoTaskFunc, NULL, &VideoTask_attributes);
    #define configTOTAL_HEAP_SIZE                    ((size_t)75000)

    Software Decoding

    The software decoding solution requires that the LibJPEG middleware is enabled from STM32CubeMX, as specified in Generator User Guide. Software decoding setup is identical for all LibJPEG capable MCUs (e.g. STM32F4, STM32F7, STM32H7).

    The TouchGFX Software Decoder expects the data decoded by LibJPEG to have BGR pixel ordering. If this setting is left as RGB, R and B color components will be swapped in your application.

    Furthermore, the size of each pixel should be 3 bytes if using a 16-bit or 24-bit video RGB buffer or 4 if using a 32-bit video RGB buffer.

    LibJPEG Configuration for a 16-bit or 24-bit video RGB buffer

  • The RGB_ORDERING setting must be configured as BGR, and the pixel size must be set according to which video RGB buffer format is used.
  • If your application uses Direct to Framebuffer strategy in a ARGB8888 (32-bit) application the pixel size must be set to 4
  • Video Data

    The current decoding solution using STM32CubeMX, TouchGFX Generator and the Video widget from TouchGFX Designer will currently only work for ST boards because the video data is linked into the application at a specific address in memory mapped flash that is already known.

    The below snippet from an EWARM project shows the extra options that are created by TouchGFX Designer and how it places the data in ExtFlashSection which is defined inside the linker script for all TouchGFX Board Packages. This will not work for non-memory-mapped flash or projects that don't define this section.

    <state>--image_input $PROJ_DIR$\..\TouchGFX\generated\videos\bin\washerdryer.bin,video_washerdryer_bin_start,ExtFlashSection,4</state>
    <state>--keep video_washerdryer_bin_start</state>

    For projects with video data in non-memory mapped memory, please read the section FileReader in this article.

    Hardware Decoding

    The Generator User Guide mentions that the JPEG IP must be enabled in STM32CubeMX to enable Hardware decoding.


    The JPEG configuration for JPEG capable STM32F7 series, e.g. STM32F769, differs slightly from the STM32H7 line. The RGB_FORMAT setting found in the JPEG configurations must respect the pixel format of the TouchGFX Framebuffer, JPEG_RGB565 in the example below.

    JPEG Parameter Settings

    Use DMA to transfer data to (memory-to-peripheral) and from (peripheral-to-memory) the JPEG peripheral through the DMA Settings tab found in JPEG configurations. Adding a DMA request for IN and one for OUT will set up the direction parameters automatically.

    JPEG DMA Settings

    This concludes the TouchGFX HAL configuration for Hardware Decoding on the JPEG capable STM32F7s (e.g. STM32F769). After Generating code from STM32CubeMX the application will be able to decode videos using the JPEG peripheral when using the Video widget from the designer.

    The RGB_FORMAT of the JPEG peripheral configuration must respect the format of the TouchGFX Framebuffer


    The only thing that seperates Hardware decoding (JPEG capable) on e.g. an STM32H750 from an STM32F769 is the way DMA transfers are configured in STM32CubeMX. Not only is the UI different, but the DMA concepts are as well.

    For the STM32H750, the JPEG peripheral can only be configured to use the MDMA peripheral rather than the regular DMA peripherals. Add an MDMA configuration for both input- and output FIFO Threshold signals as shown in the images below.

    MDMA is a much higher performance DMA engine compared to DMA1/DMA2

    JPEG DMA Settings

    JPEG DMA Settings

    FileReader interface

    When storing MJPEG videos on non-memory-mapped memories developers can specify an implementation of touchgfx::VideoDataReader that the TouchGFX Video Controllers can use to hand data for decoding to the configured decoder (Software/Hardware). The below is a simple example of such an interface that copies video data from one buffer to another.

    class MyReader : public touchgfx::VideoDataReader
    MyReader() : position(0) { }
    virtual uint32_t getDataLength() { return video_len; }
    virtual void seek(uint32_t pos) { position = pos; }
    virtual bool readData(void* dst, uint32_t bytes)
    memcpy(dst, &video_data[position], bytes);
    position += bytes;
    return true;
    uint32_t position;
    } myReader;

    Instead of pointing a Video widget to the start of the video in mapped memory, developers can configure the widget to use the data reader instead:


    Migrating TBS to support video decoding

    If you want to migrate a project created from a TouchGFX Board Setups (TBS) - with a MCU that supports video decoding - from a version of TouchGFX Designer before video decoding was supported and you want to be able to use 'Run Target' in TouchGFX Designer with that project, some manual changes to the GCC Makefile are required.

    A run-down of the required changes and why they are needed will be listed in the following sections. There are some general changes that must always be applied and some LibJPEG (software decoding) and JPEG (hardware decoding) specific changes depending on what the developers application uses. The changes are an extension to the already existing GCC Makefile from the older TBS.

    Other than updating the Makefile you will also have to setup video decoding in STM32CubeMX as described in the above scenarios.

    General changes

    Define the LIBJPEG path in your project:

    # LibJPEG path
    libjpeg_path := $(cubemx_middlewares_path)/Third_Party/LibJPEG

    The video assets input path must then be defined:

    asset_texts_input  := TouchGFX/assets/texts

    asset_videos_input := TouchGFX/assets/videos

    The video assets output path must also be defined bellow the already existing assets output paths:

    asset_images_output := $(asset_root_path)/images
    asset_fonts_output := $(asset_root_path)/fonts
    asset_texts_output := $(asset_root_path)/texts
    asset_videos_output := $(asset_root_path)/videos

    Add the video output assets to the components list:

    all_components := $(components) \
    $(asset_fonts_output) \
    $(asset_images_output) \
    $(asset_texts_output) \

    Video object files must be defined. The video object files are separated from the already existing object:

    c_source_files := $(call find, $(source_paths),*.c) $(os_source_files) $(board_c_files)
    source_files += $(board_cpp_files)

    video_object_files := $(call find, $(asset_videos_output),*.o)

    A path to the video converter script must be defined:

    textconvert_script_path := $(touchgfx_path)/framework/tools/textconvert
    textconvert_executable := $(call find, $(textconvert_script_path), *.rb)
    videoconvert_script_path := $(touchgfx_path)/framework/tools/videoconvert

    An optional echo can be added to see all video objects files. The video object files must be added to the linking stage. This line has the $(video_object_files) added together with the other object files:

    $(binary_output_path)/$(target_executable): $(object_files) $(object_asm_files)
    @echo Video Objects: $(video_object_files)
    @echo Linking $(@)
    @mkdir -p $(@D)
    @mkdir -p $(object_output_path)
    @$(file >$(build_root_path)/objects.tmp) $(foreach F,$(object_files) $(video_object_files),$(file >>$(build_root_path)/objects.tmp,$F))

    Add the video rule to the existing assets and .PHONY:

    _assets_: BitmapDatabase TextKeysAndLanguages Videos

    .PHONY: BitmapDatabase TextKeysAndLanguages Videos

    The video rule is added with the video conversion added:

    @ruby $(videoconvert_script_path)/videoconvert.rb $(asset_videos_input) $(asset_videos_output)

    Lastly update the clean rule to also remove video related output.

    @echo Cleaning: $(board_name)
    @rm -rf $(build_root_path)
    # Do not remove gui_generated
    @rm -rf $(asset_images_output)
    @rm -rf $(asset_fonts_output)
    @rm -rf $(asset_texts_output)
    @rm -rf $(asset_videos_output)
    # Create directory to avoid error if it does not exist
    @mkdir -p $(asset_root_path)
    # Remove assets folder if it is empty (i.e. no gui_generated folder)
    @rmdir --ignore-fail-on-non-empty $(asset_root_path)
    # Clean bootloader project
    @$(MAKE) -r -f ExtMem_Boot/gcc/Makefile -s $(MFLAGS) clean

    Software changes

    Add all the LIBJPEG paths to the include paths:

    include_paths := $(library_includes) \
    $(foreach comp, $(all_components), $(comp)/include) \
    $(foreach comp, $(cubemx_components), $(comp)/Inc) \
    $(foreach comp, $(touchgfx_generator_components), $(comp)/generated) \
    $(framework_includes) \
    $(cubemx_middlewares_path) \
    $(touchgfx_middlewares_path) \
    $(touchgfx_generator_components) \
    LIBJPEG/Target \
    $(libjpeg_path)/include \

    LIBJPEG source paths must be defined:

    c_source_files := $(call find, $(source_paths),*.c) $(os_source_files) $(board_c_files)
    source_files += $(board_cpp_files)

    libjpeg_source_path = Middlewares/Third_Party/LibJPEG/source

    Next all the LIBJPEG source files must be added to board_c_files:

    board_c_files := \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_bus.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_qspi.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_sdram.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_ts.c \
    $(Drivers_path)/BSP/Components/ft5336/ft5336.c \
    $(Drivers_path)/BSP/Components/ft5336/ft5336_reg.c \
    $(Drivers_path)/BSP/Components/mt25tl01g/mt25tl01g.c \
    $(Drivers_path)/BSP/Components/mt48lc4m32b2/mt48lc4m32b2.c \
    $(libjpeg_source_path)/jaricom.c \
    $(libjpeg_source_path)/jcomapi.c \
    $(libjpeg_source_path)/jdapimin.c \
    $(libjpeg_source_path)/jdapistd.c \
    $(libjpeg_source_path)/jdarith.c \
    $(libjpeg_source_path)/jdatasrc.c \
    $(libjpeg_source_path)/jdcoefct.c \
    $(libjpeg_source_path)/jdcolor.c \
    $(libjpeg_source_path)/jddctmgr.c \
    $(libjpeg_source_path)/jdhuff.c \
    $(libjpeg_source_path)/jdinput.c \
    $(libjpeg_source_path)/jdmainct.c \
    $(libjpeg_source_path)/jdmarker.c \
    $(libjpeg_source_path)/jdmaster.c \
    $(libjpeg_source_path)/jdmerge.c \
    $(libjpeg_source_path)/jdpostct.c \
    $(libjpeg_source_path)/jdsample.c \
    $(libjpeg_source_path)/jdtrans.c \
    $(libjpeg_source_path)/jerror.c \
    $(libjpeg_source_path)/jidctflt.c \
    $(libjpeg_source_path)/jidctfst.c \
    $(libjpeg_source_path)/jidctint.c \
    $(libjpeg_source_path)/jmemmgr.c \
    $(libjpeg_source_path)/jmemnobs.c \
    $(libjpeg_source_path)/jquant1.c \
    $(libjpeg_source_path)/jquant2.c \
    $(libjpeg_source_path)/jutils.c \

    The LIBJPEG source files are then added to the existing object files the same way it is done with the rest of the middleware source files:

    # Start converting paths
    object_files := $(object_files:$(touchgfx_path)/%.cpp=$(object_output_path)/touchgfx/%.o)
    object_files := $(object_files:%.cpp=$(object_output_path)/%.o)
    object_files := $(object_files:$(touchgfx_middlewares_path)/%.c=$(object_output_path)/$(touchgfx_middlewares_path)/%.o)
    object_files := $(object_files:$(cubemx_middlewares_path)/%.c=$(object_output_path)/$(cubemx_middlewares_path)/%.o)
    object_files := $(object_files:$(libjpeg_source_path)/%.c=$(object_output_path)/$(libjpeg_source_path)/%.o)
    object_files := $(object_files:$(Drivers_path)/%.c=$(object_output_path)/Drivers/%.o)
    object_files := $(object_files:%.c=$(object_output_path)/%.o)

    Hardware changes

    Add all the JPEG paths to the include paths:

    include_paths := $(library_includes) \
    $(foreach comp, $(all_components), $(comp)/include) \
    $(foreach comp, $(cubemx_components), $(comp)/Inc) \
    $(foreach comp, $(touchgfx_generator_components), $(comp)/generated) \
    $(framework_includes) \
    $(cubemx_middlewares_path) \
    $(touchgfx_middlewares_path) \
    $(touchgfx_generator_components) \

    Next all the JPEG source files must be added to board_c_files:

    board_c_files := \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_bus.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_qspi.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_sdram.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_ts.c \
    $(Drivers_path)/BSP/Components/ft5336/ft5336.c \
    $(Drivers_path)/BSP/Components/ft5336/ft5336_reg.c \
    $(Drivers_path)/BSP/Components/mt25tl01g/mt25tl01g.c \
    $(Drivers_path)/BSP/Components/mt48lc4m32b2/mt48lc4m32b2.c \

    Placing Video Buffer in External Memory

    When developers create new projects with STM32CubeMX, the linker scripts associated with the generated projects do not contain any of the possible regions used by TouchGFX. Because of this, the specific buffer used to decode MJPEG videos is placed in internal flash by the linker until developers modify their linker scripts to place the buffer elsewhere. Without such modifications developers will experience large internal memory usage and perhaps inability to fit a large video buffer, like for full screen video decoding.

    The following TBSs are preconfigured and ready to decode videos from external memory:
    1. STM32F746-DISCO
    2. STM32F769-DISCO
    3. STM32H750-DK

    The definition for the RGB buffer dedicated to JPEG decoding is generated by TouchGFX Generator when video decoding is enabled. The definition is instrumented with a location pragma that tells the linker which section the buffer should be placed in. If the linker cannot find this memory region in the linker script the buffer will be placed in internal memory.

    uint32_t videoRGBBuffer[57600] LOCATION_ATTRIBUTE_NOLOAD("Video_RGB_Buffer");

    The following compiler-specific subsections describe the modifications that developers can make to achieve placing the buffer in SDRAM. Video_RGB_Buffer represents the buffer used for video decoding. The linker script examples reserve some space in SDRAM (starting at 0xC0000000) for TouchGFX framebuffer(s).

    1. EWARM
    2. STM32CubeIDE
    3. MDK-ARM
    Further reading
    Please read the section in the Generator User Guide on configuring a project for Video Decoding.

    The following examples all reserves some space at the start of SDRAM on an STM32F746G-DISCO board (0xC0000000->0xC00FF000), allowing the application to reference framebuffers by address (e.g. 0xC0000000) without the risk of the linker overwriting framebuffer data. Each example will allow the linker to place Video_RGB_Buffer into the defined SDRAM region.

    If you're allocating the Touchgfx framebuffers rather than using directly addressing then your linker script should also place TouchGFX_Framebuffer into SDRAM.


    define symbol __ICFEDIT_region_SDRAM_start__   = 0xC00FF000;
    define symbol __ICFEDIT_region_SDRAM_end__ = 0xC0700FFF;

    define region SDRAM_region = mem:[from __ICFEDIT_region_SDRAM_start__ to __ICFEDIT_region_SDRAM_end__];

    place in SDRAM_region { first section Video_RGB_Buffer };

    After linking, EWARM\STM32F746G_DISCO\List\ contains the following placement information for Video_RGB_Buffer:
    Video_RGB_Buffer        zero     0xc00f'f000   0x3'8400  TouchGFXGeneratedHAL.o [2]
    - 0xc013'7400 0x3'8400


    SDRAM (xrw) : ORIGIN = 0xC00FF000, LENGTH = 8M

    BufferSection (NOLOAD) :
    *(Video_RGB_Buffer Video_RGB_Buffer.*)
    . = ALIGN(0x4);
    } >SDRAM

    After compilation, STM32CubeIDE\Debug\ contains the following placement information for Video_RGB_Buffer:
    BufferSection   0x00000000c00ff000    0x1c200
    *(Video_RGB_Buffer Video_RGB_Buffer.*)
    0x00000000c00ff000 0x1c200 Application/User/TouchGFX/target/generated/TouchGFXGeneratedHAL.o
    0x00000000c00ff000 videoRGBBuffer

    MDK-ARM (Keil)

    LR_IROM1 0x08000000 0x00200000  {    ; load region size_region
    ER_IROM1 0x08000000 0x00200000 { ; load address = execution address
    *.o (RESET, +First)
    .ANY (+RO)
    RW_IRAM1 0x20000000 0x00050000 { ; RW data
    .ANY (+RW +ZI)
    RW_SDRAM 0xC00FF000 UNINIT 0xC0700FFF {
    *.o (Video_RGB_Buffer)

    Make sure to include the UNINIT attribute in the section that holds the Video Buffer. This ensures leaving the memory region as uninitialized data. After linking, MDK-ARM\STM32F746G_DISCO\ contains the following placement information for Video_RGB_Buffer:
    Video_RGB_Buffer                         0xc00ff000   Section    115200  touchgfxgeneratedhal.o(Video_RGB_Buffer)