视频解码

This section shows how to configure the TouchGFX Generator to generate a TouchGFX HAL that has video decoding capabilities.

在了解该场景之前，请阅读关于在TouchGFX中使用MJPEG视频的文档和关于视频控件本身的文档。

以下场景详细说明了如何赋能TouchGFX HAL，以通过软件（LibJPEG）或硬件（JPEG）支持视频解码。它涵盖STM32F7和STM32H7，因为在STM32CubeMX配置中，支持硬件（JPEG）解码时，STM32H7的配置与STM32F7略有不同。

Be sure to read all sections in this article before configuring the MCU for your specific project.

Generally, the LibJPEG and JPEG configurations can be found in STM32CubeMX in the Middleware and Multimedia catagories:

JPEG and LibJPEG setting in STM32CubeMX

Tip
最新版本的STM32H750-DK、STM32F769-DISCO和STM32F746G-DISCO TouchGFX硬件模板（TBS）通过各自的STM32CubeMX配置支持软件和JPEG硬件解码。
Use the STM32CubeMX configurations found in these example projects as inspiration to get your specific MCU to support MJPEG Video Decoding.

支持RTOS

The Generator User Guide mentions that Single- and Double buffer decoding strategies require a CMSIS compliant RTOS, such as FreeRTOS. TouchGFX Generator generates an entry point function videoTaskFunc(), that must be associated with a Video decoding task. STM32CubeMX can generate this configuration by defining the task and entry point function in the Tasks and queues tab of the FreeRTOS Middleware configuration.

The video task stack size (defined in words for CMSIS V2) and RTOS heap size are two important factors.

For software decoding the stack size must be carefully set because LibJPEG uses dynamic memory allocation. For hardware decoding the stack size can be substantially lower because there is no software stack which dynamically allocates memory.

FreeRTOS堆对于您的一般应用 + 0xA000应该足够大。

FreeRTOS任务配置

FreeRTOS堆大小配置

Based on the above configuration, STM32CubeMX generates the following code for the video task:

main.c

/* Definitions for VideoTask */
osThreadId_t VideoTaskHandle;
const osThreadAttr_t VideoTask_attributes = {
  .name = "VideoTask",
  .stack_size = 1000 * 4,
  .priority = (osPriority_t) osPriorityLow,
};
...
void main()
{
    ...
    /* creation of VideoTask */
    VideoTaskHandle = osThreadNew(videoTaskFunc, NULL, &VideoTask_attributes);
    ...

}

FreeRTOSConfig.h

#define configTOTAL_HEAP_SIZE                    ((size_t)75000)

软件解码

The software decoding solution requires that the LibJPEG middleware is enabled from STM32CubeMX, as specified in Generator User Guide. 软件解码设置对于所有具备LibJPEG功能的MCU（如STM32F4、STM32F7、STM32H7）都是相同的。

TouchGFX软件解码器期望LibJPEG解码的数据按照BGR像素排序。如果该设置保留为RGB，R和B颜色分量将在应用中互换。

Furthermore, the size of each pixel should be 3 bytes if using a 16-bit or 24-bit video RGB buffer or 4 if using a 32-bit video RGB buffer.

LibJPEG Configuration for a 16-bit or 24-bit video RGB buffer

Caution
The RGB_ORDERING setting must be configured as BGR, and the pixel size must be set according to which video RGB buffer format is used.
If your application uses Direct to Framebuffer strategy in a ARGB8888 (32-bit) application the pixel size must be set to 4

视频数据

Caution
当前解码解决方案（使用STM32CubeMX、TouchGFX Generator、以及来自TouchGFX Designer的视频控件）目前只适用于意法半导体开发板，因为在应用程序链接时，视频数据被链接到内存映射的flash中已知的特定地址。

下面来自EWARM项目的代码片段显示了由TouchGFX设计器创建的额外选项，以及它如何将数据放在ExtFlashSection中，后者是在所有TouchGFX板级包的链接器脚本中定义的。这不适用于非内存映射的flash或未定义此部分的项目。

ewarm_project.ewp

        <option>
          <name>IlinkExtraOptions</name>
          <state>--image_input $PROJ_DIR$\..\TouchGFX\generated\videos\bin\washerdryer.bin,video_washerdryer_bin_start,ExtFlashSection,4</state>
          <state>--keep video_washerdryer_bin_start</state>
        </option>

对于视频数据位于非内存映射的flash中的项目，请阅读本文中的FileReader部分。

硬件解码

The Generator User Guide mentions that the JPEG IP must be enabled in STM32CubeMX to enable Hardware decoding.

STM32F769-DISCO

支持JPEG功能的STM32F7系列（例如STM32F769）的JPEG配置，与STM32H7系列略有不同。 The RGB_FORMAT setting found in the JPEG configurations must respect the pixel format of the TouchGFX Framebuffer, JPEG_RGB565 in the example below.

JPEG参数设置

Use DMA to transfer data to (memory-to-peripheral) and from (peripheral-to-memory) the JPEG peripheral through the DMA Settings tab found in JPEG configurations. 为IN和OUT各添加一个DMA请求将自动设置方向参数。

JPEG DMA设置

本节总结了支持JPEG功能的STM32F7（例如STM32F769）上使用硬件解码时TouchGFX HAL的配置。从STM32CubeMX生成代码后，在从Designer使用视频控件时，应用程序将能够使用JPEG外设解码视频。

Caution
JPEG外设配置的RGB_FORMAT必须遵守TouchGFX Framebuffer的格式

STM32H750-DK

唯一能将（例如）STM32H750和STM32F769上的硬件解码（支持JPEG）加以区分的因素是在STM32CubeMX中配置DMA传输的方式。不仅UI不同，DMA概念也不同。

对于STM32H750，JPEG外设只能配置为使用MDMA外设（而不是常规DMA外设）。为输入和输出FIFO阈值信号添加MDMA配置，如下图所示。

Note
与DMA1/DMA2相比，MDMA是性能高很多的DMA引擎

JPEG DMA设置

`FileReader`接口

当在非内存映射的存储器中存储MJPEG视频时，开发人员可以指定touchgfx::VideoDataReader的实现，TouchGFX视频控制器可以使用它将需要解码的数据传输到配置的解码器（软件/硬件）。下面是此接口的简单示例，它将视频数据从一个缓冲区复制到另一个缓冲区。

VideoView.cpp

class MyReader : public touchgfx::VideoDataReader
{
public:
    MyReader() : position(0) { }
    virtual uint32_t getDataLength() { return video_len; }
    virtual void seek(uint32_t pos) { position = pos; }
    virtual bool readData(void* dst, uint32_t bytes)
    {
        memcpy(dst, &video_data[position], bytes);
        position += bytes;
        return true;
    }
    private:
        uint32_t position;
} myReader;

开发人员可以配置控件以使用数据读取器，而不是将一个Video控件指向映射内存中的视频开始部分：

VideoView.cpp

video.setVideoData(myReader);

迁移TBS，以支持视频解码

If you want to migrate a project created from a TouchGFX Board Setups (TBS) - with a MCU that supports video decoding - from a version of TouchGFX Designer before video decoding was supported and you want to be able to use 'Run Target' in TouchGFX Designer with that project, some manual changes to the GCC Makefile are required.

以下部分将列出所需的变更、以及进行变更的原因。 There are some general changes that must always be applied and some LibJPEG (software decoding) and JPEG (hardware decoding) specific changes depending on what the developers application uses. 这些更改是对旧TBS中已经存在的GCC Makefile的扩展。

如上述场景所述，除了更新Makefile，您还必须在STM32CubeMX中设置视频解码。

General changes

定义您项目中的LIBJPEG路径：

# LibJPEG path
libjpeg_path := $(cubemx_middlewares_path)/Third_Party/LibJPEG

然后必须定义视频资源输入路径：

asset_texts_input  := TouchGFX/assets/texts

asset_videos_input := TouchGFX/assets/videos

The video assets output path must also be defined bellow the already existing assets output paths:

asset_images_output := $(asset_root_path)/images
asset_fonts_output := $(asset_root_path)/fonts
asset_texts_output := $(asset_root_path)/texts
asset_videos_output := $(asset_root_path)/videos

将视频输出资源添加到组件列表中：

all_components := $(components) \
    $(asset_fonts_output) \
    $(asset_images_output) \
    $(asset_texts_output)
    $(asset_texts_output) \
    $(asset_videos_output)

必须定义视频对象文件。 The video object files are separated from the already existing object:

c_source_files := $(call find, $(source_paths),*.c) $(os_source_files) $(board_c_files)
source_files += $(board_cpp_files)

video_object_files := $(call find, $(asset_videos_output),*.o)

必须定义视频转换器脚本的路径：

textconvert_script_path := $(touchgfx_path)/framework/tools/textconvert
textconvert_executable := $(call find, $(textconvert_script_path), *.rb)
videoconvert_script_path := $(touchgfx_path)/framework/tools/videoconvert

An optional echo can be added to see all video objects files. 视频对象文件必须添加到链接阶段。该行将$(video_object_files)和其他对象文件加在一起：

$(binary_output_path)/$(target_executable): $(object_files) $(object_asm_files)
    @echo Video Objects: $(video_object_files)
    @echo Linking $(@)
    @mkdir -p $(@D)
    @mkdir -p $(object_output_path)
    @$(file >$(build_root_path)/objects.tmp) $(foreach F,$(object_files) $(video_object_files),$(file >>$(build_root_path)/objects.tmp,$F))

将视频规则添加到现有资源和.PHONY：

_assets_: BitmapDatabase TextKeysAndLanguages Videos

.PHONY: BitmapDatabase TextKeysAndLanguages Videos

添加视频转换后，接着添加视频规则：

Videos:
    @ruby $(videoconvert_script_path)/videoconvert.rb $(asset_videos_input) $(asset_videos_output)

最后更新清除规则，也删除与视频相关的输出。

_clean_:
    @echo Cleaning: $(board_name)
    @rm -rf $(build_root_path)
    # Do not remove gui_generated
    @rm -rf $(asset_images_output)
    @rm -rf $(asset_fonts_output)
    @rm -rf $(asset_texts_output)
    @rm -rf $(asset_videos_output)
    # Create directory to avoid error if it does not exist
    @mkdir -p $(asset_root_path)
    # Remove assets folder if it is empty (i.e. no gui_generated folder)
    @rmdir --ignore-fail-on-non-empty $(asset_root_path)
    # Clean bootloader project
    @$(MAKE) -r -f ExtMem_Boot/gcc/Makefile -s $(MFLAGS) clean

软件变化

添加所有LIBJPEG路径到include路径：

include_paths := $(library_includes) \
    $(foreach comp, $(all_components), $(comp)/include) \
    $(foreach comp, $(cubemx_components), $(comp)/Inc) \
    $(foreach comp, $(touchgfx_generator_components), $(comp)/generated) \
    $(framework_includes) \
    $(cubemx_middlewares_path) \
    $(touchgfx_middlewares_path) \
    $(touchgfx_generator_components) \
    LIBJPEG/Target \
    $(libjpeg_path)/include \
    LIBJPEG/App

必须定义LIBJPEG源路径：

c_source_files := $(call find, $(source_paths),*.c) $(os_source_files) $(board_c_files)
source_files += $(board_cpp_files)

libjpeg_source_path = Middlewares/Third_Party/LibJPEG/source

接下来，所有LIBJPEG源文件必须添加到board_c_files：

board_c_files := \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_bus.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_qspi.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_sdram.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_ts.c \
    $(Drivers_path)/BSP/Components/ft5336/ft5336.c \
    $(Drivers_path)/BSP/Components/ft5336/ft5336_reg.c \
    $(Drivers_path)/BSP/Components/mt25tl01g/mt25tl01g.c \
    $(Drivers_path)/BSP/Components/mt48lc4m32b2/mt48lc4m32b2.c \
    $(libjpeg_source_path)/jaricom.c \
    $(libjpeg_source_path)/jcomapi.c \
    $(libjpeg_source_path)/jdapimin.c \
    $(libjpeg_source_path)/jdapistd.c \
    $(libjpeg_source_path)/jdarith.c \
    $(libjpeg_source_path)/jdatasrc.c \
    $(libjpeg_source_path)/jdcoefct.c \
    $(libjpeg_source_path)/jdcolor.c \
    $(libjpeg_source_path)/jddctmgr.c \
    $(libjpeg_source_path)/jdhuff.c \
    $(libjpeg_source_path)/jdinput.c \
    $(libjpeg_source_path)/jdmainct.c \
    $(libjpeg_source_path)/jdmarker.c \
    $(libjpeg_source_path)/jdmaster.c \
    $(libjpeg_source_path)/jdmerge.c \
    $(libjpeg_source_path)/jdpostct.c \
    $(libjpeg_source_path)/jdsample.c \
    $(libjpeg_source_path)/jdtrans.c \
    $(libjpeg_source_path)/jerror.c \
    $(libjpeg_source_path)/jidctflt.c \
    $(libjpeg_source_path)/jidctfst.c \
    $(libjpeg_source_path)/jidctint.c \
    $(libjpeg_source_path)/jmemmgr.c \
    $(libjpeg_source_path)/jmemnobs.c \
    $(libjpeg_source_path)/jquant1.c \
    $(libjpeg_source_path)/jquant2.c \
    $(libjpeg_source_path)/jutils.c \
    LIBJPEG/App/libjpeg.c

然后将LIBJPEG源文件添加到现有的目标文件中，方法与其他中间件源文件相同：

# Start converting paths
object_files := $(object_files:$(touchgfx_path)/%.cpp=$(object_output_path)/touchgfx/%.o)
object_files := $(object_files:%.cpp=$(object_output_path)/%.o)
object_files := $(object_files:$(touchgfx_middlewares_path)/%.c=$(object_output_path)/$(touchgfx_middlewares_path)/%.o)
object_files := $(object_files:$(cubemx_middlewares_path)/%.c=$(object_output_path)/$(cubemx_middlewares_path)/%.o)
object_files := $(object_files:$(libjpeg_source_path)/%.c=$(object_output_path)/$(libjpeg_source_path)/%.o)
object_files := $(object_files:$(Drivers_path)/%.c=$(object_output_path)/Drivers/%.o)
object_files := $(object_files:%.c=$(object_output_path)/%.o)

硬件变化

添加所有JPEG路径到include 路径：

include_paths := $(library_includes) \
    $(foreach comp, $(all_components), $(comp)/include) \
    $(foreach comp, $(cubemx_components), $(comp)/Inc) \
    $(foreach comp, $(touchgfx_generator_components), $(comp)/generated) \
    $(framework_includes) \
    $(cubemx_middlewares_path) \
    $(touchgfx_middlewares_path) \
    $(touchgfx_generator_components) \
    Utilities/JPEG

接下来，所有JPEG源文件必须添加到board_c_files：

board_c_files := \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_bus.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_qspi.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_sdram.c \
    $(Drivers_path)/BSP/STM32H750B-DK/stm32h750b_discovery_ts.c \
    $(Drivers_path)/BSP/Components/ft5336/ft5336.c \
    $(Drivers_path)/BSP/Components/ft5336/ft5336_reg.c \
    $(Drivers_path)/BSP/Components/mt25tl01g/mt25tl01g.c \
    $(Drivers_path)/BSP/Components/mt48lc4m32b2/mt48lc4m32b2.c \
    Utilities/JPEG/jpeg_utils.c

将视频缓冲区置于外部存储器中

When developers create new projects with STM32CubeMX, the linker scripts associated with the generated projects do not contain any of the possible regions used by TouchGFX. 因此，用于解码MJPEG视频的特定缓冲区被链接器放置在内部flash中，直至开发人员修改他们的链接器脚本，将缓冲区放置到其他地方。如果没有这样的修改，开发人员将会遇到大量内部存储被使用的情况，可能无法容纳大一点的视频缓冲区，比如全屏视频解码所需的缓冲区。

Tip
以下TBS是预先配置好的，可以从外部存储解码视频：STM32F746-DISCO
STM32F769-DISCO
STM32H750-DK

启用视频解码以后，TouchGFX Generator生成用于JPEG解码的RGB缓冲区的定义。该定义是用一个位置指示来实现的，它告诉链接器应该将缓冲区放在哪个部分。如果链接器在链接器脚本中找不到该存储区域，缓冲区将被放在内部存储中。

LOCATION_PRAGMA_NOLOAD("Video_RGB_Buffer")
uint32_t videoRGBBuffer[57600] LOCATION_ATTRIBUTE_NOLOAD("Video_RGB_Buffer");

下面特定于编译器的子章节描述了开发人员可以进行的修改，以实现在SDRAM中放置缓冲区。 Video_RGB_Buffer表示用于视频解码的缓冲区。链接器脚本示例在SDRAM中为TouchGFX帧缓存区保留了一些空间（从0xC0000000开始）。

EWARM
STM32CubeIDE
MDK-ARM

EWARM (IAR)

stm32f746xx_flash.icf

define symbol __ICFEDIT_region_SDRAM_start__   = 0xC00FF000;
define symbol __ICFEDIT_region_SDRAM_end__     = 0xC0700FFF;

define region SDRAM_region    = mem:[from __ICFEDIT_region_SDRAM_start__ to __ICFEDIT_region_SDRAM_end__];

place in SDRAM_region    { first section Video_RGB_Buffer };

链接之后，EWARM\STM32F746G_DISCO\List\STM32F746G_DISCO.map包含Video_RGB_Buffer的以下放置信息：

STM32F746G_DISCO.map

Video_RGB_Buffer        zero     0xc00f'f000   0x3'8400  TouchGFXGeneratedHAL.o [2]
                               - 0xc013'7400   0x3'8400

STM32CubeIDE

STM32F746NGHX_FLASH.ld

MEMORY
{
  ...
  SDRAM      (xrw)    : ORIGIN = 0xC00FF000,   LENGTH = 8M
}

BufferSection (NOLOAD) :
{
  *(Video_RGB_Buffer Video_RGB_Buffer.*)
  *(.gnu.linkonce.r.*)
  . = ALIGN(0x4);
} >SDRAM

编译之后，STM32CubeIDE\Debug\STM32F746G_DISCO.map 包含Video_RGB_Buffer的以下放置信息：

STM32F746G_DISCO.map

BufferSection   0x00000000c00ff000    0x1c200
 *(Video_RGB_Buffer Video_RGB_Buffer.*)
 Video_RGB_Buffer
                0x00000000c00ff000    0x1c200 Application/User/TouchGFX/target/generated/TouchGFXGeneratedHAL.o
                0x00000000c00ff000                videoRGBBuffer

MDK-ARM (Keil)

STM32F746G_DISCO.sct

LR_IROM1 0x08000000 0x00200000  {    ; load region size_region
  ER_IROM1 0x08000000 0x00200000  {  ; load address = execution address
    *.o (RESET, +First)
    *(InRoot$$Sections)
    .ANY (+RO)
  }
  RW_IRAM1 0x20000000 0x00050000  {  ; RW data
    .ANY (+RW +ZI)
  }
  RW_SDRAM 0xC00FF000 UNINIT 0xC0700FFF  {
    *.o (Video_RGB_Buffer)
  }
}

Make sure to include the UNINIT attribute in the section that holds the Video Buffer. This ensures leaving the memory region as uninitialized data. 链接之后，MDK-ARM\STM32F746G_DISCO\STM32F746G_DISCO.map包含Video_RGB_Buffer的以下放置信息：

STM32F746G_DISCO.map

Video_RGB_Buffer                         0xc00ff000   Section    115200  touchgfxgeneratedhal.o(Video_RGB_Buffer)

Tip

支持RTOS​

main.c

FreeRTOSConfig.h

软件解码​

Caution

视频数据​

Caution

ewarm_project.ewp

硬件解码​

STM32F769-DISCO​

Caution

STM32H750-DK​

Note

FileReader接口​

VideoView.cpp

VideoView.cpp

迁移TBS，以支持视频解码​

General changes​

软件变化​

硬件变化​

将视频缓冲区置于外部存储器中​

Tip

Further reading

Tip

EWARM (IAR)​

stm32f746xx_flash.icf

STM32F746G_DISCO.map

STM32CubeIDE​

STM32F746NGHX_FLASH.ld

STM32F746G_DISCO.map

MDK-ARM (Keil)​

STM32F746G_DISCO.sct

STM32F746G_DISCO.map

支持RTOS

软件解码

视频数据

硬件解码

STM32F769-DISCO

STM32H750-DK

`FileReader`接口

迁移TBS，以支持视频解码

General changes

软件变化

硬件变化

将视频缓冲区置于外部存储器中

EWARM (IAR)

STM32CubeIDE

MDK-ARM (Keil)