{"componentChunkName":"component---src-templates-acg-portal-new-template-tsx","path":"/Ympauyur4","result":{"data":{"markdownRemark":{"html":"

视频内容理解（Qwen VL 系列模型）

\n

简介

\n

QwenVLVideoUnderstandingVLLM 是基于 Daft + Ray 分布式框架的视频理解组件，利用 Qwen2.5-VL 多模态大模型通过 vLLM 推理引擎对视频内容进行理解和描述生成，适用于视频字幕生成、内容审核、视频标注等场景。

\n

功能

\n\n

输入

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
输入列名说明
videos包含视频数据的数组，元素类型为 字符串 或者 二进制。
user_prompts视频理解对应的prompt，默认为None。如果不传入，则默认使用参数中的prompt。
\n

参数

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
参数名称类型默认值描述
video_src_typestrvideo_url输入视频的格式类型。支持：\"video_url\": bos/http地址\"video_base64\": base64编码\"video_binary\": 二进制流可选值：[\"video_url\", \"video_base64\", \"video_binary\"]默认值：\"video_url\"
model_pathstr/opt/aihc/models本地模型文件存储的绝对路径。默认为容器内预置路径，当使用自定义模型时需修改此路径。默认值：\"/opt/aihc/models\"
model_namestrQwen/Qwen2.5-VL-7B-Instruct支持的视觉语言模型版本。当前仅支持Qwen2.5-VL系列模型。可选值：[\"Qwen/Qwen2.5-VL-7B-Instruct-AWQ\", \"Qwen/Qwen2.5-VL-7B-Instruct\", \"Qwen/Qwen2.5-VL-32B-Instruct-AWQ\", \"Qwen/Qwen2.5-VL-32B-Instruct\"]默认值：\"Qwen/Qwen2.5-VL-7B-Instruct\"
promptstr请给出这段视频的详细描述。用户理解视频内容的提示词，模型会根据提示词来生成视频的描述。设置为空时，建议针对每条数据传入特定的prompt。默认值：\"请给出这段视频的详细描述。\"
batch_sizeint4单次推理处理的样本数量。较大的batch_size可提升吞吐但增加显存消耗，建议根据GPU显存调整。默认值：4
dtypestrbfloat16模型推理精度选择。\"bfloat16\": 平衡精度与速度\"float16\": 更快的推理速度\"float32\": 最高精度但显存消耗最大可选值：[\"bfloat16\", \"float16\", \"float32\"]默认值：\"bfloat16\"
max_model_lenint128000支持的最大模型输入长度（token数），影响可处理视频描述的长度，不能超过128000。默认值：128000
max_num_seqsint128单批次最大序列数，影响并发推理能力。默认值：128
enable_prefix_cachingboolTrue是否启用前缀缓存以加速多轮推理。默认值：True
gpu_memory_utilizationfloat0.9单卡GPU显存利用率上限，范围0~1。默认值：0.9
enforce_eagerboolFalse是否强制使用eager模式推理，调试或特殊场景可用。默认值：False
min_pixelsint or NoneNone视频最小像素。不设置时，默认使用视频的原像素。视频像素越大，GPU显存占用越高。默认值：None
max_pixelsint or NoneNone视频最大像素。不设置时，默认使用视频的原像素。视频像素越大，GPU显存占用越高。默认值：None
fpsfloat or NoneNone视频帧率。不设置时，默认使用视频的原帧率。视频帧率越高，GPU显存占用越高。建议设置特定的帧率值。默认值：None
temperaturefloat1.0采样温度，控制生成内容的多样性。值越高生成越随机。默认值：1.0
top_pfloat0.2nucleus采样的概率阈值，控制生成内容的多样性。值越小生成越保守。默认值：0.2
repetition_penaltyfloat1.05重复惩罚系数，防止生成重复内容。值越大重复内容越少。默认值：1.05
max_tokensint8192单次生成的最大token数，影响描述长度。默认值：8192
stop_token_idslist[]生成时遇到这些token id则停止。用于自定义生成终止条件。默认值：[]
seedint42随机种子，保证推理结果可复现。默认值：42
\n

输出

\n

处理后的数组，元素为每个视频的视觉理解结果。

\n

使用示例

\n\n
\n
\n
\n Plain Text\n \n
\n
\n 
1from __future__ import annotations\n2\n3import os\n4import daft\n5from daft import col\n6from daft.aihc.common.udf import aihc_udf\n7from daft.aihc.functions.multimodal.qwen_vl_video_understanding_vllm import QwenVLVideoUnderstandingVLLM\n8\n9if __name__ == "__main__":\n10    if os.getenv("DAFT_RUNNER", "native") == "ray":\n11        import ray\n12        ray.init(dashboard_host="0.0.0.0", ignore_reinit_error=True)\n13        daft.set_runner_ray()\n14    daft.set_execution_config(actor_udf_ready_timeout=6000, min_cpu_per_task=0)\n15\n16    samples = {\n17        "video_path": [\n18            f"https://{bucket}.bj.bcebos.com/sample.mp4"\n19        ],\n20        "prompt": ["请给出视频的详细描述。"]\n21    }\n22\n23    video_src_type = "video_url"\n24    model_path = os.getenv("MODEL_PATH", "/opt/aihc/models")\n25    model_name = os.getenv("MODEL_NAME", "Qwen/Qwen2.5-VL-7B-Instruct")\n26    dtype = "bfloat16"\n27    default_prompt = None\n28    max_caption_length = 256\n29    min_pixels = 320 * 160\n30    max_pixels = 320 * 160\n31    fps = 1\n32    batch_size = 2\n33    seed = 42\n34    max_model_len = 12800\n35    max_num_seqs = 128\n36    enable_prefix_caching = True\n37    gpu_memory_utilization = 0.95\n38    enforce_eager = True\n39\n40    ds = daft.from_pydict(samples)\n41    ds = ds.with_column(\n42        "caption",\n43        aihc_udf(\n44            QwenVLVideoUnderstandingVLLM,\n45            construct_args={\n46                "video_src_type": video_src_type,\n47                "model_path": model_path,\n48                "model_name": model_name,\n49                "dtype": dtype,\n50                "prompt": default_prompt,\n51                "max_caption_length": max_caption_length,\n52                "min_pixels": min_pixels,\n53                "max_pixels": max_pixels,\n54                "fps": fps,\n55                "batch_size": batch_size,\n56                "seed": seed,\n57                "max_model_len": max_model_len,\n58                "max_num_seqs": max_num_seqs,\n59                "enable_prefix_caching": enable_prefix_caching,\n60                "gpu_memory_utilization": gpu_memory_utilization,\n61                "enforce_eager": enforce_eager,\n62            },\n63            num_gpus=1,\n64            batch_size=batch_size,\n65            concurrency=1,\n66        )(col("video_path"), col("prompt")),\n67    )\n68\n69    ds.show()\n70#╭────────────────────────────────┬────────────────────────┬─────────────────────────────────────────────────────────────╮                                                                                      \n71#│ video_path                     ┆ prompt                 ┆ caption                                                     │\n72#│ ---                            ┆ ---                    ┆ ---                                                         │\n73#│ String                         ┆ String                 ┆ String                                                      │\n74#╞════════════════════════════════╪════════════════════════╪═════════════════════════════════════════════════════════════╡\n75#│ https://{bucket}.bj.bcebos.com/┆ 请给出视频的详细描述。   ┆ 这段视频展示了一对男女在室内对话的场景。背景中可以看到一些…      │\n76#╰────────────────────────────────┴────────────────────────┴─────────────────────────────────────────────────────────────╯
\n
\n
\n
\n ","fields":{"slug":"Ympauyur4","title":"视频内容理解-QwenVL系列模型","date":"2026-05-18","extractedHeadings":[]},"headings":[{"value":"简介","depth":2},{"value":"功能","depth":2},{"value":"输入","depth":2},{"value":"参数","depth":2},{"value":"输出","depth":2},{"value":"使用示例","depth":2}]}},"pageContext":{"isCreatedByStatefulCreatePages":false,"slug":"Ympauyur4","prev":{"id":"ampauyuom","name":"图文embedding-CLIP模型","path":"ampauyuom","filePath":"操作指南/AI数据处理/算子列表/多模态/图文embedding-CLIP模型.md","seo":null,"parentIds":["ilib2qygp","Ymo88m8hi","Imob3m6so","Kmp6mooi2"],"parents":[{"id":"ilib2qygp","documentId":"bfa43a8b-968a-41a1-8c9d-906507eeaed9","name":"操作指南","repoName":"AIHC","filePath":"操作指南","disabled":false,"path":"ilib2qygp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":null},{"id":"Ymo88m8hi","documentId":"c8cb5e38-f8c5-40f4-a424-b0c7895f0c0a","name":"AI数据处理","repoName":"AIHC","filePath":"操作指南/AI数据处理","disabled":false,"path":"Ymo88m8hi","lastMergeTime":"2026-04-21 14:23:10","isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"Imob3m6so","documentId":"fe548e34-6659-4ff5-86f6-eee2c43aec90","name":"算子列表","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表","disabled":false,"path":"Imob3m6so","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"Kmp6mooi2","documentId":"28a28c63-b404-4d8f-a4f6-483f8c2c0e40","name":"多模态","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表/多模态","disabled":false,"path":"Kmp6mooi2","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""}]},"next":{"id":"3mpb58rxh","name":"视频","path":"3mpb58rxh","filePath":"操作指南/AI数据处理/算子列表/视频/视频元数据提取.md","seo":null,"parentIds":["ilib2qygp","Ymo88m8hi","Imob3m6so","6mpawyry0"],"parents":[{"id":"ilib2qygp","documentId":"bfa43a8b-968a-41a1-8c9d-906507eeaed9","name":"操作指南","repoName":"AIHC","filePath":"操作指南","disabled":false,"path":"ilib2qygp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":null},{"id":"Ymo88m8hi","documentId":"c8cb5e38-f8c5-40f4-a424-b0c7895f0c0a","name":"AI数据处理","repoName":"AIHC","filePath":"操作指南/AI数据处理","disabled":false,"path":"Ymo88m8hi","lastMergeTime":"2026-04-21 14:23:10","isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"Imob3m6so","documentId":"fe548e34-6659-4ff5-86f6-eee2c43aec90","name":"算子列表","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表","disabled":false,"path":"Imob3m6so","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"6mpawyry0","documentId":"3428d02c-e287-45b8-9d16-5a7b20881dd4","name":"视频","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表/视频","disabled":false,"path":"6mpawyry0","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""}]},"parents":[{"id":"ilib2qygp","documentId":"bfa43a8b-968a-41a1-8c9d-906507eeaed9","name":"操作指南","repoName":"AIHC","filePath":"操作指南","disabled":false,"path":"ilib2qygp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":null},{"id":"Ymo88m8hi","documentId":"c8cb5e38-f8c5-40f4-a424-b0c7895f0c0a","name":"AI数据处理","repoName":"AIHC","filePath":"操作指南/AI数据处理","disabled":false,"path":"Ymo88m8hi","lastMergeTime":"2026-04-21 14:23:10","isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"Imob3m6so","documentId":"fe548e34-6659-4ff5-86f6-eee2c43aec90","name":"算子列表","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表","disabled":false,"path":"Imob3m6so","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"Kmp6mooi2","documentId":"28a28c63-b404-4d8f-a4f6-483f8c2c0e40","name":"多模态","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表/多模态","disabled":false,"path":"Kmp6mooi2","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""}],"specificSeo":null}}}