{"componentChunkName":"component---src-templates-acg-portal-new-template-tsx","path":"/tmpbaqyho","result":{"data":{"markdownRemark":{"html":"

简介

\n

ImageViTEmbedding基于 Daft + Ray 分布式框架的图像嵌入向量提取组件，利用 ViT 类模型（如 DINOv2）将图像编码为固定维度的向量表示，适用于图像检索、分类、聚类等下游任务。

\n

功能

\n\n
\n

该算子运行需要依赖对应的模型权重文件，平台已将所需权重文件预置在公共数据集aihc_daft_public_model中，需要在创建开发负载时挂载该数据集即可正常使用。

\n
\n

参数

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
参数名称类型默认值描述
image_src_typestrimage_url输入图像的格式类型，支持：bos/http 地址(image_url)base64 编码(image_base64)二进制流(image_binary)可选值：[\"image_url\", \"image_base64\", \"image_binary\"]默认值：\"image_url\"
dtypestrfloat32模型推理精度选择：bfloat16: 平衡精度与速度（TPU上更快）float16: 更快的推理速度float32: 最高精度但显存消耗最大可选值：[\"bfloat16\", \"float16\", \"float32\"]默认值：\"float16\"
batch_sizeint32批处理大小默认值: 32
model_pathstr/opt/aihc/models模型文件存储路径默认值: \"/opt/aihc/models\"
model_namestrfacebook/dinov2-large使用的图像向量模型名称可选值: [\"google/vit-base-patch16-224-in21k\",\"google/vit-large-patch16-224-in21k\",\"facebook/dinov2-base\",\"facebook/dinov2-large\"]默认值: \"facebook/dinov2-large\"
use_cls_token_embeddingbooltrue是否使用CLS Token特征默认值: True
rankint0指定使用的GPU设备编号（多卡环境有效）。例如：0表示第一张GPU，1表示第二张GPU默认值：0
\n

输入

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
输入列名说明说明
images包含图像数据的数组，元素类型可为图像URL、Base64编码或二进制数据图像 URL 地址
\n

以 Daft DataFrame 形式传入，列名为 image。

\n

输出

\n

包含特征向量的数组，每个元素为float类型的嵌套数组，\n数组维度由模型输出决定

\n

使用示例

\n\n
\n
\n
\n Plain Text\n \n
\n
\n 
1from __future__ import annotations\n2\n3import os\n4import daft\n5from daft import col\n6from daft.aihc.common.udf import aihc_udf\n7from daft.aihc.functions.image.embedding.image_vit_embedding import ImageViTEmbedding\n8\n9if __name__ == "__main__":\n10    if os.getenv("DAFT_RUNNER", "native") == "ray":\n11        import ray\n12        ray.init(dashboard_host="0.0.0.0", ignore_reinit_error=True)\n13        daft.set_runner_ray()\n14    daft.set_execution_config(actor_udf_ready_timeout=6000, min_cpu_per_task=0)\n15\n16    image_src_type = "image_url"\n17    batch_size = 64\n18    model_path = os.getenv("MODEL_PATH", "/opt/aihc/models")\n19    model_name = "facebook/dinov2-large"\n20    dtype = "float16"\n21    use_cls_token_embedding = True\n22    rank = 0\n23    num_gpus = 1\n24\n25    samples = {\n26        "image": [\n27            "https://{bucket}.bj.bcebos.com/image.png",\n28        ]\n29    }\n30\n31    ds = daft.from_pydict(samples)\n32    ds = ds.with_column(\n33        "embedding",\n34        aihc_udf(\n35            ImageViTEmbedding,\n36            construct_args={\n37                "image_src_type": image_src_type,\n38                "batch_size": batch_size,\n39                "model_path": model_path,\n40                "model_name": model_name,\n41                "dtype": dtype,\n42                "use_cls_token_embedding": use_cls_token_embedding,\n43                "rank": rank,\n44            },\n45            num_gpus=num_gpus,\n46            batch_size=1,\n47        )(col("image")),\n48    )\n49\n50    ds.show()\n51#╭────────────────────────────────┬────────────────────────────────╮                                                                                                                                            \n52#│ image                          ┆ embedding                      │\n53#│ ---                            ┆ ---                            │\n54#│ String                         ┆ List[Float32]                  │\n55#╞════════════════════════════════╪════════════════════════════════╡\n56#│ https://{bucket}.bj.bcebos.com/┆ [0.00059747696, -0.02935791, … │\n57#╰────────────────────────────────┴────────────────────────────────╯
\n
\n
\n
\n ","fields":{"slug":"tmpbaqyho","title":"图像 Embedding（ViT 系列模型）","date":"2026-06-04","extractedHeadings":[]},"headings":[{"value":"简介","depth":2},{"value":"功能","depth":2},{"value":"参数","depth":2},{"value":"输入","depth":2},{"value":"输出","depth":2},{"value":"使用示例","depth":2}]}},"pageContext":{"isCreatedByStatefulCreatePages":false,"slug":"tmpbaqyho","prev":{"id":"Qmo9zz82x","name":"图像重采样处理器","path":"Qmo9zz82x","filePath":"操作指南/AI数据处理/算子列表/图片/图像重采样处理器.md","seo":null,"parentIds":["ilib2qygp","Ymo88m8hi","Imob3m6so","6mob45l3j"],"parents":[{"id":"ilib2qygp","documentId":"bfa43a8b-968a-41a1-8c9d-906507eeaed9","name":"操作指南","repoName":"AIHC","filePath":"操作指南","disabled":false,"path":"ilib2qygp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":null},{"id":"Ymo88m8hi","documentId":"c8cb5e38-f8c5-40f4-a424-b0c7895f0c0a","name":"AI数据处理","repoName":"AIHC","filePath":"操作指南/AI数据处理","disabled":false,"path":"Ymo88m8hi","lastMergeTime":"2026-04-21 14:23:10","isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"Imob3m6so","documentId":"fe548e34-6659-4ff5-86f6-eee2c43aec90","name":"算子列表","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表","disabled":false,"path":"Imob3m6so","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"6mob45l3j","documentId":"f121147a-d538-4da6-b212-1afecb2ecd42","name":"图片","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表/图片","disabled":false,"path":"6mob45l3j","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""}]},"next":{"id":"1moa00wo1","name":"音频","path":"1moa00wo1","filePath":"操作指南/AI数据处理/算子列表/音频/音频格式标准化处理器.md","seo":null,"parentIds":["ilib2qygp","Ymo88m8hi","Imob3m6so","zmob46e1c"],"parents":[{"id":"ilib2qygp","documentId":"bfa43a8b-968a-41a1-8c9d-906507eeaed9","name":"操作指南","repoName":"AIHC","filePath":"操作指南","disabled":false,"path":"ilib2qygp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":null},{"id":"Ymo88m8hi","documentId":"c8cb5e38-f8c5-40f4-a424-b0c7895f0c0a","name":"AI数据处理","repoName":"AIHC","filePath":"操作指南/AI数据处理","disabled":false,"path":"Ymo88m8hi","lastMergeTime":"2026-04-21 14:23:10","isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"Imob3m6so","documentId":"fe548e34-6659-4ff5-86f6-eee2c43aec90","name":"算子列表","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表","disabled":false,"path":"Imob3m6so","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"zmob46e1c","documentId":"886f1d97-777e-4f0c-81ae-e69bf16653f0","name":"音频","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表/音频","disabled":false,"path":"zmob46e1c","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""}]},"parents":[{"id":"ilib2qygp","documentId":"bfa43a8b-968a-41a1-8c9d-906507eeaed9","name":"操作指南","repoName":"AIHC","filePath":"操作指南","disabled":false,"path":"ilib2qygp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":null},{"id":"Ymo88m8hi","documentId":"c8cb5e38-f8c5-40f4-a424-b0c7895f0c0a","name":"AI数据处理","repoName":"AIHC","filePath":"操作指南/AI数据处理","disabled":false,"path":"Ymo88m8hi","lastMergeTime":"2026-04-21 14:23:10","isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"Imob3m6so","documentId":"fe548e34-6659-4ff5-86f6-eee2c43aec90","name":"算子列表","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表","disabled":false,"path":"Imob3m6so","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""},{"id":"6mob45l3j","documentId":"f121147a-d538-4da6-b212-1afecb2ecd42","name":"图片","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表/图片","disabled":false,"path":"6mob45l3j","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":""}],"specificSeo":null}}}