{"componentChunkName":"component---src-templates-acg-portal-new-template-tsx","path":"/Tmu0pyy1k","result":{"data":{"markdownRemark":{"html":"

简介

\n

视频相机位姿估计算子。抽帧后先用 MoGe-2 逐帧估计单目深度与内参，再把 (image, depth, intrinsics) 喂给 MegaSaM 的 DROID-SLAM 做跟踪与全局 bundle adjustment，输出相机内参与每帧 camera-to-world 位姿。仅支持 GPU，三个 CUDA 扩展与全部权重必须离线预置，算子内零网络零编译。

\n

功能描述

\n\n

算子参数

\n

输入

\n\n\n\n\n\n\n\n\n\n\n\n\n\n
输入含义
videos视频列。按 video_src_type 取值分别是视频路径/URL（含 bos://）、base64 字符串或二进制内容
\n

输出

\n

输出为一个 struct 列，字段如下（tensor 均指 struct<shape: list<int32>, data: list<float32>>，data 是按行优先展平的 float32；缺失时为 {\"shape\": [], \"data\": []}）：

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
输出含义
frame_countint32，输出的位姿帧数，等于 min(output_max_frames, 关键帧数)；失败时 0
image_sizelist，SLAM 分辨率下的 [H, W]，实测 [384, 512]
intrinsictensor，BA 后的像素单位内参矩阵，形状 (3,3)，fx=data[0]、fy=data[4]
cam_c2wtensor，每帧 camera-to-world 位姿，形状 (frame_count,4,4)，每个 4×4 的最后一行是 [0,0,0,1]
depthstensor，深度图 (frame_count,H,W)；if_output_depths=False（默认）时为空
npz_pathstring，落盘的 npz 绝对路径（键 images / depths / intrinsic / cam_c2w）；未设 output_npz_dir 时为空串
statusstring，success / error: no frames / error: <异常信息>
\n

参数

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
参数名称类型默认值描述
video_src_typestr\"video_url\"输入视频编码形式：video_url / video_base64 / video_binary，其他取值抛 ValueError
model_pathstr\"/opt/aihc/model\"权重根目录
moge_model_namestr\"Ruicheng/moge-2-vitl/model.pt\"相对 model_path 的 MoGe-2 权重，须是 .pt 文件；默认使用 Ruicheng/moge-2-vitl
megasam_repo_pathstr\"/opt/aihc/thirdparty/mega-sam\"mega-sam 源码根目录，须含 base/droid_slam
megasam_ckpt_namestr\"megasam/megasam_final.pth\"相对 model_path 的 DROID 权重
sample_modestr\"by_count_uniform\"抽帧模式，透传 VideoFrameSampler：by_count_uniform / by_interval_time / by_interval_frames / by_fps / by_timestamps
start_time_secfloat0.0抽帧起始秒
end_time_secfloat 或 NoneNone抽帧结束秒，None 表示到视频末尾
count_kint 或 None64by_count_uniform 模式抽帧数。默认取 64 是因为帧数过少（如 3 帧）起不了 DROID 轨迹
interval_secfloat 或 NoneNoneby_interval_time 模式间隔秒
interval_framesint 或 NoneNoneby_interval_frames 模式间隔帧
target_fpsfloat 或 NoneNoneby_fps 模式目标帧率
timestamps_seclist[float] 或 NoneNoneby_timestamps 模式时间戳列表
max_framesint 或 None300抽帧上限，显存与耗时随帧数线性增长
bufferint1024DROID 关键帧缓冲区大小，短片可下调省显存
filter_threshfloat2.0MotionFilter 运动阈值，低于此值的帧不作为关键帧
warmupint8frontend 启动所需帧数；抽帧数小于该值时算子直接报错
keyframe_threshfloat2.0关键帧剔除阈值
frontend_windowint25frontend 优化窗口
frontend_threshfloat12.0frontend 边构建距离阈值
frontend_radiusint2frontend 邻接半径
frontend_nmsint1frontend 非极大抑制
backend_threshfloat16.0backend 边构建距离阈值
backend_radiusint2backend 邻接半径
backend_nmsint3backend 非极大抑制
betafloat0.3几何 / 运动项权重
opt_intrinsicsboolTrue是否在 BA 里一并优化内参（对应 DROID terminate 的 _opt_intr）；DROID 内部若判定焦距不可观测会自动关掉
full_baboolTrue是否做全帧 BA（对应 DROID terminate 的 full_ba）
output_max_framesint1000输出帧数上限
if_output_depthsboolFalse是否把深度图写进输出列，默认关（单帧 384×512 就是 20 万个 float32）
output_npz_dirstr 或 NoneNone非空时把 images / depths / intrinsic / cam_c2w 落成 npz，目录会自动创建
video_formatstr\"mp4\"二进制 / base64 输入的容器格式
rankint0多卡场景 worker 序号，只影响 MoGe 的落卡
\n

注意事项

\n\n

调用示例

\n\n
\n
\n
\n Python\n \n
\n
\n 
1from __future__ import annotations\n2\n3import os\n4\n5import daft\n6from daft import col\n7\n8from daft.aihc.common.udf import aihc_udf\n9from daft.aihc.functions.embodied.perception.video_camera_pose import VideoCameraPose\n10\n11if __name__ == \"__main__\":\n12    if os.getenv(\"DAFT_RUNNER\", \"native\") == \"ray\":\n13        import ray\n14        ray.init(dashboard_host=\"0.0.0.0\", ignore_reinit_error=True)\n15        daft.set_runner_ray()\n16    daft.set_execution_config(actor_udf_ready_timeout=6000, min_cpu_per_task=0)\n17\n18ds = daft.from_pydict({\"video\": [\"bos://your-bucket/sample.mp4\"]})\n19    ds = ds.with_column(\n20        \"pose\",\n21        aihc_udf(\n22            VideoCameraPose,\n23            construct_args={\n24                \"model_path\": \"/path/to/models\",\n25                \"megasam_repo_path\": \"/path/to/thirdparty/mega-sam\",\n26                \"count_k\": 32,\n27                \"buffer\": 256,\n28                \"output_npz_dir\": \"/tmp/camera_pose_npz\",\n29            },\n30            num_cpus=4,\n31            num_gpus=1,\n32            concurrency=1,\n33            batch_size=1,\n34        )(col(\"video\")),\n35    )\n36    ds = ds.with_column(\"frame_count\", col(\"pose\")[\"frame_count\"])\n37    ds = ds.with_column(\"status\", col(\"pose\")[\"status\"])\n38    ds.show()
\n
\n
\n
\n ","fields":{"slug":"Tmu0pyy1k","title":"视频相机位姿估计（MegaSaM）","date":"2026-09-14","extractedHeadings":[]},"headings":[{"value":"简介","depth":2},{"value":"功能描述","depth":3},{"value":"算子参数","depth":2},{"value":"输入","depth":3},{"value":"输出","depth":3},{"value":"参数","depth":3},{"value":"注意事项","depth":2},{"value":"调用示例","depth":2}]}},"pageContext":{"isCreatedByStatefulCreatePages":false,"slug":"Tmu0pyy1k","prev":{"id":"Fmu0pxx3f","name":"图像三维人体重建（SAM 3D Body）","path":"Fmu0pxx3f","filePath":"操作指南/AI数据处理/算子列表/具身/图像三维人体重建（SAM 3D Body）.md","seo":null,"parentIds":["ilib2qygp","Ymo88m8hi","Imob3m6so","Qmob46txp"],"parents":[{"id":"ilib2qygp","documentId":"bfa43a8b-968a-41a1-8c9d-906507eeaed9","name":"操作指南","repoName":"AIHC","filePath":"操作指南","disabled":false,"path":"ilib2qygp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":null,"solutionTag":null,"solutionColor":null},{"id":"Ymo88m8hi","documentId":"c8cb5e38-f8c5-40f4-a424-b0c7895f0c0a","name":"AI数据处理","repoName":"AIHC","filePath":"操作指南/AI数据处理","disabled":false,"path":"Ymo88m8hi","lastMergeTime":"2026-04-21 14:23:10","isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":"","solutionTag":null,"solutionColor":null},{"id":"Imob3m6so","documentId":"fe548e34-6659-4ff5-86f6-eee2c43aec90","name":"算子列表","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表","disabled":false,"path":"Imob3m6so","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":"","solutionTag":null,"solutionColor":null},{"id":"Qmob46txp","documentId":"f9104826-f58e-4fe5-beaa-11c45b4333dc","name":"具身","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表/具身","disabled":false,"path":"Qmob46txp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":"","solutionTag":null,"solutionColor":null}]},"next":{"id":"Ymu0q11by","name":"旋转表示转换","path":"Ymu0q11by","filePath":"操作指南/AI数据处理/算子列表/具身/旋转表示转换.md","seo":null,"parentIds":["ilib2qygp","Ymo88m8hi","Imob3m6so","Qmob46txp"],"parents":[{"id":"ilib2qygp","documentId":"bfa43a8b-968a-41a1-8c9d-906507eeaed9","name":"操作指南","repoName":"AIHC","filePath":"操作指南","disabled":false,"path":"ilib2qygp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":null,"solutionTag":null,"solutionColor":null},{"id":"Ymo88m8hi","documentId":"c8cb5e38-f8c5-40f4-a424-b0c7895f0c0a","name":"AI数据处理","repoName":"AIHC","filePath":"操作指南/AI数据处理","disabled":false,"path":"Ymo88m8hi","lastMergeTime":"2026-04-21 14:23:10","isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":"","solutionTag":null,"solutionColor":null},{"id":"Imob3m6so","documentId":"fe548e34-6659-4ff5-86f6-eee2c43aec90","name":"算子列表","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表","disabled":false,"path":"Imob3m6so","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":"","solutionTag":null,"solutionColor":null},{"id":"Qmob46txp","documentId":"f9104826-f58e-4fe5-beaa-11c45b4333dc","name":"具身","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表/具身","disabled":false,"path":"Qmob46txp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":"","solutionTag":null,"solutionColor":null}]},"parents":[{"id":"ilib2qygp","documentId":"bfa43a8b-968a-41a1-8c9d-906507eeaed9","name":"操作指南","repoName":"AIHC","filePath":"操作指南","disabled":false,"path":"ilib2qygp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":null,"solutionTag":null,"solutionColor":null},{"id":"Ymo88m8hi","documentId":"c8cb5e38-f8c5-40f4-a424-b0c7895f0c0a","name":"AI数据处理","repoName":"AIHC","filePath":"操作指南/AI数据处理","disabled":false,"path":"Ymo88m8hi","lastMergeTime":"2026-04-21 14:23:10","isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":"","solutionTag":null,"solutionColor":null},{"id":"Imob3m6so","documentId":"fe548e34-6659-4ff5-86f6-eee2c43aec90","name":"算子列表","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表","disabled":false,"path":"Imob3m6so","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":"","solutionTag":null,"solutionColor":null},{"id":"Qmob46txp","documentId":"f9104826-f58e-4fe5-beaa-11c45b4333dc","name":"具身","repoName":"AIHC","filePath":"操作指南/AI数据处理/算子列表/具身","disabled":false,"path":"Qmob46txp","lastMergeTime":null,"isApiDoc":null,"httpMethod":null,"seo":null,"sourceOrgName":null,"sourceRepoName":null,"sourceDocumentId":"","solutionTag":null,"solutionColor":null}],"specificSeo":null}}}