lhm-metric-check

Creates and triggers an LHM (Lakehouse Migration Center) per-table metric validation task. Automatically completes: task creation (with a specified template) → per-table metric field configuration → batch saving → immediate execution.

Steps

Confirm that the aliyun CLI + aliyun-cli-lhm plugin and pyyaml are installed (see "Environment Preparation" in the root README).
Construct the --tables-json argument: a JSON array where each element is a dict containing source_table, target_table, source_partition, target_partition, source_columns, and target_columns; an empty string for a field means using the template default.
Invoke this script, passing all required parameters via the command line. Data sources support alias mode: use --src-alias and --dst-alias to resolve from data_validation_config.yaml, in which case --src-ds-id/name/type do not need to be passed.
The script calls run_metric_check() in the outer scripts/common.py to create the task and trigger execution.
On success, stdout outputs JSON: {"ok": true, "task_id": ..., "batch_id": ...}.

Pitfalls

--threshold defaults to None, meaning it is not passed to the SDK and the system uses its default value; passing none also means None.
Never pass 0.0 by default for metric checks, otherwise all numeric metrics will be falsely judged PASSED; only pass 0.0 when explicitly required.
--check-template-id defaults to 1001 (MIX); specify it explicitly if another template is needed.
When a partition field is an empty string, common.py treats it as a whole-table check (is_full_table_count=1).
MaxCompute partitioned tables usually require setting --source-global-params / --target-global-params to odps.sql.allow.fullscan=true.

Verification

After running the script, check whether the stdout output is {"ok": true, "task_id": <int>, "batch_id": <int>}.
Use lhm-poll-status or the LHM console to check the execution status of the corresponding task_id / batch_id.

Input/Output Examples

Input (command line):

python atomic-skills/lhm-metric-check/scripts/run.py \
  --task-name "逐表指标校验" \
  --src-ds-id ds-src-001 \
  --src-ds-name "源端MySQL" \
  --src-ds-type MySQL \
  --dst-ds-id ds-dst-001 \
  --dst-ds-name "目标端Hive" \
  --dst-ds-type Hive \
  --tables-json '[
    {
      "source_table": "src_db.orders",
      "target_table": "dst_db.orders",
      "source_partition": "dt=20240305",
      "target_partition": "dt=20240305",
      "source_columns": "amount,order_cnt",
      "target_columns": "amount,order_cnt"
    }
  ]' \
  --check-template-id 1001 \
  --region hangzhou

Output (stdout):

{"ok": true, "task_id": 12345, "batch_id": 67890}

Using alias mode (data sources resolved from data_validation_config.yaml):

python atomic-skills/lhm-metric-check/scripts/run.py \
  --task-name "逐表指标校验" \
  --src-alias mc_source \
  --dst-alias sr_target \
  --tables-json '[{"source_table":"src_db.orders","target_table":"dst_db.orders","source_partition":"","target_partition":"","source_columns":"amount","target_columns":"amount"}]' \
  --check-template-id 1001