返回技能市场
开发运维 安全

pai-dlc 任务

@aliyun/alibabacloud-pai-dlc-job

分布式训练DLC（Deep Learning Containers）任务API相关的能力。包括任务的创建、查询、更新和删除。任务相关的事件、日志的查询等操作。

云Skills门户 热度 91v0.0.2

PAI-DLC 深度学习作业管理

管理阿里云 PAI-DLC（AI 平台 - 深度 学习容器）服务上的深度学习训练作业。

场景说明

PAI-DLC 是阿里云 AI 平台 PAI 提供的分布式训练服务， 支持：

  • 作业创建与执行 — 为 TensorFlow、

    • PyTorch、XGBoost 及其他框架创建分布式训练作业

  • 作业监控 — 获取作业状态、日志、事件和监控指标
  • 计算健康检查 — 检查 GPU 及其他计算设备的健康状态
  • 作业管理 — 更新和停止作业

架构：PAI 工作空间 + DLC 作业 + 计算资源（ECS 公共按量付费 或灵骏专属配额）+ AIWorkSpace 目录（镜像 / 数据集 / 代码源 / 配额 / 工作空间）。

安装要求

预检查：阿里云 CLI 版本必须 >= 3.3.1
运行 aliyun version 以确认版本 >= 3.3.1。若未安装或版本过低，
请参见 [references/cli-installation-guide.md](references/cli-installation-guide.md) 中的
安装说明。
然后，[强制要求] 运行 aliyun configure set --auto-plugin-install true 以启用
自动安装插件。
关于 --user-agent 的说明： 此 skill 中每条调用 API 的 aliyun 命令都必须
include --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-dlc-job/${SESSION_ID}
（统一的 User-Agent + session-id 模板——生成规则请参见[可观测性](#observability)
）。客户端辅助命令（aliyun version、
aliyun configure ...、aliyun plugin ...、aliyun <product> --help）不会
调用远程 APIs，因此不需要该参数。
网络超时与重试（--help 未强制执行的规则）： aliyun CLI
默认连接超时为 10s / 读取超时为 10s，且不重试。对于长时间运行的流程
（列表较大、地域响应较慢），请通过全局标志显式提高超时值
--connect-timeout 15 --read-timeout 30 --retry-count 2。对于用户已确认的高风险调用，严禁依赖
默认设置（stop-job / delete-*）。
aliyun version
aliyun configure set --auto-plugin-install true
aliyun pai-dlc --help
aliyun aiworkspace --help >/dev/null 2>&1 || aliyun plugin install --names aliyun-cli-aiworkspace
aliyun plugin update

可观测性

原因：此 skill 发出的每次 PAI-DLC API 调用都必须携带统一的
User-Agent，以便平台侧链路追踪能够将请求归因到此 skill，
并关联单个 agent 会话中的所有调用。

User-Agent 模板

每条调用 API 的 aliyun 命令都必须传入：

--user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-dlc-job/{session-id}
  • 前缀 AlibabaCloud-Agent-Skills/alibabacloud-pai-dlc-job 是固定值——禁止修改。
  • {session-id} 是按下方规则生成的每会话标识符。
  • 客户端辅助命令（aliyun version、aliyun configure ...、

    • aliyun plugin ...、aliyun <product> --help）不会调用远程 APIs， 因此不需要该参数。

session-id 生成规则

  1. 在 skill 会话开始时，仅生成一次 session-id，并且要在

    2. 第一次 API 调用之前完成。

  2. 格式：小写、不含空格、不含斜杠。推荐使用

    3. uuidgen | tr 'A-Z' 'a-z' 或 $(date +%s)-$RANDOM。

  3. 会话中的每条命令都复用同一个 session-id——禁止

    4. 每次调用时重新生成。这样链路追踪才能将一个会话的所有调用归为一组。

  4. 导出一次，并在每次 API 调用中引用该 Shell 变量：
SESSION_ID=$(uuidgen | tr 'A-Z' 'a-z')
export SESSION_ID
# every API-invoking command then appends:
#   --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-dlc-job/${SESSION_ID}
aliyun pai-dlc list-jobs --region <r> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-dlc-job/${SESSION_ID}
禁止使用 aliyun configure ai-mode 设置 User-Agent。该模式
已在 skills 中弃用。始终使用 --user-agent 为每条命令显式传入
session-id 模板，使每个会话都可独立追踪。

## Environment Variables

This skill does not require any custom environment variables. Credentials are handled
by the Alibaba Cloud CLI configuration (see Authentication below). Optionally:

| Variable | Required | Purpose |
|----------|----------|---------|
| `ALIBABA_CLOUD_PROFILE` | Optional | Selects a non-default `aliyun configure` profile |
| `ALIBABA_CLOUD_REGION_ID` | Optional | Default region when `--region` is omitted (still recommended to pass `--region` explicitly) |

Do NOT export `ALIBABA_CLOUD_ACCESS_KEY_ID` / `ALIBABA_CLOUD_ACCESS_KEY_SECRET` from
within this session; configure them outside (`aliyun configure` or shell profile).

## Authentication Configuration

> **Pre-check: Alibaba Cloud Credentials Required**
>
> **Security Rules:**
> - **NEVER** read, echo, or print AK/SK values (e.g., `echo $ALIBABA_CLOUD_ACCESS_KEY_ID` is FORBIDDEN)
> - **NEVER** ask the user to input AK/SK directly in the conversation or command line
> - **NEVER** use `aliyun configure set` with literal credential values
> - **ONLY** use `aliyun configure list` to check credential status
>
> ```bash
> aliyun configure list
> ```
> Check the output for a valid profile (AK, STS, or OAuth identity).
>
> **If no valid profile exists, STOP here.**
> 1. Obtain credentials from [Alibaba Cloud Console](https://ram.console.aliyun.com/manage/ak)
> 2. Configure credentials **outside of this session** (via `aliyun configure` in terminal
>    or environment variables in shell profile)
> 3. Return and re-run after `aliyun configure list` shows a valid profile

## RAM Permissions

> **[MUST] Permission Failure Handling:** When any command or API call fails due to
> permission errors at any point during execution, follow this process:
> 1. Read `references/ram-policies.md` to get the full list of permissions required by this SKILL
> 2. Use `ram-permission-diagnose` skill to guide the user through requesting the necessary permissions
> 3. Pause and wait until the user confirms that the required permissions have been granted

For detailed permission list, see [references/ram-policies.md](references/ram-policies.md).

**Required Permissions Overview:**

| Operation | Required Permission |
|-----------|---------------------|
| Create Job | `pai:CreateJob` |
| List Jobs | `pai:ListJobs` |
| Get Job Details | `pai:GetJob` |
| Get Pod Logs | `pai:GetPodLogs` |
| Get Job Events | `pai:GetJobEvents` |
| Update Job | `pai:UpdateJob` |
| Stop Job | `pai:StopJob` |
| AIWorkSpace Resource Discovery | `paiworkspace:ListWorkspaces` / `paiimage:ListImages,GetImage` / `paidataset:ListDatasets,GetDataset` / `paicodesource:ListCodeSources,GetCodeSource` |

> **AIWorkSpace authorization note:** `Image` / `DataSourceId` / `CodeSourceId` /
> `WorkspaceId` field values for `create-job` come from the
> AIWorkSpace resource-discovery APIs. `--resource-id` (QuotaId) is manually provided by the user.
> RAM users MUST hold the corresponding
> AIWorkSpace-namespaced permissions listed above (do not abbreviate as `aiworkspace:*`).

## Parameter Confirmation

> **Authoritative parameter reference is `aliyun pai-dlc <cmd> --help`** (must-read
> before every call). This skill only documents what `--help` does **not** tell
> you: cross-field rules, cross-product dependencies, hidden behaviors, business
> labels, and reject patterns. Whenever a rule below contradicts `--help`, the
> reason is stated inline.
>
> **Confirm before call:** all user-customizable values (region, names, CIDR,
> specs, etc.) MUST be confirmed with the user — never assume defaults.

### Hard rules that override `--help`

| Rule | Why this skill overrides `--help` |
|------|-----------------------------------|
| `--workspace-id` is **always required** | `--help` marks it optional, but server silently falls back to the user's **default workspace** if omitted → job often lands in the wrong workspace. Always confirm with user. |
| `--job-specs[].Image` MUST be a verbatim `ImageUri` from `aiworkspace list-images` | Cross-product contract; `--help` only describes the field type. See §7.6 red line. |
| `--data-sources[].DataSourceId` from `aiworkspace list-datasets`; `--code-source.CodeSourceId` from `list-code-sources` | Cross-product discovery; `--help` cannot point you to the source product. |
| `--resource-id` (QuotaId) is **manually supplied** | No CLI discovery step. |

### Cross-field mutual exclusion (`--help` cannot catch these)

- `EcsSpec` ⇄ `ResourceConfig` — within a single TaskSpec, pick exactly one.
- `Uri` ⇄ `DataSourceId` — within `--data-sources[]`.
- `Uri` ⇄ `CodeSourceId` — within `--code-source`.

### `--job-type` — Worker `Type` hints per framework

`--help` lists the 9 legal enum values verbatim. What `--help` doesn't tell you
is which `JobSpecs[].Type` roles each framework expects:

| `--job-type` | Valid `JobSpecs[].Type` roles |
|---|---|
| `TFJob` | `Chief` / `PS` / `Worker` / `Evaluator` / `GraphLearn` |
| `PyTorchJob` | `Worker` (+ optional `Master`, auto-promoted) |
| `MPIJob` | `Worker` + `Master` |
| `XGBoostJob` / `OneFlowJob` / `ElasticBatchJob` | `Worker` + optional `Master` |
| `RayJob` | `Worker` |
| `SlurmJob` / `DataJuicerJob` | framework-specific roles |

> **Case-sensitive, no aliases.** `tensorflow`, `pytorch`, `tf-job`, `Pytorch`,
> `PYTORCH_JOB`, `Custom`, `CustomJob` — all rejected.
>
> **No `Custom` enum.** For single-container custom workloads, map to
> `PyTorchJob` (most permissive role set).
>
> **Locked after create:** `JobType` cannot be changed via `update-job`.

Full field reference: see [references/related-apis.md](references/related-apis.md).

## Core Workflows

### 7.1 Resource Selection Decision Guide

Before calling `create-job`, determine the resource path:

- **Public pay-as-you-go** → Use `EcsSpec` in TaskSpec; do NOT pass `--resource-id`.
  - Use cases: quick start, testing, no dedicated quota.
  - Example: `"EcsSpec": "ecs.gn6i-c4g1.xlarge"`
- **Dedicated quota** (Lingjun / enterprise quota) → Use `ResourceConfig` in TaskSpec
  AND pass `--resource-id <QuotaId>`.
  - Use cases: dedicated resource group, Lingjun smart compute, Spot bidding.
  - Example: `--resource-id quotaXXX` + `"ResourceConfig": {"CPU": "4", "Memory": "8Gi", "GPU": "1"}`

> **EcsSpec and ResourceConfig MUST NOT both appear in the same TaskSpec.**

> **Also required before `create-job`:** `--job-specs[].Image` MUST come from
> `aliyun aiworkspace list-images`; `--data-sources[].DataSourceId` from
> `list-datasets`; `--code-source.CodeSourceId` from `list-code-sources`.
> Full discovery flow → see §7.6.

**Distributed architecture choices:**

| Topology | `JobSpecs` shape |
|---|---|
| Single-node | One `Worker` only |
| TFJob PS-Worker | Both `PS` (CPU) and `Worker` (GPU) roles |
| PyTorch multi-node | One `Worker` with `PodCount > 1` |

Optional flags: `--enable-gang-scheduling true` (all-or-nothing scheduling),
`Settings.EnableRDMA: true` (high-performance network for multi-node GPU),
`Settings.EnableSanityCheck: true` (GPU health verification).

> **All commands below require `--user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-dlc-job/${SESSION_ID}`** (omitted in snippets for brevity — see [Observability](#observability)).

### 7.2 Create Training Job

Minimal single-node PyTorch job (public pay-as-you-go) parameter combination:

aliyun pai-dlc create-job --region <region> --workspace-id <ws-id> \ --display-name "my-pytorch-training" --job-type PyTorchJob \ --job-specs '[{"Type":"Worker","PodCount":1,"Image":"<ImageUri>","EcsSpec":"ecs.gn6i-c4g1.xlarge"}]' \ --user-command 'python train.py' \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-dlc-job/${SESSION_ID}


Multi-node / Spot / RDMA / data mounting — use `create-job --help`.
Subsequent snippets omit `--user-agent` for brevity — always include it.

### 7.3 List / Get Job

Use `--cli-query` to project specific fields (essential for log/event flows):

aliyun pai-dlc list-jobs --region <region> --status Running aliyun pai-dlc get-job --region <region> --job-id <id> aliyun pai-dlc get-job --region <region> --job-id <id> --cli-query "Pods[0].PodId"


### 7.4 Logs and Events

> **Always cap return size:** `--max-lines 100` (logs), `--max-events-num 50` (events).

Get `PodId` first, then query logs/events:

POD_ID=$(aliyun pai-dlc get-job --region <r> --job-id <id> --cli-query "Pods[0].PodId") aliyun pai-dlc get-pod-logs --region <r> --job-id <id> --pod-id $POD_ID --max-lines 100 aliyun pai-dlc get-pod-events --region <r> --job-id <id> --pod-id $POD_ID --max-events-num 20 aliyun pai-dlc get-job-events --region <r> --job-id <id> --max-events-num 50


**Diagnosis order:** `get-job` (status) → `get-job-events` → `get-pod-logs` → `get-pod-events`.

### 7.5 Compute Health Check

aliyun pai-dlc list-job-sanity-check-results --region <r> --job-id <id> aliyun pai-dlc get-job-sanity-check-result --region <r> --job-id <id> --sanity-check-number 1


### 7.6 Pre-Create Resource Discovery (AIWorkSpace)

**Discovery flow:** `list-workspaces` → `list-image-labels` →
`list-images` → `list-datasets` → `list-code-sources` → `pai-dlc create-job`.

> **Quota (`--resource-id`):** user-supplied. No CLI discovery step.

aliyun aiworkspace list-workspaces --region <r> # → --workspace-id aliyun aiworkspace list-image-labels --region <r> # → valid label Key=Value pairs aliyun aiworkspace list-images --region <r> --labels "K1=V1,K2=V2" # → --job-specs[].Image (use ImageUri verbatim) aliyun aiworkspace list-datasets --region <r> --workspace-id <ws> # → DataSources[].DataSourceId aliyun aiworkspace list-code-sources --region <r> --workspace-id <ws> # → CodeSource.CodeSourceId


> **Labels rules** (not in `--help`): comma-separated `Key=Value` pairs, no
> JSON / no spaces. Values MUST come from `list-image-labels` — never invent.
> Do **not** pass `--workspace-id` to `list-images` when discovering **official
> public images** (they are global). Pass `--workspace-id` only when filtering
> **custom / private images** scoped to a specific workspace.
>
> **RED LINE:** `--job-specs[].Image` MUST be a verbatim `ImageUri` (not
> `Name` / `ImageId`).
>

Field-mapping, full parameters, and error codes: see
[references/related-apis.md](references/related-apis.md) and
[references/verification-method.md](references/verification-method.md).

### 7.7 Job Lifecycle Management (Stop / Update / Web Terminal)

Stop is a **high-risk** operation. Before proceeding, query status with
`get-job`, present the result to the user, and require explicit confirmation.

> **Rules `--help` doesn't tell you (`update-job` silent-no-op family):**
>
> - **Stop Job** applies only when status is `Running` or `Queuing`.
> - **`update-job --priority`** takes effect **only** when (a) the job uses
>   **quota resources** (`--resource-id`) AND (b) status is `Creating`,
>   `Queuing`, or `EnvPreparing`. Once the job enters `Running` or later,
>   priority **cannot be modified** — the API returns `200 OK` but the change
>   is **silently NOT applied**. Always pre-check status with `get-job`.
> - **`update-job --accessibility`** takes effect immediately in any status.
> - **`update-job` does NOT expose `--display-name`** (`--help` lists only
>   `--job-id`, `--accessibility`, `--description`, `--job-specs`, `--priority`).
>   To rename a job, recreate it.

For the full pre-check + confirmation + execution templates, plus the
`update-job` low-risk path and `get-web-terminal` / `get-token` sharing
commands, see [references/job-management.md](references/job-management.md).

### 7.8 Ecs Spec Discovery

Discover available instance types; the returned `EcsSpec` value goes
verbatim into `--job-specs[].EcsSpec`.

aliyun pai-dlc list-ecs-specs --region <r> --accelerator-type GPU --resource-type ECS --page-size 20

灵骏专用：--quota-id <id>（仅限白名单用户）


> **`list-ecs-specs` does not support `--sort-by`** — even values shown as
> valid in `--help` (e.g. `CPU` / `GPU` / `Memory` / `GmtCreateTime`) are
> rejected by the server. Always omit `--sort-by` here and sort the JSON
> output client-side with `jq` — e.g.
> `... | jq '.EcsSpecs | sort_by(-.AcceleratorNumber)'`.

## Success Verification Method

For step-by-step end-to-end verification scripts (resource discovery →
CreateJob → log query → cleanup), see
[references/verification-method.md](references/verification-method.md).

**Quick verification:**

- `get-job` → Status should be `Creating` / `Queuing` / `Running` shortly after
  `create-job` returns.
- `list-jobs --status Running` → Should return the freshly created Job until it
  finishes or is stopped.
- `get-pod-logs` → Should return non-empty log content once the Pod is past
  `EnvPreparing`.

## Command Tables

The full command index (5 categories × ~40 commands, with plugin
attribution) is consolidated in
[references/related-apis.md](references/related-apis.md) §1.

## Best Practices

> Items below are **decision rules** and **operational habits** — not parameter
> values (those live in `--help`).

1. **Job naming** — use meaningful, sortable names: `project-model-date`
   (e.g. `resnet50-imagenet-20260320`). Recreate (not `update-job`) is the
   only way to rename.
2. **Resource sizing** — pick GPU type / count by model & dataset size. Verify
   availability with `list-ecs-specs --accelerator-type GPU` **before** picking
   `EcsSpec` (see §7.8).
3. **Diagnose early** — follow the order `get-job` → `get-job-events` →
   `get-pod-logs` → `get-pod-events`. Cap responses (`--max-lines 100`,
   `--max-events-num 50`) to keep agent context lean.
4. **Priority adjustment** — prefer setting `--priority` at `create-job` time.
   Post-creation `update-job --priority` only works for quota jobs in
   `Creating` / `Queuing` / `EnvPreparing` phase (§7.7); once `Running`,
   priority cannot be modified.
5. **Cost control** — use `--job-max-running-time-minutes` as an auto-stop guard
   for every long-running experiment. Spot via `SpotSpec` reduces cost at the
   risk of preemption.
6. **Health check** — enable `Settings.EnableSanityCheck: true` for GPU
   training to catch faulty devices before training starts.
7. **Resource cleanup** — `stop-job` on completed jobs to free quota.
8. **Idempotency on writes** — PAI-DLC `create-*` APIs do **NOT** expose
   `--client-token` (verified via `aliyun pai-dlc create-job --help`). Network
   retries can therefore create duplicate Jobs. Mitigation: before re-issuing
   a failed `create-*`, run `list-jobs --display-name <name>` to detect a
   half-committed prior attempt.

## Reference Links

| Reference Document | Description |
|--------------------|-------------|
| [references/related-apis.md](references/related-apis.md) | Command index, cross-product field map, lifecycle, red lines, error catalog |
| [references/ram-policies.md](references/ram-policies.md) | RAM permission policy details |
| [references/verification-method.md](references/verification-method.md) | End-to-end verification scripts |
| [references/job-management.md](references/job-management.md) | High-risk Stop/Delete/Update flow + Web Terminal |
| [references/acceptance-criteria.md](references/acceptance-criteria.md) | Skill testing acceptance criteria |
| [references/cli-installation-guide.md](references/cli-installation-guide.md) | CLI installation guide |
qianwen skills install @aliyun/alibabacloud-pai-dlc-job