Update security ML jobs to include MITRE ATT&CK framework tactics and technique metdata

[ymao1]

Proposed commit message

Adding MITRE ATT&CK framework tactic and technique codes to the custom_settings field for all security ML jobs. These mappings were derived from the prebuilt detection ML rules that include the mapping within the rule definitions. Techniques and subtechniques are stored in the same array.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Screenshots

[github-actions]

TL;DR

Check integrations ded failed while building the package because the transform manifest references a versioned ingest pipeline name that does not exist in the built package (3.1.0-ml_ded_ingest_pipeline inside package version 3.1.1). Update the transform’s versioned references to match the package version bump.

Remediation

• In packages/ded/elasticsearch/transform/pivot_transform_ea/transform.yml, update all version-coupled values from 3.1.0 to 3.1.1 (at minimum dest.pipeline; also keep dest.index and _meta.fleet_transform_version aligned with the package version convention).
• Re-run .buildkite/scripts/test_one_package.sh packages/ded origin/main <commit_sha> (or the equivalent local elastic-package check) to confirm package build resolves transform manifests successfully.

Investigation details

Root Cause

The transform definition uses a hard-coded versioned pipeline name:

• packages/ded/elasticsearch/transform/pivot_transform_ea/transform.yml:9 → pipeline: 3.1.0-ml_ded_ingest_pipeline

Buildkite attempted to build package ded/3.1.1, so it looked for 3.1.0-ml_ded_ingest_pipeline.yml in that build output and failed because that filename/version does not exist for the bumped package version.

Evidence

• Build: https://buildkite.com/elastic/integrations/builds/43546
• Job/step: Check integrations ded
• Key log excerpt:

Error: checking package failed: building package failed: resolving transform manifests failed:
failed reading transform definition file ".../packages/ded/3.1.1/elasticsearch/transform/pivot_transform_ea/transform.yml":
destination ingest pipeline file 3.1.0-ml_ded_ingest_pipeline.yml not found: incorrect version used in pipeline or unknown pipeline

Verification

• Not run in this environment; diagnosis based on Buildkite failure log and repository source inspection.

Follow-up

• After fixing transform.yml, scan packages/ded/ for any remaining 3.1.0 literals that should track the package version to avoid the same class of failure on the next patch bump.

Note

🔒 Integrity filter blocked 3 items

The following items were blocked because they don't meet the GitHub integrity level.

• #19220 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• Update security ML jobs to include MITRE ATT&CK framework tactics and technique metdata #19220 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #19220 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 25d94e4

History

• 💔 Build #43546 failed 493e224
• 💔 Build #43544 failed c080cf7