gitlab-deep-merge.md

As a Senior DevOps Engineer and Architect, I can tell you that you've hit one of the most common limitations of the GitLab CI YAML engine: GitLab does not natively support deep-merging dictionaries inside an array element (which is what workflow:rules is).

If you define a rule as a list item (e.g., - if: ...) and use !reference, GitLab treats that list item as a single block. You cannot easily inject or override variables inside it at the point of reference.

However, we can elegantly bypass this limitation by using what I call the "Nested Hash Extension" pattern.

Instead of defining your rules as an array in the common repo, we define them as a nested hash (dictionary). We then leverage GitLab's extends keyword (which does support deep-merging dictionaries, unlike arrays) to merge your custom variables in the child repository, and finally inject the resulting hash into workflow:rules using !reference.

Here is the exact architectural blueprint to achieve this.

1. The Common Repository (common-rules.yml)

In your centralized repository, define each rule as a top-level hidden job (starts with a .), containing a nested rule key.

Why the nested rule key? Because the !reference tag requires at least two arguments (the hidden job name, and the key to extract).

# common-rules.yml

.rule_master_mr_base:
  rule: # <-- We wrap the condition in this key to allow targeting via !reference
    if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"'
    variables:
      IS_MASTER_MR: "true"
      # Put any common variables here

.rule_develop_push_base:
  rule:
    if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "develop"'
    variables:
      IS_DEVELOP_PUSH: "true"

2. The Child Repository (.gitlab-ci.yml)

In your individual repositories, you include the common file. Whenever a repo needs to add custom variables to a rule, you create a local hidden block that extends the base rule.

Because variables is a dictionary, extends natively performs a deep-merge, retaining the original if condition and common variables, while appending your custom ones.

# .gitlab-ci.yml
include:
  - project: 'my-org/devops/ci-templates'
    ref: 'main'
    file: '/common-rules.yml'

# 1. Extend the base rule and merge our custom variables
.rule_master_mr_custom:
  extends: .rule_master_mr_base
  rule:
    variables:
      MY_CUSTOM_REPO_VAR: "custom_value_for_this_repo"
      DB_URL: "postgres://user:pass@localhost:5432/db"

workflow:
  rules:
    # 2. Inject the deeply-merged hash directly into the workflow:rules array
    - !reference [.rule_master_mr_custom, rule]
    
    # 3. You can still use unmodified base rules directly!
    - !reference [.rule_develop_push_base, rule]
    
    # 4. Standard fallback rules
    - when: always 

How and Why This Works Under the Hood

1. The Merge Phase: Before pipeline execution, GitLab resolves the extends keyword. It looks at .rule_master_mr_custom, grabs the if string from the base, takes the IS_MASTER_MR variable, and merges it with your MY_CUSTOM_REPO_VAR.
2. The Reference Phase: GitLab processes !reference [.rule_master_mr_custom, rule]. Because you placed a hyphen (-) before it, it correctly interprets the resulting merged hash as a single object inside the workflow:rules array.
3. The Result: The pipeline evaluates the rule exactly as if you had hardcoded:

   workflow:
     rules:
       - if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"'
         variables:
           IS_MASTER_MR: "true"
           MY_CUSTOM_REPO_VAR: "custom_value_for_this_repo"
           DB_URL: "postgres://user:pass@localhost:5432/db"

An Alternative Architecture (Variables over Rules)

If creating .custom blocks in every repo feels too verbose, the alternative architectural approach is to rely entirely on standard top-down Pipeline Variables, leaving the rules completely static.

# common-rules.yml
.rules:
  master_mr:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"'
      variables:
        IS_MASTER_MR: "true"

# child .gitlab-ci.yml
include: 'common-rules.yml'

# Define standard global variables. They will be active in the pipeline 
# if the workflow rule evaluates to true and allows the pipeline to run.
variables:
  MY_CUSTOM_REPO_VAR: "custom_value"
  DB_URL: "postgres://..."

workflow:
  rules:
    - !reference[.rules, master_mr]

When to use which?

• Use the Nested Hash Extension (first method) if you only want those custom variables injected specifically when that exact rule evaluates to true.
• Use Global Variables (second method) if the variables can safely sit in the global variables: block and you just want the rule to govern whether the pipeline runs or not.