[reproducer] Extend script argument by adding current-zuul-variables.yml

[danpawlik]

During the ZIronic deployment, the reproducer-variables.yml is generated, which might not be up-to-date what we want to achieve using Zuul job. It means, that if we want to provide some variables that should be respected by e.g. deploy-architecture.sh script, we need to add it as a last argument, then if the variable is defined earlier, it would not be respected.

In that case, we will be doing a dump of Zuul variables to current-zuul-variables.yml file then read it as last variable file.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign evallesp for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/70e24018806d48159b80e199df421436

❌ openstack-k8s-operators-content-provider NODE_FAILURE Node request 100-0008160438 failed in 0s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
❌ cifmw-pod-zuul-files NODE_FAILURE Node request 100-0008160439 failed in 0s
✔️ noop SUCCESS in 0s
❌ cifmw-pod-ansible-test NODE_FAILURE Node request 100-0008160440 failed in 0s
❌ cifmw-pod-pre-commit NODE_FAILURE Node request 100-0008160441 failed in 0s
✔️ cifmw-molecule-reproducer SUCCESS in 15m 52s

[danpawlik]

recheck

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/90bfe438c1d14553954ac0f21a4fa1b4

❌ openstack-k8s-operators-content-provider NODE_FAILURE Node request 100-0008160459 failed in 0s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
❌ cifmw-pod-zuul-files NODE_FAILURE Node request 100-0008160460 failed in 0s
✔️ noop SUCCESS in 0s
❌ cifmw-pod-ansible-test NODE_FAILURE Node request 100-0008160461 failed in 0s
❌ cifmw-pod-pre-commit NODE_FAILURE Node request 100-0008160462 failed in 0s
✔️ cifmw-molecule-reproducer SUCCESS in 16m 26s

[danpawlik]

recheck

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/0bcb33377b6143d09afb49c9249e3685

❌ openstack-k8s-operators-content-provider NODE_FAILURE Node request 100-0008160521 failed in 0s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
❌ cifmw-pod-zuul-files NODE_FAILURE Node request 100-0008160522 failed in 0s
✔️ noop SUCCESS in 0s
❌ cifmw-pod-ansible-test NODE_FAILURE Node request 100-0008160523 failed in 0s
❌ cifmw-pod-pre-commit NODE_FAILURE Node request 100-0008160524 failed in 0s
✔️ cifmw-molecule-reproducer SUCCESS in 14m 28s

[danpawlik]

Probably we don't need to create new file, just make a backup of old file and overwrite current vars.
ugly PoC for testproject:

• inventory

---
all:
  hosts:
    # NOTE: controller would be useless with Zironic. That's why
    # removing controller would be a good achievement.
    hypervisor:
      ansible_port: 22
      ansible_host: "{{ hypervisor }}"
      ansible_user: zuul
      ansible_ssh_private_key_file: /home/dpawlik/.ssh/id_rsa
    bm:
      ansible_port: 22
      ansible_host: "{{ hypervisor }}"
      ansible_user: zuul
      ansible_ssh_private_key_file: /home/dpawlik/.ssh/id_rsa
    vars:
      cifmw_artifacts_gather_edpm_logs: false
      cifmw_run_tests: false
      cifmw_test_operator_post: false
      cifmw_test_operator_pre: false
      cifmw_test_operator_stages:
      - name: tempest
        type: tempest

• play

---
- name: Overwrite reproducer-variables.yml on controller-0 with Zuul job vars
  hosts: bm
  gather_facts: false
  tasks:
    # from ci-framework-jobs/playbooks/zironic/zironic.yaml
    - name: Get customized parameters
      ansible.builtin.set_fact:
        zuul_job_overwrite_vars: >-
          {{
            hostvars['bm'] |
            dict2items |
            selectattr("key", "match",
                       "^(cifmw|pre|post)_(?!install_yamls|openshift_token|openshift_login|openshift_kubeconfig).*") |
            list | items2dict
          }}

    - name: Overwrite vars when zuul_job_overwrite_vars
      when: zuul_job_overwrite_vars | length > 0
      block:
        - name: Create artifacts with custom params
          ansible.builtin.copy:
            content: "{{ zuul_job_overwrite_vars | to_nice_yaml }}"
            mode: "0664"
            dest: "/tmp/zuul-vars-overwrite.yaml"

        - name: Create playbook for nested ansible execution
          ansible.builtin.copy:
            content: |
              {% raw %}
              - name: Run play for replacing vars
                hosts: localhost
                gather_facts: false
                tasks:
                  - name: Overwrite vars
                    delegate_to: controller-0
                    block:
                      - name: Fetch zuul overwrite vars file
                        ansible.builtin.copy:
                          src: /tmp/zuul-vars-overwrite.yaml
                          dest: /tmp/zuul-vars-overwrite.yaml

                      - name: Read reproducer-variables.yml
                        ansible.builtin.slurp:
                          src: "/home/zuul/ci-framework-data/parameters/reproducer-variables.yml"
                        register: reproducer_original

                      - name: Read override values
                        ansible.builtin.slurp:
                          src: /tmp/zuul-vars-overwrite.yaml
                        register: overrides

                      - name: Deep merge & write back
                        ansible.builtin.copy:
                          content: >-
                            {{
                              (reproducer_original.content | b64decode | from_yaml)
                              | combine(overrides.content | b64decode | from_yaml, recursive=true)
                              | to_nice_yaml
                            }}
                          dest: "/home/zuul/ci-framework-data/parameters/reproducer-variables.yml"
                          mode: '0644'
                          backup: true
              {% endraw %}
            dest: /tmp/zuul-vars-play.yml
            mode: "0664"

        - name: Run nested ansible to replace vars
          ansible.builtin.shell: >
            ansible-playbook -i /home/zuul/libvirt_inventory.yaml
            /tmp/zuul-vars-play.yml
            "$@"
          environment:
            ANSIBLE_LOG_PATH: "/tmp/zuul-vars-replace.log"