[test_operator] Add stackviz integration for Tempest test visualization #3642
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Merge Failed. This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. |
|
Merge Failed. This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. |
|
Merge Failed. This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. |
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/9eebbe61675c4dcc9d088d2c2d5ea48d ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 37m 18s |
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/8f1eb8e960fb4d9e8ca4f4a98fc80cda ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 19m 50s |
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/11f7659c426842b28edff49ed1d8f0cf ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 11m 41s |
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/75bb82277de440b3ac6855780f9c6fa9 ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 09m 12s |
sauragar
force-pushed
the
OSPRH-12077
branch
3 times, most recently
from
4000706 to
a8b3e36
Compare
|
PR Summary: Fix Stackviz Report Generation for Tempest Tests Problem Statement The stackviz integration for visualizing Tempest test results was not generating reports in Zuul CI runs. Investigation
Solution Overview This PR fixes the stackviz integration through 13 commits that address all identified issues and improve documentation: Core Fixes (5 commits)
Cleanup & Documentation (8 commits)
Foundation (3 commits)
Configuration Variables Four new variables control stackviz behavior (all documented in roles/test_operator/README.md):
Files Modified Core functionality:
Documentation:
Files Deleted
Testing & Verification ✅ Verified in Zuul CI:
Expected Output After Tempest tests complete, users will find: Single test stage: Multiple test stages (workflows): Report Features The generated stackviz HTML reports include: 📊 Core Visualization:
🎨 Theme Support:
🔍 Interaction:
Benefits
Technical Highlights Python Script Improvements:
Ansible Role Improvements:
Related Issues
|
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/1ad35fe4e1874a5d995b1b081965dc9a ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 18m 11s |
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/7f226ed574e54971acc665beca6f4750 ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 05m 33s |
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/3f99653ac2a64483bf6d1550aeaf7f6a ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 08m 27s |
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/d8974b19497244cc8715428016054050 ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 07m 15s |
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/6f2107258d6941c793b79b9718e453e3 ❌ openstack-k8s-operators-content-provider FAILURE in 6m 45s |
|
recheck |
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/f9b15630dea8425684d21ce011dd8cf3 ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 17m 07s |
sauragar
force-pushed
the
OSPRH-12077
branch
from
137e128 to
85e40e5
Compare
sauragar
force-pushed
the
OSPRH-12077
branch
from
e69acc8 to
46f7d4f
Compare
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/7359cdd1f98c4abc8ae2be9866549620 ✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 51m 04s |
sauragar
force-pushed
the
OSPRH-12077
branch
from
46f7d4f to
a364f0d
Compare
sauragar
force-pushed
the
OSPRH-12077
branch
2 times, most recently
from
88b76b2 to
79564c5
Compare
kstrenkova
left a comment
kstrenkova
left a comment
There was a problem hiding this comment.
Thank you for this PR, it seems useful and to be working correctly :D. I only have a few nitpicks.
| type: tempest | ||
| cifmw_test_operator_fail_on_test_failure: true | ||
| cifmw_test_operator_artifacts_basedir: "{{ cifmw_basedir }}/tests/test_operator" | ||
| cifmw_test_operator_generate_stackviz: true |
There was a problem hiding this comment.
(nitpick) I would name this variable cifmw_test_operator_stackviz_generate so every parameter that is related to stackviz would start with cifmw_test_operator_stackviz_ prefix.
| - name: Generate stackviz HTML report | ||
| when: | ||
| - not testpod_timed_out | ||
| - cifmw_test_operator_generate_stackviz | default(true) |
There was a problem hiding this comment.
(nitpick) I don't think this needs default(true) as the default should be set in roles/test_operator/defaults/main.yaml, which it already is :D So I think one source of truth is enough.
| ansible.builtin.shell: > | ||
| oc cp -n {{ stage_vars_dict.cifmw_test_operator_namespace }} | ||
| test-operator-logs-pod-{{ run_test_fw }}-{{ test_operator_instance_name }}:{{ pod_path }} | ||
| test-operator-logs-pod-{{ run_test_fw }}-{{ test_operator_instance_name }}:{{ pod_path }}/. |
There was a problem hiding this comment.
Not sure why this is needed (/.) 🤔 The change above (/mnt/logs...) seems like a good change as the mountpath earlier also starts with backslash.
| {% if ansible_os_family == 'Darwin' %} | ||
| open {{ cifmw_test_operator_artifacts_basedir }}/stackviz/index.html | ||
| {% else %} | ||
| xdg-open {{ cifmw_test_operator_artifacts_basedir }}/stackviz/index.html |
| - _stackviz_missing_packages | length > 0 | ||
| - cifmw_test_operator_stackviz_auto_install_deps | default(false) | ||
| block: | ||
| - name: Install missing Python packages via pip3 |
There was a problem hiding this comment.
Should be one single task installing rpm packages.
| when: _stackviz_missing_packages | length == 0 | ||
|
|
||
| # Create summary index page | ||
| - name: Create stackviz summary index |
| - _stackviz_generated_reports | length > 0 | ||
| - cifmw_test_operator_stackviz_create_index | default(true) | ||
| block: | ||
| - name: Create stackviz directory |
|
|
||
| - name: Generate summary index HTML | ||
| ansible.builtin.template: | ||
| src: stackviz-index.html.j2 |
… fixes Add comprehensive stackviz integration for Tempest test visualization with support for multi-stage test execution. This includes generation of interactive HTML reports with test results, timing analysis, and failure details. Key features: - Multi-stage stackviz report generation for main and worker jobs - Dark/light mode toggle for better viewing experience - Separate stackviz reports for tempest retry results - Support for both compressed and uncompressed subunit files - Auto-install of stackviz dependencies by default Improvements and fixes: - Fix stackviz script path for adoption jobs using cifmw_repo variable instead of playbook_dir to ensure correct script location - Fix log collection path for subunit files - Rename stackviz files for clarity using main/worker pattern - Remove unnecessary step-X directory wrapper in log collection - Clean up redundant documentation and debug tasks Documentation: - Document stackviz configuration variables in README - Add stackviz-related terms to spell-check dictionary This ensures stackviz works correctly across all job types including periodic-internal-adoption-* jobs. Signed-off-by: sauragar <sauragar@redhat.com> Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
sauragar
force-pushed
the
OSPRH-12077
branch
from
79564c5 to
edf7927
Compare
Merge upstream main branch to incorporate recent updates and keep the branch in sync with the base branch. This includes updates to common-requirements.txt. Signed-off-by: sauragar <sauragar@redhat.com>
|
Adding label DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Build failed (check pipeline). Post https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/a8f037a31df54b2d8233f253618cee1f ❌ openstack-k8s-operators-content-provider NODE_FAILURE Node request 100-0008160431 failed in 0s |
4 participants
Add automatic stackviz report generation for Tempest tests with comprehensive dependency checking and optional auto-installation.
New files:
Modified files:
Features:
Signed-off-by: Saurabh Agarwal sauragar@redhat.com