diff --git a/doc/changes/unreleased.md b/doc/changes/unreleased.md index 442d5000a..b3c34b63f 100644 --- a/doc/changes/unreleased.md +++ b/doc/changes/unreleased.md @@ -10,6 +10,7 @@ ## Documentation +* #705: Described how the versions of poetry and python are retrieved * #706: Added description how to ignore findings to the User Guide ## Refactoring diff --git a/doc/user_guide/features/github_workflows/configuration.rst b/doc/user_guide/features/github_workflows/configuration.rst deleted file mode 100644 index aa654fcaa..000000000 --- a/doc/user_guide/features/github_workflows/configuration.rst +++ /dev/null @@ -1,32 +0,0 @@ -.. _github_workflows_configuration: - -Configuration -============= - -1. Configure the GitHub project -+++++++++++++++++++++++++++++++ - -* Make sure your GitHub project has access to a deployment token for PyPi with the following name: **PYPI_TOKEN**. It should be available to the repository either as an Organization-, Repository-, or Environment-secret. - -* If your CI workflow involves slow or expensive steps you can guard these to be executed only after manual approval. The CI workflow will automatically create a GitHub environment named :code:`manual-approval`. You only need to add reviewers in (:code:`Settings/Environments/manual-approval`) and move the steps to be guarded into the related section in job :code:`slow-checks` in file :code:`.github/workflows/merge-gate.yml`. - -2. Add all workflows to your project -++++++++++++++++++++++++++++++++++++ - -.. code-block:: shell - - tbx workflow install all - -.. warning:: - - #. If you already have various workflows, you may want to run the :code:`update` command instead of the :code:`install` command. - - #. Some workflows depend on other workflows. Please ensure you have all the required workflows if you do not install all of them. - -3. Update Branch Protection -++++++++++++++++++++++++++++ - -The best and most maintainable way to have solid branch protection (:code:`Settings/Branches/main`) is to require the workflow :code:`CI / Allow Merge` to pass successfully. - -.. note:: - Setting the required status checks to pass before merging is only possible after running a CI build at **least once** on the affected branch. diff --git a/doc/user_guide/features/github_workflows/create_and_update.rst b/doc/user_guide/features/github_workflows/create_and_update.rst new file mode 100644 index 000000000..9aa3b7710 --- /dev/null +++ b/doc/user_guide/features/github_workflows/create_and_update.rst @@ -0,0 +1,66 @@ +.. _update_github_workflows: + +Creating and Updating the GitHub Workflows in Your Project +========================================================== + +PTB can initially generate the GitHub workflows in your project and also +update existing workflows. + +The workflows are based on Jinja templates with variables populated by the +PTB. The PTB reads the values from various places within your project, see +:ref:`template_variables`. + +Please note that the PTB only updates the values in the GitHub workflows when +*updating* the workflows. So, after updating the :ref:`list of Python versions +` in file ``noxconfig.py``, you need to :ref:`re-generate +` the GitHub workflows. + +Poetry Version +-------------- + +PTB has a default value for the Poetry version but you can override it in file +``noxconfig.py``, e.g. + +.. code-block:: shell + + PROJECT_CONFIG = Config( + dependency_manager=DependencyManager(name="poetry", version="2.3.0"), + ) + +.. _python_versions: + +Versions of Python and Exasol Docker DB +--------------------------------------- + +Many workflows are using a Build-matrix to iterate over multiple versions of +Python and/or the Exasol Docker DB. This is to make sure your code is valid, +free of bugs and working correctly for each combination of these items. + +PTB has a default for these versions, but you can override it in file +``noxconfig.py``, e.g. + +.. code-block:: shell + + PROJECT_CONFIG = Config( + python_versions=("3.10", "3.12), + exasol_versions=("7.1.30", "8.29.13", "2025.1.8"), + ) + +Some workflows are expected to not depend on a specific Python version and +will use only the lowest Python version in the list specified above. + +.. _update_workflows: + +Add all Workflows to Your Project +--------------------------------- + +.. code-block:: shell + + tbx workflow install all + +.. warning:: + #. If you already have various workflows, you may want to run the + :code:`update` command instead of the :code:`install` command. + + #. Some workflows depend on other workflows. Please ensure you have all + the required workflows if you do not install all of them. diff --git a/doc/user_guide/features/github_workflows/github_project_configuration.rst b/doc/user_guide/features/github_workflows/github_project_configuration.rst new file mode 100644 index 000000000..c962f2fb5 --- /dev/null +++ b/doc/user_guide/features/github_workflows/github_project_configuration.rst @@ -0,0 +1,49 @@ +.. _github_project_configuration: + +GitHub Project Configuration +============================ + +Branch Protection +----------------- + +The best and most maintainable way to have solid branch protection +(:code:`Settings/Branches/main`) is to require the workflow :code:`CI / Allow +Merge` to pass successfully. + +.. note:: + Setting the required status checks to pass before merging is only possible + after running a CI build at **least once** on the affected branch. + +Manual Approval +--------------- + +If your CI workflow involves slow or expensive steps you can guard these to be +executed only after manual approval. The CI workflow will automatically create +a GitHub environment named :code:`manual-approval`. You only need to add +reviewers in (:code:`Settings/Environments/manual-approval`) and move the +steps to be guarded into the related section in job :code:`slow-checks` in +file :code:`.github/workflows/merge-gate.yml`. + +Secrets +------- + +For accessing specific services in the Internet, your project often needs a +related *token* or other credentials. These credentials can be acquired by +registering on the service's website. + +In many cases your company or organization may manage the credentials +centrally and enable the use in multiple projects. The credentials can be +managed as Secrets in GitHub and can be made accessible to GitHub projects and +used by their workflows. + +In summary, your GitHub project may have individual secrets and/or secrets +inherited from the enclosing GitHub *Organization*. As soon as a secret is +accessible to your project, your GitHub workflows can use it and probably will +map it to *environment variables* that are used by your CI/CD automation. + +* **PYPI_TOKEN**: This secret is required to publish your project on the + `Python Package Index `_ (PyPi). Most projects will use the + org-secret. +* **SONAR_TOKEN**: See :ref:`Sonar Configuration `. + +.. _pypi: https://pypi.org/ diff --git a/doc/user_guide/features/github_workflows/index.rst b/doc/user_guide/features/github_workflows/index.rst index 52dedaf39..13a8b3d7c 100644 --- a/doc/user_guide/features/github_workflows/index.rst +++ b/doc/user_guide/features/github_workflows/index.rst @@ -1,14 +1,15 @@ .. _GitHub Workflows: -Enabling GitHub Workflows +GitHub Workflow Templates ========================= .. toctree:: - :maxdepth: 2 + :maxdepth: 1 :hidden: - configuration - troubleshooting + github_project_configuration + create_and_update + template_variables The PTB ships with configurable GitHub workflow templates covering the most common CI/CD setup variants for Python projects. The templates are defined in: @@ -26,16 +27,6 @@ workflows from the templates. In most cases, we recommend using _all_ workflows without change to ensure consistent interdependencies. For more details, see :ref:`ci_actions`. -Underlying the CLI, the PTB uses Jinja to dynamically generate project-specific -workflows. The rendering process is supported by the ``github_template_dict`` found in -your ``noxconfig.py::PROJECT_CONFIG``. This dictionary is inherited by default from -``BaseConfig.py``, ensuring a standardized baseline that can be easily overridden, if -necessary. - -.. literalinclude:: ../../../../exasol/toolbox/config.py - :language: python - :start-at: github_template_dict - Workflows --------- @@ -113,7 +104,7 @@ Pull Request When any pull request is opened, synchronized, or reopened, then the ``ci.yml`` will be triggered. -When configured as described on :ref:`github_workflows_configuration`, the +When configured as described on :ref:`github_project_configuration`, the ``run-slow-tests`` job requires a developer to manually approve executing the slower workflows, like ``slow-checks.yml``. This allows developers to update their pull request more often and to only periodically run the more time-expensive tests. diff --git a/doc/user_guide/features/github_workflows/template_variables.rst b/doc/user_guide/features/github_workflows/template_variables.rst new file mode 100644 index 000000000..f8355e3fa --- /dev/null +++ b/doc/user_guide/features/github_workflows/template_variables.rst @@ -0,0 +1,14 @@ +.. _template_variables: + +Template Variables +================== + +Underlying the CLI, the PTB uses Jinja to dynamically generate project-specific +workflows. The rendering process is supported by the ``github_template_dict`` found in +your ``noxconfig.py::PROJECT_CONFIG``. This dictionary is inherited by default from +``BaseConfig.py``, ensuring a standardized baseline that can be easily overridden, if +necessary. + +.. literalinclude:: ../../../../exasol/toolbox/config.py + :language: python + :start-at: github_template_dict diff --git a/doc/user_guide/features/github_workflows/troubleshooting.rst b/doc/user_guide/features/github_workflows/troubleshooting.rst deleted file mode 100644 index 1913048d4..000000000 --- a/doc/user_guide/features/github_workflows/troubleshooting.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _workflows_troubleshooting: - -Troubleshooting -=============== - -.. toctree:: - :maxdepth: 2 - - ../../troubleshooting/debug_github_workflows