Skip to content

Use postgres for CI run_unit_tests#286

Merged
knikolla merged 2 commits intonerc-project:mainfrom
QuanMPhm:254/postgres
Feb 12, 2026
Merged

Use postgres for CI run_unit_tests#286
knikolla merged 2 commits intonerc-project:mainfrom
QuanMPhm:254/postgres

Conversation

@QuanMPhm
Copy link
Contributor

Closes #254.

A Postgres-specific bug was found regarding creating Field Of Science entries. More details in tests/unit/test_migrate_field_of_science.py.

Copy link
Collaborator

@knikolla knikolla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this still allow to optionally run the tests with SQLite? Running tests with Postgres should be an additional thing rather than completely removing SQLite.

@QuanMPhm
Copy link
Contributor Author

QuanMPhm commented Feb 12, 2026

@knikolla I've made the one test case in src/coldfront_plugin_cloud/tests/unit/test_migrate_field_of_science.py check if DB_URL is set, in which case it will run the postgres specific SQL commands. Otherwise, all the other tests can still be ran with SQLite, you just need to not set DB_URL

Copy link
Collaborator

@knikolla knikolla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before the introduction of this change, you could run setup.sh followed by run_unit_tests.sh in your local machine without having to setup anything externally and they would work on both Mac and Linux. These changes break that.

Please preserve original behavior in the original functioning of the scripts.

For example, maybe add the postgresql user and db configuration and DB_URL variable export within the GitHub action workflow, rather than the script file itself. This way the CI in GitHub will use postgres whereas when a user is testing in their local machine and running the file they will still work as before.

The production Coldfront uses Postgres.
The tests will now more accurately reflect the production environment.
Local testing can still be done with SQLite by not
setting the `DB_URL` environment variable.

A Postgres-specific bug was found regarding creating Field Of Science
entries. More details in `tests/unit/test_migrate_field_of_science.py`.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request migrates the CI testing infrastructure from MySQL to PostgreSQL to better match the production environment. The change addresses a specific bug where Postgres enforces character limits on database fields that MySQL doesn't, which was discovered when trying to add new allocation attributes.

Changes:

  • Adds Postgres support to CI workflows by setting the DB_URL environment variable
  • Installs and configures PostgreSQL service in the CI setup script
  • Fixes a Postgres-specific sequence synchronization issue for FieldOfScience model tests
  • Consolidates CI setup across workflows to use a common setup.sh script

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
test-requirements.txt Adds psycopg2 package for PostgreSQL database connectivity
src/coldfront_plugin_cloud/tests/unit/test_migrate_field_of_science.py Adds setUp method to reset Postgres sequence for FieldOfScience IDs to prevent insertion conflicts
ci/setup.sh Starts PostgreSQL service and sets password when running in CI environment
.github/workflows/test-unit.yaml Adds DB_URL environment variable pointing to Postgres, updates step name
.github/workflows/test-functional-microstack.yaml Adds DB_URL environment variable, consolidates setup to use setup.sh script, updates step name
.github/workflows/test-functional-microshift.yaml Adds DB_URL environment variable for consistency with other workflows

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@knikolla knikolla merged commit 1a93876 into nerc-project:main Feb 12, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Change to using Postgres database for testing

2 participants