Skip to content

Add @skip-<dbtype> tags for database-specific test exclusion #302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
swissspidy merged 3 commits into from
Feb 8, 2026
+85 −2
Merged

Add @skip-<dbtype> tags for database-specific test exclusion #302

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
67f4571
Initial plan
Copilot Feb 8, 2026
7870314
Add support for @skip-mysql, @skip-mariadb, @skip-sqlite tags
Copilot Feb 8, 2026
c7eb2e7
Add test scenarios demonstrating @skip-<dbtype> tags in features/test…
Copilot Feb 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions features/testing.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,34 @@ Feature: Test that WP-CLI loads.
"""
6.3.1
"""

@skip-mysql
Scenario: Skip on MySQL databases
Given a WP install

When I run `wp eval 'echo "This should only run on MariaDB or SQLite";'`
Then STDOUT should contain:
"""
This should only run on MariaDB or SQLite
"""
Comment on lines +84 to +92
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These scenarios are intended to demonstrate/verify @skip-* behavior, but the steps/assertions are database-agnostic (they just echo a static string). If the skip-tag implementation regresses and scenarios run on the “skipped” DB, they will still pass and won’t catch it. Consider making each scenario contain a DB-specific step that would fail on the skipped DB (e.g., combine @require-mysql-or-mariadb with @skip-mysql and assert SELECT VERSION() contains MariaDB, and use @skip-sqlite with a SHOW TABLES assertion).

Copilot uses AI. Check for mistakes.

@skip-mariadb
Scenario: Skip on MariaDB databases
Given a WP install

When I run `wp eval 'echo "This should only run on MySQL or SQLite";'`
Then STDOUT should contain:
"""
This should only run on MySQL or SQLite
"""
Comment on lines +94 to +102
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This scenario’s assertion is a static echo, so it will pass on any database if @skip-mariadb is not applied correctly. To actually validate @skip-mariadb, make the scenario fail on MariaDB (or assert it is running on MySQL) using a DB-specific query/output check.

Copilot uses AI. Check for mistakes.

@skip-sqlite
Scenario: Skip on SQLite databases
Given a WP install

When I run `wp eval 'echo "This should only run on MySQL or MariaDB";'`
Then STDOUT should contain:
"""
This should only run on MySQL or MariaDB
"""
Comment on lines +104 to +112
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue here: the scenario will succeed regardless of DB type because it only checks a static string. If the goal is to ensure @skip-sqlite works, consider using a query that would fail on SQLite (e.g. SHOW TABLES) so the scenario fails if it accidentally runs under SQLite.

Copilot uses AI. Check for mistakes.

51 changes: 51 additions & 0 deletions tests/tests/TestBehatTags.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,16 +66,19 @@ public function test_behat_tags_wp_version_github_token( $env, $expected ): void
case 'mariadb':
$expected .= '&&~@require-mysql';
$expected .= '&&~@require-sqlite';
$expected .= '&&~@skip-mariadb';
break;
case 'sqlite':
$expected .= '&&~@require-mariadb';
$expected .= '&&~@require-mysql';
$expected .= '&&~@require-mysql-or-mariadb';
$expected .= '&&~@skip-sqlite';
break;
case 'mysql':
default:
$expected .= '&&~@require-mariadb';
$expected .= '&&~@require-sqlite';
$expected .= '&&~@skip-mysql';
break;
}

Expand Down Expand Up @@ -150,16 +153,19 @@ public function test_behat_tags_php_version(): void {
case 'mariadb':
$expected .= '&&~@require-mysql';
$expected .= '&&~@require-sqlite';
$expected .= '&&~@skip-mariadb';
break;
case 'sqlite':
$expected .= '&&~@require-mariadb';
$expected .= '&&~@require-mysql';
$expected .= '&&~@require-mysql-or-mariadb';
$expected .= '&&~@skip-sqlite';
break;
case 'mysql':
default:
$expected .= '&&~@require-mariadb';
$expected .= '&&~@require-sqlite';
$expected .= '&&~@skip-mysql';
break;
}

Expand Down Expand Up @@ -187,16 +193,19 @@ public function test_behat_tags_extension(): void {
case 'mariadb':
$expecteds[] = '~@require-mysql';
$expecteds[] = '~@require-sqlite';
$expecteds[] = '~@skip-mariadb';
break;
case 'sqlite':
$expecteds[] = '~@require-mariadb';
$expecteds[] = '~@require-mysql';
$expecteds[] = '~@require-mysql-or-mariadb';
$expecteds[] = '~@skip-sqlite';
break;
case 'mysql':
default:
$expecteds[] = '~@require-mariadb';
$expecteds[] = '~@require-sqlite';
$expecteds[] = '~@skip-mysql';
break;
}

Expand Down Expand Up @@ -231,19 +240,22 @@ public function test_behat_tags_db_version(): void {
$contents = "@require-mariadb-$minimum_db_version @less-than-mariadb-$db_version";
$expecteds[] = '~@require-mysql';
$expecteds[] = '~@require-sqlite';
$expecteds[] = '~@skip-mariadb';
$expecteds[] = "~@require-mariadb-$minimum_db_version";
$expecteds[] = "~@less-than-mariadb-$db_version";
break;
case 'sqlite':
$expecteds[] = '~@require-mariadb';
$expecteds[] = '~@require-mysql';
$expecteds[] = '~@require-mysql-or-mariadb';
$expecteds[] = '~@skip-sqlite';
break;
case 'mysql':
default:
$contents = "@require-mysql-$minimum_db_version @less-than-mysql-$db_version";
$expecteds[] = '~@require-mariadb';
$expecteds[] = '~@require-sqlite';
$expecteds[] = '~@skip-mysql';
$expecteds[] = "~@require-mysql-$minimum_db_version";
$expecteds[] = "~@less-than-mysql-$db_version";
break;
Expand All @@ -255,4 +267,43 @@ public function test_behat_tags_db_version(): void {
$output = exec( "cd {$this->temp_dir}; php $behat_tags" );
$this->assertSame( $expected, $output );
}

public function test_behat_tags_skip_db_type(): void {
$env_github_token = getenv( 'GITHUB_TOKEN' );
$db_type = getenv( 'WP_CLI_TEST_DBTYPE' );

putenv( 'GITHUB_TOKEN' );

$behat_tags = dirname( dirname( __DIR__ ) ) . '/utils/behat-tags.php';

file_put_contents( $this->temp_dir . '/features/skip_db.feature', '@skip-mysql @skip-mariadb @skip-sqlite' );

Comment on lines +279 to +280
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

behat-tags.php adds the DB-specific @skip-* exclusions unconditionally based on the active DB type; this test doesn’t need to write a feature file containing @skip-* tags for the output to change. Consider removing this file_put_contents() (or adding a brief comment explaining why it’s needed) to avoid implying the script parses @skip-* tags from feature files.

Suggested change
file_put_contents( $this->temp_dir . '/features/skip_db.feature', '@skip-mysql @skip-mariadb @skip-sqlite' );

Copilot uses AI. Check for mistakes.
$expecteds = array();

switch ( $db_type ) {
case 'mariadb':
$expecteds[] = '~@require-mysql';
$expecteds[] = '~@require-sqlite';
$expecteds[] = '~@skip-mariadb';
break;
case 'sqlite':
$expecteds[] = '~@require-mariadb';
$expecteds[] = '~@require-mysql';
$expecteds[] = '~@require-mysql-or-mariadb';
$expecteds[] = '~@skip-sqlite';
break;
case 'mysql':
default:
$expecteds[] = '~@require-mariadb';
$expecteds[] = '~@require-sqlite';
$expecteds[] = '~@skip-mysql';
break;
}

$expected = '--tags=' . implode( '&&', array_merge( array( '~@github-api', '~@broken' ), $expecteds ) );
$output = exec( "cd {$this->temp_dir}; php $behat_tags" );
$this->assertSame( $expected, $output );

putenv( false === $env_github_token ? 'GITHUB_TOKEN' : "GITHUB_TOKEN=$env_github_token" );
}
}
5 changes: 3 additions & 2 deletions utils/behat-tags.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ function get_db_version() {
case 'mariadb':
$skip_tags = array_merge(
$skip_tags,
[ '@require-mysql', '@require-sqlite' ],
[ '@require-mysql', '@require-sqlite', '@skip-mariadb' ],
version_tags( 'require-mariadb', $db_version, '<', $features_folder ),
version_tags( 'less-than-mariadb', $db_version, '>=', $features_folder )
);
Expand All @@ -182,12 +182,13 @@ function get_db_version() {
$skip_tags[] = '@require-mariadb';
$skip_tags[] = '@require-mysql';
$skip_tags[] = '@require-mysql-or-mariadb';
$skip_tags[] = '@skip-sqlite';
break;
case 'mysql':
default:
$skip_tags = array_merge(
$skip_tags,
[ '@require-mariadb', '@require-sqlite' ],
[ '@require-mariadb', '@require-sqlite', '@skip-mysql' ],
version_tags( 'require-mysql', $db_version, '<', $features_folder ),
version_tags( 'less-than-mysql', $db_version, '>=', $features_folder )
);
Expand Down