Migrated MariaDB, MySQL, PostgreSQL tests to use testcontainers to simplify local setup by koperagen · Pull Request #2011 · Kotlin/dataframe

koperagen · 2026-07-28T15:54:55Z

Tests still will not be run on CI for now. But it'll be now possible to run MariaDB, MySQL, PostgreSQL in one click, providing you have docker installed on PC. It's the only prerequisite.
Let's start with those and migrate imbd, mssql example after

For reference. Looks like their JUnit4 examples are a bit outdated and nowadays modern "container" classes do not implement "testrule" class, so we use manual lifecycle management instead
https://java.testcontainers.org/modules/databases/mysql/
https://java.testcontainers.org/modules/databases/mariadb/
https://java.testcontainers.org/modules/databases/postgres/

…rly named types

koperagen · 2026-07-28T15:59:27Z

-                    st.setTime(16, java.sql.Time(System.currentTimeMillis()))
+                    st.setTimestamp(14, SqlTimestamp(System.currentTimeMillis()))
+                    st.setTimestamp(15, SqlTimestamp(System.currentTimeMillis()))
+                    st.setTime(16, SqlTime(System.currentTimeMillis()))


Automatic refactorings break such fully qualified links :( I recovered it as import alias, should survive file moves etc

Copilot

🟡 Not ready to approve

It currently introduces a dependency-resolution issue in the version catalog and a Kotlin compilation error from an unused import.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR migrates the JDBC integration tests for MariaDB/MySQL/PostgreSQL to Testcontainers so contributors can run them locally with Docker, while keeping CI’s default test task free of Docker-dependent tests.

Changes:

Added Testcontainers dependencies to the version catalog and dataframe-jdbc test configuration.
Updated MariaDB/MySQL/PostgreSQL test suites to start/stop containers in @BeforeClass/@AfterClass rather than relying on localhost DBs.
Added a dedicated testcontainersTest Gradle task and excluded the Testcontainers package from the default test task.

File summaries

File	Description
gradle/libs.versions.toml	Adds Testcontainers version + catalog entries for DB modules.
dataframe-jdbc/build.gradle.kts	Adds Testcontainers test deps; excludes Docker tests from `test` and introduces `testcontainersTest`.
dataframe-jdbc/src/test/kotlin/org/jetbrains/kotlinx/dataframe/io/testcontainers/postgresTest.kt	Switches Postgres tests to a managed `PostgreSQLContainer`.
dataframe-jdbc/src/test/kotlin/org/jetbrains/kotlinx/dataframe/io/testcontainers/postgresConnectionUrlTest.kt	Reworks Postgres URL parsing tests to run against a container.
dataframe-jdbc/src/test/kotlin/org/jetbrains/kotlinx/dataframe/io/testcontainers/mysqlTest.kt	Switches MySQL tests to a managed `MySQLContainer`.
dataframe-jdbc/src/test/kotlin/org/jetbrains/kotlinx/dataframe/io/testcontainers/mariadbTest.kt	Switches MariaDB tests to a managed `MariaDBContainer`.
dataframe-jdbc/src/test/kotlin/org/jetbrains/kotlinx/dataframe/io/testcontainers/images.kt	Centralizes Docker image tags used by the Testcontainers tests.

Review details

Comments suppressed due to low confidence (2)

dataframe-jdbc/src/test/kotlin/org/jetbrains/kotlinx/dataframe/io/testcontainers/mysqlTest.kt:20

Unused import org.junit.Ignore will fail Kotlin compilation (unused imports are errors). The @Ignore annotation was removed, so this import should be removed too.
dataframe-jdbc/src/test/kotlin/org/jetbrains/kotlinx/dataframe/io/testcontainers/postgresTest.kt:30
Unused import org.junit.Ignore will fail Kotlin compilation (unused imports are errors). The @Ignore annotation was removed, so this import should be removed too.

Files reviewed: 7/7 changed files
Comments generated: 1
Review effort level: Low

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

+testcontainers-postgresql = { group = "org.testcontainers", name = "testcontainers-postgresql", version.ref = "testcontainers" }
+testcontainers-mysql = { group = "org.testcontainers", name = "testcontainers-mysql", version.ref = "testcontainers" }
+testcontainers-mariadb = { group = "org.testcontainers", name = "testcontainers-mariadb", version.ref = "testcontainers" }


zaleslaw · 2026-07-29T11:16:25Z

@copilot will this work on Windows or on Team City?
Could we refactor it and keep both the local and test container versions of tests, but call them in two forms - Docker Container and run on Local Database, don't mind about @ignore tag - it doesn't play any role

zaleslaw · 2026-07-29T11:50:32Z

https://gh.io/copilot-coding-agent-docs will this work on Windows or on Team City?

Jolanrensen · 2026-07-29T12:12:57Z

Did you test it with it already on TC?
You can configure the gradle build-step to run in a certain Docker container like dockerImage = "cimg/android:2026.02.1" I assume, like this we could make test-containers work as well.

See https://www.jetbrains.com/help/teamcity/2026.1/gradle.html?TeamCity%20Documentation#Container+Settings and https://www.jetbrains.com/help/teamcity/2026.1/container-wrapper.html?TeamCity%20Documentation

(locally, they seem to run well, btw!)

Jolanrensen · 2026-07-29T12:17:15Z

+package org.jetbrains.kotlinx.dataframe.io.testcontainers
+
+// Available image tags: https://hub.docker.com/_/mariadb/tags
+const val MARIADB_IMAGE = "mariadb:12.3.2"


I feel like these should be configurable higher up, like in Gradle with BuildConfig. If we need to bump them, it will be difficult to find them so deeply nested

I think the question here is how often we expect to update them. It's not really a dependency exposed to our users, only a test database instance we'll use to make sure our changes to JDBC module do not break anything on isolated environment. I picked all very fresh versions that are expected to be supported for 5 years.
If we're interested, i can add renovatebot with custom rule to watch over these versions right here in images.kt. That's the only common practice i could find regarding where image versions should be stored. I believe other just keep it in sync with their production database, which is not applicable to us

Shouldn't they be updated along with the library versions of the databases?

Ah no, they're different, it seems. Still, it's a test dependency from an external source, which makes it little different than other testImplementation dependencies. It can be taken offline, updated, etc. so hiding it in a source file is probably not the best way to go

I don't think so, i'd expect a very thorough backward compatibility

https://jdbc.postgresql.org/download/
This is the current version of the driver. Unless you have unusual requirements (running old applications or JVMs), this is the driver you should be using. It supports PostgreSQL 8.4 or newer and requires Java 6 or newer. It contains support for SSL and the javax.sql package.

https://mariadb.com/docs/connectors/mariadb-connector-j/about-mariadb-connector-j
MariaDB Connector/J is compatible with all MariaDB and MySQL server versions.

MariaDB Connector/J releases older than 1.2.0 may be compatible with server versions older than MySQL 5.5, but those MariaDB Connector/J releases aren't supported anymore.

https://dev.mysql.com/doc/connector-j/en/connector-j-versions.html
MySQL Server versions: Connector/J 26.7 supports MySQL 8.0 and up. ( first released 19 April 2018)

So we go in other direction and need to update only if we want to test new feature that is supported only in most recent database

Tbh i don't really mind moving it to buildconfid altogether. Mostly wanted to clarify my impression that updating those is not the same as updating usual dependencies, we probably shouldn't do it frequently even if it ends up in the version catalog

I understand :) It's comparable to us sticking to an older JUnit version since it covers all we need in our tests. Even though we don't bump it often, it will be bumped at some point, so it's nice to know where the single source-of-truth regarding versions exists in the project

koperagen · 2026-07-29T12:58:15Z

@copilot will this work on Windows or on Team City? Could we refactor it and keep both the local and test container versions of tests, but call them in two forms - Docker Container and run on Local Database, don't mind about @ignore tag - it doesn't play any role

bad copilot : ( why did it have to commit things to my branch and break CI. Besised, i'd do abstract test class and have LocalMariaDb tests + DockerMariaDb tests instead

koperagen · 2026-07-29T13:00:14Z

Did you test it with it already on TC?
will this work on Windows or on Team City?

On TeamCity eventually yes, need to configure teamcity to run on agents that have docker daemon

…nections

zaleslaw · 2026-07-29T19:49:53Z

Great job! Will test it next week on both MacOS and WIndows, locally and not, and it will be shipped!

koperagen added 7 commits July 27, 2026 19:40

Migrate postresql tests to testcontainers

df693b4

Migrate mysql tests to testcontainers

13b0747

Migrate mariadb tests to testcontainers

514283e

Move testcontainers tests under common package

50e731f

Extract image versions in common file & update to latest LTS

db02bc8

Configure task to run all testcontainer tests separately

6ced75a

Restore qualifier for JDBC types to avoid confusion with other simila…

15b656f

…rly named types

koperagen requested a review from zaleslaw July 28, 2026 15:54

koperagen commented Jul 28, 2026

View reviewed changes

zaleslaw requested a review from Copilot July 29, 2026 11:00

Copilot started reviewing on behalf of zaleslaw July 29, 2026 11:01 View session

Copilot AI reviewed Jul 29, 2026

View reviewed changes

Copilot started work on behalf of zaleslaw July 29, 2026 11:17 View session

Copilot finished work on behalf of zaleslaw July 29, 2026 11:28

Jolanrensen reviewed Jul 29, 2026

View reviewed changes

koperagen added 2 commits July 29, 2026 20:01

Refactor DB tests to allow both Docker and locally installed DBMS con…

3ce079d

…nections

Move image versions to libs.versions.toml + BuildConfig

7af1db1

koperagen force-pushed the testcontainers branch from 89fcba0 to 7af1db1 Compare July 29, 2026 17:27

Uh oh!

Conversation

koperagen commented Jul 28, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Not ready to approve

Comments suppressed due to low confidence (2)

Uh oh!

zaleslaw commented Jul 29, 2026

Uh oh!

zaleslaw commented Jul 29, 2026

Uh oh!

Jolanrensen commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

koperagen Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

koperagen commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

koperagen commented Jul 29, 2026

Uh oh!

zaleslaw commented Jul 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Jolanrensen commented Jul 29, 2026 •

edited

Loading

koperagen Jul 29, 2026 •

edited

Loading

koperagen commented Jul 29, 2026 •

edited

Loading