Don’t Invent Data for Integration Tests, Synthesize It
 | Ivan Ponomarev
|
What do we want from tests?
Clean & neat
Fast running
Easy to write and maintain
What do we want from tests?
Clean & neat
Fast running
Easy to write and maintain
The tech stack
RDBMS (PostgreSQL or others)
Backend: Java/Kotlin + Spring Boot
Restful API, frontend/other services: out of scope
A very traditional architecture
A slippery terminology: E2E test
A slippery terminology: integration test
(sometimes called E2E)
Unit Test
A slippery terminology: component test
(sometimes called integration or even unit test)
Testcontainers
Application under test
Methods under test
TalkDao.getTalksByConference(Conference conference)
Must return a set of Talk object together with related
Speaker objects and Conference object
Methods under test
TalkService.changeStatus(in talkId, Status status)
No talk should be rejected without a feedback!
Approach #1: The Golden Dump
Spinning up a subset of beans
Spinning up a subset of beans
Spinning up a subset of beans
Spinning up a subset of beans
The Golden Dump Pros and Cons
A straighforward approach
Fragile: init script can significantly diverge from the actual schema
The Golden Dump Pros and Cons
- Obscure: what does this mean?
insert into talkspeakers (talkid, speakerid) values (1001, 1004);
insert into talkspeakers (talkid, speakerid) values (1002, 1002);
insert into talkspeakers (talkid, speakerid) values (1002, 1003);The Golden Dump Pros and Cons
- Poor cohesion: tests rely on initialization script which is in a separate file far away.
Test setup relying on the script:
//A talk with a feedback
int id = 1001;
//A talk without a feedback
int id = 1002;The Golden Dump Pros and Cons
- Poor cohesion: tests rely on initialization script which is in a separate file far away.
Assertions relying on the script:
assertThat(talk.getName()).isEqualTo(
"Reactive, or not reactive: that is the question");
assertThat(talk.getSpeakers().stream().map(Speaker::getName))
.containsExactlyInAnyOrder("Evgeny Borisov", "Kirill Tolkachev");
assertThat(talk.getStatus()).isEqualTo(IN_REVIEW);The Golden Dump Pros and Cons
- Might be easy to write for the fist time, but time consuming to maintain.
Let’s count lines of code
Golden Dump | |
SQL | 56 |
DaoTest | 58 |
ServiceTest | 40 |
Total | 154 |
Approach #2: Object Mother
Object Mother Pros and Cons
- No SQL scripts, total decoupling from schema migrations.
- Type safe!
Object Mother Pros and Cons
- Good cohesion: you can give your fixtures the meaningful names, it’s easy to understand what you test.
service.changeStatus(
talkWithFeedback().getId(),
Status.REJECTED);Object Mother Pros and Cons
We have to pre-fill database with objects before the tests
We use our own DAOs- We have to care about cleaning up the database after each test
(which is ok, but can be slow)
Object Mother Pros and Cons
- It’s relatively easy to maintain ObjectMother class (thanks to type safety),
but "inventing" objects with all their attributes can be difficult
Let’s count lines of code
Golden Dump | Object Mother | |
SQL | 56 | |
ObjectMother | 50 | |
DaoTest | 58 | 64 |
ServiceTest | 40 | 65 |
Total | 154 | 179 |
Approach #3: synthesized data
Synthesized Data Pros and Cons
- We are using all the benefits of Object Mother approach, and
- no need to "invent" test examples
- no need to pre-fill the database using our own DAOs
Synthesized Data Pros and Cons
- We have to provide the config for TDK, but it’s less prone to changes.
- Little or no control for what’s actually in your database, we have to modify values occasionally using our DAO
Let’s count lines of code
Golden Dump | Object Mother | TDK | |
Config | 15 | ||
SQL | 56 | ||
ObjectMother | 50 | 36 | |
DaoTest | 58 | 64 | 47 |
ServiceTest | 40 | 65 | 44 |
Total | 154 | 179 | 142 |
TDK features
No-op
# Just copy everything from the source to the target
default_config:
mode: KEEPData subsetting
# Take 50% of the rows from the source database,
# do not transform the data
default_config:
mode: KEEP
target_ratio: 0.5
table_truncation_mode: TRUNCATE
schema_creation_mode: CREATE_IF_NOT_EXISTSData subsetting
# Take 50% of the rows and mask them,
# but keep public.productlines as is
default_config:
mode: MASKING
target_ratio: 0.5
tables:
- table_name_with_schema: "public.productlines"
mode: "KEEP"
target_ratio: 1
table_truncation_mode: TRUNCATE
schema_creation_mode: CREATE_IF_NOT_EXISTS
safety_mode: "RELAXED"Data masking: Before
| CUSTOMERNUMBER | CUSTOMERNAME | CONTACTLASTNAME | CONTACTFIRSTNAME | PHONE | ADDRESSLINE1 |
|---|---|---|---|---|---|
103 | Atelier graphique | Schmitt | Carine | 40.32.2555 | 54, rue Royale |
112 | Signal Gift Stores | King | Jean | 5551838 | 8489 Strong St. |
114 | Australian Collectors, Co. | Ferguson | Peter | 03 9520 4555 | 636 St Kilda Road |
119 | La Rochelle Gifts | Labrune | Janine | 40.67.8555 | 67, rue des Cinquante Otages |
Data masking: After
| CUSTOMERNUMBER | CUSTOMERNAME | CONTACTLASTNAME | CONTACTFIRSTNAME | PHONE | ADDRESSLINE1 |
|---|---|---|---|---|---|
7604416 | Fwomnubri Jdorefqpfjn Vepbwfoe | Iylapd | Wqvj | 7010-370953 | Lmmejweaooyha 954 |
29919216 | Ubkcg & Hhitqfm Wz | Hjpxftbrdybfxev | Rgykous | (72) 478-1967 | um. Akseehys 97 |
65286067 | Pwaibpu Lmypt | Llsrsmu | Grrnbqbi | 281-942530 | Sqy Teyfbglp qo Cdqo 21 |
68022786 | Fudqrgjjycmz Sau Gsct Dms. | Ywsyew | Uurxn | 5919048124 | 7833 Xtuvvhu Gb. |
Data generation
# Take the source database
# and generate the database twice as big with the same schema
default_config:
mode: GENERATION
target_ratio: 2
table_truncation_mode: TRUNCATE
schema_creation_mode: CREATE_IF_NOT_EXISTS
safety_mode: "RELAXED"Data generation
# Same as above,
# but use specific generator for `public.products` table
default_config:
mode: GENERATION
target_ratio: 2
tables:
- table_name_with_schema: "public.products"
mode: "GENERATION"
target_ratio: 2
transformations:
- columns: [ "productname" ]
params:
type: "formatted_string_generator"
pattern: "[A-Z]{16}"A couple of dozens generators
Pagila Demo
Thanks for listening!
Give tdk-tc a 
!
@inponomarev