owned this note
owned this note
Published
Linked with GitHub
# 1. 添加的依賴
* 使用 **postgresql 的 testcontainers**依賴
```xml=
<!-- https://mvnrepository.com/artifact/org.testcontainers/postgresql -->
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>postgresql</artifactId>
<version>1.19.3</version>
<scope>test</scope>
</dependency>
```
# 2. 其他準備
* 需要安裝docker
* 可提前下載對應DB的images
# 3. 基本範例
* **每一次個@Test測試都會重新啟動container**
* **類註解**
* **@SpringBootTest**:告訴 Spring Boot 測試框架啟動整個 Spring 應用程式上下文
* 自帶 @ExtendWith({SpringExtension.class})
* **SpringExtension**:**JUnit 5 + Spring 的整合入口**
* **它告訴 JUnit 5:「這個測試類需要 Spring 的支援」**
* 啟用 Spring TestContext Framework,才能使用
* @Autowired 注入 Bean
* @TestConfiguration 或 @Configuration 配置的 Bean
* Spring 的生命週期管理(初始化/銷毀 Bean)
* **@ActiveProfiles**:設定該測試類要使用的properties
* **@AutoConfigureMockMvc**:告訴 Spring Boot 在測試中自動設定一個 MockMvc 實例。
* MockMvc:模擬發HTTP Request
* **測試類步驟**
1. 標記類註解
2. 設定要啟動的幾個DB容器
3. 啟動並對容器做設定,使用 **@BeforeAll**
* postgres.**withDatabaseName**:設定DB名
* postgres.**withInitScript**:啟動後,要執行的SQL
3. 獲取 DB容器的信息,來設定properties
* 方法為**static**
* 方法 添加 **@DynamicPropertySource**註解
5. 寫測試
6. 關閉容器 **@AfterAll**
* **測試方法**
* 標記 **@Test**
* 使用 **@Sql** 初始化資料、清空資料
* **@SqlGroup** 整合多個 **@Sql**
* **@Sql**:
* **value[]**:設定sql檔案路徑(多個)
* **statements**:寫SQL
* **config**:設定該 **@Sql** 的DB相關設定
* 設定使用那一個數據源(BeanName):**@SqlConfig(dataSource = "CoreDatasource")**
* **executionPhase**:
* **Sql.ExecutionPhase.AFTER_TEST_METHOD**:方法執行前 執行
* **Sql.ExecutionPhase.AFTER_TEST_METHO**:方法執行後 執行
```java=
package com.eunodata.eunoai.configTemplate;
import com.eunodata.eunoai.core.adapter.dto.response.ConfigTemplateResDTO;
import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.springframework.test.context.jdbc.Sql;
import org.springframework.test.context.jdbc.SqlConfig;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
import org.testcontainers.containers.PostgreSQLContainer;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
@SpringBootTest
@ActiveProfiles({"test", "log", "source"})
@AutoConfigureMockMvc
public class ConfigTemplateControllerTest {
// 1. 建立DB容器,使用的 images >> "postgres:14-alpine"
// 1.1 建立DB容器一
static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>(
"postgres:14-alpine"
);
// 1.2 建立DB容器二
static PostgreSQLContainer<?> postgresAggregator = new PostgreSQLContainer<>(
"postgres:14-alpine"
);
//2. 設定DB 容器的相關設定
@BeforeAll
static void beforeAll() {
//2.1 指定DB 名稱
postgres.withDatabaseName("eunoai");
postgresAggregator.withDatabaseName("eunoai_aggregator");
//2.2 設定要執行的sql
//不需要添加 classpath:,底層使用 URL的getResource()方法,掃描 src/test/resources/**
postgres.withInitScript("schema.sql");
//3. 啟動容器
postgres.start();
postgresAggregator.start();
}
//5. 關閉容器
@AfterAll
static void afterAll() {
postgres.stop();
postgresAggregator.stop();
}
@Autowired
MockMvc mockMvc;
@TempDir // 測試時創建臨時目錄或文件,測試結束後會自動刪除。
protected static Path tempDirBy;
protected static Path tempDir;
// 3. 獲取 DB容器的信息,來設定properties
@DynamicPropertySource
static void configureProperties(DynamicPropertyRegistry registry) {
registry.add("spring.eunodata.eunoai.datasource.aggregator.jdbc-url", postgresAggregator::getJdbcUrl);
registry.add("spring.eunodata.eunoai.datasource.aggregator.username", postgresAggregator::getUsername);
registry.add("spring.eunodata.eunoai.datasource.aggregator.password", postgresAggregator::getPassword);
registry.add("spring.eunodata.eunoai.datasource.jdbc-url", postgres::getJdbcUrl);
registry.add("spring.eunodata.eunoai.datasource.username", postgres::getUsername);
registry.add("spring.eunodata.eunoai.datasource.password", postgres::getPassword);
registry.add("spring.eunodata.eunoai.datasource.log.jdbc-url", postgres::getJdbcUrl);
registry.add("spring.eunodata.eunoai.datasource.log.username", postgres::getUsername);
registry.add("spring.eunodata.eunoai.datasource.log.password", postgres::getPassword);
registry.add("spring.eunodata.datasource.source.jdbc-url", postgres::getJdbcUrl);
registry.add("spring.eunodata.datasource.source.username", postgres::getUsername);
registry.add("spring.eunodata.datasource.source.password", postgres::getPassword);
// 確保整個測試,路徑是一樣
if (tempDir == null) {
tempDir = tempDirBy;
}
registry.add("model.inventory", () -> tempDir + "/model-file/");
}
@Test
@Sql(value = "classpath:data/data.sql", config = @SqlConfig(dataSource = "CoreDatasource"))
@Sql(statements = "TRUNCATE TABLE config_template", config = @SqlConfig(dataSource = "CoreDatasource"), executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
void foo() throws Exception {
// [Arrange] 預期回傳的值
List<ConfigTemplateResDTO> expectedList = new ArrayList();
Map<String, Integer> optParam = new HashMap<>();
optParam.put("iteration", 60);
optParam.put("cross_validation", 5);
optParam.put("random_state", 27);
Map<String, List<Object>> paramDict = new HashMap<>();
paramDict.put("max_depth", List.of(2, 3, 4));
paramDict.put("learning_rate", List.of(0.01, 0.1, 0.2));
paramDict.put("n_estimators", List.of(100, 150, 200));
ConfigTemplateResDTO build = ConfigTemplateResDTO.builder()
.templateId("d6bdbcc156764b7a9c6b98d27e27fad8")
.templateName("house-XGB-dep-4")
.scaler("Standard Scaler")
.categoryName("REGRESSION")
.algorithmId(203L)
.algorithmName("XGBoost Regressor")
.paramDict(paramDict)
.optMethod("Grid Search")
.optParam(optParam)
.build();
expectedList.add(build);
// [Act] 模擬網路呼叫[GET] /api/todos
String returnString = mockMvc.perform(MockMvcRequestBuilders.get("/eunoapis/configTemplate/")
.accept(MediaType.APPLICATION_JSON))
.andExpect(status().isOk())
.andReturn().getResponse().getContentAsString(StandardCharsets.UTF_8);
List<ConfigTemplateResDTO> actualList = new Gson().fromJson(returnString, new TypeToken<List<ConfigTemplateResDTO>>() {
}.getType());
// [Assert] 判定回傳的body是否跟預期的一樣
assertEquals(expectedList.size(), actualList.size());
}
}
```
# 4. 單例範例
* **作用**:**一個測試類只啟動一次容器**
## 4.1 建立抽象類
* **用於容器設定\啟動**
```java=
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.containers.PostgreSQLContainer;
public abstract class AbstractIntegrationTest {
static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>(
"postgres:14-alpine"
);
static PostgreSQLContainer<?> postgresAggregator = new PostgreSQLContainer<>(
"postgres:14-alpine"
);
static {
postgres.withDatabaseName("eunoai");
postgresAggregator.withDatabaseName("eunoai_aggregator");
postgres.withInitScript("data/schema.sql");//不需要添加 classpath:,底層使用 URL的getResource()方法
postgresAggregator.start();
postgres.start();
}
@DynamicPropertySource
static void configureProperties(DynamicPropertyRegistry registry) {
registry.add("spring.eunodata.eunoai.datasource.aggregator.jdbc-url", postgresAggregator::getJdbcUrl);
registry.add("spring.eunodata.eunoai.datasource.aggregator.username", postgresAggregator::getUsername);
registry.add("spring.eunodata.eunoai.datasource.aggregator.password", postgresAggregator::getPassword);
registry.add("spring.eunodata.eunoai.datasource.jdbc-url", postgres::getJdbcUrl);
registry.add("spring.eunodata.eunoai.datasource.username", postgres::getUsername);
registry.add("spring.eunodata.eunoai.datasource.password", postgres::getPassword);
registry.add("spring.eunodata.eunoai.datasource.log.jdbc-url", postgres::getJdbcUrl);
registry.add("spring.eunodata.eunoai.datasource.log.username", postgres::getUsername);
registry.add("spring.eunodata.eunoai.datasource.log.password", postgres::getPassword);
registry.add("spring.eunodata.datasource.source.jdbc-url", postgres::getJdbcUrl);
registry.add("spring.eunodata.datasource.source.username", postgres::getUsername);
registry.add("spring.eunodata.datasource.source.password", postgres::getPassword);
}
}
```
## 4.2 繼承抽象類
* **開發測試方法**
```java=
import com.eunodata.eunoai.core.adapter.dto.response.ConfigTemplateResDTO;
import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.jdbc.Sql;
import org.springframework.test.context.jdbc.SqlConfig;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
@SpringBootTest
@ActiveProfiles({"test", "log", "source"})
@AutoConfigureMockMvc
public class ConfigTemplateControllerTest extends AbstractIntegrationTest {
@Autowired
MockMvc mockMvc;
@Test
@Sql(value = "classpath:data/data.sql", config = @SqlConfig(dataSource = "CoreDatasource"))
@Sql(statements = "TRUNCATE TABLE config_template", config = @SqlConfig(dataSource = "CoreDatasource"), executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
void foo1() throws Exception {
// [Arrange] 預期回傳的值
List<ConfigTemplateResDTO> expectedList = new ArrayList();
Map<String, Integer> optParam = new HashMap<>();
optParam.put("iteration", 60);
optParam.put("cross_validation", 5);
optParam.put("random_state", 27);
Map<String, List<Object>> paramDict = new HashMap<>();
paramDict.put("max_depth", List.of(2.0, 3.0, 4.0));
paramDict.put("learning_rate", List.of(0.01, 0.1, 0.2));
paramDict.put("n_estimators", List.of(100.0, 150.0, 200.0));
ConfigTemplateResDTO build = ConfigTemplateResDTO.builder()
.templateId("d6bdbcc156764b7a9c6b98d27e27fad8")
.templateName("house-XGB-dep-4")
.scaler("Standard Scaler")
.categoryName("REGRESSION")
.algorithmId(203L)
.algorithmName("XGBoost Regressor")
.paramDict(paramDict)
.optMethod("Grid Search")
.optParam(optParam)
.build();
expectedList.add(build);
// [Act] 模擬網路呼叫[GET] /api/todos
String returnString = mockMvc.perform(MockMvcRequestBuilders.get("/eunoapis/configTemplate/")
.accept(MediaType.APPLICATION_JSON))
.andExpect(status().isOk())
.andReturn().getResponse().getContentAsString(StandardCharsets.UTF_8);
List<ConfigTemplateResDTO> actualList = new Gson().fromJson(returnString, new TypeToken<List<ConfigTemplateResDTO>>() {
}.getType());
// [Assert] 判定回傳的body是否跟預期的一樣
assert expectedList.containsAll(actualList);
}
}
```
# 5. 參考網址
* **基本範例**:https://testcontainers.com/guides/testing-spring-boot-rest-api-using-testcontainers/
* 每一個測試類都會啟動\關閉容器
* **單例範例**:https://testcontainers.com/guides/testcontainers-container-lifecycle/
* 測試只啟動一次容器
* **gitLab CI/CD**:https://java.testcontainers.org/supported_docker_environment/continuous_integration/gitlab_ci/
# 6. 其他
## 6.1 @sql的執行時間點
* 發現點:在@BeforeAll 創建檔案,並由@sql讀取
* @sql 無法讀取到檔案
* @sql的默認的執行點 **早於** @BeforeEach
## 6.2 @Sql 多數據源下使用
* 要求:指定 數據源\事務控制器
* 不同的ORM架構下,需要**使用對應的事物控制器**,且`其他的配置需為符合該ORM`
* **錯誤範例**:使用 JPA的事物控制器(`JpaTransactionManager`),卻未配置EntityMangerFactory
* **結果**:當使用getBean()`取出該事物控制器的實例`時,該配置的datasource`不是原本設定的datasource`,而是其他完整配置JAP中所使用的datasource。
### 6.2.1 基本使用
* **前提**:共配置**兩個數據源**,`一個JAP,一個JDBC`
* **JAP**:
* 數據源的BeanName:**CoreDatasource**
* 事務控制器:
* BeanName:**CoreTransactionManager**
* 使用的事物控制器:**JpaTransactionManager**
* **JDBC**:
* 數據源的BeanName:**CoreSecondDatasource**
* 事務控制器:
* BeanName:**CoreSecondTransactionManager**
* 使用的事物控制器:**DataSourceTransactionManager** (`Mybatis也使用`)
```java=
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.jdbc.Sql;
import org.springframework.test.context.jdbc.SqlConfig;
import org.springframework.test.context.jdbc.SqlGroup;
@SpringBootTest
public class DatasetControllerTest {
@Test
@SqlGroup(value = {
@Sql(scripts = "classpath:data/drop.sql",
config = @SqlConfig(dataSource = "CoreDatasource",
transactionManager = "CoreTransactionManager",
transactionMode = SqlConfig.TransactionMode.ISOLATED),
executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD),
@Sql(scripts ="classpath:data/drop_aggregator.sql" ,
config = @SqlConfig(dataSource = "CoreSecondDatasource",
transactionManager = "CoreSecondTransactionManager",
transactionMode = SqlConfig.TransactionMode.ISOLATED),
executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
})
void getAllDataset() throws Exception {
...
}
}
```
## 6.3 testcontiner的URL
* **本機**:localhost
* **gitLab的CI/CD**:docker
* JAVA中獲取URL+port的正則:`[a-z]+:[1-9]\\d+`
## 6.4 Gitlab CI/CD 配置
* 在需要執行 test的job中要添加
* **services屬性**,屬性值:**docker:dind**
```yml=
build-mvn:
stage: build
image: maven:3.9.5-eclipse-temurin-11-alpine
services:
- docker:dind
```
* 完整配置
```yml
services:
- name: docker:dind
command: ["--tls=false"]
variables:
DOCKER_HOST: "tcp://docker:2375"
DOCKER_TLS_CERTDIR: ""
DOCKER_DRIVER: overlay2
MAVEN_OPTS: >-
-Dhttps.protocols=TLSv1.2
-Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository
-Dorg.slf4j.simpleLogger.showDateTime=true
-Djava.awt.headless=true
MAVEN_CLI_OPTS: >-
--batch-mode
--errors
--fail-at-end
--show-version
--no-transfer-progress
image: maven:3-eclipse-temurin-19
cache:
paths:
- .m2/repository
test:
stage: test
script:
- 'mvn $MAVEN_CLI_OPTS verify'
```
* DOCKER_HOST: "tcp://docker:2375"
* DOCKER_DRIVER: overlay2
###### tags:`SpringBoot` `Test` `TestContainer ` `Mockit`