fix:sample/plate 之前的开发
This commit is contained in:
彭帅
290
docs/architecture/02-phenotyping-data-flow.md
Normal file
290
docs/architecture/02-phenotyping-data-flow.md
Normal file
@@ -0,0 +1,290 @@
|
||||
# Phenotyping 模块数据流与表关系
|
||||
|
||||
本文档分析 Phenotyping 模块的数据录入顺序、核心表关系，以及它与 Core 模块 `study/trial/program/crop` 的衔接方式。
|
||||
|
||||
## 结论
|
||||
|
||||
Phenotyping 模块的数据主线是：
|
||||
|
||||
```text
|
||||
core study -> ontology -> trait / method / scale -> observation_variable -> observation_unit -> event / image -> observation
|
||||
```
|
||||
|
||||
更贴近业务录入的顺序可以理解为：
|
||||
|
||||
```text
|
||||
1. 先有 Core 数据：crop、program、trial、study
|
||||
2. 录入 Ontology / Trait / Method / Scale
|
||||
3. 组装 ObservationVariable
|
||||
4. 录入 ObservationUnit
|
||||
5. 录入 Event 和 Image
|
||||
6. 录入 Observation
|
||||
```
|
||||
|
||||
初始化脚本中与 Phenotyping 相关的执行顺序是：
|
||||
|
||||
```text
|
||||
R__init_data_14_observation_units.sql
|
||||
R__init_data_17_events.sql
|
||||
R__init_data_18_images.sql
|
||||
R__init_data_19_observation_variables.sql
|
||||
R__init_data_20_observations.sql
|
||||
R__init_data_26_observation_variables2.sql
|
||||
R__init_data_26_observation_variables3.sql
|
||||
```
|
||||
|
||||
注意：初始化脚本为了构造演示数据，`observation_unit` 早于 `observation_variable` 插入；从业务建模角度看，二者都依赖已存在的 `study`，而真正的观测值 `observation` 需要同时引用 `observation_unit` 和 `observation_variable`。
|
||||
|
||||
## 核心表说明
|
||||
|
||||
| 表 | 作用 | 主要上游依赖 | 主要下游 |
|
||||
| --- | --- | --- | --- |
|
||||
| `ontology` | 本体信息，定义术语来源 | 无 | `trait`、`method`、`scale`、`observation_variable` |
|
||||
| `ontology_ref` | 本体引用项 | 可独立录入 | `trait_ontology_reference`、`method_ontology_reference`、`scale_ontology_reference` |
|
||||
| `trait` | 性状定义，描述“测什么” | 可选 `ontology` | `observation_variable` |
|
||||
| `method` | 测量方法，描述“怎么测” | 可选 `ontology` | `observation_variable` |
|
||||
| `scale` | 标尺/单位/数据类型，描述“用什么尺度表达” | 可选 `ontology` | `observation_variable`、`scale_valid_value_category` |
|
||||
| `observation_variable` | 观测变量，由 trait/method/scale 组成 | `crop`、`trait`、`method`、`scale`、`ontology` | `observation`、`study_variable` |
|
||||
| `observation_unit` | 观测对象，如 plot/plant/block | `crop`、`program`、`trial`、`study`，可选 germplasm/seed_lot/cross | `observation`、`event_observation_units`、`image` |
|
||||
| `event` | 田间事件，如施肥、灌溉、采样等 | `study` | `event_param`、`event_observation_units` |
|
||||
| `event_param` | 事件参数 | `event` | `event_parameter_entity_values_by_date` |
|
||||
| `image` | 图片/影像数据 | 可选 `observation_unit`、`geojson` | `image_observations` |
|
||||
| `observation` | 实际观测值 | `observation_unit`、`observation_variable`、`study`、可选 `season` | `image_observations` |
|
||||
|
||||
## 建议录入顺序
|
||||
|
||||
### 1. 准备 Core 上游数据
|
||||
|
||||
Phenotyping 数据通常挂在 Core 的层级下面：
|
||||
|
||||
```text
|
||||
crop -> program -> trial -> study
|
||||
```
|
||||
|
||||
其中 `study` 是 Phenotyping 的入口节点。`observation_unit`、`event`、`observation` 都会直接或间接关联到 `study`。
|
||||
|
||||
### 2. 录入 Ontology
|
||||
|
||||
先录入 `ontology` 和需要的 `ontology_ref`。
|
||||
|
||||
`ontology` 用来标识术语体系来源，后续 `trait`、`method`、`scale`、`observation_variable` 都可以挂载本体信息。
|
||||
|
||||
### 3. 录入 Trait / Method / Scale
|
||||
|
||||
这三类数据共同描述一个观测指标：
|
||||
|
||||
```text
|
||||
Trait = 测什么，例如 plant height
|
||||
Method = 怎么测，例如 ruler measurement
|
||||
Scale = 用什么单位/数据类型表达，例如 cm、numeric
|
||||
```
|
||||
|
||||
`scale` 如果有枚举或分类值，还会录入：
|
||||
|
||||
```text
|
||||
scale_valid_value_category
|
||||
```
|
||||
|
||||
### 4. 组装 ObservationVariable
|
||||
|
||||
录入 `observation_variable`，它会引用：
|
||||
|
||||
```text
|
||||
crop
|
||||
trait
|
||||
method
|
||||
scale
|
||||
ontology
|
||||
```
|
||||
|
||||
业务上它相当于“可被采集的一项指标”。例如“株高-尺测法-cm”。
|
||||
|
||||
`study_variable` 是 `study` 和 `observation_variable` 的多对多关系，表示某个 study 会采集哪些变量。
|
||||
|
||||
### 5. 录入 ObservationUnit
|
||||
|
||||
录入 `observation_unit`，它表示被观测对象，例如 field、block、plot、plant。
|
||||
|
||||
它通常会引用：
|
||||
|
||||
```text
|
||||
crop
|
||||
program
|
||||
trial
|
||||
study
|
||||
```
|
||||
|
||||
并且可选关联：
|
||||
|
||||
```text
|
||||
germplasm
|
||||
seed_lot
|
||||
cross
|
||||
observation_unit_position
|
||||
observation_unit_treatment
|
||||
observation_unit_level
|
||||
```
|
||||
|
||||
### 6. 录入 Event
|
||||
|
||||
录入 `event`，用于表达发生在 study 或 observation unit 上的事件。
|
||||
|
||||
常见关系：
|
||||
|
||||
```text
|
||||
event -> study
|
||||
event_observation_units -> observation_unit
|
||||
event_param -> event
|
||||
```
|
||||
|
||||
### 7. 录入 Image
|
||||
|
||||
录入 `image`，图片可以直接关联 `observation_unit`，也可以通过 `image_observations` 关联一个或多个 `observation`。
|
||||
|
||||
图片坐标使用 `geojson/coordinate` 扩展。
|
||||
|
||||
### 8. 录入 Observation
|
||||
|
||||
最后录入 `observation`，这是 Phenotyping 模块的核心事实数据。
|
||||
|
||||
一条 observation 通常同时引用：
|
||||
|
||||
```text
|
||||
observation_unit
|
||||
observation_variable
|
||||
study
|
||||
trial
|
||||
program
|
||||
crop
|
||||
season
|
||||
```
|
||||
|
||||
代码里 `ObservationEntity.setObservationUnit(...)` 会从 observation unit 继承 study/trial/program/crop，因此 observation 的上下文可以由 observation unit 自动带出。
|
||||
|
||||
## Phenotyping 数据流图
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
C["Core: crop"] --> P["Core: program"]
|
||||
P --> T["Core: trial"]
|
||||
T --> S["Core: study"]
|
||||
|
||||
O["ontology 本体"] --> TR["trait 性状"]
|
||||
O --> M["method 方法"]
|
||||
O --> SC["scale 标尺"]
|
||||
OR["ontology_ref 本体引用"] --> TR
|
||||
OR --> M
|
||||
OR --> SC
|
||||
|
||||
C --> OV["observation_variable 观测变量"]
|
||||
TR --> OV
|
||||
M --> OV
|
||||
SC --> OV
|
||||
O --> OV
|
||||
|
||||
S --> SV["study_variable 研究-变量"]
|
||||
OV --> SV
|
||||
|
||||
C --> OU["observation_unit 观测单元"]
|
||||
P --> OU
|
||||
T --> OU
|
||||
S --> OU
|
||||
G["Germplasm/SeedLot/Cross 可选"] --> OU
|
||||
OU --> OUP["position / treatment / level"]
|
||||
|
||||
S --> E["event 事件"]
|
||||
E --> EP["event_param 事件参数"]
|
||||
E --> EOU["event_observation_units"]
|
||||
OU --> EOU
|
||||
|
||||
OU --> IMG["image 图像"]
|
||||
GEO["geojson / coordinate"] --> IMG
|
||||
|
||||
OU --> OB["observation 观测值"]
|
||||
OV --> OB
|
||||
S --> OB
|
||||
T --> OB
|
||||
P --> OB
|
||||
C --> OB
|
||||
SE["Core: season"] --> OB
|
||||
|
||||
IMG --> IO["image_observations"]
|
||||
OB --> IO
|
||||
```
|
||||
|
||||
## Phenotyping ER 关系图
|
||||
|
||||
```mermaid
|
||||
erDiagram
|
||||
crop ||--o{ observation_variable : "crop_id"
|
||||
crop ||--o{ observation_unit : "crop_id"
|
||||
crop ||--o{ observation : "crop_id"
|
||||
|
||||
program ||--o{ observation_unit : "program_id"
|
||||
program ||--o{ observation : "program_id"
|
||||
|
||||
trial ||--o{ observation_unit : "trial_id"
|
||||
trial ||--o{ observation : "trial_id"
|
||||
|
||||
study ||--o{ observation_unit : "study_id"
|
||||
study ||--o{ event : "study_id"
|
||||
study ||--o{ observation : "study_id"
|
||||
study ||--o{ study_variable : "study_db_id"
|
||||
|
||||
ontology ||--o{ trait : "ontology_id"
|
||||
ontology ||--o{ method : "ontology_id"
|
||||
ontology ||--o{ scale : "ontology_id"
|
||||
ontology ||--o{ observation_variable : "ontology_id"
|
||||
|
||||
ontology_ref ||--o{ trait_ontology_reference : "ontology_reference_id"
|
||||
ontology_ref ||--o{ method_ontology_reference : "ontology_reference_id"
|
||||
ontology_ref ||--o{ scale_ontology_reference : "ontology_reference_id"
|
||||
|
||||
trait ||--o{ observation_variable : "trait_id"
|
||||
method ||--o{ observation_variable : "method_id"
|
||||
scale ||--o{ observation_variable : "scale_id"
|
||||
scale ||--o{ scale_valid_value_category : "scale_id"
|
||||
|
||||
observation_variable ||--o{ observation : "observation_variable_id"
|
||||
observation_variable ||--o{ study_variable : "variable_db_id"
|
||||
|
||||
observation_unit ||--o{ observation : "observation_unit_id"
|
||||
observation_unit ||--o{ observation_unit_position : "observation_unit_id"
|
||||
observation_unit ||--o{ observation_unit_treatment : "observation_unit_id"
|
||||
observation_unit ||--o{ observation_unit_level : "observation_unit_id"
|
||||
|
||||
event ||--o{ event_param : "event_id"
|
||||
event ||--o{ event_observation_units : "event_entity_id"
|
||||
observation_unit ||--o{ event_observation_units : "observation_units_id"
|
||||
|
||||
image ||--o{ image_observations : "image_entity_id"
|
||||
observation ||--o{ image_observations : "observations_id"
|
||||
observation_unit ||--o{ image : "observation_unit_id"
|
||||
|
||||
season ||--o{ observation : "season_id"
|
||||
```
|
||||
|
||||
## API 与表的对应关系
|
||||
|
||||
| API | 主表 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `/brapi/v2/ontologies` | `ontology` | 本体查询、新增 |
|
||||
| `/brapi/v2/traits` | `trait` | 性状定义 |
|
||||
| `/brapi/v2/methods` | `method` | 测量方法 |
|
||||
| `/brapi/v2/scales` | `scale` | 标尺、单位、数据类型 |
|
||||
| `/brapi/v2/variables` | `observation_variable` | 观测变量，由 trait/method/scale 组合 |
|
||||
| `/brapi/v2/observationunits` | `observation_unit` | 观测单元 |
|
||||
| `/brapi/v2/events` | `event` | 田间/实验事件 |
|
||||
| `/brapi/v2/images` | `image` | 图像数据 |
|
||||
| `/brapi/v2/observations` | `observation` | 实际观测值 |
|
||||
|
||||
## 关键注意点
|
||||
|
||||
1. `study` 是 Phenotyping 与 Core 的连接点，大多数表型数据都应挂到具体 study。
|
||||
2. `observation_variable` 不是单独的数值，它是“性状 + 方法 + 标尺 + 本体”的指标定义。
|
||||
3. `observation_unit` 是被观测对象，`observation` 是对这个对象在某个变量上的一次测量结果。
|
||||
4. `event` 可以绑定多个 `observation_unit`，适合记录施肥、灌溉、采样等动作。
|
||||
5. `image` 可以直接绑定 `observation_unit`，也可以通过 `image_observations` 与观测值关联。
|
||||
6. `trait/method/scale/observation_variable` 都有 `*_additional_info` 和 `*_external_references` 扩展表，用于补充业务字段和外部引用。
|
||||
7. `observation` 冗余保存了 `crop/program/trial/study` 上下文，代码中会从 `observation_unit` 或 `study` 向上继承这些上下文，方便查询。
|
||||
|
||||
Reference in New Issue
Block a user