This repository contains data accompanying the development (training and testing) and evaluation (external validation) of a reinforcement learning (RL) algorithm to recommend settings of the mechanical ventilator in intensive care patients accompanying the publication **PUBNAME** by **PUBAUTHORS** (**PUBDOI**).

Development was done from local ICU patient treatment trajectories (Department of Anesthesiology and Intensive Care Medicine, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Local Ethical Committee Approval: BO-EK-423082021). External validation was performed on data extracted and preprocessed from [MIMIC-IV](https://physionet.org/content/mimiciv/3.1/). 

Source code of the developed RL algorithm including training and evaluation pipeline is published at [github](https://github.com/NIMI-research/rl-acute-respiratory-failure-dss).

# Data structure and parameters
State vectors and recommendations are stored in colon-separater format including the state vector, the actions taken by the clinicians and the recommendations (action space) as well as the respective training/test splits with the following directory structure:

- **dev** Development dataset including test and training split
- **eva** External validation dataset
- **configs** Example config files

ICU stays are identified by the column `caseid`, throughout all files for development (folder `dev`) and external validation (`eva`) sets.

The columns `min duration` and `max duration` mark the beginning and end time in minute,s since the beginning of identified controlled invasive mechanical ventilation episode. For each time window median of the available data was taken.

## Demographics
Demographics data considered within the project are:

| Parameter | Description                      | Unit |
| --------- | -------------------------------- | ---- |
| sex       | female (1), male (2)             | -    |
| weight    | body weight                      | kg   |
| height    | body height                      | cm   |
| los       | length of stay                   | days |
| survived  | ICU survival status (TRUE/FALSE) | -    |

## State vector
The following parameters were used as input for the RL-algorithm:

| Parameter  | Unit            | Description                                          | Column name |
| ---------- | --------------- | ---------------------------------------------------- | ----------- |
| $H_aCO_3$  | $mmol/l$        | arterial bicarbonat                                  | HaCO3       |
| $Hb$       | $mmol/l$        | arterial haemoglobin                                 | Hba         |
| $HR$       | $min^{-1}$      | heart rate                                           | HR          |
| $MAP$      | $mmHg$          | mean arterial blood pressure                         | ART         |
| $MV$       | $l$             | minute ventilation                                   | MV          |
| $P_aCO_2$  | $mmHg$          | partial pressure of carbon dioxide in arterial blood | PaCO2       |
| $P_aO_2$   | $mmHg$          | partial pressure of oxygen in arterial blood         | PaO2        |
| $pH$       | $a.u.$          |                                                      | pHa         |
| $P_{insp}$ | $cmH_2O$        | inspiratory airway pressure                          | Pinsp       |
| $S_aO_2$   | $\%$            | oxygen saturation in arterial blood                  | SaO2        |
| $S_pO_2$   | $\%$            | peripheral oxygen saturation                         | SpO2        |
| $V_T$      | $\frac{ml}{kg}$ | Tidal volume normalised to                           | VT          |

## Action space
Clinician (dataset) therapy decisions and AI (policy) recommendations may be found in files starting with `dataset_` and `policy_`, respectively.

The following actions defined and their binning was derived from development data set statistics:

| Parameter | Unit       | Description                        | Level 0        | Level 1           | Level 2            | Level 3             | Level 4            |
| --------- | ---------- | ---------------------------------- | -------------- | ----------------- | ------------------ | ------------------- | ------------------ |
| $F_iO_2$  | $\%$       | fraction of oxygen in inspired air | $[21\dots35]$  | $(35\dots40]$     | $(40\dots45]$      | $(45\dots60]$       | $(60\dots100]$     |
| $PEEP$    | $cmH_2O$   | positive end-expiratory pressure   | $[0\dots7.14]$ | $(7.14\dots8.16]$ | $(8.16\dots10.02]$ | $(10.02\dots12.24]$ | $(12.24\dots45.9]$ |
| $RR$      | $min^{-1}$ | respiratory rate                   | $[2\dots18]$   | $(18\dots22]$     | $(22\dots25]$      | $(25\dots29]$       | $(29\dots60]$      |

with ranges $()$ excluding or $[]$ including the boundary values.

Additional columns are:

- $1^{st}$  column: running index among all time points
- `time_step` time index of the identified interval of mechanical ventilation,  starting with $1$ for each ICU case.
## Q-values
Q-values can be found in files ending with `q_values.csv` each containing one column termed `q_value` containing the corresponding Q-values for each time step of the *Action space*.
# Particularities
## Development data (dev)
Data for development (*train*) and internal validation (*test*) where acquired from single centre ICU therapy data from the TUD Dresden University of Technology Faculty of Medicine and University Hospital Carl Gustav Carus, Department of Anesthesiology and Intensive Care Medicine, Dresden, Germany.
### De-identification

In accordance with current data protection laws, the following steps were undertaken to de-identify the dataset - meaning to anonymise patient data to make identification of individuals impossible (or at least very unlikely):

- removed directly identifying data (name, address, credit card/social security/passport/etc. numbers, and raw time/date data)
- Pseudonomysation: ICU cases were assigned random numbers in the range of $100000…199999$.
- k-anonymisation was performed on variables age, gender, height, weight, and survival using R package `sdcMicro` gaining  k-anonymity of $k=5$.
### Data split and Preprocessing

The local data set was split into training (train) and internal validation (test) in the ratio of 80/20 % of total cases.

After down sampling to $10$ minute intervals (see `dev/data_non-imputed.csv`), missing values were imputed using last observation carried forward (LOCF).
## External validation data (eva)
External validation was done on a subset of the MIMIC-IV database, for which episodes of invasive mechanical ventilation longer than $4 h$ were extracted for a random number ob patients.

##  Training + Validation
To run training, testing and external validation the respective file locations for `train.csv`, `test.csv`, and / or `state_vector.csv` need to be specified in the `config/dataset_config.yml`, [see also](https://github.com/NIMI-research/rl-acute-respiratory-failure-dss).

Example config files can be found in `configs/`.