# Reproducibility

`reproducibility.Rmd`

Results of the **MAMS** package for studies involving
more than 2 stages are seed-dependent (as the Gaussian quadrature
integration of the multivariate normal distribution relies on
probabilities estimated by means of the randomised Quasi-Monte-Carlo
procedure of Genz and Bretz in `mvtnorm::pmvnorm()`

).

Results are reproducible if a seed is set before the evaluation of a
function of the **MAMS** package (typically by means of the
function `set.seed`

):

- When
`parallel=TRUE`

, the future package assigns independent streams of L’Ecuyer pseudo-random numbers to each parallelised task, allowing results to be reproducible when a seed is set, even when using a different parallelisation strategy and/or a different number of workers.

- When
`parallel=FALSE`

, the random number generation is handled by base R directly instead of by the future package, so that, if the number of stages is larger than 2, evaluations using the same seed will not lead to the same exact results with`parallel=FALSE`

and`parallel=TRUE`

.

### Sequential computation

```
set.seed(2910)
m_seq <- mams(K = 3, J = 3, p = 0.65, p0 = 0.55, r = 1:3, r0 = 1:3, alpha = 0.05,
power = 0.9, parallel = FALSE)
```

`m_seq`

```
## Design parameters for a 3 stage trial with 3 treatments
##
## Stage 1 Stage 2 Stage 3
## Cumulative sample size per stage (control): 29 58 87
## Cumulative sample size per stage (active): 29 58 87
##
## Maximum total sample size: 348
##
## Stage 1 Stage 2 Stage 3
## Upper bound: 3.611 2.554 2.085
## Lower bound: 0.000 0.000 2.085
```

### Parallel computation

```
set.seed(2910)
m_par <- mams(K = 3, J = 3, p = 0.65, p0 = 0.55, r = 1:3, r0 = 1:3, alpha = 0.05,
power = 0.9, parallel = TRUE)
```

`m_par`

```
## Design parameters for a 3 stage trial with 3 treatments
##
## Stage 1 Stage 2 Stage 3
## Cumulative sample size per stage (control): 29 58 87
## Cumulative sample size per stage (active): 29 58 87
##
## Maximum total sample size: 348
##
## Stage 1 Stage 2 Stage 3
## Upper bound: 3.612 2.554 2.085
## Lower bound: 0.000 0.000 2.085
```

The results of sequential and parallel computation in this case slightly differ for the upper bound at Stage 1.