1
%% -*- mode: noweb; noweb-default-code-mode: R-mode; -*-
3
\documentclass{article}
7
This is a quick discussion of the statistical characteristics and
8
corresponding sample size requirements for an HVTN/EuroVac trial to
9
evaluate 2 issues with respect to related MVA and NYVAC products.
10
Larry Corey has initially proposed two Phase II trials. The first is
11
to examine the differences between the two pox vector vaccines; the
12
second to look at the effect of ordering with an MVA and DNA in a
15
\section{Proposed Trial Designs}
16
\label{SEC:prop-trial-designs}
18
This section contains the proposed trials designs along with some of
19
the reasoning behind the approaches.
21
Statistician's conjectures:
23
\item we should see cellular as well as humoral responses; however,
24
the design will be based on cellular immune response assays with
25
qualitative (binary) responses.
26
\item it will take at least a 15\% difference in immune response
27
(i.e. 15\% vs 30\%, or 40\% vs. 55\%, or 60\% vs 75\%) for us to
28
consider one to be better than the other.
29
\item we need the lower bound of the 95\% confidence interval to be at
30
least 30\% to move forward to a Phase IIb or III trial.
33
\subsection{Pox Vector Comparisons}
34
\label{SEC:pox-vect-comp}
36
The primary goal of this study is to compare the effectiveness of
37
NYVAC with MVA when considering immunological response. The proposed
38
schedule is a 3 visit regimen (for example, 0, 1, 3) with the primary
39
immunological endpoints being recorded after the 3rd visit.
41
Questions that the study will look at (ordered in terms of importance):
43
\item Is one delivery vector (NYVAC, MVA) uniformly better than the
44
other (qualitative cellular immune response, for either a single
45
time point or for a cumulative probability of response).
46
\item Do any of the arms have sufficient immune response
47
characteristics to warrant a Phase 3 trial?
48
\item Do the gene inserts play a role in immune response?
49
\item If the last two are true, which factor has a stronger effect on
53
We propose the following schema based on a conservative approach for
54
answering the first question. This will have 80\% power to detect at
55
minimum of a 15\% difference in immunological response (40\% vs. 55\%)
56
at a single time point (or for cumulative response), with an
57
adjustment of 6 additional to cover possible 10\% attrition over the
58
course of the study. We have more power as the probability of immune
59
response moves away (higher or lower) from 50\%.
63
\begin{tabular}{l|rr|c}
64
Group & Vector & Clade & N \\ \hline
65
A1 & MVA & B/B & 64 \\
66
B1 & MVA & C/C & 64 \\
67
C1 & MVA & B/C & 64 \\
68
D1 & NYVAC & B/B & 64 \\
69
E1 & NYVAC & C/C & 64 \\
70
F1 & NYVAC & B/C & 64 \\
71
G1 & CTRL & & 60 \\ \hline
74
\caption{Trial 1 Schema. For 90\% power, increase the group size to
76
\label{tab:trial1:schema}
79
We pay dearly to reduce the difference in immune response to 10\% from
80
15\%; this would require more than doubling each arm, i.e. to 144 per
81
group, for a total of $6 \times 144 = 864$ plus a suitable control
84
Since these products could be stand-alone vaccines in their own right,
85
we describe the information that will be obtained from this trial, for
86
characterizing each arm for moving forward to a subsequence Phase III
87
study. With respect to characterizing each arm, we will have at least
88
80\% power to determine that the lower bound of an 95\% confidence
89
interval for the net (adjusted for false-positive) immunological
90
response is greater than 30\%, if the observed rate is at least 61\%
91
(i.e. if this was the true situation, we would expect of 64 vaccinees,
92
that $64 \times 0.61 = 39$ positive immune responses would be observed
93
in the study, prior to adjusting for the false positives in the
94
control arm). This computation assumes a 10\% false positive rate (6
95
positives in the $N=60$ control arm). \textbf{If we don't think that
96
we can pull off a 61\% response, it might be worth increasing the
99
For the first primary hypothesis (which delivery vector is better?),
100
the statistical analysis plan is to compare 2 proportions, one
101
combining groups A1,B1,C1 against groups D1,E1,F1.
103
For the second primary hypothesis (which arms are worth bringing
104
forward to a Phase III trial), the characterization will be done by
105
looking at each of the arms (A1,B1,C1,D1,E1,F1) and comparing each
106
with the control arm (G1).
108
Assuming no vector effect, we will have 80\% power (79.4\%) to detect
109
a 20\% difference in the gene inserts (B,C, and the B/C combination)
110
for each vaccine. A priori, it seems doubtful (to me) that the
111
inserts will have this much effect. This was computed via a
112
Bonferroni adjustment to the 2-sample power calculation.
114
I have not characterized the other secondary goals (selection of best
115
response, etc), but these would definitely be part of the statistical
118
\subsection{Prime-Boost Comparison}
119
\label{SEC:prime-boost-comp}
121
The goal of this study is to evaluate the immunogenicity of 3
122
prime-boost strategies using DNA and NYVAC vaccine products, using
123
regimens which employ 3 vaccinations.
125
Questions that the study will look at (ordered in terms of importance):
127
\item Which DNA-NYVAC prime-boost strategy produces the highest immune
128
response, 2 weeks after the third vaccination?
129
\item Do any of the arms meet the criteria for moving forward?
132
We propose the following schema based on a conservative approach, with
133
80\% power to detect at minimum of a 15\% difference in immunological
134
response (40\% vs. 55\%) at a single time point (or for cumulative
135
response), with an adjustment of 6 additional to cover possible 10\%
136
attrition over the course of the study. We have more power as the
137
true probability of immune response moves away (higher or lower) from
142
\begin{tabular}{r|ccc|c}
143
Group & M0 & M1 & M2 & N \\ \hline
144
A2 & NYVAC & NYVAC & DNA & 85 \\
145
B2 & DNA & NYVAC & NYVAC & 85 \\
146
C2 & DNA & DNA & NYVAC & 85 \\
147
D2 & CTRL & CTRL & CTRL & 80 \\ \hline
151
\caption{DNA/NYVAC prime-boost study. For 90\% power, increase the
152
size of each arm to 110 (with a corresponding increase in the
154
\label{tab:primeboost:schema}
157
The larger sample sizes are due to the primary analysis being a
158
3-group rather than 2-group comparison. It might be possible to
159
combine B2 and C2 for determination of the primary question (order of
160
products), and lower the numbers, but this assumes that the regimens
161
are pretty much equivalent, i.e. that it does not matter whether NYVAC
162
or DNA is given at the second vaccination.
164
For characterization purposes, we have moved from 61\% to 58\% as far
165
as what the true target to achieve is, given 10\% a false positive
168
See the first trial for increased verbiage trying to describe how we
169
are looking at the first and second questions of interest.
175
\centerline{\Large \textbf{The following can be ignored}}
177
(unless you are curious about some of the numbers, or other options;
178
included primarily for future HVTN Stat group discussion).
181
<<initialize,echo=FALSE>>=
182
exists.HVTNDESIGN <- require(hvtndesign)
183
exists.STATMOD <- require(statmod)
186
\section{Computations for the proposed designs}
187
\label{SEC:comp-upon-with}
190
\subsection{Pox Vector Trial Calculations}
191
\label{SEC:calculations:poxvector}
193
The following considers a trial to conservatively detect 15\%
194
difference at 80\% power.
196
<<trial 1 vector comparison>>=
197
trial1.characteristics <- power.prop.test(p1=0.40,p2=0.55,power=0.8)
198
print(trial1.characteristics)
200
N is the number assigned to MVA and assigned to NYVAC,
201
which then will be divided into 3 to form the comparison,
202
assuming that the gene insert has no effect
203
<<trial 1 vector comparison>>=
204
NperArm <- trial1.characteristics$n / 3
206
so the number per arm assuming no loss
207
<<trial 1 vector comparison>>=
210
and the number per arm assuming 10\% loss to followup
211
<<trial 1 vector comparison>>=
212
ceiling(NperArm * 1.1)
215
If we consider a scenario with 80\% power to detect a 10\% difference,
216
we have the following:
218
<<trial 1 vector comparison>>=
219
trial1.characteristics.2 <- power.prop.test(p1=0.45,p2=0.55,power=0.8)
220
print(trial1.characteristics.2)
222
N is the number assigned to MVA and assigned to NYVAC,
223
which then will be divided into 3 to form the comparison,
224
assuming that the gene insert has no effect
225
<<trial 1 vector comparison>>=
226
NperArm <- trial1.characteristics.2$n / 3
228
so the number per arm assuming no loss
229
<<trial 1 vector comparison>>=
232
and the number per arm assuming 10\% loss to followup
233
<<trial 1 vector comparison>>=
234
ceiling(NperArm * 1.1)
237
and we end up having to pay dearly to achieve be able to detect a 10\%
241
The following considers a trial to conservatively detect 15\%
242
difference at 90\% power.
244
<<trial 1 vector comparison>>=
245
trial1.characteristics3 <- power.prop.test(p1=0.40,p2=0.55,power=0.9)
246
print(trial1.characteristics3)
248
N is the number assigned to MVA and assigned to NYVAC,
249
which then will be divided into 3 to form the comparison,
250
assuming that the gene insert has no effect
251
<<trial 1 vector comparison>>=
252
NperArm3 <- trial1.characteristics3$n / 3
254
so the number per arm assuming no loss
255
<<trial 1 vector comparison>>=
258
and the number per arm assuming 10\% loss to followup
259
<<trial 1 vector comparison>>=
260
ceiling(NperArm3 * 1.1)
266
We describe the 3-way comparison of inserts by adjusting the significance
267
level via a Bonferroni correction, and compute via:
268
<<Three-way pox insert comparison>>=
269
power.prop.test(n=128,p1=0.40,p2=0.6,sig.level=0.0166)
275
\subsection{Prime-Boost Trial Calculations}
276
\label{SEC:calculations:primeboost}
279
trial2.characteristics <- power.prop.test(p1=0.40,p2=0.55,power=0.8,sig.level=0.0166)
280
print(trial2.characteristics)
282
N is the number assigned to MVA and assigned to NYVAC,
283
which then will be divided into 3 to form the comparison,
284
assuming that the gene insert has no effect
285
<<trial 2 vector comparison>>=
286
NperArm2 <- trial2.characteristics$n / 3
288
so the number per arm assuming no loss
289
<<trial 2 vector comparison>>=
292
and the number per arm assuming 10\% loss to followup
293
<<trial 2 vector comparison>>=
294
ceiling(NperArm2 * 1.1)
297
At 90\% power, we have:
299
trial2.characteristics2 <- power.prop.test(p1=0.40,p2=0.55,power=0.9,sig.level=0.0166)
300
print(trial2.characteristics2)
302
N is the number assigned to MVA and assigned to NYVAC,
303
which then will be divided into 3 to form the comparison,
304
assuming that the gene insert has no effect
305
<<trial 2 vector comparison>>=
306
NperArm4 <- trial2.characteristics2$n / 3
308
so the number per arm assuming no loss
309
<<trial 2 vector comparison>>=
312
and the number per arm assuming 10\% loss to followup
313
<<trial 2 vector comparison>>=
314
ceiling(NperArm4 * 1.1)
319
\section{General Computations}
320
\label{SEC:computations}
323
\subsection{2-sample tests of proportions}
324
\label{SEC:2-sample-tests}
326
We present here the computations used to generate the numbers for
327
discussion. The characteristics were computed using R 1.7.1, with the
328
following code below.
330
We will be conservative and will consider that the insert has no
331
effect, and use the ``worst'' situation (to be calculated below).
333
<<PowerDataFrame,echo=FALSE,eval=TRUE>>=
334
PowerDataFrame <- function(PowerList=seq(from=0.8,to=0.9,by=0.05),
335
LowerResponseList=seq(from=0.1,to=0.6,by=0.05),
336
DeltaList=seq(from=0.1,to=0.2,by=0.05)) {
338
power.results <- NULL
339
for (Power in PowerList) {
340
for (Delta in DeltaList) {
341
for (LowerResponse in LowerResponseList) {
342
power.temp <- power.prop.test(p1=LowerResponse,
343
p2=LowerResponse + Delta,
347
power.results <- rbind(power.results,
350
power.temp$p2 - power.temp$p1,
356
colnames(power.results) <- c("Low","High",
359
as.data.frame(power.results)
361
@ %def PowerDataFrame
363
<<compute power for Pox Trial vector comparison>>=
364
power.dataframe <- PowerDataFrame(PowerList=c(0.8))
367
The results suggest that.
369
<<display Pox Trial power for vector comparison,results=tex,echo=FALSE>>=
370
if (require(xtable)) {
371
xtable(power.dataframe,label="tab:80power:5diff",
372
caption=paste("Scenarios used for considering 80\\% power: ",
373
"based on test of binomial proportions, ",
374
"multiply n by 2 for whole trial, divide by 3 for each group"))
379
\subsection{Characterization in the presence of False Positives}
380
\label{sec:char-pres-false}
383
<<False Positive Characterization Function>>=
384
ci.false.positive <- function(nrep=50000, #nrep
389
H0VacMCtl = 0.3) ## null hypthosis for net rate
392
Ctl.resp <- rbinom(nrep,nCtl,pCtl)
393
Vac.resp <- rbinom(nrep,nVac,pVac)
395
phat.Ctl <- Ctl.resp / nCtl
396
phat.Vac <- Vac.resp / nVac
398
pDiff <- phat.Vac - phat.Ctl
401
vCtl <- phat.Ctl * (1 - phat.Ctl) / nCtl
402
vVac <- phat.Vac * (1 - phat.Vac) / nVac
405
results.asympt <- 1.0 - (sum(H0VacMCtl > (pDiff - (1.96 * sqrt(vDiff))) &
406
H0VacMCtl < (pDiff + (1.96 * sqrt(vDiff)))) /
409
results.emp <- quantile(pDiff,c(0.025,0.975))
410
list(PowerAsympt=results.asympt,EmpiricalCI=results.emp)
412
@ %def ci.false.positive
416
<<Characterizing Arms in Pox Vector study>>=
417
ci.false.positive(nCtl=60,nVac=64)
418
ci.false.positive(nCtl=60,nVac=64,pVac=0.55)
419
ci.false.positive(nCtl=60,nVac=64,pVac=0.60)
420
ci.false.positive(nCtl=60,nVac=64,pVac=0.61)
421
ci.false.positive(nCtl=60,nVac=64,pVac=0.65)
427
<<Characterizing Arms in Pox Vector study>>=
428
ci.false.positive(nCtl=80,nVac=85)
429
ci.false.positive(nCtl=80,nVac=85,pVac=0.55)
430
ci.false.positive(nCtl=80,nVac=85,pVac=0.58)
431
ci.false.positive(nCtl=85,nVac=85,pVac=0.58)
433
ci.false.positive(nCtl=80,nVac=85,pVac=0.60)
434
ci.false.positive(nCtl=80,nVac=85,pVac=0.61)
435
ci.false.positive(nCtl=80,nVac=85,pVac=0.65)
439
\subsection{Selection}
440
\label{SEC:selection}
443
<<SelectionProb Function>>=
444
Selection.prob <- function(N=10000,
446
probs=c(0.5,rep(0.2,3)),
449
SPACE.PROBLEMS=TRUE) {
451
if (!(length(n) == length(probs))) {
452
stop("Length of n and probs do not match.")
454
set.seed(random.seed)
458
if (SPACE.PROBLEMS) {
459
## loop for computation
462
if (DEBUG) cat("\n*****\n","trial ",i,"\n")
464
trial <- rbinom(k,n,probs)
466
cat("N per trial :",n,"\n")
467
cat("Trial results:",trial,"\n")
468
cat("True Probs:",probs,"\n")
469
cat("Best Trial/Prob",
470
which(trial == max(trial)),
471
which(probs == max(probs)))
473
## Do we need to handle the "multiple max" problem?
474
if (DEBUG) cat("\nResult: ")
475
if (which(probs == max(probs),
477
which(trial == max(trial),
479
select <- c(select,TRUE)
480
if (DEBUG) cat("TRUE\n")
482
select <- c(select,FALSE)
483
if (DEBUG) cat("FALSE\n")
487
## compute in a single matrix
495
<<SelectionProbTest>>=
496
Selection.prob(n=c(30,30,30,30,30),probs=c(0.2,0.2,0.2,0.2,0.5))
497
Selection.prob(n=c(rep(64,6),60),probs=c(rep(0.4,5),0.5,0.1))
498
Selection.prob(n=c(rep(64,6),60),probs=c(rep(0.42,5),0.58,0.1))
added
removed
etc/useR-2006-ESS/Lab2/EuroVac.Rnw
