Zoom Logo

The Statistics Debate! - Shared screen with speaker view
Glenn Johnson
29:43
There is no program at this time. This session will begin shortly. We are just letting everyone regsitered get into the room. Thank you for attending!
Glenn Johnson
29:57
There is no program at this time. This session will begin shortly. We are just letting everyone regsitered get into the room. Thank you for attending!
Glenn Johnson
30:15
A recording of this event will be made available in the NEWS section of the niss.org website as soon as the files are processed, usually within one day or so.
Glenn Johnson
30:36
There is no program at this time. This session will begin shortly. We are just letting everyone regsitered get into the room. Thank you for attending!
Glenn Johnson
31:01
There is no program at this time. This session will begin shortly. We are just letting everyone regsitered get into the room. Thank you for attending!
Glenn Johnson
31:27
A recording of this event will be made available in the NEWS section of the niss.org website as soon as the files are processed, usually within one day or so.
Glenn Johnson
31:43
A recording of this event will be made available in the NEWS section of the niss.org website as soon as the files are processed, usually within one day or so.
Glenn Johnson
31:56
There is no program at this time. This session will begin shortly. We are just letting everyone regsitered get into the room. Thank you for attending!
PAOLA ZANINOTTO
32:20
Thank you
Andy Wiesner
32:47
Silly question but I assume we are muted correct? I have construction going on and don’t want it to be disruptive
James L Rosenberger
33:22
Yes, attendees are all muted
Glenn Johnson
36:28
Given the issues surrounding the misuses and abuse of p-values, do you think they should continue to be used or not? Why or why not?
Glenn Johnson
39:53
Hi Yiling, not sure what the url will be just yet, but it will be found at the top of the NEWS section when it appears. Does that help?
James L Rosenberger
43:35
previous Please add your questions to the Q&A
Glenn Johnson
44:15
Hello Chris, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Glenn Johnson
44:45
Hello Cheng, please use the Q&A tool to post your questions. It will be a lot easier for us to track all of the questions there. Thanx!
Glenn Johnson
45:03
No problem. Enjoy!
Jessica Li
46:30
'
Glenn Johnson
46:44
No Worries, comments get lost in the chat. In the Q&A handles them better... Cheers, Glenn
Glenn Johnson
49:08
Question 2: Should practitioners avoid the use of thresholds (e.g., P-value thresholds) in interpreting data? If so, does this preclude testing?
David Teplow
54:11
DT: How do you argue against the fact that p-values HAVE been useful in practice, not theory.
James L Rosenberger
54:27
Questions intended for the panelists should be put in the Q&A and will be addressed during the discussion after the break.
Glenn Johnson
54:33
Hello Charles, please use the Q&A tool to post your questions. It will be a lot easier for us to track all of the questions there. Thanx!
Glenn Johnson
54:59
Hello David, please use the Q&A tool to post your questions. It will be a lot easier for us to track all of the questions there. Thanx!
Glenn Johnson
57:36
Hi Davis, yes, all of the attendees are all on mute - so no worries...
William Peden
57:49
David Tramifow says that the assumptions of the model used for testing with p-values are wrong, so p-values should not be used for interpreting test. But isn't the aim of using the assumptions and p-value to create a test that is a hard type of test to pass, not to be a true description of reality? False assumptions about sampling etc. don't necessarily make the test less hard - it depends on the assumptions and the other features of the test.
Glenn Johnson
58:10
Hello William, please use the Q&A tool to post your questions. It will be a lot easier for us to track all of the questions there. Thanx!
William Peden
01:00:11
Sorry - I lost track of which box I was using.
Glenn Johnson
01:00:35
No worries - just trying to keep things in order... Enjoy!
Glenn Johnson
01:02:05
Question 3: Is there a role for sharp null hypotheses or should we be thinking about interval nulls?
Glenn Johnson
01:04:47
Hello Yling, please use the Q&A tool to post your questions. It will be a lot easier for us to track all of the questions there. Thanx!
James L Rosenberger
01:06:47
We can capture the comments in the chat, but if you want the panelists to answer a question, also add it to the Q&A.
James L Rosenberger
01:07:44
You can toggle between gallery view and speaker view on the Zoom screen.
William Peden
01:08:17
(Double-click anywhere on the screen.)
James L Rosenberger
01:08:41
We will allow questions if there is time, after the break.
James L Rosenberger
01:09:23
I see you raised hand, but will likely not have time for live questions, until the very end.
Pamela Reinagel
01:10:57
no we cannot toggle only you can
James L Rosenberger
01:12:49
Is it now in gallery view? or Speaker view?
Glenn Johnson
01:15:50
Question 4: Should we be teaching hypothesis testing anymore, or should we be focusing on point estimation and interval estimation?
Glenn Johnson
01:17:34
Thanx Esra - I sent his to Jim...
Shenae Lee
01:17:44
Shenae Lee (You): Hello! Could you share your opinions on using Bayesian approach when you have a few observed event data (e.g. 10 component failures in 20 years). Thanks!
Glenn Johnson
01:18:07
Hello Shenae, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
David Colquhoun
01:21:29
I'd love a reference to the Thai vaccine trial that Jim Berger mentioned
Glenn Johnson
01:26:16
Question 5: What are your reasons for or against the use of Bayes Factors?
David Colquhoun
01:31:30
I love -epln(p) -beautifully simple
Richard Lewis
01:33:34
The view I am coming to about threshold-based tests is that their fundamental problem is that they conflate estimation (what should I conclude from the data about parameters in models) and decision (should I run another experiment? should I build on this model for new theory rather than another? should I encourage a phd student to pursue the idea? etc) The former activity seems on relatively solid ground and I at least want to get this much out of my analysis! The latter--making decisions--depends on the specific decision, subjective utility, costs/gains of hits/misses/false alarms etc. I would welcome panelists' thoughts on this.
Glenn Johnson
01:33:57
Hello Richard, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Richard Lewis
01:34:21
Will do. thanks.
Glenn Johnson
01:37:59
QUESTION 6: With so much examination of if/why the usual nominal type I error .05 is appropriate, should there be similar questions about the usual nominal type II error .2?
Glenn Johnson
01:41:04
Hello Murali, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Tim Brodkorb
01:47:28
is Bayes theorem a Model? if so, then it can describe data but also could potentially be incredibly wrong. Any examples the speakers know of?
Glenn Johnson
01:47:58
Hello Tim, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Shenae Lee
01:48:08
i just did posted my edited comment on Q&A:) Thanks
Glenn Johnson
01:50:15
QUESTION 7: What are the problems that lead to the reproducibility crisis and what are the most important things we should do to address it?
Jian Gao
01:50:23
Yes, models can be wrong. It is the investigators’ job to make the models ‘right’.The possibility of a model being wrong does not warrant eyeballing everything. Eyeballing the data does not make the models right either. The following article addresses much of the issues discussed today:https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-020-01051-6
Glenn Johnson
01:50:42
Hello Jian, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Uli Widmaier
01:52:25
Deborah claimed that David's a priori procedure doesn[t stand on its own but rather imports confidence intervals through the back door, as it were. I'd like to hear more about that, and I'd like to hear David's response.
Bill Jefferys
01:52:39
G.E.P. Box: "All models are wrong, but some models are useful." So the question I find of interest is, how do we decide whether a model is *useful* or not? Which then says that a decision has to be made (shall we use this model, provisionally, in the future?). More generally, it is hard for me to imagine a situation where you do hypothesis testing just to publish a number (p-value or Bayes factor or something else). Don't we do these things because ultimately we need to make a decision? Is this vaccine something that we should use? Does the evidence from this accelerator experiment lead us to want to expend additional time and expense looking for some related effect? And so on. All of which tells me that there may be a role of decision theory in all of this. Can the panel members address this issue?
Glenn Johnson
01:52:55
Hello Uli, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Glenn Johnson
01:53:03
Hello Bill, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
jacksonfoote
01:53:35
Would be nice to replicate this discussion for a broader, less educated audience on reddit.com
Cheng Cheng
01:55:39
Causes of reproducibility:
Ron Kenett
01:56:15
claims derived from statistical analysis are presented in one way or another. Discussing presentation and generalization of findings should precede the p value discussion . Somehow this is ignored https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID3526850_code1304617.pdf?abstractid=3035070&mirid=1
Glenn Johnson
01:56:35
Hello Ron, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Cheng Cheng
01:57:25
Poor study design; change the hypothesis/model after peaking into data; P hacking; misinterpretation of significance test
Betsy Becker
01:57:33
The papers on using methods other thaa.
Glenn Johnson
01:57:47
Hello Cheng, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Betsy Becker
01:58:45
other than p-values for assessing replications date back to the 1980s with papers by Humphreys critiquing Buriel, in the psychology lit.
David Firth
01:59:08
[copied here from the Q&A channel] I wonder if I am understanding correctly David Trafimow's position, that a p-value measures only the evidence against the *model*. That is not correct, surely. A statistical model is a family of distributions, and a null hypothesis is a subfamily (for example, the subfamily comprising only those distributions in the model that have mean zero). So in what sense does a p-value for the evidence against a null hypothesis measure the strength of evidence against the whole *model*? (My own answer: there is no such sense.)
Cheng Cheng
02:04:10
Revision:
Jian Gao
02:05:05
In healthcare research and practice, it is rare that a treatment works for everybody, researchers and physicians are forced to balance the benefit and risk. So, it is crucial that we know the probability that a treatment will or will not work. The question is, what is the best statistic for that?
Cheng Cheng
02:05:10
Causes of reproducibility crisis:Poor study design; change the hypothesis/model after peaking into data; P hacking; misinterpretation of significance test. Banning significance test does not solve the problem.
Glenn Johnson
02:05:34
Hello Jian, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Glenn Johnson
02:05:50
Hello Cheng, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
David Colquhoun
02:12:12
Lot's of journals now sccrpt
David Colquhoun
02:12:21
oops
David Colquhoun
02:13:10
Lots of journals now accept registered reports.
Stephen Senn
02:13:40
In drug regulation elaborate protocols and statistical analysis planes are established before starting (for the former) and before unblinding (for the latter). This is much rarer in academia.
Glenn Johnson
02:14:48
Hello Stephen, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Glenn Johnson
02:16:05
Hello David, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Glenn Johnson
02:16:29
Hello Yoav, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
Bhaskar Thakur
02:21:05
Can we keep/fix some margin (such as superiority/equivalence) in observational studies like randomized controlled trial to get our results in the form of effect size with p-value?
Glenn Johnson
02:21:30
Hello Bhaskar, please use the Q&A tool to post your questions/comments. It will be a lot easier for us to track all of the questions there. Thanx!
David Colquhoun
02:26:14
I fear that the failure of statisticians to agree on how to improve on p values means that efforts to improve things have failed almost completely.
Grace Chiu
02:27:24
Thanks, panelists and hosts!
Pamela Reinagel
02:27:25
Thanks to all the speakers and moderator, this was great
Emma Mills
02:27:32
thank you
Cilene Oliveira
02:27:42
thank you!