Menu
Log in


Research Integrity

  • 3 Jul 2023 3:11 PM
    Reply # 13222907 on 13222068
    John Maindonald wrote:

    There are issues here that I hope SSA will pursue.  The bigger issue is perhaps the extent to which a race to get papers published is working against research quality, in study design, execution, and data analysis.


    I am thinking hard how to word my thoughts diplomatically. I have  been lucky enough to go to at least 4 or 5 top international level schools and programs. Even then I feel there are other factors at play in the world which work against quality

    I have similar difficult to express issues about research ethics and justification of studies etc

    I will check with ChatGPT how to word them diplomatically and respectfully get back to you

    One small point in relation to student studies. They are generally hard to justify for any other reason than hopefully a bit of a learning exercise so hopefully etc. Note I did go the effort of checking my Honours Ethics application to vaguely reacquaint myself. How many harmless animals were inconvenienced for a bit of academic credit. Here is the exact question "Describe the potential benefits (to participants, researcher, institutions, community groups, and/or society at large) and explain how these are sufficient to justify the following: etc" No further comment but some of us would not be very welcome on Ethics committees.

    My concerns would be more for all the "real" research

    I would never lay claim to have reached particularly high pinnacles of statistical expertise but etc. Sometimes I feel some  of us have highly informed and justified concerns about such issues but have difficulty raising them respectfully. Many of us also were never raised or exposed to the kind of language regarded as respectful and are often pulled up over it. 


    Last modified: 3 Jul 2023 4:42 PM | Duncan Lowes
  • 1 Jul 2023 4:07 PM
    Reply # 13222505 on 13222068

    Were I again in my ANU office of 1998-2001 working as a consultant, I might post on my door something akin to the attached graph with its accompanying title and subtitle.  Overly small sample sizes and effect sizes (which typically involve a larger or smaller component of guesswork) make experimental results no more than a slight improvement on a game of chance. 

    In the 54 social psychology papers in the “Reproducibility: Psychology” reported in 2015 (Reproducibility: Psychology), an average estimated effect size (difference of interest, divided by SD) in the original dropped to 0.15 in the replicate.  Other such studies have not, in most cases, been a lot more encouraging.

    When I was working at ANU, there were occasions where the Ethics Committee had required the student researcher to reduce their sample size (in at least one case, in a context where I could discern no harm to animal or human) to a level that made the experiment near useless.  If animal lives are involved, the case is surely to have fewer experimenters doing larger carefully planned and executed experiments.

    1 file
    Last modified: 1 Jul 2023 4:10 PM | John Maindonald
  • 1 Jul 2023 7:20 AM
    Reply # 13222372 on 13222068

    Thanks John

    One change that the stats community could pursue is a requirement for statisticians to be on human ethics committees. This policy was championed by the late Sue Wilson a few years ago, and the society supported her, but there was no change to policy. I recently spoke with the chair of an ethics committee in South Australia who added a statistician and they felt it was a great success, as they've prevented many badly designed studies from starting. They called it "Putting the ambulance at the top of the cliff." But are there enough statisticians in Australia and New Zealand for this to become a national policy? I know there's a potential change here in Australia of fewer ethics committees who take on more work, so that would make it more feasible. It's also not an easy job trying to work out what some researchers are aiming to achieve, as researchers often have poor stats knowledge (using cut-and-paste methods) and aren't able to clearly describe their design.

    Adrian

  • 30 Jun 2023 7:28 PM
    Message # 13222068

    At this afternoon's Statistical Consulting Network meetup on Zoom, I mentioned Adrian Barnett's recent article in 'The Conversation'.  Links to this and to articles in the Campus Morning Mail that have more detail are
    Article in The Conversation
    Campus Morning Mail, May 8
    Article in Campus Morning Mail May 11

    There are issues here that I hope SSA will pursue.  The bigger issue is perhaps the extent to which a race to get papers published is working against research quality, in study design, execution, and data analysis.

Powered by Wild Apricot Membership Software