Canberra Branch SSA, Victorian Branch SSA & Statistical Consulting Network, August meeting

Over 120 SSA and NZSA members joined online to hear from four statistical consultants, in a webinar jointly hosted by the Canberra and Victorian Branches, and the Statistical Consulting Network. 

Elyse Corless (Data Analysis Australia) drew on her commercial experience in dealing with the challenges of An abundance of data where quantity should not be taken to imply quality.  While large amounts of data can be overwhelming, they can also draw the inquisitive consultant in and be potentially distracting.  Commercial consultants often need to juggle multiple projects each with their own time and budget constraints, while considering the client’s knowledge and understanding of statistical methods and concepts. Elyse emphasised the importance of careful project scoping and planning, and of understanding the value of the project to the client facilitated by regular communication.  She provided tips for improving data quality including inspecting and visualising data, sampling snippets of very large data sets, and using coding in the data review process.  Routine use of coding should be used to support reproducibility of both data checking processes and subsequent analytics.  Elyse shared examples from a range of her recent projects.

Graham Hepworth (Statistical Consulting Centre, University of Melbourne) discussed The art of the possible.  Among important qualities of a statistical consultant is being able to make a judgement of what is needed by the client; this also requires judgement about the capacity of the client to deal with the complexity of the analysis provided.  Graham shared some findings of a client-consultant survey from the Statistical Consulting Centre, and showed that within generally good levels of satisfaction, clients tended to be more satisfied when the problem involved was less complex.  However clients’ perception of the quality of service (overall very positive) varied little in terms of their statistical competence.  Graham suggested that this reflected good judgement on the part of the consultant in adapting to client capacity; he illustrated the range of options a consultant might consider in the analysis of an ordinal outcome variable and discussed his experience in weighing up these options.

Hwan-Jin Yoon’s (Statistical Consulting Unit, Australian National University) contribution focussed on Statistical consulting in the university: some challenges.  Jin contrasted problem-minded versus solution-minded focus, and characterised four different types of roles that might be adopted by a consultant depending degree of activity and engagement on the part of both the consultant and the client.  These roles were helper, leader, teacher and collaborator.  Jin shared examples of how clients’ questions can shape expectations about the statistician’s role, and he discussed the drawbacks inherent in some roles.  Ideally the consultant takes on a collaborative role, with learning on both sides; this is Jin’s “joy of statistics”.  In addition to dealing with client expectations, challenges in academic statistical consulting included the variability in the type of statistical questions and client background in statistics, and the need to deal with the human side of the interactions.  Ultimately, the success of the consultation come down to the human side – the empathy on the part of the consultant.

Doug Zahn (Professor Emeritus, Florida State University) invited participants let him know What is your most troublesome stumbling block?  This elicited a very generous response from over 50 people, and so Doug focussed on four stumbling blocks that resonated with many of the examples provided.  These were:

• When the data have already been collected and can't answer the question of interest.
• Being left out of the study design, taken in too late.
• Eliciting the actual research question from the researcher.
• What to do when I don't know what to do Knowing how to say “I don’t know!”

An effective 21st century consultation relies on open communication that is co-operative and collaborative. It results in an agreed plan of shared work that is robust to scrutiny, and that is carried out in a timely way.  Seeing value in a consultant’s skills and contribution, the client identifies a resource for future work.

It results in an agreed plan of shared work that is robust to scrutiny, and that is carried out in a timely way.  Seeing value in a consultant’s skills and contribution, the client identifies a resource for future work.

In a wanted conversation the consultant endeavours to find out what the client wants from the consultation, rather than making assumptions or, worse, guessing.  Doug emphasised the expanded wanted conversation where a consultant strives to elicit all that the clients wants, in an iterative process. Importantly, there should be agreement about the final list of all wants, and this should reflect consideration of relevant stakeholder requirements and application of the project results.

Consultants need to be honest about their capacity to meet client needs, and about the limitations of a client’s project; Doug advised we look for velvet gloves in delivering these messages!

Doug will be the Keynote Speaker in December at the Statistical Consulting Network 2020 meeting.  This is a virtual event where statistical consultants can connect, present their ideas, discuss best practice and more.  You can find more information here and you can register via the SSA website.

There was a lot of lively discussion in the chat during the webinar.  In brief, here is a summary of some contributions.

On negotiating authorship, or the offer of co-authorship in lieu of payment

• Every university in Australia has an authorship policy. Consultants working in unis should be aware of it and make sure their clients are aware of it.
• For students I would be referring them to various guidelines about what warrants authorship... there are a lot of resources available and education is key.
• The SSA Statistical Consulting Network could also come up with a statement on co-authorship specific to statistical consultants to publish on the website.  Will let you know plans as they develop.
• We find it helpful to have a policy document on co-authorship that can be produced.
• Even if co-authorship is not on the table, appropriate acknowledgement of consulting support should be explicitly discussed.  We also have statements around this when clients come to our service.
• Here is the Australian code of conduct around research authorship.
• ICMJE guidelines on co-authorship can/could also be applied to other substantive areas of application

Is it better to do something wrong (at client's direction) and get paid or do nothing and not get paid?

• Sometimes an older, stubborn client may be persuaded with issues of reputation; that is one thing they are concerned about.  "The technique you want to use is no longer respected." (expressed politely)
• If you can walk away then you may have to do so - otherwise you may suffer reputationally
• I think Dunning (a psychologist) had studied on why older/ specialised client/people refused to take in advice, I cannot remember the exact study sorry. only thing I can remember is that they did trials with London taxi drivers and proved that when someone knew something very well, they tend to question new/conflicting information strongly.
• When I was early in my career in medical/hospital consulting, I put in my mind that often these people came in with an offsider. So, in that situation I convinced myself that they were more scared of me than I was of them (helped me relax with a bit of an internal smile to myself)
• Other
• This could be useful re correct versus useful reporting of frequentist results: https://discourse.datamethods.org/t/language-for-communicating-frequentist-results-about-treatment-effects/934

SSA members can watch the recording of this webinar here.

Sue Finch
Co-chair Statistical Consulting Network