Hi all,

We've discussed this issue fairly regularly at the monthly Statistical Consulting Network meetings (last Wednesday of the month, 12-1 AEST). Personally I haven't seen hard data on this yet, but anecdotally things seem to have been changing rapidly over the last year or so, the main things we have noticed in academic consulting (PhD students etc) are:

- Maybe less people come for assistance than before, because they can answer their questions themselves. Whether or not those questions were answered correctly we can't know.

- Some consults involve less workload than they used to. In particular, implementing suggested procedures involves less work for the consultant - often we used to write example code to show clients how to do something but often we can point people in the right direction and they can sort it out themselves. If we do code ourselves that is also much faster.

- As mentioned previously there is a new type of client who comes with pages of code and ask if it is right. This would be much easier to assess if they also brought chat histories used to generate the code so you can see how the conversation developed rather than coming in right at the end. You could then also advise on prompting.

We developed an SSA statement on the use of generative AI largely in response to this last issue, emphasising the importance of saving chats and bringing them to consultations. As AI evolves our response to it should do so thoughts on updates to the statement would always be welcome :)

One final point I want to make is that, at least in academic consulting, a lot of time and energy is often spent establishing what the research question is. Well, what an answerable and interesting question is. This classic client type gets limited benefits from AI because the help it offers is a function of the quality of the questions you pose! A key part of the gig has always been helping clients to ask better questions, and that is true now more than ever.

Thanks for the thought provoking discussion! I've been following this closely, as well as the impact AI is having across white-collar professions.

Paul, I really appreciate you sharing your personal experience. I too have heard that AI may reduce demand for entry-level roles while leaving senior positions relatively affected.  My main concern with that is that today's senior people build their experience by grinding through the mundane/routine tasks early in their career. If AI takes away those foundational activities, it will be interesting, perhaps one might say concerning, to see what senior expertise look like in 10 years' time.

One under appreciated task that statisticians do is data checking and cleaning... I'm not sure how capable AI is at taking a messy colour coded spreadsheet and turning it into a dataset ready for analysis, but if AI could be trusted to do this I'm all for it! Unfortunately, I think this area is being ignored and it could be a case of garbage in, garbage out. Also, as others have said, checking whether the data fits the statistical models is tricky, since it could mean uploading potentially confidential data.

We’re seeing a few big changes – two in particular.

Joanna has already mentioned the first – the good ole “I’ve already done it in AI – I just need you to check it for me”. Which they expect to be quick. The problem is that it often isn’t quick at all. The AI code is often far harder to check than human written code (even impossible without further work) as it’s more verbose/complex than it needs to be, with no commenting, and a tendency to overwrite data without checking that processing/wrangling has been done right.  The client also often doesn’t really understand what or why AI has done what it has. So we need to spend time figuring that out with them. Compared to the past when clients had usually spent some time figuring out the basics and had enough confidence in their code/analysis that we didn’t need to check all of it – just the parts they didn’t understand (which they could point out and ask what they were worried about).

The 2nd is people turning up with AI advice which is completely wrong. So now we need to spend one consult convincing them its wrong and they need to do some of the fundamental thinking about their research questions and what’s possible with their study design and data. And then a 2^nd (which used to be the 1^st) to give them good advice. Attached to this is that AI often doesn’t check assumptions, so people are showing up already having written up results from a model that doesn’t fit their data and is thus wrong. This can become quite a barrier to convincing them – as they don’t want to acknowledge they have wasted all that time writing up! And means we are seeing people later in the research cycle – often close to deadlines meaning they have less time for better analysis and wind up with a less insightful understanding.

For us the immediate problem has been explaining that AI is leading to more work, not less!

So what does this mean going fwd?

I think the key is that people are going to need greater statistical literacy in order to ask AI the right Q’s and then check it has done things correctly. Which means universities might need to start offering courses where they teach researchers to be ‘architects/builders’ rather than ‘carpenters’.

For consultants I think it will mean more work at the beginning– to ensure the analysis fits the RQ and study design. Particularly for people without the requisite statistical literacy. And we guide people to do much more complex analysis than they would have been able to do in the past – rather than us doing that analysis for them. So regular meetings where we set them tasks which they use AI to help them do, and we check what they did from last time.

Practically this might mean businesses need to have at least 2 services. A “doing” service and a “consultant” service which is a much higher hourly rate. (Which might even turn out to be more lucrative if people need more help than they expect!)

On the topic more generally of "providing things that AI cannot", I find the interview with Virginia Dignum interesting, regarding her recent book ."The AI Paradox: How to make sense of a complex future." (Princeton University Press, 2026.)
https://press.princeton.edu/ideas/virginia-dignum-on-the-ai-paradox
One question in particular requires human input -- What assumptions can/should be made about the way that the data ha been sample, as they affect the intended use of the result from the analysis?