I ran a linear model with orthogonal polynomials and a couple of factors. I then ran it through step() and was surprised to see the coefficients of the polynomials changed, very slightly when some powers were dropped. They are supposed to be orthogonal aren’t they?

After about 15 seconds, I realised they were orthogonal to each other but not orthogonal to the factors. They were very slightly correlated. So of course they could change (by a small amount since the correlations were small).

I asked both Copilot and ChatGPT5 if my thinking was correct. Not only Copilot but Chat got it wrong. Both of them told me that the reason was that the orthogonal polynomials for poly(x,10) and poly(x,9) would not be the same, when there were other factors in the model. Clearly they were sourcing the same erroneous reddit conversation.

Even when I pointed out that poly() can be run outside lm() both dug in, for a while, but eventually admitted that I was right. And both eventually pointed out that the problem was that they were correlated with the factors, as if they had told me something new!

The moral of the story is not only that AI can be wrong, completely. I think we all knew that already. The moral I think is that you should treat it as a research assistant with a severe case of Aspergers. Very clever but likely to cling to an idea. The conversation was focussed and stimulating and I came out at the end understanding better and also with the right answer, but only because I push back where required.

How this works when non-experts, such as my students, ask the same kinds of questions is another matter….