I think my recent article may help, seeing as the outcome is binary.

“Understanding between-cluster variation in prevalence and limits for how much variation is plausible”, by myself and Daniel Farewell, in Statistical Methods in Medical Research.

https://journals.sagepub.com/doi/full/10.1177/0962280220951831.

Abstract

In clinical trials and observational studies of clustered binary data, understanding between-cluster variation is essential: in sample size and power calculations of cluster randomised trials, for example, the intra-cluster correlation coefficient is often specified. However, quantifications of between-cluster variation can be unintuitive, and an intra-cluster correlation coefficient as low as 0.04 may correspond to surprisingly large between-cluster differences. We suggest that understanding is improved through visualising the implied distribution of true cluster prevalences – possibly by assuming they follow a beta distribution – or by calculating their standard deviation, which is more readily interpretable than the intra-cluster correlation coefficient. Even so, the bounded nature of binary data complicates the interpretation of variances as primary measures of uncertainty, and entropy offers an attractive alternative. Appealing to maximum entropy theory, we propose the following rule of thumb: that plausible intra-cluster correlation coefficients and standard deviations of true cluster prevalences are both bounded above by the overall prevalence, its complement, and one third. We also provide corresponding bounds for the coefficient of variation, and for a different standard deviation and intra-cluster correlation defined on the log odds scale. Using previously published data, we observe the quantities defined on the log odds scale to be more transportable between studies with different outcomes with different prevalences than the intra-cluster correlation and coefficient of variation. The latter increase and decrease, respectively, as prevalence increases from 0% to 50%, and the same is true for our bounds. Our work will help clinical trialists better understand between-cluster variation and avoid specifying implausibly high values for the intra-cluster correlation in sample size and power calculations.

Dear ANZStaters,

I am trying to find out the relationship between a. the ICC for surgeons and the b. the variation due to surgeons.

In Udyavar,2018 (The impact of individual physicians on outcomes after trauma: is it the system or the surgeon?") both the ICC ("Surgeons with higher mortality rates were not clustered at specific hospitals, as the intraclass correlation for surgeon level mortality rates was 0.02") and the quote in the subject above were given. I am doing a systematic review and wonder how the ICC and this particular variation are related (the relationship is not that the ICC is the square of the variation).

The reason is that some papers list the variation due to the surgeon while other papers show the ICC (Intra-class correlation coefficient).

(The ICC is important as even a small ICC can have a substantial design effect - if you cluster a randomized controlled trial by practitioner - surgeon for example - you will need substantially more patients to gain sufficient statistical power than if the ICC is nil).

Does anybody know how the ICC for a practitioner and the variation due to a practitioner are related?

I am looking for a relationship between the one-way random-effects ICC and the patient outcome variation due to the surgeon in a multilevel (two levels or three levels with hospital, surgeon and patient) model.

Cheers,

Christoph