Reviewing manuscript for labmate, and they use truncated bar graphs

[gravenewworld]

So I'm currently trying to review a manuscript for my labmate, who I have good working dynamics with, and I've been slowly combing through the rough draft and find that many of the bar graphs he has in the manuscript have truncated Y axes. He claims to have statistical significance between control and treated groups, yet the fold changes in several cases is quite small. The bar graphs representing these changes have truncated axes. Should I say anything about this? I feel like I'm walking on eggshells with this. There are small, but statistically significant data, that will look small if it is plotted as a fold change from 0-1 on the Y-scale. I'm going to give him the benefit of the doubt and assume it is just truncated to show that there is a change and not to mislead about how large the size of the effect is. How should a small, but apparently statistically significant fold change be represented? Should I say anything about this to him with the risk of sounding like I'm trying to undermine his work on cheap technical issues? Or should I simply privately message my PI and have him ask him about this?

 

[AlephZero]

Just my personal opinion: plots are meant to convey information. If the information content is that something varies over a small range and the variation is statistically significant, I think I would want to see a plot with an axis where I can read the values off the chart, not one with a scale that shows nothing except "all the values are nearly the same".

On the other hand, it may depend on the target audience for the document. Professional scientists should be able to draw their own conclusions so long as the plots are correct and clearly labeled. On the other hand a press release might need a different standard of "idiot-proofing."

 

[ParticleGrl]

What are you using to make the plots? Y-axis truncation is (for example) the default in excel. He might have just made the plots using default settings and not realize what he did.

If the statistical error bars are plotted with the data, I'd have no issue with a truncated graph (often its the easiest way effectively plot small differences)

 

[gravenewworld]

He is using Graphpad. I don't know, I just have a hard time swallowing a graph that shows a fold change with a Y-scale of 0.8-1, and the treated group only at around .9-.92 with SEM bars +/- 0.03-0.05. I mean is something that is a fold change of 0.04-0.1 really meaningful in biology?

 

[AlephZero]

I thought the question was "is the plot an accurate representation of the data and the statistical analysis of it" - and my answer tends towards "yes", from what you have said here.

The question "are the results and the analysis valid, and even if they are statistically significant according to the letter of the analysis methods used, are they of any real scientific interest" is a completely different issue - and I don't have any expertise to answer it.

 

[Andy Resnick]

<snip>Should I say anything about this to him with the risk of sounding like I'm trying to undermine his work on cheap technical issues? Or should I simply privately message my PI and have him ask him about this?

There are many reasons why you should first bring your concerns to your labmate- and based on your post, you should at least have an informal discussion. Your labmate's response to your concerns will dictate whether or not you then need to bring your concerns to the PI.

 

[EricVT]

Any time I make deliberate changes to the scale of presented data in a graph I specify the changes in the figure captions. For example "Range of ordinate axis set to 0.5 to 0.7 to emphasize variations in measurement results." or something along those lines. I don't think there is anything inherently wrong with the practice as long as it is done transparently.