1
$\begingroup$

i am testing soil chemistry in soil that has been samples in fields subjected grazing. So two levels ( grazing treatment and soil depth).

I have used a two-way ANOVA after testing for normalisation etc. I want to express how to report the data. I am using boxplots with the letters so letters indicate difference between treatment , depth and their interaction and i am reporting it as something like this:

"Two-way ANOVA of soil properties across three different soil depths under three grazing treatments with Tukey’s post-hoc test for multiple comparisons. Result is reported as the mean ± SE (n = 15). Different uppercase indicate the significant differences between graze treatment at each soil layer, different lowercase letters indicate the significant differences (P<0.05) along soil depth at each grazing treatment and letters with an asterisk indicate differences between the interaction".

I am not sure if this is an appropriate way to word it?

Improve this question
$\endgroup$
4
  • $\begingroup$ Could you maybe provide an example of the kind of plot you're using ? I'm a little perplexed about using capital letters and small letters for different factors, and also about then using asterisks. ... But this may a good way to display your results... $\endgroup$
    – Sal Mangiafico
    Commented Feb 13, 2023 at 19:14
  • $\begingroup$ I don't have a plot at the moment but i was thinking of using captital letters to show differences between graze treatment and lowercase letters to show differences in depth only when there is no significant interaction because it is not appropriate to report or discuss the underlying single terms in that case. Where there is an interaction i was thinking of using a capital letter with an asterisk. $\endgroup$
    – martyn
    Commented Feb 14, 2023 at 13:45
  • $\begingroup$ I guess my best advice is to try to mock-up a plot with your lettering scheme, and run it by a few people, to see how clear the presentation is. ... Perhaps a typical interaction plot will work fine for your purpose. $\endgroup$
    – Sal Mangiafico
    Commented Feb 19, 2023 at 20:21
  • $\begingroup$ To answer part of the question, often the wording is "[groups] sharing a letter are not significantly different ( p > 0.05)." The reason to prefer this over "different letters indicate ... significant difference..." is that often when constructing a compact letter display, you get results like "a, ab, b". You want it to be clear that "a" is not different from "ab", but "a" is different from "b". $\endgroup$
    – Sal Mangiafico
    Commented Feb 19, 2023 at 20:25

1 Answer 1

Reset to default
2
$\begingroup$

This is related to so-called "compact letter displays" for distinguishing among outcomes. Russ Lenth discusses this in one of the vignettes of his emmeans package:

Another way to depict comparisons is by compact letter displays, whereby two EMMs sharing one or more grouping symbols are not “significantly” different ... I really recommend against this kind of display, though, and decline to illustrate it. These displays promote visually the idea that two means that are “not significantly different” are to be judged as being equal; and that is a very wrong interpretation. In addition, they draw an artificial “bright line” between P values on either side of alpha, even ones that are very close.

That issue extends to your decision to choose a type of display "only when there is no significant interaction." That puts undue weight on a sharp cutoff of interaction P-values. You've presumably done all pairwise comparisons in your Tukey test. Show them.

The vignette linked above and the interactions vignette suggest alternatives for displaying such results. If you want to compare P-values in particular, consider the "Pairwise P-value plots" that were developed as an alternative to compact letter displays.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.