Double Slit Experiment: Dumb question that needs to be asked

[PeterDonis]

I don't think of it as separate.

In other words, you're ignoring the actual physics. The whole point is that the pattern you get with orthogonal polarizers at each slit (i.e., "which-way information") is different from the pattern you get with no polarizers. That's why we call the first a "diffraction pattern" (or two of them, one for each slit) and the second an "interference pattern". Thinking of them as "the same" is a misrepresentation of the physics.