Consider the product:
<br />
\left\langle p_{1}, \ldots, p_{m} | q_{1}, \ldots q_{n} \right\rangle \equiv\left\langle 0 | a_{p_{1}} \, \ldots a_{p_{m}} \, a^{\dagger}_{q_{n}} \ldots a^{\dagger}_{q_{1}} | 0 \right\rangle<br />
Let us start moving a_{p_{m}} to the right. With each creation operator it gives a contraction:
<br />
a_{p_{m}} \, a^{\dagger}_{q_{k}} = \delta_{p_{m}, q_{k}} + \zeta \, a^{\dagger}_{q_{k}} \, a_{p_{m}}<br />
and one factor of \zeta when the flipping occurs. When it reaches the furthest right, it annihilates the vacuum giving zero a_{p_{m}} \, | 0 \rangle = 0. Finally, we are left with the following sum:
<br />
\sum_{k_{m} = 1}^{n}{\zeta^{k_{m}} \, \delta_{p_{m}, q_{k}} \, \langle 0 | a_{p_{1}} \, \ldots a_{p_{m - 1}} \,\ldots a^{\dagger}_{q_{k_{m} + 1}} \, a^{\dagger}_{q_{k_{m} - 1}} \ldots | 0 \rangle}<br />
The brackets in the sum differ from the original one by a a_{p_{m}} and a a_{q_{k_{m}}} and each one has a factor of \zeta^{k_{m}}, where \zeta = +1 for bosons and \zeta = -1 for fermions.
We may repeat this procedure for a_{p_{m - 1}} for each of the terms in the sum. Then, we will get a double sum with one pair of creation-annihilation operators less and some power of \zeta. Continuing this procedure, we will be left with either a bracket of the form:
<br />
\langle 0 | a_{p_{1}} | 0 \rangle = 0, \; m > n<br />
or:
<br />
\langle 0 | a^{\dagger}_{q_{j_{n - m}}} \, a^{\dagger}_{q_{j_{n - m - 1}}} \, a^{\dagger}_{q_{j_{1}}} | 0 \rangle = 0, \; n > m<br />
So, we can only get a non-zero matrix if m = n, i.e. we have the same number of annihilation operators as creation operators. Furthermore, due to the Kronecker deltas, we must have the single-particle states p_{1}, \ldots , p_{n} be some permutation of the states q_{1}, \ldots q_{n}. Finally, the power of \zeta is equal to the number of transopsitions necessary to bring the sequence \{q_{1}, \ldots , q_{n}\} to be identical to the sequence \{p_{1}, \ldots, p_{n} (notice that \zeta^{2} = 1 for any kind of particle).
You could have used this rule if you wrote the one-body operator as:
<br />
O = \sum_{m, m'}{(m | O | m') a^{\dagger}_{m} \, a_{m'}} = \zeta \, \sum_{m, m'}{(m' | O | m) a^{\dagger}_{m} \, a_{m'}} - \zeta \, \sum_{m}{(m | O |m)}<br />
EDIT:
I noticed that you wrote the one-body operator with two creation operators. If this is so, then the matrix element is zero.