PQ = (8, -8, -8), so (3/8)PQ = (3, -3, -3), which doesn't get you a point on that line. What you can do, though, is calculate OP + (3/8)PQ = (-4, 2, 6) + (3, -3, -3) = (-1, -1, 3), which is the same as the answer shown in your problem.
What I've done above is to look at things from a vector perspective. OP is a vector from the origin to point P. PQ is a vector from P to Q, but it is equivalent to the vector from the origin to the point (8, -8, -8). To get to a point 3/8 of the way from P to Q, we have go from the origin to point P, and then 3/8 of the distance from P to Q.
Whoever solved the problem used the midpoint formula three times, not twice: the first time to get a point M halfway between P and Q; the second time to get N, a point halfway between P and M (and a quarter of the way from P to Q); and the third time to get a point R halfway between N and M (3/8 of the distance from P to Q).
Maybe this "graphic" will help you understand better.
P----------------------------------------------------Q
P-----------------------M
P---------N------------M
P---------------R
d(P,M) = (1/2)d(P,Q)
d(P,N) = (1/4)d(P,Q) = d(N,M)
d(P,R) = (1/4)d(P,Q) + (1/2)d(N,M) = (1/4)d(P,Q) + (1/8)d(P,Q) = (3/8)d(P,Q)