This relies on the law of conservation of momentum, initial momentum = final momentum.
In the initial conditions, the only thing that is moving is the bowling ball, moving "up" the lane at 12 m/s. We know that the bowling ball is 5 times the mass of the pin (m_p). So we can use this to calculate the initial momentum as,
(12 m/s)*(5 m_p) = 60 m_p up.
So we know that the final momentum must equal this.
Say you put a coridinate grid out on the bowling lane, with up (the original direction of travel by the bowling ball) in the +y, and right as +x.
The final conditions saw the pin flies off at 80* from the original bowling ball direction (we will say right, in the +x direction), meaning the pin has some momentum in the +x direction. This was not the case in the original conditions, all the momentum was stright up, so in order to cancel this out, the bowling ball must be traveling at some angle to the left in the -x direction.
Also we know the pin [and bolwing ball] have some final momentum in the stright up (+y) direction. We know that the ball's and the pin's momentums must add together to give the final upward momentum, which is equal to the initial upward momentum.
I will leave all the math up to you.