So the way we learned to solve elastic collisions is to use the center of mass reference frame. I calculated that the COM is moving at v=1.66 m/s relative to the lab frame. Next I calculated the velocity of the white ball to be +1.66 m/s relative to the COM frame and the black ball to have velocity of -1.66 m/s relative to the COM.

I know that after the elastic collision the velocities relative to the COM just switch signs so therefore the velocity of the white ball should be moving away from the COM at 1.66 m/s at an angle of 28 degrees with a negative x component. To calculate the x and y components relative to the COM I did x = -1.66cos(28) and y = 1.66sin(28). Then when I convert it back into the lab frame I get x_lab = -1.66cos(28) + 1.66 and y_lab = 1.66sin(28) + 0. Finally I use the Pythagorean theorem to find the final velocity and its wrong.

I have been trying this problem all day and I can't figure out what is wrong.

If the collision were "head-on" then the velocities would just reverse direction in the COM frame. But, you have a glancing collision. If the angle for the final velocity for the white ball is 28 degrees in the lab frame, it will not be 28 degrees in the COM frame. Construct a vector addition diagram showing how the final velocity of the white ball in the lab frame is related to the final velocity in the COM frame.