Ok,whenever distributionm of mass changes,the CM will in general change.
But that's a geometrical problem.Here you are dealing with DYNAMICS.
And since CM was stationary w.r.t. water initially,and F_ext acting on the total system=0,the position of CM w.r.t. water will be the same all the way.
Because,to change the position of CM relative to water,you are to make it move.So,ther must be an initial acceleration,so that F_ext is not zero.If you are still not convinced,cross this point.
I think what I did was correct...but,the logic should be different.
Let me choose the origin of co-ordinates as the geometrical midpoint of the boat.The boat moves relative to water and the origin of co-ordinates will change relative to a frame attached to water.Whereas the position of CM will be stationary relative to water.That is,the boat will move,but, CM will not move relative to water.It is also clear from the statement of the problem.
So they key to this problem is to find the position of the CM relative to the center of the boat, both before and after A and B move. The difference in the positions of CM relative to the center of the boat must be the distance the boat moved in the water to keep the CM at the same location in the water.
So... let's do "after" first. When they are both at the center of the boat, the CM is in the middle, relative to the middle, its position is zero!
Ok, now let's find the cm when they are at opposite ends of the boat. We need to consider the masses and positions of all three objects: A, B and M (the mass of the boat) relative to the center of the boat.
calc the pos of cm with:
x_cm = ( 1 / total mass) ( m1x1 + m2x2 + m3x3 )
person A m = 50, x = -2 meters
person B m = 60, x = 2 meterrs
boat 40 kg,x=0
xcm = ( 1 / 50 + 60 + 40) ( 50*-2 + 60*2 + 40*0 ) =
= ( 1 / 150 ) ( 20 ) = 0.133 meters
So,here it is the way to do it.The approach really does not make any problem because,we are interested in the difference.We have used a frame relative to which the CM changes,but,relative to water CM does not change at all.