You seem to be confused about some concepts. (The exact same stuff I was confused about less than 4 months ago!). Hopefully my explanation fixes things for you.
Lets walkthrough part a,
What do we know? There is no friction between the ground and m2, but friction between m1 and m2. Think about this as if you were to glue two boxes together and then push them on ice.
There is a friction force on m1 opposite in direction to Fa. And by Newtons third law, on m2, there is the same friction force but opposite direction.
The question tells us to find the force applied to m1 given m1 and m2 do not slip (a fancy way of saying m1 and m2 move with the same acceleration, a1=a2).
Draw a FBD (always do this) for m1 using the standard x/y coordinate system
In the y-axis we have: gravity, normal force
In the x-axis we have: applied force (say +x), friction (-x)
From F=ma in the y direction, we know N = mg
The mass doesn't slip ==> static friction
Ff = u_s N = u_s mg
F=ma in the x direction, Fa = m1gu_s + m1a1 = m1gu_s + m1a (1)
Now for the second mass, draw the FBD. From Newtons third law, we know Ff (between m1 and m2) acts in the +x direction, as well as gravity and normal force.
F=ma in x direction, Ff = m2a2 ==> a2 = a1 = a = m1gu_s/m2 (2)
Now sub (2) into (1) and get Fa = m1gu_s(1 + m1/m2) (the same thing you got).
I think i got part a correct, double check that for me,
Yes, you did.
but I am stumped on part b. if you are increasing the force, then now there is kinetic friction. would M2 (the mass on the bottom ) be moving at all? wouldn't the acceleration be 0 because there is so much applied force that it would overcome the static force.
As you can see from my solution of part a, the mass would be moving! Draw a separate FBD. The friction from m1 on m2 makes it move in the +x direction. The acceleration is NOT zero. (But the acceleration of m1 with respect to m2 is indeed zero, they move as if they were one object IN PART A ONLY)
also do we assume that the second applied force (twice the size of the first one) is applied after the first one is applied or do we disregard the first force completely? part a is done in the first image, and I started part b by drawing the free body diagram in the second picture.
You can pretty much ignore part a, the only information you need is that the two masses slip and move with different accelerations!
Read my solution of part a, fix your misconceptions, and you'll see part b is just as easy as part a.
Note that for part a we found Fa with the MAXIMUM static friction force. In part 2, since Fa is doubled, you know for sure it slips, and thus kinetic friction is used.
