X/Y does follow a beta distribution! (Assuming they have the same second parameter. This is very important). It's called the beta distribution of the second kind with parameters alpha_x and alpha_y. The F distribution is simply b*X/Y where b>0.
I'll show you why X/Y is called the beta distribution of the second kind.
Suppose X~\Gamma(\alpha_1,\beta) and Y~\Gamma(\alpha_2,\beta), X and Y independent. What is the distribution of U_1=\frac{X}{Y}?
Now this is a multivariate transformation, (
http://www.ma.ic.ac.uk/~ayoung/m2s1/Multivariatetransformations.PDF see here if you don't know how to do these), so we will use U_2=Y as an auxillary equation.
So, g_1(x,y)=x/y and g_2(x,y)=y where x,y are positive reals (because they come from a gamma distribution) now it should be clear to see that g_1^{-1}(x,y)=xy and g_2^{-1}(x,y)=y. Note how g_1 and g_2 have range (0,+infty).
Therefore, f_{(U_1,U_2)}(u_1,u_2)=f_{(x,y)}(g_1^{-1}(u_1,u_2),g_2^{-1}(u_1,u_2))|J|. As an exercise you can show that |J|=u_2
Since X and Y are independent f_{(X,Y)}=f_X f_Y
Now f_{(U_1,U_2)}(u_1,u_2)=f_x(u_1u_2)f_y(u_2)u_2=\frac{e^{-\frac{1}{\beta}(1+u_1)u_2}u_1^{\alpha_1-1}u_2^{\alpha_1+\alpha_2-1}}{\beta^{\alpha_1+\alpha_2}\Gamma(\alpha_1)\Gamma(\alpha_2)}
(I have done some simplifying)
Now, we don't want the pdf of (U_1,U_2) we want the pdf of U_1, so we integrate over the joint to get the marginal distribution of U_1.
f_{U_1}(u_1)=\frac{u_1^{\alpha_1-1}}{\beta^{\alpha_1+\alpha_2}\Gamma(\alpha_1)\Gamma(\alpha_2)}\int_0^{+\infty}u_2^{\alpha_1+\alpha_2-1}e^{-\frac{1}{\beta}(1+u_1)u_2}du_2
But the integral is just a gamma function (after we change variables). So this means that \int_0^{+\infty}u_2^{\alpha_1+\alpha_2-1}e^{-\frac{1}{\beta}(1+u_1)u_2}du_2=\frac{\Gamma(\alpha_1+\alpha_2)\beta^{\alpha_1+\alpha_2}}{(1+u_1)^{\alpha_1+\alpha_2}}.
Plugging this in we get f_{U_1}(u_1)=\frac{\Gamma(\alpha_1+\alpha_2)u_1^{\alpha_1-1}}{\Gamma(\alpha_1)\Gamma(\alpha_2)(1+u_1)^{\alpha_1+\alpha_2}}=\frac{u_1^{\alpha_1-1}}{\beta(\alpha_1,\alpha_2)(1+u_1)^{\alpha_1+\alpha_2}}
So there we go! U_1=X/Y is distributed as that. Now why is this called a beta distribution of the second kind? If you do some transformations you should see that \beta(\alpha_1,\alpha_2)=\int_0^1x^{\alpha_1}(1-x)^{\alpha_2-1}dx=\int_0^{+\infty}\frac{x^{\alpha_1-1}}{(1+x)^{\alpha_1+\alpha_2}}dx
I hope someone finds this interesting ;0