Well, your question is very easily answered. The answer has two parts:
(a) Relativistic mass is an idea that is outdated since 1907 and should not be used and taught anymore. It leads to confusion. The mass in relativity is alwasy understood as the invariant mass of an object and as the name tells you it's a scalar, i.e., independent of the frame of reference.
(b) ##\vec{F}=m \vec{a}## doesn't hold in general, even not in Newtonian physics. The correct law, already stated by Newton, is ##\vec{F}=\mathrm{d} \vec{p}/\mathrm{d} t##. Now in the theory of relativity this is a very complicated quantity, and it is easier to use proper time (assuming we have a massive particle; the massless case is more complicated and of no practical relevance since the only massless particle-like objects are photons and shouldn't be treated as classical particles to begin with). The proper time of a particle is defined by
$$\mathrm{d} \tau=\mathrm{d} t \sqrt{1-\vec{v}^2/c^2},$$
where ##t## is the coordinate time wrt. an inertial reference frame. Now you define the four-vector
$$p^{\mu}=m \frac{\mathrm{d} x^{\mu}}{\mathrm{d} \tau}.$$
This is the four-momentum vector. It's time component is the energy of the particle (up to a factor ##c## which is due to the unfortunate joice of SI units), including its rest energy
$$p^0=m c \gamma=\frac{m c}{\sqrt{1-\vec{v}^2/c^2}}.$$
The three spatial components are relativistic momentum
$$\vec{p}=\frac{m \vec{v}}{\sqrt{1-\vec{v}^2/c^2}}.$$
The mass is a scalar and the covariant energy-momentum relation of a classical particle is given by the "on-shell condition"
$$p_{\mu} p^{\mu}=m^2 c^2,$$
which leads to the energy-momentum relation
$$p^0=\frac{E}{c}=\sqrt{m^2 c^2 +\vec{p}^2}.$$
This shows that the relativistic energy includes the rest energy of the particle
$$E_0=c p^0|_{\vec{p}=0}=m c^2.$$
