Acceleration, like almost everything else in relativity, is a relative concept. Although everyone agrees what zero acceleration is, they disagree on the magnitude of a non-zero acceleration.
Thus an acceleration that is constant according the object feeling the acceleration (constant "proper acceleration") would, according to an inertial observer, be a decreasing acceleration.
It turns out, surprisingly, that,
for linear motion of a constant mass, the "force" is invariant, i.e. all inertial observers agree on its value. (The appropriate relativistic equation for force is \textbf{f} = d\textbf{p}/dt, not \textbf{f} = m \, d^2\textbf{x}/dt^2.)
Thus a constant force applied to a constant mass in the same direction as its velocity causes constant proper acceleration but decreasing "coordinate acceleration" relative to an inertial observer.
The two accelerations are linked by
a_{proper} = \gamma^3 \, a_{coord}
\gamma = 1/\sqrt{1-v^2/c^2}
This is proved in
this thread, posts #13,#14,#15 equation (12) (with a correction in post #28).
For an example of a force acting on a non-constant mass, consider friction. Friction may cause a body to heat up, and that gain in energy is accounted for as an increase in mass (i.e. rest mass). Another, more obvious, example is a rocket ejecting burnt fuel and therefore losing mass.
_______________
Proof that, for linear motion of a constant mass, force is invariant:
In the notation of the posts I referred to above
p = m \, dx/d\tau = mc \, \sinh\phi
\frac {dp}{d\tau} = mc \, \frac{d\phi}{d\tau} \, \cosh\phi
f = \frac {dp}{dt} = \frac {dp/d\tau}{dt/d\tau} = mc \, \frac{d\phi}{d\tau} = m \, a_{proper}