As has been pointed out, it is possible to make the metric [tex]g_{\mu \nu}[/tex] be equal to the Minkowski metric [tex]\eta_{\mu \nu}[/tex] at any given (nonsingular) point, and to make all of its first derivatives vanish there, by using Riemann normal coordinates. However, the tensor [tex]R \indices{^{\mu}_{\nu}_{\sigma}_{\rho}}[/tex] (the Riemann curvature tensor) is an isometric invariant of the manifold, which is to say that if you keep the metric [tex]g_{\mu \nu}[/tex], then the curvature stays, too.
However, if we are only dealing with weak gravitational fields, then we may regard general relativity as the theory of a symmetric tensor [tex]h_{\mu \nu}[/tex] propagating against a flat, Minkowskian background; this is called linearized gravity, and is used to study gravitational waves. Specifically, we can write [tex]g_{\mu \nu} = \eta_{\mu \nu} + h_{\mu \nu}[/tex], where the perturbation [tex]h_{\mu \nu}[/tex] is assumed to contribute significantly to measurable quantities only to first order (this is the "weak-field" assumption). We then have [tex]g^{\mu \nu} = \eta^{\mu \nu} - h^{\mu \nu}[/tex] (again, to first order), and we can raise and lower indices using [tex]\eta[/tex] (in fact, [tex]h^{\mu \nu}[/tex] is defined here as [tex]\eta^{\mu \sigma} \eta^{\nu \rho} h_{\sigma \rho}[/tex]). We can then go on to derive the Riemann, Ricci, and Einstein tensors, and the Ricci scalar, to get the field equations. Alternatively, and in keeping with the viewpoint of a field theory on a flat background, we can define the Lagrangian
[tex]\displaystyle \mathcal{L} = \frac{1}{4} [2h \indices{^{\mu}^{\nu}_{,\mu}} h_{,\nu} - 2h \indices{^{\rho}^{\sigma}_{,\mu}} h \indices{^{\mu}_{\sigma}} + \eta^{\mu \nu} h \indices{^{\rho}^{\sigma}_{,\mu}} h_{\rho \sigma, \nu} - \eta^{\mu \nu} h_{,\mu} h_{,\nu} ] \textrm{,}[/tex]
which, when varied with respect to [tex]h_{\mu \nu}[/tex], gives the linearized Einstein equations.