Thanks again to
@robphy for his good advice and suggested prompt. Really helped.
I also thank all the others who have contributed to this thread.
A summary (as I understand it):
Roughly, Fermi claims that given a curve, not necessarily a geodesic, and a point P on it, there exist coordinate charts in a finite neighborhood of P (finite in the sense of not-infinitesimal), such that the metric is ##~\delta_{\mu\nu}~## (or ##~\eta_{\mu\nu}~##) and the ##\Gamma##'s vanish, at any point on the curve inside that neighborhood.
Fermi's 1922 work, appeared as 3 separated "notes". The original text in Italian is available for free on the net. An English translation by D. H. Delphenich,
is available at the translator's website.
Fermi defined his law of transport (that we now refer to as "Fermi-Walker") as follows: at an arbitrary point on the curve, erect an orthogonal "n-tetrad" ##~\{\mathbf{\lambda}_{(\mu)}\}~## (the brackets on the sub-index indicate that these are not components) such that one of them, say ##~\mathbf{\lambda}_{(0)}~##, is tangent to the curve. Then parallel-transport the n-tetrad to the desired point on the curve, and follow with a rotation (or a boost, depending on the case) that will align the parallel-transport of ##~\mathbf{\lambda}_{(0)}~## with the tangent at the new point.
Then Fermi constructs his "normal coordinates". He does not define them by geodesics perpendicular to the curve. Rather, he considers immersion in an Euclidean space, and takes the projections of the straight lines. As I understand his description, the two definitions will agree only to first order in the deviation (of the point from the curve). This allows Fermi to deduce the form of the metric on and around the curve (as equations 13.69a+13.71 in MTW). He doesn't demonstrate explicitly that all the other "spatial" derivatives of the metric vanish. Later proofs of this point (like Levi-Civita's, and equation 13.69b in MTW) employ the "geodesicness" of the coordinates.
Then Fermi argues that for any curve ##L## and its "normal directions", there exists a transformation that will map it (as a whole or by parts) to a curve ##L^*## in Euclidean space of the same dimensions, such that the relations are preserved. He shows that the number of equations at hand equals the number of conditions to be satisfied. From this, he infers the claim stated above. He doesn't explicitly follow it all the way through. My understanding is that the normal coordinates ##~y^{*\mu}~## around ##L^*## are functions of the Cartesian coordinates ##~x^{*\mu}~##, and can be inverted. If we define new coordinates ##~z^\mu~## as functions of the normal coordinates ##~y^\mu~## such that ##~z(y)~## has the same functional form as ##~x^*(y^*)~##, the metric in the ##z##'s on ##L## will become ##~\delta_{\mu\nu}~## and the ##\Gamma##'s will vanish (since the ##~x^*##'s are Cartesian in an Euclidean space).
I'll refer to the coordinates that satisfy this condition as "Fermi Zero Coordinates" (FZC), and to the familiar ones from the F-W transport as "Fermi-Walker Coordinates" (FWC). When the base curve is a geodesic, both transport laws (parallel and F-W) are the same, and both FZC and FWC reduce to the well known Fermi Normal Coordinates (FNC). All further coordinate transformations that agree with the original coordinates up to second order in the deviations, will satisfy the same conditions for the metric and the ##\Gamma##'s on the curve. From the Physics point of view, FWC are more interesting and discussed in many texts. Levi-Civita's paper discusses FZC, and that's why I wanted to read it.
Levi-Civita's paper, "Sur l'écart géodésique" ("On the geodesic deviation", see OP), was published shortly before the second edition of his book, which was translated to English as "The Absolute Differential Calculus". Most of the introduction (of the paper) and sections 6-10, appear in the translated book (chapter VII). For some reasons, Levi-Civita omitted sections 1-5. He only quotes the results needed for reading 6-10 and provides a reference to the paper. Section 1 reproduces Riemann's normal coordinates around a single point. In the book, L-C discusses it in an earlier chapter, without the construction of normal coordinates (exercise 13.3 in MTW, as opposed to section 11.6 there). He also mentions FZC in the book, but rather than including the full construction of section 4, he resorts to a counting of equations vs. conditions on a footnote.
In his book "Non-Riemannian Geometry", Eisenhart extended Levi-Civita's construction (section 4 in the paper) to spaces with symmetric connections, with or without a metric (section 25). In particular, his proof holds in Riemannian spaces. While the math is not hard to follow, Eisenhart doesn't discuss the guiding ideas behind it, and it seemed (to me) somewhat like a magician's trick: it works, but you don't understand how it was obtained. The lack of a metric in the proof, further obscures the underlying motivation.
So, it all converges to reading section 4 in Levi-Civita's paper (and the previous ones, as a preparation). As far as I've managed to find out, no English translation of it is publicly available (over a century!). Eisehart's extension seems to be the closest thing.
I attach a translation of the introduction and sections 1-5 as html in text files. The translation is based on Claude's output. It is flawed, but readable and understandable. The main differences from the construction for FWC are the use of parallel transport of the n-tetrad rather than F-W, and allowing the normal coordinates to obtain non-zero values on the curve (required for satisfying the conditions). After reading it, Eisenhart's extension becomes clearer.
From the practical aspect, in F-W transport the unit tangent to the curve is a "bone" (as the "bein" in "vierbein"...) of the n-tetrad (n-bein). We need to find the transport of n-2 bones, since the remaining one can be obtained from orthogonality. In parallel transport, we need to find n-1 bones.
Levi-Civita and Eisenhart provide an explicit scheme for the construction of the old coordinates as functions of the new ("normal") ones. In many cases we know the coordinates of the curve as functions of its affine parameter (e.g. the Boyer-Lindquist coordinates of a static observer in Kerr spacetime). So naturally we may be interested in the inversion: finding the FWC or FZC as functions of the original coordinates. It seems that this problem was ignored for decades.
This paper addresses it.
Edit: for a reason that I don't understand, the .txt files don't show after submission of the post. Why?