My answer to the question in the thread title:
There's no theory-independent answer to questions like these. Physical terms are defined differently in different theories. The term "inertial frame of reference" is especially tricky because there are least two different definitions in GR (and I suppose, also in SR). It can refer to a coordinate system or a frame field. I will only be talking about coordinate systems.
The closest thing to a theory-independent answer that I can think of is this:
An inertial frame of reference is a coordinate system that the theory associates with a pair (p,C) where p is an event in spacetime, and C is a curve through p such that an accelerometer moving as described by C would show 0 acceleration at p.
The exact details of how this association is made are however different in different theories. This answer works for pre-relativistic classical mechanics, SR and GR. The only caveat is that GR may associate
many coordinate systems with a pair (p,C), not just one.
A coordinate system is a function from a subset of spacetime into ##\mathbb R^4##. If x is a coordinate system and p is an event in the domain of x, then x(p) is a 4-tuple of real numbers, called the coordinates of p in x.
I'm a big fan of the following approach: Suppose that we would like to find all theories of physics such that
- The theory's model of space and time is ##\mathbb R^4##.
- The theory is consistent with the principle of relativity
- The theory is consistent with the principle of rotational invariance of space.
- Inertial coordinate systems are defined on all of spacetime.
Then we can make progress by trying to find all the functions of the form ##x\circ y^{-1}## where x and y are inertial coordinate systems. These are functions that change coordinates from one inertial coordinate system to another. (Note that we have ##(x\circ y^{-1})(y(p))=x(p)##). To proceed, we must interpret the statements on the list as mathematical statements. Nothing could be more natural than to require that these functions are permutations of ##\mathbb R^4## that take straight lines to straight lines. This ensures that all inertial coordinate systems agree about which curves represent constant-velocity motion. We interpret the principle of relativity as saying that these functions should form a group, and we interpret the principle of rotational invariance as saying that this group should have the rotations of space as a subgroup.
One can then show (after some minor additional assumptions) that the group is either the group of Galilean transformations or the group of Poincaré transformations. Unfortunately the proof is very difficult. I think that this would have become a standard part of every introduction to SR if it had been easy.
So what does this have to do with this thread? Well, it suggests an answer to the question, at least for Newtonian mechanics and SR. Since there's exactly one inertial coordinate system for each element of the group, it makes perfect sense to think of the transformations themselves as inertial coordinate system. The identity map on ##\mathbb R^4## is
my inertial coordinate system, and every other Galilean/Poincaré transformation is someone else's.