Your proposition is correct if one only wants to travel to an instantaneous moment in the past. After that moment, though, the particles wouldn't necessarily behave in the same way as they did in the past.