It depends on what you mean by "exist". Personally, I don't have a problem with the existing or not existing of physical terms. Just because the Maths runs out of meaning at infinity doesn't mean we can't use the Maths to describe and (more importantly) predict things. There is not a single experiment on anything that can start off with a truly static situation; you have to bring the kit into the Lab and turn it all on and then you have to turn it off and de-rig it, at the end. Your 'fixed' conditions are not actually fixed but we do not worry about it. But you still need to be aware of the time to wait before doing your actual experiment. The power supply volts have to stabilise, so does the floor of the lab and the temperature in the room - all to within the accuracy you are working.This seems to be an interesting and profound argument.
Are you saying that the electrostatic / magnetostatic field - one basic topic of classical electromagnetism may not actually exist ?
If the above doesn't upset you and if you are prepared to consider your experiment is valid then the deeper existence or non-existence of an 'electrostatic' field need not bother you either. I don't think this is any big deal - it's just practical.
Edit. People can get too philosophical about such matters and it's easy to treat Science as a Religion. Our scientific beliefs should be restricted to things that can be verified or falsified by experiment. More than that is outside the realm of Science.