Here is why it is problematic. You simultaneously assume that
1) The measured system (particle) exists even before measurement.
2) The dynamics is local.
3) The random decision happens when the detector clicks (not before).
Indeed, each assumption by itself seems reasonable. But the problem is that they cannot all be simultaneously true. At least one must be wrong. You must give up at least one of them.
Let me explain why they cannot all be true. From 3) and 1) it follows that, immediately before the click, the system exists not only near one detector, but near both of them. But then, puff, at the time of click, the system suddenly ceases to exist near the detector that didn't click. How did this part of the system knew that the click happened near the other part? Since the two parts are spatially separated, there must have been some non-local (even if random) mechanism, which contradicts 2). Hence assumptions 1) and 3) contradict 2), which implies that it is not possible that all three assumptions are true.
And yet, you seem not be ready to give up any of the three assumptions. That's the problem.