Single wire Earth return pertains only to the primary (19 kv, etc.) circuit from the substation. Each customer still has a neutral wire (grounded/earthed conductor) in the respective secondary circuit from the pole transformer to the building fuse/breaker panel.
In a SWER system one pole transformer can also server multiple customers.
In rural areas the overall amount of current drawn by the primary circuit (at primary circuit voltages) is usually small enough that the voltage drop across the Earth return is not too great. If the current draw got too large for the Earth conductivity, as would happen in an urban or suburban setting, then the primary voltage and therefore the customers' secondary service voltage would drop noticeably. Because the current draw could vary greatly over the course of a day or even an hour, automatic voltage regulation would be needed.