Yes, it would radiate like an antenna. You can make a transmission line behave like this by putting the voltage source in series. Exciting the 2 conductors in differential mode will allow it to behave like a TEM transmission line. Superconducting power lines with very low resistance should then be able to transfer power without having a coupled multi conductor system, like HV transmission lines do (they can use the axial electric field rather than TEM).
Not sure if this is correct though.
No, it isn't correct.
DC will not radiate from a line no matter how long it is.
If you connected just the positive terminal of a battery to a very long wire, eventually the whole wire would be charged positively WITH RESPECT TO THE NEGATIVE TERMINAL but not with respect to anything else. There would be no power available without a connection back to the negative terminal.
AC will not radiate either unless it has a return path. You cannot join one side of a generator to a long wire and expect it to radiate unless you connect the other side of the generator to a ground connection or another long wire going in a different direction.
Then it becomes like a dipole or a centre fed very long wire and it would radiate.
Capturing this radiated power would not be easy, though, since you would need another very long wire very close to the first one. This is because the power would be radiated equally in all directions at right angles to the first wire and another wire would only intercept a small portion of it.
Feeding ac into one end of a pair of parallel wires results in an inefficient radiator. That is what transmission line are used for. They are not perfect, though, and some power is radiated. The further apart they are, the less their fields due to opposite currents cancel out, so the more they radiate.