We could say so. But, as you have realized, that would render the notion of median meaningless in this case.
One could make a sneaky argument that the conventional interpretation of median is the arithmetic mean of the upper and lower bound on the set of values that divide the set in half. If all values in the range [0,1] divide the set [0,1] in half then it follows that the "median" is the mean of 0 and 1, i.e. 0.5. But this does not lead to a very robust notion of "median", so let's discard that argument.
What you really need for a meaningful notion of median in the case of uncountable sets is a way to compare "how many" set elements have values greater than the median with "how many" set elements have values less than the median. Using cardinality to compare "how many" isn't very good for this. So you need a different "measure".
An obvious measure to use for this particular application would be interval length or, equivalently, Lebesgue measure. So the median value is the one that divides the interval [0,1] into two sub-intervals of equal length.