But if you mean the sky was clear instead of blue, then I would imagine the sea to appear black since light can only reflect from so far down. It would be clear towards the shore (you would see the ground under teh water).
I think that it would still be primarily blue-green, since those are the wavelengths that propogate best through water. If you take a pure white (halogen, for instance) light source to a depth where sunlight doesn't exist, things are going to have that teal tinge. The situation shouldn't change just because of a lack of atmosphere, since at depth it's irrelevant. As whozum said, however, that's just my opinion.