The Casimir effect is not about the vacuum, because there are always charges involved. In the most simple textbook treatment you simplify the presence of the charges in the uncharged plates by applying appropriate boundary conditions, but from a microscopic point of view there are charges present.
The vacuum is, as was posted in one posting before, the ground state of a quantum field theory. There are of course many subtleties with this idea. Usually what we consider are free fields in empty space and built the bosonic or fermionic Fock space out of occupation-number basis-vectors (i.e., totally antisymmetrized product states of N one-body basis states; usually chosen as momentum-spin or energy-angular-momentum states).
Already when you consider external classical fields, e.g., the famous case of a strong electrostatic field, interesting features concerning vacuum states occur. In this case, the socalled Schwinger-pair-creation mechanism is predicted but not yet experimentally confirmed: there are spontaneously electron-positron pairs created in this electrostatic field, because you have different in- and out-vacuum states that are connected by a Bogoliubov transformation. Of course, again here you don't deal with empty space but with space + a classical electric field, which itself has to be created somehow by charges.