It would matter in the case of traction, but I don't think in ideal circumstances, the distribution of mass affects the translational speed. I know it does in rotational speed, which would require some look into the moments of inertia. Different mass distributions (e.g. a solid cylinder, a hollow cylinder, a ball...etc) have different moments of inertia, which affect how much energy is required to get them to a particular speed. This is relevant when you pick your wheels/tires. You'll want to minimize your moment of inertia so that it takes the least amount of energy to get it up to speed. Off the top of my head, a hollow cylinder probably has the smallest, but that's just your typical tire. The effect might be negligible, but maybe try and make your wheels hollow in the center. Low inertia means it's easier to change speed, therefore it would get fast faster and slow down faster, which fits your goals of going fast and then stopping in a short amount of time.
Edit: Maybe make your car in the shape of a big hollow cylinder some how and exploit this fact. If the cylinder were wide, you wouldn't have much trouble going in a straight line, and you'd effectively have one big tire. If nothing else, it would look cool ;).