^ It depends on the field and what you mean by "dealing with numbers." If you mean computations, then sure. If you mean literally "seeing" numbers, you'll be getting quite used to them in particular applied math fields (numerical analysis, analysis, PDEs, ODEs, etc.). You usually don't deal with numbers until you actually get down to the computations. In the theory of these fields, it's still mostly abstractions (elements, sets, mappings, etc.).
But in other fields as mentioned, you almost never use numbers to deal with anything. Rather you're almost never working in the set of reals again. You simply work with mappings and properties of new definitions, seeing what comes from it (geometry/topology, algebras, logic, etc.).