If you somehow had "solid lumps" of exotic matter, they would fall down, just like ordinary matter. The reason is the equivalence principle - with a negative gravitational mass, exotic matter would also have a negative inertial mass. Thus exotic matter would be repelled by the Earth's gravitational field, and hence fall towards the Earth.
But not all is lost. While you couldn't make anything actually "fall up", if you could safely handle exotic matter, you could make a heavy object lighter (lower mass) by adding exotic matter to it, to reduce the total mass of your spaceship/plane/whatever.
By putting together two large masses , one of exotic matter, and one of normal matter, you can even build something that would accelerate indefinitely. There will be a repulsive gravitational force between the two masses - but the normal matter will move in the direction of the force, while the exotic matter will move away from the force. If the two masses are EXACTLY equal, they will maintain a constant separation, and accelerate indefinietly. If gravity does not provide enough force, some sort of spring could be used to generate the force, or they could be charged to generate the force. Probably some sort of active triming will be needed to keep the two masses exactly equal - otherwise the separation between masses will not remain constant, and one will "pull away" from the other.
Unfortunately, exotic matter if it existed in bulk form would probably not be able to be handled safely. One of the other interesting features it would have would be a negative temperature. This means that if one had a lump of exotic matter that acted thermally with normal matter, one would have a run-away heat transfer. The kinetic energy of the exotic matter due to molecular motion (assuming it has molecules or something roughly equivalent, since we are assuming it occurs in a "lump") would become more and more negative. The kinetic energy of the normal matter would become more and more positve. The total energy would remain at zero, of course. The result would be fun as long as it lasted, but eventually the negative energy level of the exotic matter would become so high that it would implode any possible sort of container. Then this exotic matter would continue to exchange energy with its surroundings, until eventually it got a high enough velocity to escape the Earth's gravitational field.
This totally bizare behavior is so unphysical that I personally don't think bulk lumps of exotic matter are likely to exist.