Toolmaker here, (machinist) when grinding or sanding materials, the abrasive will cut small chips as it scratches the surface of the material. Size and shape of the chips will vary depending upon the material, coarseness of the abrasive etc.
Let's look at steel for example. the sparks you see flying off of the steel when grinding are small pieces of the steel being scraped off the surface and burning in the air from the frictional heat being generated and the high surface to volume ratio of the steel.
Using high pressure and sharp abrasives, can make steel chips large enough to easily see the resulting shape. Most will look tiny fish hooks and steel wool. Factor in the wear of the abrasive, oxidation of the alloying elements in the steel, etc. there is a fair amount of particles that could be nasty if they get in your lungs.
The good news, is the particles usually are large enough to be easily filtered out with a good respirator and proper ventilation. The bad news is, it's not the large particles that are the problem, most of those can be dealt with by the body's defenses. The smaller particles that get inhaled deep into the lungs are what causes silicosis, lung cancer etc if enough are inhaled for long enough.
It's also a matter of the shape of the particles, Asbestos is most problematic because it breaks up into microscopic needle shaped bits. They can go deep into the lungs and every time the lungs expand and contract they move and destroy cells. The repeated damage builds up and eventually creates health issues. But with the proper Personal Protective Equipment or PPE, even asbestos can be dealt with.
That said, we are constantly breathing dust, pollen, carbon from smoke etc. Almost everything around us is some form of an oxide, rust, silica, aluminum oxide and so on. Yet we survive for the most part without problems.
People live in super dusty environments all over the world, think of the desert people, surrounded by massive amounts of silicon dioxide sand blowing around constantly. Mostly unnoticed and unconcerning.
So small particles from a scratched surface probably aren't going to cause any problems unless in large quantities, with limited ventilation. Not counting clearly super dangerous materials such as Plutonium, mercury, arsenic etc. Those you really want to avoid, obviously.
