The sound comes from the ball vibrating over different surfaces - the vibration will be different for different textures ... pebbly surfaces make a different noise to a steel grate for example. So it stands to reason that it should be different over something soft too, like felt.
So what is the problem?
When you walk on carpet, your steps sound different from when you walk on wood, or concrete, or a metal catwalk ... does that bother you the same?
If you shout in a bare room, you get an echoey effect (very noticeable if you record it) but, if the room is carpeted and furnished, there is almost no echo, and if you line the walls and ceiling with soft material or foam rubber shapes then the echo can be eliminated completely. Why is this?
Have a think about these things - what sort of things will affect how you hear sound waves - and you should be able to figure it out.