Have you already accepted the other offer? If so I would be more worried about turning that one down. I think it's far more rude to turn down an offer you've already accepted than it is to try and accept an offer you've already turned down. Put yourself in their shoes by imagining if they did that to you. If you were accepted into a position and then they said "well, actually we slept on it and decided to go with this other guy instead," you would probably be pretty insulted/upset. On the other hand, if they turned you down and later said "actually, we DO want you," you might not accept the position, but you probably wouldn't be overly upset about them doing that.
So if you haven't accepted the other offer yet, it won't hurt to try. They might say no, but you may as well try. If you've already accepted the other offer, think a lot more carefully about it, because they probably won't be very happy about you accepting and then suddenly turning them down. In any case, the sooner you act, the better.