Try using ADS (http://adsabs.harvard.edu/abstract_service.html). Look up the author(s) to see if you can find the paper. In the search results, the green 'X' means it's on arxiv.org, 'E' means the article can be read online, 'F' means there's a pdf version you can download (you may need journal access for these, you don't need it for 'X'). The title on the left includes where it was published (if it just says arXiv and a number, it has either not been submitted, not yet been accepted, or was turned down but posted anyway).
Make sure to click the 'physics' box on the first ADS page to search more journals.
It depends what field the article is in: you could use spires (http://www-spires.fnal.gov/spires/) for example. Alternatively, a lot of authors put a comment into the arxiv article with a journal reference when they have had it accepted for publication somewhere, so it may be sufficient to check the comments, and then search on the journal webpage.