Deriving RA/Dec from a known sky to an unknown sky

solarblast

I hope I can make sense out of this question. Suppose I have an image of the sky and know on that image say the RA/Dec and x,y position of 20 stars, and the time the image was taken. The zenith is somewhere in the middle. Now suppose one hour later I have another image of the sky, and would like to determine the RA/Dec of an unknown object. (x,y) would be known. I would think that I could find the az/al for the 20 stars in the first image, and apply them to the unknown object. Next apply a transform from az/el to RA/Dec in the second image--noting the time change. Would that work?