Basically, Google ranks pages based on the text that is visible on a page, and on the links that point to a page.
Google doesn't give out further details of its ranking methods because they don't want to give clues to spammers. But they do say that text on different parts of a page can have different weight (plain body text, page titles, section headings, etc.). And they do a lot of analysis of link patterns and other things to try to favor links that people make because they think a site is good, versus links that website owners exchange or buy outright just to try to increase their ranking.
Many links to PF probably come from .edu sites which are probably considered as generally more "trustworthy" than links from .com sites. Also, it probably helps that the name itself is simply "physics forums".