Hi vishugupta,
The bible for the industry is Parker's O-ring Handbook. They provide standard gland dimensions for all types of applications and a section on material compatibility. The book can be downloaded here:
http://www.parker.com/literature/ORD 5700 Parker_O-Ring_Handbook.pdf
As you've noted, the standard sizes are listed per AS568. Generally, there's no need to go outside of these dimensions, though I've often done so, even for pressure as high as 15,000 psi. The standard gland dimensions need not always be applied and sometimes there are reasons to go outside the box, especially for high pressure applications.
There are also non-standard O-rings designed to fit the same dimensions as these standards, but the O-rings aren't round. Quad-seals for example, are X shaped and work great for dynamic applications.
If your seal is a static one or has very limited movement, preferably with a lubricant, and as long as temperature and pressure are within limits, then an O-ring is a great way to go. If however, your application is a dynamic one that has to last a long time such as piston rings, or is outside the pressure and temperature limits, then there are other solutions to sealing that will be better than using elastomeric O-rings.