You can start by evaluating whether a redox couple exists in the molecule. Look for atoms in an unusually high oxidation state together with atoms that are in their normal or low oxidation state. Organic perchlorate salts would be likely candidates. Look at likely products of a combustion or redox process. If many moles of gaseous products are produced per mole of explosive, this adds to the gibbs free energy of the event (entropy). Azide decomposition (air bag deployment compounds) is an example of this.
Look for fuel and oxidant in the same structure. TNT is an example as are most of the organic nitro compounds. In TNT, 7 carbons are in their normal oxidation state with three nitrogens (from nitro) in a +5 oxidation state. 6 Oxygens are available to react with the carbons.
Unusually strained molecules can release the strain energy in an explosive event. This energy is not usually enough to supply the entire explosive energy but does add some energy to the energy of an otherwise high energy species.
Look for an energetic event that does not depend too strongly on diffusion. An exothermic ring opening event or the pressure/heat induced decomposition of part structures to yield high energy intermediates (like in TNT) will cause something to explode.
If you are planning to make some compound and you are worried that it could explosively decompose, examine the literature for compounds that have similar structures. It is likely that the similar compound will behave similarly.