I have heard of carbon compounds in the form of rings or circles like cyclohexane and cyclopentane and it set me wondering of why carbon atoms do not form square or triangular rings. Could someone explain the reason?
They do form triangular and square compounds, but they are often unstable. It has to do with the geometry of the orbitals/location of their electrons. There are certain "idealized" bonding angles and triangular compounds and square compounds deviate severely from them. As a result, they do some "weird" bonding.
However, this is a broad generalization. Cyclopropene cation is actually quite stable and can be stored at room temperature with no problems, for example.
" Besides forming chains, carbon atoms also form rings. The simplest of the cyclic alkanes (general formula CnH2n,,) is cyclopropane (C3H6), shown in Fig. 22.5(a). Since the carbon atoms in cyclopropane form an equilateral triangle with 60° bond angles, their Sp3 hybrid orbitals do not overlap head-on as in normal alkanes [Fig. 22.5(b)]. This results in unusually weak, or strained, C-C bonds; thus the cyclopropane molecule is much more reactive than straight-chain propane. The carbon atoms in cyclobutane (C4HS) form a square with 90° bond angles, and cyclobutane is also quite reactive.
The next two members of the series, cyclopentane (C5H10) and cyclohexane (C6Hd, are quite stable, because their rings have bond angles very close to the tetra-hedral angles, which allows the Sp3 hybrid orbitals on adjacent carbon atoms to over-lap head-on and form normal C-C bonds, which are quite strong. To attain the tetrahedral angles, the cyclohexane ling must "pucker" -that is, become nonplanar. Cyclohexane can exist in two forms, the chair and the boat forms, are shown in Fig. 22.6. The two hydrogen atoms above the ring in the boat form are quite close to each other, and the resulting repulsion between these atoms causes the chair form to be preferred. At 25°C more than 99% of cyclohexane exists in the chair form. "
From Zumdahl chemestry text book