When dealing with the fundamental and adjoint representations, it is more useful to use tensor methods. I will below explain the methods for [itex]SU(2)[/itex] and [itex]SO(3)[/itex].
[itex]SU(2)[/itex]: Here the fundamental representation (i.e., the smallest non-trivial space on which the elements of [itex]SU(2)[/itex] act) is spanned by 2-component vector [itex]X_{a}, a = 1,2[/itex]. We often write [itex][n][/itex] for the n-dimensional (irreducible) representation space spanned by objects having n independent components. Now, let us consider the tensor product [itex]X_{a}Y_{b}[/itex] which represents [itex][2] \otimes [2][/itex]. We write
[tex]
X_{a}Y_{b} = \frac{1}{2}(X_{a}Y_{b} + X_{b}Y_{a}) + \frac{1}{2}(X_{a}Y_{b} - X_{b}Y_{a}). \ \ (1)[/tex]
Since the index a and b take values in the set {1,2}, the first object on the right-hand-side has only 3 independent elements; [itex]\{X_{1}Y_{1}, (1/2)(X_{1}Y_{2}+X_{2}Y_{1}),X_{2}Y_{2}\}[/itex]. We think of it either as 3-component vector [itex]V_{i}, i = 1,2,3[/itex] or as [itex]2 \times 2[/itex] symmetric matrix [itex]G_{ab}=G_{ba}[/itex]. Both span the same representation space [itex][3][/itex] of [itex]SU(2)[/itex]. The 2nd object in Eq(1) has only one independent element; [itex]\{(1/2)(X_{1}Y_{2}-X_{2}Y_{1})\} = (1/2)\epsilon^{ab}X_{a}Y_{b}[/itex]. Thus, it belongs to the trivial (invariant) representation space [itex][1][/itex]; notice that it is proportional to the invariant [itex]SU(2)[/itex] “metric”. So, we can rewrite Eq(1) as
[tex]
X_{a}Y_{b} = G_{ab} + (1/2) \epsilon_{ab} \epsilon^{cd} X_{c}Y_{d}.[/tex]
This shows that
[tex][2]\otimes [2] = [3] \oplus [1].[/tex]
Next, let us decompose the tensor product [itex][3] \otimes [2][/itex]. For this we need to break the tensor [itex]G_{ab}X_{c}[/itex] into symmetric and anti-symmetric parts. So, we write
[tex]
G_{ab}X_{c} = \frac{1}{3}(G_{ab}X_{c} + G_{ca}X_{b} + G_{bc}X_{a}) + \frac{1}{3}(G_{ab}X_{c}-G_{bc}X_{a}) + \frac{1}{3}(G_{ab}X_{c}-G_{ca}X_{b}). \ \ (2)[/tex]
So what do we have here? The 1st object on the right-hand-side is a rank-3 totally symmetric tensor [itex]T_{(abc)}[/itex] in 2-dimension. Such tensor has only 4 independent components; [itex]\{T_{(111)},T_{(112)},T_{(122)},T_{(222)}\}[/itex]. We can think of this as a vector spanning 4-dimensional vector space [itex][4][/itex].
The 2nd + 3rd objects on the RHS of Eq(2) can be written as
[tex]X_{abc}\equiv \epsilon_{ac}Z_{b} + \epsilon_{cb}Z_{a},[/tex]
where
[tex]Z_{a} = G_{ab}\epsilon^{bc}X_{c}.[/tex]
Now, using the identity
[tex]\epsilon_{ab}Z_{c} + \epsilon_{ca}Z_{b} + \epsilon_{bc}Z_{a} = 0,[/tex]
we finally find
[tex]G_{ab}X_{c} = T_{(abc)} + \epsilon_{ab}Z_{c},[/tex]
which proves
[tex][3] \otimes [2] = [4] \oplus [2].[/tex]
[itex][SO(3)][/itex]: I will give you the result of decomposing the tensor product [itex][3] \otimes [3][/itex] and leave you to fill in the details to conclude that
[tex][3] \otimes [3] = [5] \oplus [3] \oplus [1],[/tex]
follows from
[tex]U^{i}V^{j} = \frac{1}{2}(g^{ij} - \frac{1}{3}\delta^{ij}g) + \frac{1}{2}A^{ij} + \frac{1}{6}\delta^{ij}g,[/tex]
where [itex]g^{ij} = g^{ji}, A^{ij} = -A^{ji}[/itex] and [itex]g = \delta_{ij}g^{ij}[/itex].
Question for you: Why didn’t we subtract a trace from the symmetric [itex]SU(2)[/itex] tensor [itex]G_{ab}[/itex]?
Sam