Let us denote $a_n=(n^2+11n-4)\cdot n!+33\cdot 13^n+4$. If $n\ge 4$, then 8 divides $n!$. Hence, $a_n\equiv 33\cdot 13^n+4\equiv 5^n+4 \pmod 8$. As $5^2\equiv 1 \pmod 8$, then $5^2\equiv 1 \pmod 8$ for all even $n$. Therefore, $a_n\equiv 5 \pmod 8$, if $n\ge 4$ and $n$ is even. But perfect squares leave remainders 0, 1 or 4 when dividing by 8.
Secondly, when $n\ge 7$, then 7 divides $n!$. So $a_n\equiv 33 \cdot 13^n+4\equiv 5\cdot (-1)^n+4 5 \pmod 7$. Therefore, for odd $n\ge 7$, $a_n\equiv =-5+4=-1 \pmod 7$. But perfect squares leave remainders 0, 1, 4 or 2 when dividing by 7.
We are left with possible candidates $n=1,\,2,\,3$ and $n=5$. For $n=5$, $a_n$ is not a perfect square because $a_5=33\cdot 13^5+4\equiv 3^6-1\equiv 3 \pmod 5$, but perfect squares leave remainder 0, 1 or 4 when divided by 5. Also, $a_n$ is not a perfect ssquare for $n=3$ because $a_3=(9+33-4)\cdot 6+33\cdot 13^3+4\equiv 3+3^4-1\equiv 3\pmod 5$. Finally, we can check that $a_1=(1+11-4)\cdot 1+33\cdot 13+4=441=21^2$ and $a_2=(4+22-4)\cdot 2+33\cdot 169+4=5625=75^2$.