THere is always one homomorphism between any two groups. The same comments apply as for rings, but this time finite things are interesting. However it is a computationally hard problem to even determine if two given gruops of the same cardinality are isomorphic.
That might sound odd, but I'm first assuming that the two groups are merely given as generators and relations, which is about the best you can hope for. For instance, consider the groups
G=<s,t: s^2=t^3=1, sts=t^{-1}>
H=<x_1,x_2,x_3 :(x_r)^2=1, for r=1,2,3 and (x_i)(x_j)(x_i)^{-1}=x_k {i,j,k}=(1,2,3}>
are these two groups isomorphic? (They both have order 6, if I did it correctly.)
Now imagine that there were dozens of generators...
So if you can't find the number of isomorphisms between two given groups then you can't hope to find the number of homs between two groups in general.
Mind you, in it is actually more a case of: we can obviously know when there are no nontrivial homs, but if there might be some, we struggle.
There is for instance only one hom from G to H if the orders are coprime.
Plus the kernel of any hom is a normal subgroup, so there might be a generic way to rule out lots of possibilities, assuming that you know enough about the source group.
If all you have is a set of matrices though then you ain't going to get very far.