Yes the group will have order 10. If you label a vertex A and label a vertex adjacent to A B. Then there are 5 positions that A can be moved to and each of those allows exactly 2 positions for B.
If you center your pentagram about the origin then the reflection across the x-axis would be a reflectional symmetry. You should be able to generate your group from that reflection and the rotation of 72 degrees about the origin.
Perhaps you should try to construct a subgroup of order 30. Such a subgroup is normal so you can constuct one by taking the union of conjugacy classes...