Your approach sounds good, I think you probably just made an algebraic mistake somewhere along the way. Use conservation of momentum to find the velocity of the bullet+block system in terms of the bullet's initial velocity, then use conservation of energy to relate the velocity of the bullet+block to the displacement of the spring.
Edit: Just for reference...
Conservation of momentum: [tex]m_i v_i = m_f v_f[/tex]
Kinetic energy: [tex]\frac{1}{2} m v^2[/tex]
Spring potential energy: [tex]\frac{1}{2} k x^2[/tex]