It might be just a feature of the times. I don't know much about Conan Doyle, but there are many other rationalists of the time who also seemed swayed by spiritualism, such as William James.
It could be that the belief in spirits or telepathy were encouraged by scientific advance in fact - once you start seeing a material/mechanical base to everything, then this makes ideas about ectoplasm, a spirtual plane or whatever, a "more rational" belief than some immaterial idea of soul.
If you want to believe in religious teachings, then a rationalist would want to find a substantial explanation. It may seem kooky from our modern standpoint, but it would have been the more scientific view in Victorian times - if you were taking the arch-materialist position of the reductionist thinker.
Exactly the same dynamic still operates today when it comes to things like quantum approaches to consciousness. People are looking for a substantial
explanation of something they feel needs a properly material answer.