Actually, I've just done the expansions of S^(-1)=exp(-sA) and S^+=exp(sA^+), and comparing these brings me to exactly that conclusion, that A^(+)=(-A), ie. A must be antihermitian.
Adding the requirement, det(exp(sA))=1, would leave me concluding that A must be any traceless, antihermitian...