In short: yes (with pivoting for numerical stability), but it's far from the only way, and in certain cases other methods may be better (for example, some method based on conjugate gradients is good when the matrix is sparse, and matrix decomposition is useful for when you want to solve Ax = b for several different b).