The short answer is "it depends"
There are 2 main interpretations of the quadratic forms with infinite matrices:
The first one is that while we have infinite matrix, we only consider vectors with finitely many non-zero coordinates. The quadratic form is defined for all such vectors. In this case "positive definite" means that (A x, x)>0 for all non-zero vectors x with finitely many non-zero coordinates.
In this case the Silvester's criterion works, i.e. the form is positive definite if and only if all finite principle minors are positive.
The other interpretation of the quadratic form (A x, x) with infinite matrix A is to assume that A is a matrix of a bounded operator in \ell^2 (the space of sequences x=\{x_k\}_{k=1}^n such that \sum_{k=1}^\infty|x_k|^2<\infty).
In this case if all finite principal minors are positive, we can only conclude that the matrix A is positive-semidefinite, i.e. that (Ax,x)\ge0 for all x\in\ell^2 (and "positive definite" means that (Ax, x)>0 for all non-zero x\in\ell^2). And it is not hard to construct an example where all finite principal minors are positive, but (Ax, x)=0 for some non-zero x\in\ell^2.
There can be more complicated interpretations of the quadratic forms with infinite matrices, but I am not going to discuss these right now.