I find your scan pretty good. I prefer to write ##p## for the four-vector ##\vec{p}## for its spatial components (everything in a given fixed inertial reference frame defining the basis of four Minkowski-orthogonal unit vectors).
You are dealing with free quantum fields and thus can decompose it in terms of the momentum-spin (momentum-polarization) annihilation and creation operators of the corresponding single-particle basis.
Let's take the most simple case of scalar bosons first, i.e., the spin is 0 and thus there's only a momentum for the single-particle basis. The field operator fulfills the Klein-Gordon equation
$$(\Box + m^2) \hat{\phi}(x)=0,$$
which implies that the four-momentum's time component is completely determined by the three spacelike components and by definition we always choose the positive solution to label the plane-wave field modes, i.e.,
$$p^0=+\sqrt{\vec{p}^2+m^2}=\omega(\vec{p)}.$$
Note that I work in natural units with ##\hbar=c=1##, and I use the west-coast convention for the Minkowski pseudometric, i.e., ##(\eta_{\mu \nu})=(\eta^{\mu \nu})=\mathrm{diag}(1,-1,-1,-1)##.
Now since the KG equation allows field modes with both positive and negative frequencies, the most general solution is (with ##x^0=t##) [Edit: Typo corrected in view of #5]
$$\hat{\phi}(x)=\hat{\phi}(t,\vec{x}) = \int_{\mathbb{R}^3} \frac{\mathrm{d}^3 \vec{p}}{\sqrt{(2 \pi)^3 2 \omega(\vec{p})}} \left [ \hat{a}(\vec{p}) \exp(-\mathrm{i} p \cdot x) + \hat{b}^{\dagger}(\vec{p}) \exp(+\mathrm{i} p \cdot x \right]_{k^0 =\omega(\vec{k})}.$$
The factor in front is just for convenience and chosen such that the non-trivial commutator relations for the annihilation and creation operators become as simple as possible (i.e., such that the one-particle states created by ##\hat{a}^{\dagger}(\vec{p})## and one-antiparticle states created by ##\hat{b}^{\dagger}(\vec{p})## from the vacuum state are normalized by the ##\delta## distribution without any additional factors; that's what obviously also your book does, but not all authors follow this convention):
$$[\hat{a}(\vec{p}),\hat{a}(\vec{p}')]=\delta^{(3)}(\vec{p}-\vec{p}').$$
The same holds for the photon operator, but here you also have polarization vectors. Again each wave-number four-vector obeys the on-shell condition. In this case since photons are massless you have
$$k^2=0 \; \Rightarrow \; k^0=\omega(\vec{k})=\sqrt{\vec{k}^2}=|\vec{k}|.$$
Now obviously the book follows the Gupta-Bleuler method, i.e., it defines four polarization tensors (leaving the gauge before quantization completely unfixed).
Then you need to define four polarization four-vectors. Conveniently they are chosen as a "four-bein", i.e., three Minkowski-orthonormal vectors but dependent on the vector ##\vec{k}##, such that
$$(\epsilon_0^{\mu}(\vec{k}))=(1,0,0,0), \quad \epsilon_3^{\mu}(\vec{k})=(0,\vec{k}/|\vec{k}|), \quad \epsilon_{j}^{\mu}(\vec{k})=(0,\vec{\epsilon}_j(\vec{k}))$$
with ##j \in \{1,2,\}## and the ##\vec{\epsilon}_j(\vec{k})## both orthogonal to ##\vec{k}## and orthonormal to each other,
$$\vec{k} \cdot \vec{\epsilon}_j(\vec{k})=0, \quad \vec{\epsilon}_j(\vec{k}) \cdot \vec{\epsilon}_{j'}(\vec{k})=\delta_{jj'}, \quad j \in \{1,2\},$$
such that
$$\vec{\epsilon}_1(\vec{k}) \times \vec{\epsilon}_2(\vec{k})=\vec{\epsilon}_3(\vec{k})=\frac{\vec{k}}{|\vec{k}|}.$$
Then the ##\epsilon_0^{\mu}(\vec{k})## is called the time-like polarization, ##\epsilon_3^{\mu}(\vec{k})## the longitudinal and ##\epsilon_j(\vec{k})## with ##j \in \{1,2\}## the transverse polarization state.
It's of course clear that only the latter transverse polarization states are physical. As a massless field the photon field has only two physical polarization states, namely those of the transversal electromagnetic fields. For convenience of notation I've choosen real polarization vectors. Physically the transverse ones are just the usual linear polarization vectors in orthogonal polarization directions.
The field operators now are given by (taking into account that the classical em. four-potential are real four-vectors and thus the corresponding field operators have to be self-adjoint) [Edit: Typo corrected in view of #5]
$$\hat{A}^{\mu}(x) = \sum_{r=0}^3 \int_{\mathbb{R}^3} \frac{\mathrm{d}^3 \vec{k}}{\sqrt{(2 \pi)^3 2 |\vec{k}|}} \epsilon_r^{\mu}(\vec{k}) \left [\hat{a}_r(\vec{k}) \exp(-\mathrm{i} k \cdot x) + \hat{a}_r^{\dagger}(\vec{k}) \exp(+\mathrm{i} k \cdot x) \right]_{k^0=|\vec{k}|}.$$
I hope, now the notation is clarified.