I will immediately prove the second part of the challenge, the first part being a special case.
We first prove the lemma that if $[a, b)$ is an interval with $ka < b$ for integers $a, b, k > 1$ then $[a, b)$ contains a power of $k$. We first note that $k^0 = 1 < a$. Now suppose $k^r < a$ for any $r \in \mathbb{N}$. Then there are three possibilities: either $k^{r + 1} < a$, $k^{r + 1} \geq b$, or $k^{r + 1} \in [a, b)$. Now $k^{r + 1} \geq b$ is impossible, since $ka < b$ and $k^r < a$ hence $k(k^r) = k^{r + 1} < b$. If $k^{r + 1} < a$, then we can repeat the argument until $k^{r + s} \in [a, b)$ for some $s > 0$, which will happen as the sequence $(k^n)_{n = 0}^\infty$ is strictly increasing.
Since $\lfloor \sqrt{1} \rfloor = 1$ and $\lfloor \sqrt{x} \rfloor < x$ for $x > 1$ since $\lfloor \sqrt{x} \rfloor^2 \leq x$, it follows that repeatedly applying rule $(2)$ starting from any integer will eventually yield 1. Since $S$ is nonempty, there exists at least one element in $S$, and thus $1 \in S$ by the above reasoning. It therefore suffices to show that $1 \in S \implies x \in S$ for all $x \in \mathbb{N}$ to prove the claim. Denote $I_n$ the interval $[x^{2^n}, (x + 1)^{2^n})$. This interval $I_n$ can be made arbitrarily large by increasing $n$, as:
$$\lim_{n \to \infty} \frac{(x + 1)^{2^n}}{x^{2^n}} = \lim_{n \to \infty} \left ( \frac{x + 1}{x} \right)^{2^n} = \infty$$
In other words, the upper bound of the interval grows faster than the lower bound, and hence $I_n$ is asymptotically unbounded in size.
Now let $k \in \mathbb{N}$ such that $x \in S \implies kx \in S$. Since $1 \in S$, repeatedly applying this rule amounts to saying that $k^r \in S$ for all $r \in \mathbb{N}$. We have seen that $I_n$ can be made arbitrarily large, thus there exists an $n$ such that $kx^{2^n} < (x + 1)^{2^n}$ for any $k$. Hence by the lemma, $k^r \in [ x^{2^n}, (x + 1)^{2^n} )$ for some $r \in \mathbb{N}$. At this point, since $k^r \in S$, it suffices to repeatedly apply rule $(2)$ on $k^r$ which is guaranteed to eventually yield $x$ by monotonicity of the square root floor function, and we conclude that $x \in S$. We have only needed $k > 1$ (for the lemma) and therefore $S = \mathbb{N}$ for all (integer) $k > 1$, including $k = 4$. $\blacksquare$