At the end of the inductive step, you write "Therefore, $5^k-1$ is a multiple of 4". Could you explain why you used the word "therefore"? How does it follow from what comes earlier? In fact, what is the status of this claim: were you proving it, or did you accept it for granted?
The rest seems OK, but I would tweak the style a little. In my opinion, every proof by induction should have the following clearly marked elements.
(1) What I call the induction statement $P(n)$, a propety of natural numbers that you are supposed to prove for all $n$. Writing this property explicitly is important because it makes it easier writing the induction hypothesis and the claim that needs to be proved in the induction step. Besides, in more complicate proofs by induction, the induction statement $P(n)$ may be different from the property $Q(n)$ that appears in the problem statement: "Prove $Q(n)$ for all $n\ge0$". It may happen that $Q(n)$ is too weak for the induction to go through, and one needs to find a suitable strengthening, or generalization. This is a nontrivial step and should definitely be recorded.
(2) The base case (you did this correctly).
(3) The induction step (you have not marked it).
(4) The induction hypothesis (you wrote it, but did not label it as such).
(5) The content of the induction step, i.e., a proof of $P(k+1)$ from $P(k)$. The application of the induction hypothesis $P(k)$ should be clearly marked (you have not done this).
P.S. As a non-native English speaker, I am never sure whether to use "inductive: or "induction" as an adjective. Sorry for any grammatical errors.