No. Define the map [itex]f:\mathbb{R} \rightarrow \mathbb{R}[/itex] by setting [itex]f(x) = 0[/itex] for all [itex]x \in \mathbb{R}[/itex]. Now for every [itex]0 < \varepsilon[/itex] take [itex]\delta = \varepsilon^{-1}[/itex]. Then [itex]|f(x)| < \varepsilon[/itex] whenever [itex]|x| < \delta[/itex].
In some senses this is the only counter-example. To prove this let [itex]f:\mathbb{R} \rightarrow \mathbb{R}[/itex] be a function continuous at [itex]x_0 \in \mathbb{R}[/itex] in the following way: for every [itex]0 < \varepsilon[/itex] the inequality [itex]|x-x_0| < \varepsilon^{-1}[/itex] implies [itex]|f(x)-f(x_0)| < \varepsilon[/itex]. If there exists [itex]y_0 \in \mathbb{R}[/itex] such that [itex]f(y_0) \neq f(x_0)[/itex], then set [itex]0 < c = \min\{|f(y_0)-f(x_0)|,|y_0-x_0|^{-1}\}[/itex]. Since [itex]|y_0 - x_0| < c^{-1}[/itex], this implies that [itex]|f(y_0) - f(x_0)| < c[/itex], which is a contradiction. Therefore, [itex]f(x) = f(x_0)[/itex] for all [itex]x \in \mathbb{R}[/itex].