Actually, in the general theory of integration, sums are integrals too.
I'll loosely introduce the idea of a Lesbegue integral, applying it to your coin flipping example.
Here, you have a topological space, X, which is simply the positive integers with the discrete topology.
You have a measure μ -- a function which takes subsets of X to real numbers that satisfies some axioms that encapsulate the idea of "size".
Here, μ can be defined by its values on singleton sets: μ({n}) = 2^-n. μ is actually a special sort of measure called a probability measure. (I think simply because μ(X) = 1)
To define an integral with respect to this measure, you first start with characteristic functions: for any subset S of X, define
<br />
\chi_S(x) := \left\{<br />
\begin{tabular}{ll}<br />
1 & x \in S \\<br />
0 & x \notin S<br />
\end{tabular}<br />
It's easy to define the integral of these things:
<br />
\int_T \chi_S \, d\mu := \mu(S \cap T)<br />
You then extend the integral in the obvious way to any linear combination of character functions. (for technical reasons, you do it only for nonnegative ones) Call these step functions.
You can then extend the integral to any positive "measurable" function. (I'll omit the definition) If f is a nonnegative measurable function, consider all possible step functions that are less than or equal to f. Then, you take the integral of all of them, and the sup is the integral of f.
That is:
<br />
\int_T f \, d\mu := \sup_{\phi <= f} \int_T \phi \, d\mu<br />
In this particular example, you can prove that it is equal to:
<br />
\int_T f \, d\mu := \sum_{t \in T} f(t) \mu({t})<br />
whether T is finite or infinite.
In the case where dμ is the usual measure on the real numbers, and f is a nonnegative Riemann integrable function, you can show that the integral defined above has the same value as the Riemann integral.
Then, you continue extending the definition of the integral to other cases (such as allowing functions to be negative), and you get the Lesbegue integral.