A very simple example is the best I can do here.
Suppose you have the differential equation y"- y= 0 with boundary conditions y(0)= 1, y(1)= 2. Divide the interval from 0 to 1 into 4 equal parts (I SAID "simple"). We can approximate the derivative y' (which is limit (y(x+h)- y(x))/h with h going to 0)by the "finite difference" (y(x+y)- y(x))/h. Obviously, the smaller h is, the more accurate this is- here h= 0.25 which isn't all that accurate. The more partitions, the more accurate.
There are a variety of possible "finite difference" formulas for the second derivative- the best is the "centered difference": y" is approximately ((y(x)- y(x-h))/h - (y(x+y)- y(x))/h)/h= (2y(x)- y(x+h)-y(x-h))/h2. Let yi be y(x) at each of the "partition" points (also called "knots") and plug into the differential equation using the finite difference equation. That will give an equation (linear if the de is linear) for the "unknowns" yi. Because we are using the centered difference we can only do that at the "internal" knots, not the endpoints. If we have n intervals, there will be n-2 internal knots and so that only gives n-2 equations for the n yi values. Fortunately, we are given the values at the endpoints so we have n (linear) equations for the n values.
In the example I gave, The equations would be
y1= 1
-y1+ 2y2- y3= 0
-y2+ 2y3- y4= 0
y4= 2
Notice, by the way, that, for each i, the equation involves only yi, yi-1, and yi+1: 3 values only out of the possible n values. Written as a matrix, we would have a matrix in which only the central 3 diagonals have non-zero entries. That's a "tri-diagonal" matrix, a special case that is well studied. There are methods for solving tri-diagonal matrix equations using only the three diagonal values. If we divided into, say, 10000 intervals to get really good accuracy, that would give us 10000 equations in 10000 unknowns- but the three central diagons have 10000+ 9991+ 9991= 29998 entries- still large but only 0.03% of 10000x10000 possible entries- a "sparce" matrix.
