Python DEAP Library -- List index out of range error

[Prof. 27]

Homework Statement

So I'm currently modifying an example Symbolic Regression program that uses genetic programming, a type of evolutionary algorithm. I'm using the python deap library. Not sure if anyone on here is familiar with it, but I think that the error is more general to a misuse of python. I've "plotted" a series of values from the function x^2 to test the program on before working on more difficult problems in a list. I get a this error:

Traceback (most recent call last):
File "C:/Users/Anonymous/Downloads/WinPython-64bit-3.4.3.4/python-3.4.3.amd64/Lib/idlelib/Projects/GP SymbReg/Draft2.py", line 81, in <module>
main()
File "C:/Users/Anonymous/Downloads/WinPython-64bit-3.4.3.4/python-3.4.3.amd64/Lib/idlelib/Projects/GP SymbReg/Draft2.py", line 77, in main
halloffame=hof, verbose=True)
File "C:\Users\Anonymous\Downloads\WinPython-64bit-3.4.3.4\python-3.4.3.amd64\lib\site-packages\deap\algorithms.py", line 148, in eaSimple
for ind, fit in zip(invalid_ind, fitnesses):
File "C:/Users/Anonymous/Downloads/WinPython-64bit-3.4.3.4/python-3.4.3.amd64/Lib/idlelib/Projects/GP SymbReg/Draft2.py", line 47, in evalSymbReg
diff = ((func(testvals[index]) - values[index])**2)
IndexError: list index out of range

I assume this is exactly what it means, that the index variable achieves greater values than the index of one of my two lists. The trouble is, after an extensive review I am unable to find how this could be the case.

Here is the relevant code (as far as I'm aware, if you need the whole program, let me know.):

#Set data (x^2) and test vals.
values = [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
L = []
testvals = L[0:len(values)-1:1]
#Evaluate Function
def evalSymbReg(individual):
    index=0
    # Transform the tree expression in a callable function
    func = toolbox.compile(expr=individual)
    # Evaluate the mean squared error between the function and the sample values.
    diff = ((func(testvals[index]) - values[index])**2)
    if index < len(values):
        index+1
        return diff,
    else:
        pass

Any ideas on why I'm getting the error?

Homework Equations

None

The Attempt at a Solution

A review of the material on stack overflow
Python Documentation on Lists, len()
DEAP Documentation

 

[jedishrfu]

Try adding debug prints before the point of fail to see what the index value is and what the table size is? From there you should be able to determine the problem.

 

[Prof. 27]

Thanks, that's a good idea.

 

[Prof. 27]

I added a bunch of print statements, but I just get the errors before they print. The only thing it does print is func. I'm assuming that's indicative that something else besides my index gets too high is wrong with my program. I'll post my entire code here, its not that much different than the DEAP GP Example, but anyone is free to use it (if they can fix it from my bad modifications. Lol).

import operator
import math
import random
import numpy
from deap import algorithms
from deap import base
from deap import creator
from deap import tools
from deap import gp
# Define new functions
def protectedDiv(left, right):
    try:
        return left / right
    except ZeroDivisionError:
        return 1
#Make Primitive Set
pset = gp.PrimitiveSet("MAIN", 1)
pset.addPrimitive(operator.add, 2)
pset.addPrimitive(operator.sub, 2)
pset.addPrimitive(operator.mul, 2)
pset.addPrimitive(protectedDiv, 2)
pset.addPrimitive(operator.neg, 1)
pset.addPrimitive(math.cos, 1)
pset.addPrimitive(math.sin, 1)
pset.addEphemeralConstant("rand101", lambda: random.randint(-100,100))
pset.renameArguments(ARG0='x')
#Create
creator.create("FitnessMin", base.Fitness, weights=(-1.0,))
creator.create("Individual", gp.PrimitiveTree, fitness=creator.FitnessMin)
#Register base.Toolbox() and assign to toolbox
toolbox = base.Toolbox()
toolbox.register("expr", gp.genHalfAndHalf, pset=pset, min_=1, max_=2)
toolbox.register("individual", tools.initIterate, creator.Individual, toolbox.expr)
toolbox.register("population", tools.initRepeat, list, toolbox.individual)
toolbox.register("compile", gp.compile, pset=pset)
#Set data (x^2) and test vals.
values = [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
L = []
testvals = L[0:len(values)-1:1]
#Evaluate Function
def evalSymbReg(individual):
    index=0
    # Transform the tree expression in a callable function
    func = toolbox.compile(expr=individual)
    print(func)
    # Evaluate the mean squared error between the expression
    # and the real function : x**4 + x**3 + x**2 + x
    diff = ((func(testvals[index]) - values[index])**2)
    if index < values.length:
        index+1
        print(index)
        return diff,
    else:
        pass
        print(index)
    print(index)
#Register genetic operators
toolbox.register("evaluate", evalSymbReg)
toolbox.register("select", tools.selTournament, tournsize=3)
toolbox.register("mate", gp.cxOnePoint)
toolbox.register("expr_mut", gp.genFull, min_=0, max_=2)
toolbox.register("mutate", gp.mutUniform, expr=toolbox.expr_mut, pset=pset)
toolbox.decorate("mate", gp.staticLimit(key=operator.attrgetter("height"), max_value=17))
toolbox.decorate("mutate", gp.staticLimit(key=operator.attrgetter("height"), max_value=17))
#Main method
def main():
    random.seed(318)
#Create population and assign hof to tools.HallOfFame(1)
    pop = toolbox.population(n=300)
    hof = tools.HallOfFame(1)
#Statistics
    stats_fit = tools.Statistics(lambda ind: ind.fitness.values)
    stats_size = tools.Statistics(len)
    mstats = tools.MultiStatistics(fitness=stats_fit, size=stats_size)
    mstats.register("avg", numpy.mean)
    mstats.register("std", numpy.std)
    mstats.register("min", numpy.min)
    mstats.register("max", numpy.max)
#Set algorithm
    pop, log = algorithms.eaSimple(pop, toolbox, 0.5, 0.1, 40, stats=mstats,
                                   halloffame=hof, verbose=True)
    # print log
    return pop, log, hof
if __name__ == "__main__":
    main()

 

[Mark44]

#Set data (x^2) and test vals.
values = [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
L = []
testvals = L[0:len(values)-1:1]  <---
#Evaluate Function
def evalSymbReg(individual):
    index=0
    # Transform the tree expression in a callable function
    func = toolbox.compile(expr=individual)
    # Evaluate the mean squared error between the function and the sample values.
    diff = ((func(testvals[index]) - values[index])**2)
    if index < len(values):
        index+1  <---
        return diff,  <--
    else:
        pass

I have marked three lines of your code with "<---". In the first line I marked, L is an unintialized array. In the following line, you are setting testvals with the elements in the array L. I don't believe this is valid.

In the second line I marked, you have "index + 1". Your intent seems to be to increment index, but your code doesn't do that. It simply evaluated index, adds 1, and then does nothing to change the value of index. You should have either "index = index + 1" or "index++".

In the third line, you have a comma at the end of "return diff,". I don't believe that should be there.

 

[Prof. 27]

Thanks for the pointers Mark44. Much appreciated.

 

[Prof. 27]

The code now works thanks to your first two suggestions. I'm going to start working on a sympy version now.

Thanks for the help all