1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Python DEAP Library -- List index out of range error

  1. Jul 19, 2015 #1
    1. The problem statement, all variables and given/known data
    So I'm currently modifying an example Symbolic Regression program that uses genetic programming, a type of evolutionary algorithm. I'm using the python deap library. Not sure if anyone on here is familiar with it, but I think that the error is more general to a misuse of python. I've "plotted" a series of values from the function x^2 to test the program on before working on more difficult problems in a list. I get a this error:

    Traceback (most recent call last):
    File "C:/Users/Anonymous/Downloads/WinPython-64bit- SymbReg/Draft2.py", line 81, in <module>
    File "C:/Users/Anonymous/Downloads/WinPython-64bit- SymbReg/Draft2.py", line 77, in main
    halloffame=hof, verbose=True)
    File "C:\Users\Anonymous\Downloads\WinPython-64bit-\python-3.4.3.amd64\lib\site-packages\deap\algorithms.py", line 148, in eaSimple
    for ind, fit in zip(invalid_ind, fitnesses):
    File "C:/Users/Anonymous/Downloads/WinPython-64bit- SymbReg/Draft2.py", line 47, in evalSymbReg
    diff = ((func(testvals[index]) - values[index])**2)
    IndexError: list index out of range

    I assume this is exactly what it means, that the index variable achieves greater values than the index of one of my two lists. The trouble is, after an extensive review I am unable to find how this could be the case.

    Here is the relevant code (as far as I'm aware, if you need the whole program, let me know.):

    Code (Python):
    #Set data (x^2) and test vals.
    values = [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
    L = []
    testvals = L[0:len(values)-1:1]
    #Evaluate Function
    def evalSymbReg(individual):
        # Transform the tree expression in a callable function
        func = toolbox.compile(expr=individual)
        # Evaluate the mean squared error between the function and the sample values.
        diff = ((func(testvals[index]) - values[index])**2)
        if index < len(values):
            return diff,
    Any ideas on why I'm getting the error?

    2. Relevant equations

    3. The attempt at a solution
    A review of the material on stack overflow
    Python Documentation on Lists, len()
    DEAP Documentation
  2. jcsd
  3. Jul 20, 2015 #2


    Staff: Mentor

    Try adding debug prints before the point of fail to see what the index value is and what the table size is? From there you should be able to determine the problem.
  4. Jul 20, 2015 #3
    Thanks, that's a good idea.
  5. Jul 20, 2015 #4
    I added a bunch of print statements, but I just get the errors before they print. The only thing it does print is func. I'm assuming that's indicative that something else besides my index gets too high is wrong with my program. I'll post my entire code here, its not that much different than the DEAP GP Example, but anyone is free to use it (if they can fix it from my bad modifications. Lol).

    Code (Python):
    import operator
    import math
    import random
    import numpy
    from deap import algorithms
    from deap import base
    from deap import creator
    from deap import tools
    from deap import gp
    # Define new functions
    def protectedDiv(left, right):
            return left / right
        except ZeroDivisionError:
            return 1
    #Make Primitive Set
    pset = gp.PrimitiveSet("MAIN", 1)
    pset.addPrimitive(operator.add, 2)
    pset.addPrimitive(operator.sub, 2)
    pset.addPrimitive(operator.mul, 2)
    pset.addPrimitive(protectedDiv, 2)
    pset.addPrimitive(operator.neg, 1)
    pset.addPrimitive(math.cos, 1)
    pset.addPrimitive(math.sin, 1)
    pset.addEphemeralConstant("rand101", lambda: random.randint(-100,100))
    creator.create("FitnessMin", base.Fitness, weights=(-1.0,))
    creator.create("Individual", gp.PrimitiveTree, fitness=creator.FitnessMin)
    #Register base.Toolbox() and assign to toolbox
    toolbox = base.Toolbox()
    toolbox.register("expr", gp.genHalfAndHalf, pset=pset, min_=1, max_=2)
    toolbox.register("individual", tools.initIterate, creator.Individual, toolbox.expr)
    toolbox.register("population", tools.initRepeat, list, toolbox.individual)
    toolbox.register("compile", gp.compile, pset=pset)
    #Set data (x^2) and test vals.
    values = [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
    L = []
    testvals = L[0:len(values)-1:1]
    #Evaluate Function
    def evalSymbReg(individual):
        # Transform the tree expression in a callable function
        func = toolbox.compile(expr=individual)
        # Evaluate the mean squared error between the expression
        # and the real function : x**4 + x**3 + x**2 + x
        diff = ((func(testvals[index]) - values[index])**2)
        if index < values.length:
            return diff,
    #Register genetic operators
    toolbox.register("evaluate", evalSymbReg)
    toolbox.register("select", tools.selTournament, tournsize=3)
    toolbox.register("mate", gp.cxOnePoint)
    toolbox.register("expr_mut", gp.genFull, min_=0, max_=2)
    toolbox.register("mutate", gp.mutUniform, expr=toolbox.expr_mut, pset=pset)
    toolbox.decorate("mate", gp.staticLimit(key=operator.attrgetter("height"), max_value=17))
    toolbox.decorate("mutate", gp.staticLimit(key=operator.attrgetter("height"), max_value=17))
    #Main method
    def main():
    #Create population and assign hof to tools.HallOfFame(1)
        pop = toolbox.population(n=300)
        hof = tools.HallOfFame(1)
        stats_fit = tools.Statistics(lambda ind: ind.fitness.values)
        stats_size = tools.Statistics(len)
        mstats = tools.MultiStatistics(fitness=stats_fit, size=stats_size)
        mstats.register("avg", numpy.mean)
        mstats.register("std", numpy.std)
        mstats.register("min", numpy.min)
        mstats.register("max", numpy.max)
    #Set algorithm
        pop, log = algorithms.eaSimple(pop, toolbox, 0.5, 0.1, 40, stats=mstats,
                                       halloffame=hof, verbose=True)
        # print log
        return pop, log, hof
    if __name__ == "__main__":
  6. Jul 20, 2015 #5


    Staff: Mentor

    I have marked three lines of your code with "<---". In the first line I marked, L is an unintialized array. In the following line, you are setting testvals with the elements in the array L. I don't believe this is valid.

    In the second line I marked, you have "index + 1". Your intent seems to be to increment index, but your code doesn't do that. It simply evaluated index, adds 1, and then does nothing to change the value of index. You should have either "index = index + 1" or "index++".

    In the third line, you have a comma at the end of "return diff,". I don't believe that should be there.
  7. Jul 20, 2015 #6
    Thanks for the pointers Mark44. Much appreciated.
  8. Jul 20, 2015 #7
    The code now works thanks to your first two suggestions. I'm going to start working on a sympy version now.

    Thanks for the help all
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted