Troubleshooting Nested Loops in Python: Unexpected Output Explained

[cepheid]

My code isn't working, and the reason seems to be because nested for loops in python don't work according to my expectations. If I have some code like this:

f1 = open('name_of_file1.txt', 'r')
f2 = open('name_of_file2.txt', 'r')

for line1 in f1:
    #stuff at this level should happen once for every line in file1. Right?
    print line1
    for line2 in f2:
            #stuff at this level should happen once for every line in file2. Right?
            print line2

My expectation for the output:

line 1 from file1
.
.
.
(all lines from file2 here)
.
.
.
line 2 from file1
.
.
.
(all lines from file2 here)
.
.
.
line 3 from file1
.
.
.
(all lines from file2 here)
.
.
.
et cetera

The output that I ACTUALLY get:

line 1 from file1
.
.
.
(all lines from file2 here)
.
.
.
(all remaining lines from file1 in sequence)

Huh?

 

[jtbell]

It looks like when the program tries to print the lines from file2 the second time, it's still "stuck" at the end of file2 from the first time.

Try opening file2 inside the outer loop right before you start to read lines from it. Then close it after the inner loop (but before the outer loop ends).

 

[cepheid]

Ohh, okay. I will try that, thanks.

 

[D H]

#stuff at this level should happen once for every line in file2. Right?

Wrong.

Python can't read your mind. How is it to know that you want file2 rewound?

Issue a file2.seek(0) command before the start of your inner loop to rewind the file to the start. Unlike jtbell's suggestion, you only need one open command for file2 if you rewind via seek(). Opens are expensive. Seeks are relatively cheap.

 

[cepheid]

Wrong.

Python can't read your mind. How is it to know that you want file2 rewound?

Issue a file2.seek(0) command before the start of your inner loop to rewind the file to the start. Unlike jtbell's suggestion, you only need one open command for file2 if you rewind via seek(). Opens are expensive. Seeks are relatively cheap.

Got it. And yes, this turned out to be the story. Thanks to both of you for the help.

 

[jtbell]

Issue a file2.seek(0) command before the start of your inner loop to rewind the file to the start.

Yes, that's better than my idea. I didn't know whether python had some sort of "rewind" operation, so I played it safe with the close / (re)open trick.

 

[Borek]

Same here - problem was obvious, I just wasn't sure how to implement it in Python other than by close/save. But JT was faster

 

[jhae2.718]

I would do something like this:

[color=#408080][i]#!/usr/bin/env python2[/i][/color]
[color=#408080][i]# -*- coding: utf-8 -*-[/i][/color]
[color=#008000][b]from[/b][/color] [color=#0000FF][b]__future__[/b][/color] [color=#008000][b]import[/b][/color] print_function
[color=#008000][b]from[/b][/color] [color=#0000FF][b]__future__[/b][/color] [color=#008000][b]import[/b][/color] unicode_literals

[color=#008000][b]with[/b][/color] [color=#008000]open[/color]([color=#BA2121]'file1.txt'[/color], [color=#BA2121]'r'[/color]) [color=#008000][b]as[/b][/color] f1:
    [color=#008000][b]for[/b][/color] line [color=#AA22FF][b]in[/b][/color] f1[color=#666666].[/color]readlines():
        [color=#008000][b]print[/b][/color](line)
        [color=#008000][b]with[/b][/color] [color=#008000]open[/color]([color=#BA2121]'file2.txt'[/color], [color=#BA2121]'r'[/color]) [color=#008000][b]as[/b][/color] f2:
            [color=#008000][b]for[/b][/color] line [color=#AA22FF][b]in[/b][/color] f2[color=#666666].[/color]readlines():
                [color=#008000][b]print[/b][/color](line)