Python: leaking in for loop and lazy evaluation

written by @marc_lelarge

used to check python prerequisites of the deep learning course dataflowr. Before reading this post, you should start with the quiz

If you are not familiar with binding (i.e. assignment) and mutation of variables in Python you can have a look at Binding vs Mutation.

1.0 Basics on scope

Before looking at the leaking problem, we illustrate basic properties of scope in python:

def print_11():
    x = 11            # new local variable inside the scope of print_11
    print(x)          # will print the local variable x = 11

def print_x_outside():
    print(x)          # will print the global variable x (not defined yet)

x = 10                # global variable x = 10

def main():

    def print_x_inside():
        print(x)      # will print the variable x in the scope of main (not defined yet)

    x = 11            # variable x = 11 in the scope of main
    print_11()        # print 11
    print_x_inside()  # print 11

    x = 12            # new binding for the variable x
    print_x_outside() # print 10
    print_11()        # print 11
    print_x_inside()  # print 12

main()

1.1 Scope and binding (i.e. assignment)

Assume, we want to print the old value of x and then modify it. The following code will produce an error:

def main():
      
  def print_modify(new_value):
    print(x)
    x = new_value
      
  x = 10
  print_modify(11)
    
main()

The last statement in print_modify assigns a new value to x, the compiler recognizes it as a local variable. Consequently when the earlier print(x) attempts to print the uninitialized local variable and an error results, see Why am I getting an UnboundLocalError when the variable has a value?
Probably the easiest way to do this:

def main():
    def easy_print_modify(x,new_value):
        print(x)
        return new_value
    
    x = 10
    x = easy_print_modify(x,11)
    print(x)

main()

or you can use nonlocal :

def main():
    def other_print_modify(new_value):
        nonlocal x
        print(x)
        x = new_value
    
    x = 10
    other_print_modify(11)
    print(x)
    
main()

1.2 Scope and mutation

def print_12():
    x = [1, 2]
    print(x)          

def print_x_outside():
    x[0] = 3
    print(x)          

x = ['a', 'b']                

def main():
    
    def print_x_inside():
        x[1] = 'b'
        print(x)     

    x = [1, 2]                  
    print_x_inside()   # print [1, 'b']
    print(x)           # print [1, 'b']
    print_12()         # print [1, 2]
    
    x = [1,2,3]
    print_x_outside()  # print [3, 'b']
    print_x_inside()   # print [1, 'b', 3]
    print(x)           # print [1, 'b', 3]
    print_12()         # print [1, 2]

main()
print(x)               # print [3, 'b']

2.1 Leaking in for loop

basic example: even the variable _ becomes a variable accessible outside the for loop!

for _ in range(10):
    foo = 2
print(_, foo) # print 9 2

confusing variables:

i = 20

list_a = []
for i in range(10):
    list_a.append(2*i)

print(i)   # print 9 - leaking -

2.2 List comprehension to avoid leaking

i = 20

litst_b = [2*i for i in range(10)]

print(i)   # print 20 - no leaking -

Takeaways: List comprehensions have their own scope, loops don't.

3.1 Lazy evaluation

Variables in functions are evaluated only when called:

list_fun = []
for i in range(10):
     list_fun.append(lambda: i)

for f in list_fun:
     print(f())
# print 9 ten times        
print(i) # print 9

i = 20
for f in list_fun:
     print(f())
# print 20 ten times

Here i is not local to the lambdas but is defined in the outer scope, and it is accessed when the lambda is called — not when it is defined. At the end of the loop, the value of i is 9, so all the functions now return 9.
See also Why do lambdas defined in a loop with different values all return the same result?

With list comprehension, there is a local scope (as seen above) so now i is local but since the function is not called at creation, the local variable i is not evaluated only when it is called but then i=9.

list_fun_comp = [lambda: i for i in range(10)] # variable i is in the local scope
for f in list_fun_comp:
     print(f())
# print 9 ten times        

i = 20  # this variable is in the global scope
for f in list_fun_comp:
     print(f())
# print 9 ten times        

We now present several ways to solve the problem:

3.2 hack

Using an argument with a default value thanks to i=i below, will creates a new variable i local to the lambda and computed when the lambda is defined:

list_fun_hack = [lambda i=i: i for i in range(10)] 

for f in list_fun_hack:
     print(f())
# print 0 1 2 3 4 5 6 7 8 9

3.3 currying

In mathematics and computer science, currying is the technique of converting a function that takes multiple arguments into a sequence of functions that each takes a single argument. Here this is very simple since

$f(i,y)=i$ we can rewrite it as

$f(i,y)=h(i)(y)$ with

$h(i)=(y\to i)$, i.e.

$h(i)$ is the constant function equal to

$i$. Note that

$h:\mathbb{R}\mapsto {\mathbb{R}}^{\mathbb{R}}$ below is called with variable i in the for loop, the local variable i is evaluated:

list_fun_curry = [(lambda x :(lambda: x))(i) for i in range(10)]

for f in list_fun_curry:
     print(f())
# print 0 1 2 3 4 5 6 7 8 9

3.4 lazy generator

Variables used in the generator expression are evaluated lazily see Generator expressions so you can use a generator:

def fun_gen(n):
    i = 0
    while i < n:
        yield (lambda: i)
        i += 1

for f in fun_gen(10):
    print(f())
# print 0 1 2 3 4 5 6 7 8 9

But this code will not work as intended:

for f in list(fun_gen(10)):
    print(f())
# print 10 ten times

More references

Python Oddities Explained by Trey Hunner

Common Gotchas from The Hitchhiker’s Guide to Python!

Back to the deep learning course dataflowr

tags: public python dataflowr