Python's versus Lua's closures

cplusplus.cpp

#include <functional>
#include <iostream>
#include <vector>

int main()
{
    std::vector<std::function<int (int)>> l;
    for(int i = 0; i < 10; ++i)
        l.push_back([=](int x) { return i + x; });
    std::cout << l[2](3) << std::endl; // 5
    return 0;
}

lua.lua

l = {}
for i = 0, 10 do
    l[i] = function(x) return x + i end
end
print(l[2](3))
-- 5

python.py

l = []

for i in range(10):
    l.append(lambda x: x + i)

print(l[2](3))
# 12

l= []

for i in xrange(10):
    l.append(lambda x, i=i: x+i)

print l[2](3)
#5

Yep, or use a wrapper function:

def accgen(n):
    return lambda x: x+n
l = [accgen(i) for i in xrange(10)]

print l[2](3)
#5

Important difference though: you can mutate the upvalue in Lua (and C++ if you add the 'mutable' keyword), not in Python:
see https://gist.github.com/remram44/5834233 ;-)

Of course you can mutate an upvalue in Python, see nonlocal keyword: https://docs.python.org/3.5/reference/simple_stmts.html#grammar-token-nonlocal-stmt

That doesn't mutate the captured up-value but the actual global variable in the parent scope.

"actual global variable in the parent scope"? Heh.

Here's the code using nonlocal (needed additional changes to be able to use it, but it is structurally similar):

l = []

def main():
    for i in range(10):
        def _lambda(x):
            nonlocal i
            return x + i
        l.append(_lambda)

    print(l[2](3))
    # 12

if __name__ == '__main__':
    main()

go ahead and run it, you will find that the result is still 12. nonlocal doesn't change in any way how the closure is created, the single i variable, global to the main() scope, is still shared by all of them.

the single i variable, global to the main() scope

That's confusing again. There only one global scope - at the level of the module, outside any function. So, what you're talking is that i is local to main() scope (and explicit, and thus mutable, upvalue in _lambda).

nonlocal works as expected in Python (the language which doesn't have nested scopes within a function, there's only one scope for the entire function). The issue you describe is not with mutating an upvalue per se (and that's what my original comment referred to), but with a way a loop control variable is captured in a closure. There're indeed different choices, the one in Python is a logical for its single-scope-within-function design. This issue is covered in more detail with examples for various languages in e.g. http://craftinginterpreters.com/closures.html#design-note (at the bottom of that chapter).

Congratulations, you get the point of this Gist, which is to show the different behaviors between those languages. I did not claim nonlocal did not work as expected or did anything to help in this situation, you did.

Indeed, perhaps I got confused by comment above stating:

you can mutate the upvalue in Lua (and C++ if you add the 'mutable' keyword), not in Python:

Indeed, maybe it says only about the specific upvalue in the specific code snipper, not about an upvalue at all. (You may quite guess that my native language doesn't have definite/indefinite articles.)

Anyway, thanks for both posting the gist and being around to discuss it. My coming here is based merely on a google search regarding whether people use term "upvalue" in regard to Python (the official docs don't use it). And I just couldn't resist commenting on a seemingly incorrect statement, for the sake of beginners who may come here ;-).

I don't think Lua upvalues actually work differently to Python's in the respect you demonstrated, but their for loops do. E.g.:

l = {}
local i = 0
while true do
    l[i] = function(x) return x + i end
    if i + 1 > 9 then break end
    i = i + 1
end
print(l[2](3))
-- 12

Each i is a different local in the for loop.

That's one way to look at it, sure

Anyway, I also found this while curious about whether Python uses the term upvalue :P

The Python documentation refers to them as "cells", see for example in data model and PyFunction.

Cool, thank you!