Reputation: 537
In the code below, gen_window
has a print statement in it but when I run the code the print statement doesn't get executed. Why is that and then how should I go about debugging such lambda functions? (Even the debugger ignores the breakpoint in these functions.)
getpairs = rdd.flatMap(lambda xi: gen_window(xi, n))
def gen_window(xi, n):
x, i = xi
l = []
for offset in range(n):
print("-->", (i - offset, (i, x)))
l.append((i - offset, (i, x)))
return l
Upvotes: 0
Views: 483
Reputation: 1
Lambda functions always return an iterator hence you would have to wrap that lambda call into a list
def string_length(s): length = len(s) print(f"The length of '{s}' is {length}") return length
strings = ["hello", "world", "python"]
lengths = list(map(lambda x: string_length(x), strings))
print(lengths) This would execute the print statements
def string_length(s): length = len(s) print(f"The length of '{s}' is {length}") return length
strings = ["hello", "world", "python"]
lengths = map(lambda x: string_length(x), strings)
print(lengths)
But this would not
Upvotes: 0
Reputation: 51683
Works:
def gen_window(xi, n):
x, i = xi
l = []
for offset in range(n):
print("-->", (i - offset, (i, x)))
l.append((i - offset, (i, x)))
return l
xi = [3,5]
n = 3
gen_window(xi, n)
Lambdas only get executed wenn you actually use them - If you get no output you are probably never using it.
Output:
--> (5, (5, 3))
--> (4, (5, 3))
--> (3, (5, 3))
Upvotes: 1