CODEWITHSUNDEEP
pythonfunctioniterator
beoliver
beoliver

Reputation: 5759

python now, next, n iteration

writing a general function that can iterate over any iterable returning now, next pairs.

def now_nxt(iterable):
    iterator = iter(iterable)
    nxt = iterator.__next__()
    for x in iterator:
        now = nxt
        nxt = x
        yield (now,nxt) 

for i in now_nxt("hello world"):
    print(i)

('h', 'e')
('e', 'l')
('l', 'l')
('l', 'o')
('o', ' ')
(' ', 'w')
('w', 'o')
('o', 'r')
('r', 'l')
('l', 'd')

I have been thinking about the best way to write a function where the number of items in each tuple can be set.

for example if it was

func("hello",n=3)

the result would be:

('h','e','l')
('e','l','l')
('l','l','o')

I am new to using timeit, so please point out if I doing anything wrong here:

import timeit

def n1(iterable, n=1):
    #now_nxt_deque
    from collections import deque
    deq = deque(maxlen=n)
    for i in iterable:
        deq.append(i)
        if len(deq) == n:
            yield tuple(deq)

def n2(sequence, n=2):
    # now_next
    from itertools import tee
    iterators = tee(iter(sequence), n)
    for i, iterator in enumerate(iterators):
        for j in range(i):
            iterator.__next__()
    return zip(*iterators)

def n3(gen, n=2):
    from itertools import tee, islice
    gens = tee(gen, n)
    gens = list(gens)
    for i, gen in enumerate(gens):
        gens[i] = islice(gens[i], i, None) 
    return zip(*gens)


def prin(func):
    for x in func:
        yield x

string = "Lorem ipsum tellivizzle for sure ghetto, consectetuer adipiscing elit."

print("func 1: %f" %timeit.Timer("prin(n1(string, 5))", "from __main__ import n1, string, prin").timeit(100000))
print("func 2: %f" %timeit.Timer("prin(n2(string, 5))", "from __main__ import n2, string, prin").timeit(100000))
print("func 3: %f" %timeit.Timer("prin(n3(string, 5))", "from __main__ import n3, string, prin").timeit(100000))

results:

$  py time_this_function.py 
func 1: 0.163129
func 2: 2.383288
func 3: 1.908363

Upvotes: 6

Views: 601

Answers (5)

Eric
Eric

Reputation: 97565

A one-liner based on cravoori's answer:

from itertools import tee, islice, izip

def now_next(gen, n=2):
    return izip(*(islice(g, i, None) for i, g in enumerate(tee(gen, n))))

Upvotes: 0

iruvar
iruvar

Reputation: 23374

A variation on Eric's technique that uses slicing

from itertools import tee, islice, izip

def now_next(gen, n=2):
  gens = tee(gen, n)
  gens = list(gens)
  for i, gen in enumerate(gens):
    gens[i] = islice(gens[i], i, None) 
  return izip(*gens)

for x in now_next((1,2,3,4,5,6,7)):
  print x

Upvotes: 1

Eric
Eric

Reputation: 97565

Here's a really simple way to do it:

  • Clone your iterator n times using using itertools.tee
  • Advance the ith iterator i times
  • izip them all together
import itertools

def now_next(sequence, n=2):
    iterators = itertools.tee(iter(sequence), n)
    for i, iterator in enumerate(iterators):
        for j in range(i):
            iterator.next()
    return itertools.izip(*iterators)

Upvotes: 5

dav1d
dav1d

Reputation: 6055

My solution:

def nn(itr, n):
    iterable = iter(itr)

    last = tuple(next(iterable, None) for _ in xrange(n))
    yield last
    for _ in xrange(len(itr)):
        last = tuple(chain(last[1:], [next(iterable)]))
        yield last

This was made for Python 2, if you wanna use it with Python 3, replace xrange with range.

next, has a great default parameter, which will be returned instead of raising a StopIteration, you could also add this default-parameter to your function like so:

def nn(itr, n, default=None):
    iterable = iter(itr)

    last = tuple(next(iterable, default) for _ in xrange(n))
    yield last
    for _ in xrange(len(itr)):
        last = tuple(chain(last[1:], [next(iterable, default)]))
        yield last

I played some more with it, e.g. using itr.__class__() as default, but that seems wrong for lists and tuples, well it just makes sense for strings.

Upvotes: 2

beoliver
beoliver

Reputation: 5759

My proposal would be,

from collections import deque

def now_nxt_deque(iterable, n=1):
    deq = deque(maxlen=n)
    for i in iterable:
        deq.append(i)
        if len(deq) == n:
            yield tuple(deq)

for i in now_nxt_deque("hello world", 3):
    print(i)

('h', 'e', 'l')
('e', 'l', 'l')
('l', 'l', 'o')
('l', 'o', ' ')
('o', ' ', 'w')
(' ', 'w', 'o')
('w', 'o', 'r')
('o', 'r', 'l')
('r', 'l', 'd')

Upvotes: 5

Related Questions