Python split list into n chunks

Question

I know this question has been covered many times but my requirement is different.

I have a list like: range(1, 26). I want to divide this list into a fixed number n. Assuming n = 6.

>>> x
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25]
>>> l = [ x [i:i + 6] for i in range(0, len(x), 6) ]
>>> l
[[1, 2, 3, 4, 5, 6], [7, 8, 9, 10, 11, 12], [13, 14, 15, 16, 17, 18], [19, 20, 21, 22, 23, 24], [25]]

As you can see I didn't get 6 chunks (six sublists with elements of original list). How do I divide a list in such a way that I get exactly n chunks which may be even or uneven

Answer 1

Assuming all chunks, but possibly the last one, have the same size, the list xs can be written as the sum of two lists:

xs = equal_chunks + last_chunk

Equivalently,

len(xs) = len(equal_chunks) + len(last_chunk)
        = chunk_size*(chunks-1) + (chunk_size + remainder)

When the remainder is zero, the list xs can be partitioned exactly into N chunks.

The following function decomposes a list into the above sum of two lists:

def chunk_list(xs, chunks):
    chunk_size, remainder = divmod(len(xs), chunks)
    end_equal_chunks = chunk_size*(chunks-1)

    return [
        xs[idx:idx+chunk_size] 
        for idx in range(0, end_equal_chunks, chunk_size)
    ] + [xs[end_equal_chunks:]]

And given as input the list in the question produces the desired output:

[
 [ 1,  2,  3,  4], 
 [ 5,  6,  7,  8], 
 [ 9, 10, 11, 12], 
 [13, 14, 15, 16], 
 [17, 18, 19, 20], 
 [21, 22, 23, 24, 25]
]

Answer 2

Easy hardcode-ish solution, here I'm going to split a list into quarters

k, m = divmod(len(yourInputList), 4)
partitionedList = list(yourInputList[i*k+min(i, m):(i+1)*k+min(i+1, m)] for i in range(4))

print(partitionedList[0])
print(partitionedList[1])
print(partitionedList[2])
print(partitionedList[3])

The number 4 is used two times here, and you can substitute your own number in this hardcode solution, as well as your own list instead of yourInputList

Answer 3

Try this:

ls = [1, 2, 3, 4, 5, 6 ,7 ,8 ,9]

size = 3

splited = [ls[x:x+splitedSize] for x in range(0, len(ls), size)]

print(splited)

Answer 4

This accepts generators, without consuming it at once. If we know the size of the generator, the binsize can be calculated by max(1, size // n_chunks).

from time import sleep

def chunks(items, binsize):
    lst = []
    for item in items:
        lst.append(item)
        if len(lst) == binsize:
            yield lst
            lst = []
    if len(lst) > 0:
        yield lst


def g():
    for item in [1, 2, 3, 4, 5, 6, 7]:
        print("accessed:", item)
        sleep(1)
        yield item


for a in chunks(g(), 3):
    print("chunk:", list(a), "\n")

Answer 5

I came up with the following solution:

l = [x[i::n] for i in range(n)]

For example:

n = 6
x = list(range(26))

l = [x[i::n] for i in range(n)]
print(l)

Output:

[[0, 6, 12, 18, 24], [1, 7, 13, 19, 25], [2, 8, 14, 20], [3, 9, 15, 21], [4, 10, 16, 22], [5, 11, 17, 23]]

As you can see, the output consists from n chunks, which have roughly the same number of elements.

---

How it works?

The trick is to use list slice step (the number after two semicolons) and to increment the offset of stepped slicing. First, it takes every n element starting from the first, then every n element starting from the second and so on. This completes the task.

Answer 6

I would simply do (let's say you want n chunks)

import numpy as np

# convert x to numpy.ndarray
x = np.array(x)
l = np.array_split(x, n)

It works and it's only 2 lines.

Example:

# your list
x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25]
# amount of chunks you want
n = 6
x = np.array(x)
l = np.array_split(x, n)
print(l)

>> [array([1, 2, 3, 4, 5]), array([6, 7, 8, 9]), array([10, 11, 12, 13]), array([14, 15, 16, 17]), array([18, 19, 20, 21]), array([22, 23, 24, 25])]

And if you want a list of list:

l = [list(elem) for elem in l]
print(l)

 >> [[1, 2, 3, 4, 5], [6, 7, 8, 9], [10, 11, 12, 13], [14, 15, 16, 17], [18, 19, 20, 21], [22, 23, 24, 25]]

Answer 7

This function will return the list of lists with the set maximum amount of values in one list (chunk).

def chuncker(list_to_split, chunk_size):
    list_of_chunks =[]
    start_chunk = 0
    end_chunk = start_chunk+chunk_size
    while end_chunk <= len(list_to_split)+chunk_size:
        chunk_ls = list_to_split[start_chunk: end_chunk]
        list_of_chunks.append(chunk_ls)
        start_chunk = start_chunk +chunk_size
        end_chunk = end_chunk+chunk_size    
    return list_of_chunks

Example:

ls = list(range(20))

chuncker(list_to_split = ls, chunk_size = 6)

output:

[[0, 1, 2, 3, 4, 5], [6, 7, 8, 9, 10, 11], [12, 13, 14, 15, 16, 17], [18, 19]]

Answer 8

If you want to have the chunks as evenly sized as possible:

def chunk_ranges(items: int, chunks: int) -> List[Tuple[int, int]]:
    """
    Split the items by best effort into equally-sized chunks.
    
    If there are fewer items than chunks, each chunk contains an item and 
    there are fewer returned chunk indices than the argument `chunks`.

    :param items: number of items in the batch.
    :param chunks: number of chunks
    :return: list of (chunk begin inclusive, chunk end exclusive)
    """
    assert chunks > 0, \
        "Unexpected non-positive chunk count: {}".format(chunks)

    result = []  # type: List[Tuple[int, int]]
    if items <= chunks:
        for i in range(0, items):
            result.append((i, i + 1))
        return result

    chunk_size, extras = divmod(items, chunks)

    start = 0
    for i in range(0, chunks):
        if i < extras:
            end = start + chunk_size + 1
        else:
            end = start + chunk_size

        result.append((start, end))
        start = end

    return result

Test case:

def test_chunk_ranges(self):
    self.assertListEqual(chunk_ranges(items=8, chunks=1),
                         [(0, 8)])

    self.assertListEqual(chunk_ranges(items=8, chunks=2),
                         [(0, 4), (4, 8)])

    self.assertListEqual(chunk_ranges(items=8, chunks=3),
                         [(0, 3), (3, 6), (6, 8)])

    self.assertListEqual(chunk_ranges(items=8, chunks=5),
                         [(0, 2), (2, 4), (4, 6), (6, 7), (7, 8)])

    self.assertListEqual(chunk_ranges(items=8, chunks=6),
                         [(0, 2), (2, 4), (4, 5), (5, 6), (6, 7), (7, 8)])

    self.assertListEqual(
        chunk_ranges(items=8, chunks=7),
        [(0, 2), (2, 3), (3, 4), (4, 5), (5, 6), (6, 7), (7, 8)])

    self.assertListEqual(
        chunk_ranges(items=8, chunks=9),
        [(0, 1), (1, 2), (2, 3), (3, 4), (4, 5), (5, 6), (6, 7), (7, 8)])

Answer 9

My answer is to simply use python built-in Slice:

# Assume x is our list which we wish to slice
x = range(1, 26)
# Assume we want to slice it to 6 equal chunks
result = []
for i in range(0, len(x), 6):
    slice_item = slice(i, i + 6, 1)
    result.append(x[slice_item])

# Result would be equal to 

[[0,1,2,3,4,5], [6,7,8,9,10,11], [12,13,14,15,16,17],[18,19,20,21,22,23], [24, 25]]

Answer 10

This solution is based on the zip "grouper" pattern from the Python 3 docs. The small addition is that if N does not divide the list length evenly, all the extra items are placed into the first chunk.

import itertools

def segment_list(l, N):
    chunk_size, remainder = divmod(len(l), N)
    first, rest = l[:chunk_size + remainder], l[chunk_size + remainder:]
    return itertools.chain([first], zip(*[iter(rest)] * chunk_size))

Example usage:

>>> my_list = list(range(10))
>>> segment_list(my_list, 2)
[[0, 1, 2, 3, 4], (5, 6, 7, 8, 9)]
>>> segment_list(my_list, 3)
[[0, 1, 2, 3], (4, 5, 6), (7, 8, 9)]
>>>

The advantages of this solution are that it preserves the order of the original list, and is written in a functional style that lazily evaluates the list only once when called.

Note that because it returns an iterator, the result can only be consumed once. If you want the convenience of a non-lazy list, you can wrap the result in list:

>>> x = list(segment_list(my_list, 2))
>>> x
[[0, 1, 2, 3, 4], (5, 6, 7, 8, 9)]
>>> x
[[0, 1, 2, 3, 4], (5, 6, 7, 8, 9)]
>>>

Answer 11

In cases, where your list contains elements of different types or iterable objects that store values of different types (f.e. some elements are integers, and some are strings), if you use array_split function from numpy package to split it, you will get chunks with elements of same type:

import numpy as np

data1 = [(1, 2), ('a', 'b'), (3, 4), (5, 6), ('c', 'd'), ('e', 'f')]
chunks = np.array_split(data1, 3)
print(chunks)
# [array([['1', '2'],
#        ['a', 'b']], dtype='<U11'), array([['3', '4'],
#        ['5', '6']], dtype='<U11'), array([['c', 'd'],
#        ['e', 'f']], dtype='<U11')]

data2 = [1, 2, 'a', 'b', 3, 4, 5, 6, 'c', 'd', 'e', 'f']
chunks = np.array_split(data2, 3)
print(chunks)
# [array(['1', '2', 'a', 'b'], dtype='<U11'), array(['3', '4', '5', '6'], dtype='<U11'),
#  array(['c', 'd', 'e', 'f'], dtype='<U11')]

If you would like to have initial types of elements in chunks after splitting of list, you can modify source code of array_split function from numpy package or use this implementation:

from itertools import accumulate

def list_split(input_list, num_of_chunks):
    n_total = len(input_list)
    n_each_chunk, extras = divmod(n_total, num_of_chunks)
    chunk_sizes = ([0] + extras * [n_each_chunk + 1] + (num_of_chunks - extras) * [n_each_chunk])
    div_points = list(accumulate(chunk_sizes))
    sub_lists = []
    for i in range(num_of_chunks):
        start = div_points[i]
        end = div_points[i + 1]
        sub_lists.append(input_list[start:end])
    return (sub_list for sub_list in sub_lists)

result = list(list_split(data1, 3))
print(result)
# [[(1, 2), ('a', 'b')], [(3, 4), (5, 6)], [('c', 'd'), ('e', 'f')]]

result = list(list_split(data2, 3))
print(result)
# [[1, 2, 'a', 'b'], [3, 4, 5, 6], ['c', 'd', 'e', 'f']]

Answer 12

arr1=[-20, 20, -10, 0, 4, 8, 10, 6, 15, 9, 18, 35, 40, -30, -90, 99]
n=4
final = [arr1[i * n:(i + 1) * n] for i in range((len(arr1) + n - 1) // n )]
print(final)

Output:

[[-20, 20, -10, 0], [4, 8, 10, 6], [15, 9, 18, 35], [40, -30, -90, 99]]

Answer 13

Here take my 2 cents..

from math import ceil

size = 3
seq = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]

chunks = [
    seq[i * size:(i * size) + size]
    for i in range(ceil(len(seq) / size))
]

# [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11]]

Answer 14

The solution(s) below have many advantages:

• Uses generator to yield the result.
• No imports.
• Lists are balanced (you never end up with 4 lists of size 4 and one list of size 1 if you split a list of length 17 into 5).

def chunks(l, n):
    """Yield n number of striped chunks from l."""
    for i in range(0, n):
        yield l[i::n]

The code above produces the below output for l = range(16) and n = 6:

[0, 6, 12]
[1, 7, 13]
[2, 8, 14]
[3, 9, 15]
[4, 10]
[5, 11]

If you need the chunks to be sequential instead of striped use this:

def chunks(l, n):
    """Yield n number of sequential chunks from l."""
    d, r = divmod(len(l), n)
    for i in range(n):
        si = (d+1)*(i if i < r else r) + d*(0 if i < r else i - r)
        yield l[si:si+(d+1 if i < r else d)]

Which for l = range(16) and n = 6 produces:

[0, 1, 2]
[3, 4, 5]
[6, 7, 8]
[9, 10, 11]
[12, 13]
[14, 15]

See this stackoverflow link for more information on the advantages of generators.

Answer 15

For people looking for an answer in python 3(.6) without imports.
x is the list to be split.
n is the length of chunks.
L is the new list.

n = 6
L = [x[i:i + int(n)] for i in range(0, (n - 1) * int(n), int(n))]

#[[1, 2, 3, 4, 5, 6], [7, 8, 9, 10, 11, 12], [13, 14, 15, 16, 17, 18], [19, 20, 21, 22, 23, 24], [25]]

Answer 16

Hint:

• x is the string to be split.

• k is number of chunks

  n = len(x)/k
  
  [x[i:i+n] for i in range(0, len(x), n)]
  

Answer 17

more_itertools.divide is one approach to solve this problem:

import more_itertools as mit


iterable = range(1, 26)
[list(c) for c in mit.divide(6, iterable)]

Output

[[ 1,  2,  3,  4, 5],                       # remaining item
 [ 6,  7,  8,  9],
 [10, 11, 12, 13],
 [14, 15, 16, 17],
 [18, 19, 20, 21],
 [22, 23, 24, 25]]

As shown, if the iterable is not evenly divisible, the remaining items are distributed from the first to the last chunk.

See more about the more_itertools library here.

Answer 18

If order doesn't matter:

def chunker_list(seq, size):
    return (seq[i::size] for i in range(size))

print(list(chunker_list([1, 2, 3, 4, 5], 2)))
>>> [[1, 3, 5], [2, 4]]

print(list(chunker_list([1, 2, 3, 4, 5], 3)))
>>> [[1, 4], [2, 5], [3]]

print(list(chunker_list([1, 2, 3, 4, 5], 4)))
>>> [[1, 5], [2], [3], [4]]

print(list(chunker_list([1, 2, 3, 4, 5], 5)))
>>> [[1], [2], [3], [4], [5]]

print(list(chunker_list([1, 2, 3, 4, 5], 6)))
>>> [[1], [2], [3], [4], [5], []]

Answer 19

Use numpy

>>> import numpy
>>> x = range(25)
>>> l = numpy.array_split(numpy.array(x),6)

or

>>> import numpy
>>> x = numpy.arange(25)
>>> l = numpy.array_split(x,6);

You can also use numpy.split but that one throws in error if the length is not exactly divisible.

Answer 20

Assuming you want to divide into n chunks:

n = 6
num = float(len(x))/n
l = [ x [i:i + int(num)] for i in range(0, (n-1)*int(num), int(num))]
l.append(x[(n-1)*int(num):])

This method simply divides the length of the list by the number of chunks and, in case the length is not a multiple of the number, adds the extra elements in the last list.

Answer 21

Try this:

from __future__ import division

import math

def chunked(iterable, n):
    """ Split iterable into ``n`` iterables of similar size

    Examples::
        >>> l = [1, 2, 3, 4]
        >>> list(chunked(l, 4))
        [[1], [2], [3], [4]]

        >>> l = [1, 2, 3]
        >>> list(chunked(l, 4))
        [[1], [2], [3], []]

        >>> l = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
        >>> list(chunked(l, 4))
        [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10]]

    """
    chunksize = int(math.ceil(len(iterable) / n))
    return (iterable[i * chunksize:i * chunksize + chunksize]
            for i in range(n))

It returns an iterator instead of a list for efficiency (I'm assuming you want to loop over the chunks), but you can replace that with a list comprehension if you want. When the number of items is not divisible by number of chunks, the last chunk is smaller than the others.

EDIT: Fixed second example to show that it doesn't handle one edge case

Answer 22

One way would be to make the last list uneven and the rest even. This can be done as follows:

>>> x
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25]
>>> m = len(x) // 6
>>> test = [x[i:i+m] for i in range(0, len(x), m)]
>>> test[-2:] = [test[-2] + test[-1]]
>>> test
[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16], [17, 18, 19, 20], [21, 22, 23, 24, 25]]

Answer 23

x=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25]
chunk = len(x)/6

l=[]
i=0
while i<len(x):
    if len(l)<=4:
        l.append(x [i:i + chunk])
    else:
        l.append(x [i:])
        break
    i+=chunk   

print l

#output=[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16], [17, 18, 19, 20], [21, 22, 23, 24, 25]]