Removing items in an array in arbitrary multiples

Question

I have an array which is a list of times (it's the number of days in 10 years), I want to remove a certain selection of this list and then perform calculations based off the remaining days. This is for an astrophysical simulation and I'm trying to pick certain patterns of 'observing' another dataset.

My array is:

t = [0,1,2,3,...,3650]

For my purposes, I want to keep the first 180 days in each year and remove the rest leaving me with something like:

t_new = [1,2,...,178,179,365,366,...]

I have achieved this by:

a = 180
b = 365 - a

t_new = [x for i, x in enumerate(t) if i%(a+b) < a]

Having a and b as variables lets me play with being able to keep the different amounts of data in each year.

Now what I want to do is keep the first 10 of every 30 of the remaining data. This will simulate only being able to observe on the first 10 days of every month, and the first 6 months of every year.

I've tried:

c = 10
d = 30-c

t_new2 = [x for i, x in enumerate(t_new) if i%(c+d) < c]

This should result in:

t_new2 = [0,1,2,3,4,5,6,7,8,9,30,31,...,178,179,365,366,...]

So this is the days remaining after we've removed the last 20 of every 30 days (every month), and the last 185 of 365 (every year).

But I'm getting the error:

ValueError: too many values to unpack

Is there a better of doing this logic, perhaps combining it into just one line?

Thanks!

Follow-up question:

The solution so far is,

t_new = [365*year+day for year in range(10) for day in range(180) if day%30 < 10]

Which edits the original array based off the values of each index. I'd like to keep only the exact same indices in a second array whose values are effectively random, is there a way to turn the above into an index operation?

Answer 1

Note: Note that you oversimplify the calendar here. Leap years are ignored here.

You can do this as follows:

[day for year in range(10) for day in range(365*year,365*year+180)]

The code works as follows: first we iterate over the years: a range from 0 (included) to 10 (excluded), so 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9.

Next for every year we iterate with day from 365*year (the day at which the "virtual year" starts) to 365*year+180 (the day at which our 180 days end).

And for each such day, we add it to the list.

This is also more efficient, than using an if filter statement in the list comprehension: here only "valid" days are generated. A filter can of course obtain the same functionality, but you will first generate a (possibly large) set of values you have to withdraw later.

Now what I want to do is keep the first 10 of every 30 of the remaining data.

We can do this by adding an additional filter here:

[365*year+day for year in range(10) for day in range(180) if day%30 < 10]

This will generate:

>>> [365*year+day for year in range(2) for day in range(180) if day%30 < 10]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524]

(I generated a list for the first two years only)

EDIT: if you want the indices of an array for the above stated indices, then you can simply use:

[a[365*year+day] for year in range(10) for day in range(180) if day%30 < 10]

where a is you original list with values.

Answer 2

An alternative approach using a generator to save some memory, and maybe more pythonic...

""" This approach (using a function with yield statement) is called a generator
It allows to create series of data without reserving memory for the entire thing
only the current few parameters are stored in memory"""

def iterfunc(start_no, elem_count, period):
    """this is the generator"""
    i = start_no
    while 1:
        if i%period<=elem_count and i%period>=start_no :
            yield i
        i+=1

""" Need to create an instance of the generator,
so that this instance remembers our parameters,
also this way we can create more instances with 
different parameters in each one
"""
day = iterfunc(1, 3, 10)

""" Here we print the days, but equally well we can append each to a list """
for x in range(30):
    print(day.next())

Answer 3

to be able to generate a list with sets of consecutive values repeating over a larger period like [1,2,3,11,12,13,21,22,23,...], we need to provide 4 parameters

• starting number of the consecutive part start_no
• number of consecutive elements elem_count
• the period (number) period
• total number of days you consider tot

sample code

start_no = 1
elem_count = 3
period = 10
tot = 3650
t_new = [x for x in range (tot) if x%period<=elem_count and x%period>=start_no]
print(t_new)