Exhaustive random number generator

Question

In one of my project I encountered a need to generate a set of numbers in a given range that will be:

• Exhaustive, which means that it will cover the most of the given range without any repetition.

• It will guarantee determinism (every time the sequence will be the same). This can be probably achieved with a fixed seed.

• It will be random (I am not very versed into Random Number Theory, but I guess there is a bunch of rules that describes randomness. From perspective something like 0,1,2..N is not random).

Ranges I am talking about can be ranges of integers, or of real numbers.

For example, if I used standard C# random generator to generate 10 numbers in range [0, 9] I will get this:

0 0 1 2 0 1 5 6 2 6  

As you can see, a big part of given range still remains 'unexplored' and there are many repetitions.

Of course, input space can be very large, so remembering previously chosen values is not an option.

What would be the right way to tackle this problem?

Thanks.

After the comments: Ok i agree that the random is not the right word, but I hope that you understood what I am trying to achieve. I want to explore given range that can be big so in memory list is not an option. If a range is (0, 10) and i want three numbers i want to guarantee that those numbers will be different and that they will 'describe the range' (i.e. They wont all be in a lower half etc).

Determinism part means that i would like to use something like standard rng with a fixed seed, so I can fully control the sequence.

I hope i made things a bit clearer.

Thanks.

Answer 1

Here's three options with different tradeoffs:

1. Generate a list of numbers ahead of time, and shuffle them using the fisher-yates shuffle. Select from the list as needed. O(n) total memory, and O(1) time per element. Randomness is as good as the PRNG you used to do the shuffle. The simplest of the three alternatives, too.
2. Use a Linear Feedback Shift Register, which will generate every value in its sequence exactly once before repeating. O(log n) total memory, and O(1) time per element. It's easy to determine future values based on the present value, however, and LFSRs are most easily constructed for power of 2 periods (but you can pick the next biggest power of 2, and skip any out of range values).
3. Use a secure permutation based on a block cipher. Usable for any power of 2 period, and with a little extra trickery, any arbitrary period. O(log n) total space and O(1) time per element, randomness is as good as the block cipher. The most complex of the three to implement.

Answer 2

Do not use a random number generator to select numbers in a range. What will eventually happen is that you have one number left to fill, and your random number generator will cycle repeatedly until it selects that number. Depending on the random number generator, there is no guarantee that will ever happen.

What you should do is generate a list of numbers on the desired range, then use a random number generator to shuffle the list. The shuffle is known as the Fisher-Yates shuffle, or sometimes called the Knuth shuffle. Here's pseudocode to shuffle an array x of n elements with indices from 0 to n-1:

for i from n-1 to 1
    j = random integer such that 0 ≤ j ≤ i
    swap x[i] and x[j]

Answer 3

Generate an array that contains the range, in order. So the array contains [0, 1, 2, 3, 4, 5, ... N]. Then use a Fisher-Yates Shuffle to scramble the array. You can then iterate over the array to get your random numbers.

If you need repeatability, seed your random number generator with the same value at the start of the shuffle.

Answer 4

If you just need something, what about something like this?

maxint = 16
step = 7
sequence = 7, 14, 5, 12, 3, 10, 1, 8, 15, 6, 13, 4, 11, 2, 9, 0

If you pick step right, it will generate the entire interval before repeating. You can play around with different values of step to get something that "looks" good. The "seed" here is where you start in the sequence.

Is this random? Of course not. Will it look random according to a statistical test of randomness? It might depend on the step, but likely this will not look very statistically random at all. However, it certainly picks the numbers in the range, not in their original order, and without any memory of the numbers picked so far.

In fact, you could make this look even better by making a list of factors - like [1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15, 16] - and using shuffled versions of those to compute step * factor (mod maxint). Let's say we shuffled the example factors lists like [3, 2, 4, 5, 1], [6, 8, 9, 10, 7], [13, 16, 12, 11, 14, 15]. then we'd get the sequence

5, 14, 12, 3, 7, 10, 8, 15, 6, 1, 11, 0,  4, 13, 2, 9

The size of the factors list is completely tunable, so you can store as much memory as you like. Bigger factor lists, more randomness. No repeats regardless of factor list size. When you exhaust a factor list, generating a new one is as easy as counting and shuffling.

Answer 5

It is my impression that what you are looking for is a randomly-ordered list of numbers, not a random list of numbers. You should be able to get this with the following pseudocode. Better math-ies may be able to tell me if this is in fact not random:

list = [ 1 .. 100 ]
for item,index in list:
  location = random_integer_below(list.length - index)
  list.switch(index,location+index)

Basically, go through the list and pick a random item from the rest of the list to use in the position you are at. This should randomly arrange the items in your list. If you need to reproduce the same random order each time, consider saving the array, or ensuring somehow that random_integer_below always returns numbers in the same order given some seed.