Students often have a "pre-conceived" idea of what they think "Random" looks like. Which if you think about it for a second is interesting in itself. (Go look up Apophenia)
The correct answer is: When then sequence appears to have no pattern perceivable by the participant!
Note that this is not the same as "No pattern perceivable to the researcher". The researcher is often conditioned to the sequence simply by their understanding of the research design and having tested the experiment a few (or hundreds) of times. Their brain is already trying to detect patterns in the sequence. This is what brains do.
This is always in contrast to a naive participant who will only experience the sequence once. (If you design calls for repeated measures... different design)
What are the options?
Random Sequence With replacement
Imagine a bag that contains all the possible sequence items, reach in and take an item without looking, record the item and then return it to the bag. Repeat as needed.
This method uses a sampling mechanism of reaching blindly into a bag. This means that each sample is independant.
For example, if we have possible items in the bag (5,9,4,2,7), then a sequence of five samples could be any of the following:
5,9,4,2,7
5,5,5,5,5
7,7,7,7,2
5,9,5,9,5
While these sequences that appear to be patterns are "legal" and can quite possibly be generated using this mechanism, there are 5 * 5 * 5 * 5 * 5 (3125) possible different sequences that could be generated.
Much like a coin-walk, the distribution of the items in this random system will approch being perfectly even.... as the sample size grows... but it will not be anything like even at small sample sizes. Keep in mind that for any participant, who expereinces the sequence once, this is a sample size of 1... I.e lots of noise in the distribution. . Due to this fact, these sequences that appear to have a pattern are generally not what the researchers "want" to see. And while this is still a very effective mechanism for generating the stimuli sequence, the possiblity of there appearing to be a pattern can mess with the researchers head.
Random Sequence Without Replacement
Imagine a bag that contains all the possible sequence items, reach in and take an item without looking, record that item but do not return it to the bag. Now pick the next item from those remaining in the bag. Obviously the bag will eventually be exhausted. (At which point your sequence may be finished, or you may return all items to the bag and start again)For instance, again with our items of ( 5,9,4,2,7), some sequences could be
7,5,9,2,4
2,4,5,7,9
There are 5 * 4 * 3 * 2 * 1 (120) possible sequences if we are creating five item sequences.
If we are creating say a ten item sequence, then we would run this method twice.
i.e 2,4,5,7,9,7,5,9,2,4
Which would give us a possible 120 * 120 ( 14400 ) possible sequences.
This mechanism will generate sequences that do not have the same possibility of repetition of items as the first mechanism.
The distribution of items in this method is much closer to even at small sample sizes and so creates a safe feeling for the researchers. However, keep in mind that where the number of items in the bag is small, this can create a repeating sequence that the participants can still detect. (7 plus or minus 2 being a useful rule of thumb for the number of items a person can remember )
Generally, if the number of sequence items is less than 9, the participant can start to anticipate by elimination what item may come next. This is unavoidable, as its part of the "normal" function of the brain. The longer the sequence runs... the better they will feel at this. (This does not mean they will be right... but over time, it can be better than chance for people who are good at this task)
So what is the best mechanism to generate your stimuli sequence?
Well this gets tricky because often stimuli sequences involve repetition of some items, some contains distractor items with their own frequency, others may include cueing rules and other rules for follow order. Some sequences include "idiot check" items, some are trying to cause patterns and anticipation, while others are trying to control for these effects.... some have blocks of similar stimuli, some have intruders, control blocks and neutral effect stimuli. There are as many permutations as there are researchers and research design.
Find some experts and talk it over. There is a sequence generator for you out there.
The biggest problem however is when the right generator has been build and verified and the researcher says "I don't think its random enough...."
