Wednesday, 30 July 2014

Research Instrument Design and Anonymity

Here's a scenario...

Your research design means you are collecting data using an online survey instrument. You have ethical clearance to collect data from your participant population anonymously. You then embed a question in the survey asking the participants to provide their email address if they would like a summary of the results of the research; as per your ethical obligation.

Later....

When looking at the raw data from your instrument, you see a row of the responses provided by the participant... next to their email address.

Now tell me how this is anonymous... and show your working!

Why is this a problem?

1) You know the population that was invited to participate.
2) You know some or all of the email addresses of those participants. By elimination you may be able to guess more identities. With additional demographic information you can further refine your guesses.
3) You now need to store and secure the data from your instrument at a much higher level of security.
4) You are now required to store ALL this data for seven years in such a way that it can be reivewed by third parties (who you do not know) at any time in the future.
5) This data exists on multiple computer systems already.
6) This data may leak in a number of obvious and in-obvious ways.
7) You are ledgislated to handle and secure this data as of 12/3/2014.
8) It's really hard to be sure that data is actually deleted. Sooner or later, a search engine will find it.

How is this ethical research? How are you in compliance with the NHMRC guidelines? How are you in compliance with the Australian Privacy Legislation? How are you in compliance with the University Policies?

The government has passed some updates to the privacy laws and now they have real and specific applicability to this data.

Penalties for non-compliance

You are individually liable for penalties if you do not comply (up to $220,000 for individuals, $1.7 Million for organisations... I.e your employer, the University) See the Privacy Legislation below.

What is Personal Information?

Personal information has the meaning as set out in s 6 of the Privacy Act:

information or an opinion (including information or an opinion forming part of a database), whether true or not, and whether recorded in a material form or not, about an individual whose identity is apparent, or can reasonably be ascertained, from the information or opinion.

Sensitive information is a subset of personal information. The Privacy Act defines sensitive information as:

information or an opinion about an individual’s:
1. racial or ethnic origin; or
2. political opinions; or
3. membership of a political association; or
4. religious beliefs or affiliations; or
5. philosophical beliefs; or
6. membership of a professional or trade association; or
7. membership of a trade union; or
8. sexual preferences or practices; or
9. criminal record;
that is also personal information; or
health information about an individual; or
genetic information about an individual that is not otherwise health information.

What are your obligations once you collect Personal Information in a research data set?

Securely storing the data (Copying, publishing, data requests, integrity investigations)
Securely destroy the data
Authentication of access to the data
Manditory reporting of data breaches
Transfer of control of the data
Hosting the data on foreign servers
Implications of the Freedom of Information Act

The list of obligations and the cost for compliance is quite substantial..... are you sure you want to collect this information for your research project?

Academic Email is subject to the Freedom of Information Act under Australian Law. This means that if your data set has been emailed (say between the student and their supervisor) then it could be leaked via that mechanims.

What can Students and Supervisors do?

Avoid this whole mess by not collecting personally identifying information (Email Addresses specifically) as part of the data set. Design around this potential risk.

Do not sample very small, specific, known populations.

Beware when the Ethics review hands back a requirement for this kind of mechanism in your study. Be prepared to push back with some alternate design strategies to avoid this problem.

Be aware of the legislation and the implications of compliance.

Design research to seperate identity and data, if the participants are known. Do not embed their identities in the data set or research materials (which must then be stored and shared)

Alternate Design Strategies

Case 1 - Ethical Requirement for optional feedback of research results to research participants.

The recomended (by me) strategy is to provide the particiants with a contact email address (researcher or supervisor) from whom they can request a copy of the results of the research.
This strategy avoids the issue of collecting and holding a list of email addresses with their associated cost and the risk of violating the anonymity of the participants.

Case 2 - Repeated measures design requiring followup contact with participants.

Request participants to contact the researcher and be added to a pool prior to the data collection starting. Then the researcher can broadcast to this list an anonymous link to the data collection instrument at each measure time.

This provides a dis-connect between the participants activity and their identity. Unless the researcher has a very small pool or makes other attempts to link the particiant and their data... there is no way to identify who has provided which data record.

This then allows the list of email addresses to be stored seperatly and destroyed securely independ of the data set that results from the research.

Thursday, 5 June 2014

Random Stimuli Sequences

When is a sequence of stimuli "Random" enough?

Students often have a "pre-conceived" idea of what they think "Random" looks like. Which if you think about it for a second is interesting in itself. (Go look up Apophenia)

The correct answer is: When then sequence appears to have no pattern perceivable by the participant!

Note that this is not the same as "No pattern perceivable to the researcher". The researcher is often conditioned to the sequence simply by their understanding of the research design and having tested the experiment a few (or hundreds) of times. Their brain is already trying to detect patterns in the sequence. This is what brains do.

This is always in contrast to a naive participant who will only experience the sequence once. (If you design calls for repeated measures... different design)

What are the options?

Random Sequence With replacement

Imagine a bag that contains all the possible sequence items, reach in and take an item without looking, record the item and then return it to the bag. Repeat as needed.

This method uses a sampling mechanism of reaching blindly into a bag. This means that each sample is independant.

For example, if we have possible items in the bag (5,9,4,2,7), then a sequence of five samples could be any of the following:

5,9,4,2,7

5,5,5,5,5

7,7,7,7,2

5,9,5,9,5

While these sequences that appear to be patterns are "legal" and can quite possibly be generated using this mechanism, there are 5 * 5 * 5 * 5 * 5 (3125) possible different sequences that could be generated.

Much like a coin-walk, the distribution of the items in this random system will approch being perfectly even.... as the sample size grows... but it will not be anything like even at small sample sizes. Keep in mind that for any participant, who expereinces the sequence once, this is a sample size of 1... I.e lots of noise in the distribution. . Due to this fact, these sequences that appear to have a pattern are generally not what the researchers "want" to see. And while this is still a very effective mechanism for generating the stimuli sequence, the possiblity of there appearing to be a pattern can mess with the researchers head.

Random Sequence Without Replacement

Imagine a bag that contains all the possible sequence items, reach in and take an item without looking, record that item but do not return it to the bag. Now pick the next item from those remaining in the bag. Obviously the bag will eventually be exhausted. (At which point your sequence may be finished, or you may return all items to the bag and start again)

For instance, again with our items of ( 5,9,4,2,7), some sequences could be

7,5,9,2,4

2,4,5,7,9

There are 5 * 4 * 3 * 2 * 1 (120) possible sequences if we are creating five item sequences.

If we are creating say a ten item sequence, then we would run this method twice.

i.e 2,4,5,7,9,7,5,9,2,4

Which would give us a possible 120 * 120 ( 14400 ) possible sequences.

This mechanism will generate sequences that do not have the same possibility of repetition of items as the first mechanism.

The distribution of items in this method is much closer to even at small sample sizes and so creates a safe feeling for the researchers. However, keep in mind that where the number of items in the bag is small, this can create a repeating sequence that the participants can still detect. (7 plus or minus 2 being a useful rule of thumb for the number of items a person can remember )

Generally, if the number of sequence items is less than 9, the participant can start to anticipate by elimination what item may come next. This is unavoidable, as its part of the "normal" function of the brain. The longer the sequence runs... the better they will feel at this. (This does not mean they will be right... but over time, it can be better than chance for people who are good at this task)

So what is the best mechanism to generate your stimuli sequence?

Well this gets tricky because often stimuli sequences involve repetition of some items, some contains distractor items with their own frequency, others may include cueing rules and other rules for follow order. Some sequences include "idiot check" items, some are trying to cause patterns and anticipation, while others are trying to control for these effects.... some have blocks of similar stimuli, some have intruders, control blocks and neutral effect stimuli. There are as many permutations as there are researchers and research design.

Find some experts and talk it over. There is a sequence generator for you out there.

The biggest problem however is when the right generator has been build and verified and the researcher says "I don't think its random enough...."

Wednesday, 4 June 2014

SCU Interlectual Property Rights in Research Projects

http://policies.scu.edu.au/view.current.php?id=00017

Optical Illustion Resources

http://news.distractify.com/culture/mind-blowing-optical-illusions/?v=1

Copyright, Video and Model Releases for Research Purposes

The current copyright legal framework. The general rule is you need to obtain permission to use from the rights holder. This may include payment for the license to use. There are some exemptions under the copyright act for education and research purposes.

COPYRIGHT ACT 1968 - SECT 40

Fair dealing for purpose of research or study

There are various issues that have arisen since the copyright act was written (digital technology) that are more complex to argue. There are proposals to change the copyright system, but it has not yet reached the legislation stage.

ALRC Proposals to change Copyright - Educational Use section

Fair Use and various examples...

Taking Photographs/Video for use in research projects need an SCU model release form signed by the model. This needs to be archived with your project records.

Talent Release /Permission to Use Form

Tuesday, 3 June 2014

Survey Design - Participant Information Sheets

The following is a good template for a functional participant information sheet: (Your milage may vary) This follows the format in the ethics application docs.

Research Title: SOME TITLE HERE

My name is *Researcher Name* and I am conducting research, under the supervision of *Suerpvisor Name*, as part of the Honours year of the Bachelor of Psychology degree at Southern Cross University. My research project investigates the *very brief description*

What does this research involve?

*Description of the process from the participants point of view...* This questionnaire will include questions about your .... This questionnaires should take approximately 20 minutes so overall. All data collection can be done from any computer with internet access. What time of day you decide to complete the questionnaire is completely up to you.

Responsibilities of the Researcher

It is our responsibility as the researchers to provide you with sufficient information to understand the implications of participation in the research. Your participation in the study is voluntary. To ensure anonymity...... Demographic details, such as age and sex, will be linked to the data. We will keep data and consent forms securely in a Psychology Office at SCU for seven years, after which time they will be destroyed. On completion of participation in the study, we are happy to provide you with a summary of the results if you provide contact details. We are unable to provide information on individual scores or provide any clinical advice.

Responsibilities of the Participant

The measures used in the study require that participants have normal or corrected-to-normal vision and hearing. Although there have been no reports of adverse reactions to the methods used in this project, if at any time you are bothered by the experience of recording your emotions or thinking about personality variables during the online component we are asking that you access support as you require. We will give you contact details for the SCU Counselling Service, but you may choose to contact your GP or other sources of support if necessary. As the tasks involved in this project require that you log on the internet, we are looking for participants who:

have access to a computer
are fluent in English
are at least 18 years of age.

Possible Discomforts and Risks

This research is considered very low risk. It is possible that reflecting on your mood or personality may cause you concern. If you are distressed as a result of participating in this research, you are advised to contact a counsellor (e.g., SCU Counselling Service, (02) 6659 3263, or by email on counselling@scu.edu.au) or other forms of support of your choosing. You are welcome to discontinue your involvement at any time without any negative consequences.

Publication of Results of this Research

The findings of this research may be submitted for publication in peer-reviewed journals or presented at conferences. A complete account of the findings of this research will be available at the Southern Cross University Library at a later stage. No identifying information will be included as all participants will just be referred to by a number.

Informed Consent

Participation in this research is voluntary. Your agreement to participate will be assumed from your completion of the survey. You are free to withdraw your consent and participation from the study at any time. Your decision will be treated with respect, no questions will be asked and there will be no negative consequences associated with your withdrawal. All your data will be destroyed immediately.

Inquiries

Any questions you may have regarding this research can be directed to the researcher or research supervisor.
*Supervisor Name here*
Lecturer
Psychology
School of Health and Human Sciences
Southern Cross University,
Coffs Harbour, NSW 2450
Ph: *****
Email:*****

Student Researcher Details
*Researcher Name Here*
Psychology
Southern Cross University
Coffs Harbour Campus NSW 2450
Email: *****

This research has been approved by the Southern Cross University Human Research Ethics Committee. The approval number is: ******.

If you have concerns about the ethical conduct of the research, please write to:
The Ethics Complaints Officer, Southern Cross University
PO Box 157
Lismore
NSW 2480
email: ******

Do you give Consent

By pressing the "Next" button, you consent to your data being used in this project. Remember you can withdraw at any time simply by closing the survey window.

Survey Questions on Gender

Question: How to ask demographics questions about gender.

The scenario: You are creating a survey that includes a demographics section and asks about the gender of the participant. You are interested in equity and diversity as dictated by the policy of the University and respecting your participants rights under state and federal Law....

Old form of the question:

What is your gender?
A) Female
B) Male

Whats changed? The Australian High Court has upheld the right of individuals to be identified as being of "non-specific" gender. Note that this is a NSW specific issue, but other states are working on their own legislation.

http://www.abc.net.au/news/2014-04-02/high-court-recognises-gender-neutral/5361362

New form of the question:

What is your gender?
A) Female
B) Male
C) Non-specific

Please note that this is not nearly as inclusive as it could be, but its a step forward. There are more issues here that need attention if you are a social researchers.

Distinction between public and private gender
Self selected terms used to identify current gender state
Distinction between physical gender expression and social gender/role expression
Distinction between biological gender (DNA) and physical gender expression
Gender Role descriptions

As always, with survey design, there is a tention between asking a question to shape the answers given for the conveinence of the researcher and asking a question to be as inclusive as possible. If your question format is categorical (multiple choice) then you may need to include an "Other" text entry box to fully capture the range of answers your sample population may choose to respond with.

Coffs Harbour SCU Psychology Labs