UMBC "Substance Use" survey shows incredible bias
Faulty questions and religious bias
posted almost 14 years ago
This appeared on the myUMBC spotlights today (2/13):
"College Substance Survey! A chance to win a $50 or $100 Visa gift card! Take the College Substance Survey today! -Must be a UMBC student. -Must be 18 years or older If you are interested in participating, please click the Visit Website button"
So, I clicked the button that took me to Survey Monkey (can you say unprofessional?), read the disclaimer, and authorized participation. I noted that the survey was sponsored by Dr. DiClemente at the UMBC Psychology Department HABITs lab, so I figured it was legit.
After taking the survey, I felt obliged to write this LONGGG response to Dr. DiClemente and his lab. Definitely do NOT take this survey. If anyone else agrees that what I listed below are serious problems, please consider also writing to the department. Here's my email:
Hello Dr. DiClemente,
I recently took the substance abuse survey from a link on myUMBC. I'm writing because I have several problems with the direction that the survey took as it progressed, and with the phrasing of several questions. As I continued to take it I became increasingly unhappy with the survey. I'm not sure who is in charge of the research that this survey is conducting, but if this isn't relevant to you, please forward these comments to whoever is directing this study. Your name appeared on the home page of the survey. I have also CCed the email address that appeared on the flyer.
As a full disclosure, I am an atheist, and the president of the Secular Student Alliance at UMBC, a group that seeks to improve the image of nontheist students and promote secular humanism. Some studies have shown that in the age group of 18-25, up to 21% of the population identifies as atheist, agnostic, or nonreligious. This means that up to one in five respondents to your survey may be nontheists.
First of all, without first asking in the survey for respondents to identify their religious affiliation, the survey entered a section in which respondents should identify the level to which they accept and find God in their daily lives. The survey does not identify to which "God" it is referring, but contains the following disclaimer:
"The list that follows includes items that you may or may not experience. Please consider how often you directly have this experience, and try to disregard whether you feel you should or should not have these experiences. A number of items use the word 'God.' If this word is not a comfortable one for you, please substitute another word which calls to mind the divine or holy for you."
To me, this is nonsense. Nothing is divine or holy to me, and so this section was both irrelevant and difficult to answer. There was no way to answer "not applicable", just "never" or "refuse to answer." I do experience a connection to all of life, but only because I study the natural sciences and have an understanding of how humans as animals are connected to all other living organisms on the planet. But the bold posing of questions in this list, such as "I feel God's love for me, directly", "I feel thankful for my blessings", "I feel God's presence", and "I desire to be closer to God or in union with the divine" is presuppositional and shameful for a scientific research study. Another question asks "In general, how close do you feel to God?". I responded "not at all," but presumably this can be interpreted to mean that I acknowledge a god exists and that I do not feel close to such a being-- perhaps because of my substance use behavior? As I do not acknowledge any divine entities exist, this question is nonsense to me. With no "not applicable" response, this and other questions are totally faulty.
I can imagine how such results will be applied-- I expect that if the results find a negative correlation between amount of spirituality and substance use, this will be a successful result for the survey. But the questions are utterly biased and misleading. There was no answer in this section that I could comfortably respond to, none that I identified with that did not explicitly require religious belief. I will find it very distasteful if the results of this research lead to the declaration that having religious belief leads students to less instances of substance abuse. It would be an embarrassment for the department, honestly.
In the "choices and likes" section, more problems arose. Not just due to issues of religious belief, but in simple survey structure and general substance knowledge. Many of these choices present false dichotomies and incredible biases. Question #13 asks:
A) I find stimulants make me uncomfortable
B) I often like to get high (drinking liquor or smoking marijuana).
C) Refuse to answer
The most glaring fallacy of this question is that liquor and marijuana are NOT STIMULANTS. They are primarily depressants. What if a respondent who likes the way depressants make them feel (slowing heartrate, etc) but dislikes the way stimulants make them feel? The question also leaves no room for respondents who occasionally like to "get high", but not often. This question should be thrown out-- it is useless.
Question #22 is perhaps the worst in this entire survey, and reveals the incredible bias of this research.
A) I would like to meet some persons who are homosexual (men or women).
B) I stay away from anyone I suspect of being "gay" or "lesbian."
C) Refuse to answer.
First of all, I already KNOW many people who identify as homosexual and are proud to call them my friends. And what about the wording of response B-- anyone I "suspect" of being gay or lesbian? As if it's a crime, the committers of which deserve to be avoided? Homosexuality is a spectrum of personal identity. Other than identifying as gay or lesbian, sexual identify also typically includes people who are bisexual or transgendered. What if a student who identified as gay or lesbian were taking this survey? How should they interpret this question? Many UMBC students are homosexual. This question is offensive and biased.
Question #29:
A) I like to date persons who share my values [I suppose this means religious values, again, as a subtext]
B) I like to date persons who are physically exciting
Is this a realistic dichotomy? That we must settle for people who we do not find physically attractive so that they share our values, or people we are attracted to physically who do not agree with us in any way? There is no compromise in this question.
Question #36 is simply judgmental.
A) I feel best after taking a few drinks
B) Something is wrong with people who need liquor to feel good.
Why not phrase the responses like this?
A) I feel like I need to drink alcohol to feel good
B) I like having a few drinks socially
B) I prefer not to drink, but I don't mind when others do in moderation
C) I don't drink, and I prefer not to be around people who drink
These are just a few of the glaring problems with this survey.
I would have left these comments on a "comments about this survey page", if there had been one, but this was another problem with the survey. It left no opportunity for feedback, or to rescind responses at the end. One cannot even go backwards in the survey to amend responses.
Your survey advertises itself as a college student substance use survey, but it is far from any kind of respectable scientific research. I'm honestly shocked that the IRB approved this study. I'm looking forward to hearing how the HABITS lab responds to these issues, as I don't think any of the data that is collected from this survey is useable.
Thanks for your time,
(edited almost 14 years ago)