3/22: Doing Research

Warning: today's class is long and Thursday's is short. Plan your reading time accordingly.

Last Thursday, we started talking about P2, the proposal and annotated bibliography. To recap: P2 is a proposal for what you want to do your future research paper (P3) about. The proposal is a distinct assignment, with its own due dates and peer review and such.

Next week (3/28 to 4/1) we'll have one-on-one meetings via Zoom to discuss your topics, progress, and ideas. Those meetings will replace class, so there won't be lesson posts next week. A signup sheet for these Zoom appointments will come out this Thursday. You don't have to meet with me live--you can sign up on that same sheet for regular draft feedback instead--but it is recommended.

Your job this week is to pick a few topics you might be interested in and begin doing preliminary research. There's a Slack discussion this week where you need to share a few of these ideas and--very importantly--give some of your peers feedback on their shared ideas. The Slack channel p2topicideas is already available.

The other thing you need to do is turn in an early draft of your proposal before our Zoom meeting. On Canvas, the box says "March 28" but in actuality, you can turn it in (or even submit and resubmit a document) up until your one-on-one Zoom with me. Since the meeting will be about your ideas and preliminary research, it's helpful to have a draft, even if you end up changing ideas entirely (it's only 2 paragraphs--way easier to change now than when you've finished the whole thing.)

Today, we're going to talk about research approaches and research methods generally. You can always ask me questions about your specific research needs, but today's class is about basic strategies.

Expert or Scholarly Sources

Your proposal should (eventually) be joined with a list of about 8 or more sources, at least 5 of which should be expert or scholarly. What does this mean?

A scholarly or expert source is a document written by an expert in the subject matter, usually for other experts in that area. Typical signs that you’re seeing a scholarly source are:

The author is named, and their credentials are legitimate. (i.e. they’re a Doctor of Medicine and the article is about medical things they did work on).
The source attempts to be objective and fair, even if it has a point it’s trying to explain. (i.e. it’s not an advertisement or a biased argument, but actually trying to discuss a complex thing.)
It is peer reviewed, meaning it’s published in a place where, in order to get published, other experts have looked at it.
The language is geared to an audience of interested or expert people.

You’ll notice that points 1, 3, and 4 disqualify Wikipedia as an expert source. Wikipedia is a valuable research tool, but what it actually is is an inside-out annotated bibliography. Instead of a list of sources with summaries, it’s a summary of a whole field with a list of its sources. Still useful as summary, but not expert on its own.

Point #4 also disqualifies stuff like Procon.org and news websites describing scientific studies, since they’re “boiling stuff down” to basic terms, which is good for quick understanding but bad if you want to know more.

Starting on Wikipedia = Yes

Wikipedia is your friend, but it's the kind of friend you need to take with a small grain of salt. Wikipedia is a summary of expert knowledge on a topic or area, and usually carefully cites its sources at the bottom. This makes it an excellent place to start researching, since you can read the article for a basic understanding, then find the article's sources, read those, and then read those source's sources. This is sometimes called drilling down into research, or rabbit-hole-ing. It is a research pro-gamer move, and almost always a good idea.

In general, though, you shouldn't cite Wikipedia as a source in itself, since it's not: it's a research tool that helped you find sources*. Citing Wikipedia is like citing your parents on your college math exam because they taught you numbers when you were 2 years old.

*You can cite Wikipedia though, if you want to talk about how Wikipedia talks about things. For instance, in my field, there's a big problem: there's a Wikipedia entry that is constantly being edited by some jerk to push his own weird views, and so I once quoted these revisions while giving a lecture at a conference, to illustrate how this was a problem (since non-experts and students thought those weird views were serious because they were on Wikipedia.) Confusing, I know. Modern academia is weird.

That weird dude thankfully wasn't at the conference, but the person he kept editing-over was, so I made a new friend.

Getting More Sources: Drilling Down

You’re enrolled at a university that has one of the best research libraries on this coast. Thankfully, library.udel.edu is still fully operational, so if you can't physically get to the library most stuff is available electronically. This means you can do all your research from home! (If you're on campus though, a browse in the shelves is always useful).

Things You Should Know About Being in the Library

Texts in the library are not organized by Dewey Decimal system. Instead, they use Library of Congress organization (because it's a "research library" not a public one). This means that texts are grouped by field and topic, so if you could physically locate one text about a thing you want to research, you can go to where it is on the shelf, and the stuff around it will also be about that thing. This is way cooler than it sounds.
You can look up any book on the library website, and there’s a code that tells you where the book is physically located in the library.

On a side note, Google Scholar is often a very useful tool, but it’s not always possible to get the actual source through it. This is where the library can help. If you have the author(s)’ names, the title, and what book or journal or magazine the source is in from a previous search, you can search for it on the library website and usually get a PDF of the whole article.

If they don’t have it, you can contact Interlibrary Loan and they will get it for you electronically. You must be logged in to the library website to do this, but the login is the same as for your email and Canvas and stuff.

Watch the video below for how to use the library to find and get sources. It's just me screen-sharing, so fullscreen it so you can see what my mouse is doing.

This video is 13 minutes long--sorry, but the library has some detailed stuff!

Things to note:

The general search-bar search is okay, but often returns a lot of not-very-relevant results.
Each major field has a research guide that will show you important journals, research sources, and websites for academic work in that field. This is a good way to get the pulse of a field.
Each field also has an assigned research librarian, who is an expert on finding info in that field. You should know who your major's librarian is, since they're just rad and crazy helpful.
You may have worked with research databases in high school, which is good, but many of the ones you worked with earlier are generalist databases. For specific topics, each field often has its own database tailored to that interest.
Side note: your log-in credentials for those high school databases may still work, but be very aware that databases are a paid product and have access tiers--your high school login might take you to the baby's-first-database version, which can dumb-down your results or even conceal "inappropriate" topics that are valid research.
Try searching for your topic with multiple different terms or phrases. Some ideas get phrased differently over time, so the terms to search for change.

Citing Sources

You should choose the citation style appropriate to your field of interest. If you’re not sure, or not comfortable, MLA style is the vanilla ice cream of citation styles. The Purdue OWL has online guides for most citation styles. In an annotated bibliography, you’re expected to properly cite the sources you list, in addition to very briefly describing what they are about and what they do for you.

At this stage of research, it is often a good idea to skim expert sources, rather than reading them beginning-to-end. Scholars rarely read the entirety of stuff in real life–most good sources are structured for skimming and so experts can read the parts they’re interested in and skip the rest. Learn to do this. Take notes on what you read and your questions about it, and keep these safe.

Record the authors and titles of sources–don’t just save links somewhere, since links tend to break. Seriously.

A source that’s very badly formatted, not-skimmable, and hard to read is a red flag--it may not be expert or scholarly at all. More on sketchy sources in a minute.

You should also be able to tell what kind of source a thing is. Here’s some common forms of scholarly source:

Books and eBooks: scholarly ones are often published by a University Press. Some university presses are more fancy than others, but generally have “University Press” or “UP” in the publisher’s name. Some, like "Bloomsbury Academic" are just a normal publisher's academic press--these are also scholarly.
Journal Articles: articles about a topic are often published in journals. For example, the Arak Journal from earlier in the course is a collection of articles. For a list of prestigious journals in specific fields, the library has guides by major and field. It is a good idea to know the cool journals in your own major.
Databases: these are not actually sources–they’re collections of journals and newspaper articles, curated by field and topic. You generally do not cite databases–you just use them to get books and journal articles.

Books and ebooks take a long time to write (1-5 years), and often include summaries of the current conversation around an issue as a chapter. Books are therefore good for broad or big-picture knowledge, getting a general sense of a whole area, and for finding more sources (since they always have source lists.) Books are often geared towards “noobs” or to a larger audience of experts from several related fields.

Articles in journals take less time to write (2-10 weeks), tend to include a little less summary, and are often more specific and cutting-edge. Journal articles are how academic-types keep each other up to date on the latest and craziest stuff. These are great sources once you know what you want to talk about, and for getting deep or specific knowledge. Journals are for “nerds” of a particular topic, and tend to use more jargon without defining it.

Non-Scholarly Sources

The reason why I only require 5 of the 8 sources to be expert is because non-expert sources are often valuable, even if they aren’t as solid and convincing as expert stuff. If you’re doing a project in a weird or obscure area, too, the definition of who an expert is might change. Non-scholarly sources include things like:

Interviews or testimonials from “normal” people.
Informal surveys.
Explanatory or educational texts, like Buzzfeed reports, news articles, blog posts, tweets, lectures on YouTube, TED Talks, and Instagram posts. Sometimes these are written by experts--those are in an expert/not-expert gray area, but still often informative and valuable.
Documentaries.
Wikipedia entries.

Non-scholarly sources are often good for establishing what people think about an issue, what the popular discourse is, or providing reader-accessible, simple definitions of key concepts and terms. Sometimes, however, they're written by biased nonexperts and might be a little off, so always take them with a realistic grain of salt.

Likewise, textbooks and interviews with professors are expert sources, but work best when backed up by some published Serious-Pants sources.

Other Kinds of Sources

Interviews, literary texts (like books, games, and movies) that you might analyze in an essay, first-hand accounts of an incident, historical artifacts, and personal narratives are called primary texts, meaning they are eye-witness accounts or artifacts of the time. This isn’t a relevant category for most research, but I wanted you to know what they were called, just in case. For instance, the sample proposal on Canvas is for a literature research paper, so it has a primary source section in the annotated bibliography. In its case, "primary sources" are the literary texts being analyzed.

Bad Sources

Sadly, there have always been special interest groups who try to push their agenda by pretending to have the credibility of experts, or by inventing their own fake experts. These groups will often adopt ambiguous or misleading names and cite experts who have fake degrees, or just as bad, have advanced degrees but not in the area they're asked to discuss. This is a profitable exercise, and so it is often difficult to identify bad sources.

This is why research guides and databases are important--these things are peer reviewed and policed by experts, who seek out and remove bad sources. Google Scholar does not do this, for example. This is why it's sometimes important to Google a journal or author you've found, to make sure they didn't just pop into existence 5 minutes ago.

Here's a list of things to look out for in your research findings:

The author/scholar is not affiliated with an institution or organization at all (you can google legitimate academics and find them--if nothing else, you'll find a record from when they were a grad student).
They don't prove the things they say they prove, and don't properly cite sources.
The author is a respected expert, but not discussing something within their expertise (this is sometimes called reputation farming, like when a celebrity endorses a product they would never actually use, for money.)
Ad-like language--academic prose, even when it has a point to prove, usually doesn't sound like it's enthusiastically promoting something. We express our excitement through rigor, specific examples, and good writing, not by over-selling ideas or speaking in broad generalizations.
They refer to people by their first names or use titles wrong. For example, no actual psychiatrist would go by Dr. Phil professionally, which should be your first clue that Dr. Phil isn't an accredited doctor of anything. As another example, a "Natural Doctor" sounds weird because it is--it's not a real doctoral degree, but some health/lifestyle gurus say they have these.
Keep an eye on the STAR criteria. They can clue you in, too.
"Research" institutions with extremely vague names (what would the "Institute of Family Values" even study? What field is that?) or names that weirdly avoid using the word "university" (i.e. the YouTube propaganda channel PragerU*) are trying to skirt regulations about naming because they're not legitimate sources of information.

If you paid attention to the ideology discussion last week, you'll have also noticed another key give-away for bad sources: self-fulfilling proof. In other words, any source that purports that all opposing evidence is fake and conspiring to oppress it is probably bad. In fact, it's actually probably a conspiracy theory.

A final prominent indicator that something is shady is the use of stigmatized knowledge. Stigmatized knowledge is a rhetorical move where information is presented as "something they don't want you to know"--this move makes the audience feel special and included, but also prevents the information from ever being contested. After all, they could be working to prevent you from knowing! If you use the STAR criteria, this kind of evidence will normally fail Typicality, and tries to conceal it by accusing everyone else of being malicious.

Exercise: whenever some person or source accuses a large, abstract group of conspiring to make them look wrong, replace that large, abstract group with another one. If the new statement sounds just bonkers, the original one is ridiculous too. Every time a politician says "the media," replace it with "vegetarians" or "dentists" or something and you'll see what I mean.

*Fun Internet Drama: PragerU (originally "Prager University") has been in trouble with Youtube a few times because it violates Community Guidelines on using the term "University" when it's not an educational institution (it's a political non-profit organization founded by a guy named Prager, files taxes in the US as such, and is legally forbidden from pretending it's something else). Its supporters have accused YouTube of censoring their views each time, even though much of the trouble is due to deceptive naming and not because it promulgates extremist, pseudo-academic "discussions" of US history and politics (these things are problematic, too, but technically "free speech" most of the time).

To counter and parody this, a bunch of people from the other end of the political spectrum formed the Gravel Institute--it does the same thing, but notably "Institute" isn't a lie the way "University" would be, so they get in trouble less. It also helps that nobody from the Gravel Institute tried to storm the Capitol on January 6th, unlike several PU fans.

The internet is weird sometimes. Bad sources can be dangerous.

For more on how to detect bad behavior in sources on the internet and in the news, I recommend DataJournalism's short handbook Investigating Disinformation and Media Manipulation, which you can read here for free. You could also just watch CrashCourse's class in Media Literacy on YouTube. Neither of these are required reading for this class, but simply personal enrichment suggestions you might do on a boring Sunday afternoon sometime.

Homework

Identify 1-3 potential topics, theses, or questions you might want to do research on, and share them in this week's Slack discussion channel. Then, circle back later and contribute to the discussion with your classmates. If someone has 3 ideas and you think #2 is cool, tell them!

Don't forget the quiz!

Read some Wikipedia articles, maybe do some early research. More on this on Thursday.