Source Evaluation and Information Literacy Study Guide
Why We Evaluate Sources
Validity of Information: Ensuring the information presented is factually correct and sound.
Bias of Information: Recognizing underlying prejudices or perspectives that may influence the content.
Preventing Misinformation: Actively working to avoid sharing false or inaccurate data.
Accuracy of Conclusions: Verifying that any conclusions drawn are supported by the presented data.
Data Support: Confirming that the evidence and data provided genuinely back up the claims made.
Avoiding Propaganda: Identifying and resisting information designed to promote a specific political cause or point of view, often through misleading or emotionally manipulative means.
Expert Analysis: Ensuring data has been competently analyzed by qualified individuals.
Verification: Confirming that data and conclusions have been corroborated by multiple reliable sources or individuals.
Reputability: Distinguishing between credible sources and those driven by commercial interests, AI-generated spam, or "pay-to-play" schemes.
Expert Authorship: Prioritizing work produced by recognized experts in the field over unsubstantiated claims from non-specialists.
Differentiating Information Sources: AI, Search Engines, and Research Databases
AI (Artificial Intelligence)
Source Dependency: AI's output is limited by the quality of its training data; it's "only as good as its sources."
Lack of Oversight: AI systems do not inherently have checks or oversight mechanisms; they merely compile and parrot information from their training.
No Understanding of Relevance: AI can generate text based on patterns but lacks true comprehension of "relevance" or context.
Training Process: AI "learns" by being "trained" on vast datasets, which may include non-reputable information (e.g., social media images, uncredited art).
Compilation, Not Creation: AI results are compilations of recognized patterns and information, not original thought or analysis.
Risk of Hallucinations/Fabrications: AI can generate entirely false information, as seen in cases like lawyers citing fake court cases.
Search Engines
Algorithm-Driven: Search engines use complex algorithms to rank and present links.
No Inherent Accuracy Checks: They do not verify the accuracy of the links or information themselves; users are responsible for evaluating trustworthiness.
Evolving Algorithms: Algorithms have shifted from simple keyword matching to incorporating user behavior and preferences.
AI Integration: Search engines increasingly integrate AI into their algorithms and search results, often displaying AI-generated snippets or summaries on the first page.
SEO Influence: Search Engine Optimization (SEO) heavily influences search result order, prioritizing factors like link popularity and click-through rates over accuracy.
"Relevance" vs. Accuracy: Search engine algorithms define "relevance" based on various factors, but this does not equate to factual accuracy (e.g., ragebaiting/clickbait).
Bias: Algorithms can create a "bias machine," tailoring results to what the user "wants to hear" based on past behavior and demographic data.
Research Databases (Academic Databases)
Peer-Reviewed Content: Should primarily contain articles and scholarly works that have undergone peer review.
Community Vetted: The content is generally considered reputable by the academic or scientific community.
"Pay-to-Play" Caveat: Awareness is needed regarding some publications that prioritize payment over rigorous review.
Information Retrieval: While the information is generally reliable, finding the most relevant content within vast databases can be challenging.
Understanding Bias in Sources
Ubiquitous Agenda: Every source, individual, or organization has an agenda, whether overt or subtle.
Internal Biases: Everyone possesses internal biases that can unconsciously influence their perspective.
Impact of Bias: Bias can affect all aspects of information dissemination, including where it's shared, how it's framed, and the language used (e.g., passive voice in journalism when discussing victims).
High-Stakes Topics: Bias is particularly prevalent and problematic in "hot topics" where the stakes are high, leading to potential misrepresentation or dishonesty.
Types of Bias:Confirmation Bias: Tendency to search for, interpret, favor, and recall information in a way that confirms one's preexisting beliefs.
Demographic Bias: Distortions in coverage due to factors like race, gender, ethnicity, culture, or economic class.
Distance Bias: Potential for bias based on geographical or cultural distance from the subject.
"Big Story" Bias: Journalists' judgment clouded by the perceived importance of an event.
Corporate Bias: Influence of a news outlet's business interests (advertisers, parent company).
Neutrality Bias: Overly zealous attempts to appear unbiased, leading to factual misrepresentation.
Partisan Bias: Journalists' political views affecting coverage.
Differentiating News from Opinion: Recognize that "straight news" aims for impartiality, while opinion columns, editorials, and op-eds are designed to express a viewpoint.
Forms of Bias:Absence of Fairness and Balance: Failure to present all relevant viewpoints accurately and impartially.
Framing: The approach and organization of a news report.
Sourcing: Which voices and perspectives are included or excluded.
Story Selection: What a news organization deems newsworthy.
Tone: The specific words and phrases used.
Recognizing Personal Biases: Our own preconceptions can lead us to misinterpret news, make assumptions about journalists, or dismiss challenging information. Bias is often unintentional and exists on a spectrum.
Critiquing Bias Charts: Be wary of media bias charts that may have methodological shortcomings, such as small samples, mixing news with opinion, or equating credible sources with disinformation outlets.
How to Evaluate Sources: The CRAAP Test
The CRAAP test is a widely used framework for evaluating the credibility and reliability of information sources.

C - Currency:When was the information published or posted?
Has the information been revised or updated?
Is the information current enough for your topic? (Especially important for rapidly changing fields like science or technology).
R - Relevance:Does the information relate to your research question or topic?
Is the information at an appropriate level (i.e., not too elementary or too advanced)?
Is it an in-depth treatment or a brief overview?
Who is the intended audience?
A - Authority:Who is the author/publisher/source/sponsor?
What are the author's credentials or organizational affiliations?
Are they qualified to write on the topic?
Is there contact information, such as a publisher or email address?
Does the URL reveal anything about the source (e.g., .edu, .gov, .org, .com)?
A - Accuracy:Where does the information come from?
Is the information supported by evidence?
Has the information been reviewed or refereed?
Can you verify any of the information in another source or from personal knowledge?
Does the language or tone seem unbiased and free of emotion?
Are there spelling, grammar, or typographical errors?
P - Purpose:What is the purpose of the information (to inform, teach, sell, entertain, persuade)?
Do the authors/publishers make their intentions clear?
Is the information fact, opinion, or propaganda?
Does the point of view appear objective and impartial?
Are there political, ideological, cultural, religious, institutional, or personal biases?
Quiz: Source Evaluation and Information Literacy
Instructions: Answer each question in 2-3 sentences.

Why is it crucial to evaluate sources even in an age where information is readily available online?
Briefly explain one key difference between how Artificial Intelligence (AI) and search engines provide information.
What is the primary advantage of using academic research databases compared to a general search engine?
Describe one significant way in which AI's training process can lead to inaccurate or unreliable outputs.
How do search engine algorithms influence the "relevance" of links, and why is this not the same as accuracy?
Explain what "pay-to-play" publications are and why researchers should be cautious of them.
Define "confirmation bias" and explain how it can impact a person's perception of news.
According to the CRAAP test, what does "Authority" refer to, and why is it important to consider?
Give an example of "corporate bias" in news media and how it might manifest in reporting.
Why is it more helpful to think about bias in news as a "spectrum" rather than a binary (biased vs. unbiased)?
Quiz Answer Key
Why is it crucial to evaluate sources even in an age where information is readily available online? It is crucial because anyone can publish anything on the internet, leading to widespread misinformation, propaganda, and unverified claims. Evaluating sources ensures the validity, accuracy, and unbiased nature of the information we consume and share.
Briefly explain one key difference between how Artificial Intelligence (AI) and search engines provide information. AI compiles and parrots information from its training data without inherent checks or understanding of relevance, sometimes even fabricating content. Search engines, conversely, use algorithms to rank existing links on the internet, leaving the accuracy verification up to the user.
What is the primary advantage of using academic research databases compared to a general search engine? Academic research databases primarily contain articles and studies that have been peer-reviewed and vetted by experts in their field. This significantly increases the likelihood that the information found is credible, accurate, and supported by rigorous research, unlike the unverified content often found via general search engines.
Describe one significant way in which AI's training process can lead to inaccurate or unreliable outputs. AI is trained on vast amounts of data, which may include non-reputable sources like uncredited social media content. If its training data contains biases or inaccuracies, the AI will simply compile and reproduce these flaws, leading to "hallucinations" or factually incorrect information.
How do search engine algorithms influence the "relevance" of links, and why is this not the same as accuracy? Search engine algorithms determine "relevance" based on factors like keywords, user behavior, click rates, and SEO, which prioritize visibility and engagement. This algorithmic "relevance" is not synonymous with accuracy, as an algorithm might promote clickbait or popular but factually incorrect content over credible sources.
Explain what "pay-to-play" publications are and why researchers should be cautious of them. "Pay-to-play" publications are journals or platforms where authors can pay a fee to have their work published, potentially bypassing rigorous peer review standards. Researchers should be cautious because this model can compromise the quality and credibility of the research, as publication might be prioritized over scientific merit.
Define "confirmation bias" and explain how it can impact a person's perception of news. Confirmation bias is the tendency to seek out, interpret, and favor information that confirms one's existing beliefs or hypotheses. This can cause a person to selectively consume news from sources that align with their views and dismiss information that challenges them, leading to a reinforced, potentially unbalanced, understanding of issues.
According to the CRAAP test, what does "Authority" refer to, and why is it important to consider? "Authority" in the CRAAP test refers to the credibility of the author, publisher, or source, including their qualifications and affiliations. It's important because knowing the author's expertise helps determine if they are knowledgeable enough to provide reliable information on the topic, ensuring the source has legitimate standing.
Give an example of "corporate bias" in news media and how it might manifest in reporting. Corporate bias could manifest if a news outlet owned by a large energy company downplays the environmental impacts of fossil fuels or gives disproportionate airtime to experts who support such industries. This bias serves the financial interests of the parent company, influencing story selection or the tone of coverage.
Why is it more helpful to think about bias in news as a "spectrum" rather than a binary (biased vs. unbiased)? Thinking about bias as a spectrum acknowledges that nothing humans produce can be entirely free of potential bias. It allows for a more nuanced understanding, recognizing varying degrees of bias and encouraging critical assessment of where a source falls on that spectrum, rather than simply dismissing it as "biased" or accepting it as "unbiased."
Essay Format Questions (No Answers Supplied)
Compare and contrast the strengths and weaknesses of using AI, search engines, and academic databases for research, emphasizing the specific evaluation challenges each presents.
Discuss the various forms and types of media bias identified in the source material. Provide examples of how these biases can manifest in news reporting and explain their potential impact on public understanding of complex issues.
Explain the importance of the CRAAP test in evaluating sources. Choose three components of the CRAAP test and elaborate on why each is critical for determining a source's reliability and credibility in today's information landscape.
Analyze the statement, "In this day and age anyone can say anything on the internet, it's no longer that info is checked before shared." How does this statement highlight the necessity of information literacy, and what specific challenges does it pose for consumers of online information?
Reflect on the idea that "everyone has an agenda" and "everyone has internal biases." How do these inherent human traits necessitate a critical approach to all information sources, and what steps can individuals take to mitigate the influence of both external and internal biases in their own research?
Glossary of Key Terms
Accuracy: The quality or state of being correct or precise; a key component of the CRAAP test assessing the factual correctness of a source.
Academic Databases: Specialized collections of scholarly articles, journals, and research papers, often peer-reviewed, used by researchers.
AI (Artificial Intelligence): Systems that simulate human intelligence, in this context, compiling and generating information based on vast training data, but without true understanding or oversight.
Algorithm: A set of rules or instructions followed by a computer, particularly used by search engines to rank and display information.
Authority: A component of the CRAAP test that assesses the credibility and qualifications of the author, publisher, or source.
Bias: A prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair. Can be conscious or unconscious.
"Big Story" Bias: A type of media bias where journalists' judgment is influenced by their perception of an event's major importance.
Confirmation Bias: The tendency to interpret new evidence as confirmation of one's existing beliefs or theories.
Corporate Bias: A type of media bias where the business interests of a news outlet (including advertisers and parent company) influence coverage.
CRAAP Test: An acronym for a method used to evaluate sources based on Currency, Relevance, Authority, Accuracy, and Purpose.
Currency: A component of the CRAAP test that assesses the timeliness of the information.
Demographic Bias: Distortions in news coverage due to factors such as race, gender, ethnicity, culture, or economic class.
Framing: How a news report is approached and organized, influencing audience perception.
Misinformation: False or inaccurate information, especially that which is intended to deceive.
Neutrality Bias: A type of media bias where an attempt to appear unbiased leads to misrepresentation of facts.
Op-eds (Opposite the Editorial Page): Opinion pieces typically written by someone not affiliated with the news organization's editorial board.
Partisan Bias: A type of media bias where journalists' own political views affect news coverage.
"Pay-to-Play" Publications: Journals or platforms where publication is contingent on a fee, potentially compromising peer review and quality.
Peer Review: The evaluation of a scientific, academic, or professional work by others working in the same field.
Propaganda: Information, especially of a biased or misleading nature, used to promote or publicize a particular political cause or point of view.
Purpose: A component of the CRAAP test assessing the reason the information exists (e.g., to inform, persuade, sell).
Ragebaiting/Clickbait: Content designed to provoke an emotional response or entice clicks, often at the expense of accuracy or substance.
Relevance: A component of the CRAAP test assessing the applicability and appropriateness of the information to the research topic.
Research Databases: See Academic Databases.
Search Engine: A program that searches for and identifies items in a database that correspond to keywords or characters specified by the user, used especially for finding particular sites on the World Wide Web.
Search Engine Optimization (SEO): The process of maximizing the number of visitors to a particular website by ensuring that the site appears high on the list of results returned by a search engine.
Sourcing: The selection of voices and perspectives included in a news story.
Story Selection: The decision by a news organization about what is newsworthy enough to cover.
Tone: The general character or attitude of a piece of writing, influencing how information is perceived.
Validity: The quality of being logically or factually sound; soundness or cogency.