Learning outcomes:
- Explain the differences between AI, search engines, and academic databases as information sources
- Evaluate a source using the CRAAP test criteria and identify potential bias
- Compare news coverage of the same topic across different sources and describe how bias affects the information presented
Key Takeaways:
- Evaluate sources using the CRAAP criteria: Currency, Relevance, Authority, Accuracy, and Purpose
- Appearing in a search result, or being stated by an AI, does not make information true or reliable
- Bias is not always obvious; understanding who funded a study, who wrote it, and who the intended audience is helps you see it
- Primary sources, peer-reviewed journals, and academic databases are held to higher verification standards than general web content
- Comparing how different sources cover the same topic is one of the most effective ways to identify bias and missing perspectives
Would you like to download my PowerPoint to follow along?
- Why we evaluate sources
- Validity of information
- Bias of information
- Making sure we aren't sharing misinformation
- Checking that the conclusions we have are accurate
- Making sure that the data supports the conclusions shared
- Being careful to not fall under propaganda
- Example: Climate change
- In this day and age anyone can say anything on the internet, it's no longer that info is checked before shared
- There is a huge issue with AI giving information that is junk and inaccurate so we need to find where the info is coming from and make sure it's been checked by someone knowledgeable in that arena
- Why it's important to use good sources
- Making sure that the data has been analyzed properly by someone that knows what they are doing
- Making sure that the data and conclusions were verified by more than one person
- Making sure the source is reputable not just AI driven spam or a company trying to get money
- For example: some companies publish information that is inaccurate just to sell their products
- Some journals and publications are pay to play
- Making sure the work was done by experts in the field, not just random keyboard warriors
- Harvard on why sources matter
- The difference between AI, Search Engines and research databases
- AI
- It's only as good as it's sources
- It has no checks or oversight, only can compile what was put in
- Doesn't understand "relevance" only can parrot back info
- Rutgers university on AI and research includes a breakdown some of some AI options and how they could be used
- Search Engines
- Use an algorithm to decide which links to give you
- No checks on accuracy of links or info, you have to decide if you think the link/writer is trustworthy
- Santa Clara University Bias in Search Engines And Algorithms
- Research Databases
- Should only have articles that have been checked and peer reviewed
- Have to be careful of Pay to Play publications
- Example of databases available at NECC
- Brown University on search engines vs academic databases for research
- AI sources and how AI gets info
- AI technically doesn't learn, it's trained by looking through hundreds and thousands of pieces of information during the training phase
- Training is done by giving it info from anything the creators can get their hands on, not always reputable (such as images from social media and art from artists without asking)
- The results are just compilation of info and patterns recognized
- Example: Lawyer used AI for their arguments and cited fake court cases
- When AI Gets It Wrong: Addressing AI Hallucinations and Bias
- Search Engines and how they give links
- Search engines go through what's available on the internet and then adds it to its internal list and uses an algorithm to decide what it thinks are the most important
- Algorithms have changed over the years, from the simple keyword searches of the 90s, to the more tailored versions of today that take user behavior into account
- Search engines are also incorporating more and more AI into both the algorithms and what it shows on page 1, most people don't go past page 1
- SEO (Search Engine Optimization) affects the order things come up in, and it changes all the time, it's not based on accuracy of data mostly it's based on how many links link there, or how often people click the link for that search
- Search engine algorithms are what decide what's "relevant" or not, but that's not the same thing as accurate.
- The 'bias machine': How Google tells you what you want to hear
- Example: Ragebaiting or click bait
- Academic databases and news sources
- Should be peer reviewed and checked, not just anyone can post
- Community should have already decided if the source and data is reputable
- Knowing how to find the info that's relevant is more important, the knowledge has been checked, but it can be hard to find what you want
- Should I use library databases to find research instead of Google? Includes Pros and cons
- New source bias
- 35 Media Bias Examples for Students
- Media Literacy Guide: How to Detect Bias in News Media
- Is social media biased or balanced?
- In brief: News media bias and also there is an Infographic
- Bias and why it matters in sources
- Everyone has an agenda, some are more obvious than others
- Everyone has internal biases, even if we don't think we do, we do
- Bias can affect everything from where the information is shared to even the words used for sharing
- An unfortunate example of this is the passive voice in journalism, and how journalists tend to talk about victims and children
- Bias is also an issue when it's a hot topic, because the larger the stakes the more people are inclined to lie or misconstrue results
- There are a lot of types of bias, such as confirmation bias, demographic bias, and distance bias
- Examples:
- How to evaluate sources
- CRAAP Test
- C - Currency
- R - Relevance
- A - Authority
- A - Accuracy
- P - Purpose
- Designed by Sarah Blakeslee and their team Original Publication
- Evaluating Resources and Misinformation using the CRAAP test includes questions to ask about each source to make sure you're evaluating it thoroughly
Suggested Activities and Discussion Topics:
- Activity: Try out a CRAAP test on a news article you've found. Make sure to try finding a news article through a search engine, an AI, and an academic database. Where the CRAAP tests different? Easier to do with one? Different results between them? This should take about 20–30 minutes.
- Activity: Listen to This Podcast That was created using AI from these materials. Transcript for the Podcast What are your thoughts? Did the AI do a good job representing the materials? Did you find any mistakes? This should take about 20–25 minutes.
- Activity: Go through This AI generated study guide, what do you think? Did it capture the week materials well? How did you do on the self quiz? Do you know all the vocab used?
- Activity: Go through this AI-generated key terms list for this module. How many of the terms did you already know? Were there any definitions you would update or expand based on the lecture notes? Did the AI miss any important terms? This should take about 10–15 minutes.
- Activity: Take a quiz on internal bias such as this one from harvard on implicit associations What did you find? Where you surprised? Is it different then the people around you? This should take about 15–20 minutes.
- Activity: Watch This AI generated video the Transcript for the Video is here and answer the following questions. This should take about 20–30 minutes.
- Did the AI accurately represent the original materials?
- Did you find any mistakes, inaccuracies, or "hallucinations"?
- Did it capture the essence of the material well, or did it miss nuances?
- Do you think anything was missed from the original content?
- Discussion: Pick an AI such as Gemini or ChatGPT, and request that it generate some content for you based on this material. This could be a summary, a short informational paragraph or an infographic.Now you're going to give it a review based on the following questions:
- Did the AI accurately represent the original materials?
- Did you find any mistakes, inaccuracies, or "hallucinations"?
- Did it capture the essence of the material well, or did it miss nuances?
- What does this exercise reveal about the limitations of AI as a primary information source?
- Discussion: Find two news articles on the same controversial or "hot topic" from different news outlets.Try to choose sources that might have different perspectives. Answer the following questions
- What forms of bias did you see? For example, were there issues with absence of fairness and balance, framing, sourcing, story selection,or tone?
- From the article 35 Media Bias Examples for Students, find at least 2 kinds of bias, name them and explain why you drew the conclusions you did, some examples could be "Big story" bias, Corporate bias, Demographic bias, Neutrality bias, Partisan bias, or Confirmation bias.
- What internal bias do you have that might contribute to your options?
- Activity: Choose an AI tool of your choice such as ChatGPT, Gemini, or Claude. Find a recent news headline that makes a bold claim and paste it into the AI, then ask the AI to evaluate the claim using the CRAAP test criteria (Currency, Relevance, Authority, Accuracy, Purpose). Read its evaluation carefully and compare it to your own quick evaluation of the same claim. Answer the following: Did the AI catch any issues you missed? Did it miss anything you noticed? Did it hallucinate any sources or facts? What does this tell you about using AI as a source evaluation tool? This should take about 5–10 minutes.
- Activity: Read this thorough CRAAP test checklist and this poorly completed CRAAP test checklist, both evaluating sources on the same general topic. Compare and contrast the two by answering the following questions:
- Which specific criteria does the bad example fail to actually evaluate, even though it marks them as passing?
- What red flags about the source does the bad checklist ignore or dismiss? List at least three.
- What does the good checklist do that the bad one does not? Think about things like verification, depth of reasoning, and acknowledging limitations.
- If you were a reader relying on the bad checklist, what false conclusions might you draw about that source?
- Why do you think someone might rush through a CRAAP test and still feel confident they did it correctly?
- Activity: Using the same poorly completed CRAAP test checklist, rewrite it as a thorough and honest evaluation. You do not need to find the actual website; use the information already in the checklist and think critically about what it reveals. For each of the five criteria, write at least two to three sentences that genuinely evaluate the source rather than just accepting it. When you are done, compare your rewritten version to the good example checklist: What did you catch that the original missed? Did you end up with a different overall rating? What does this tell you about the difference between completing a checklist and actually evaluating a source? This should take about 20–30 minutes.
- Activity: Both the good checklist and the bad checklist reference specific sources. Try to locate each one independently using the information provided in the checklists. For the academic article, search JSTOR or another database available through your library. For the company website, try a direct search. Document everything you find or do not find, and answer the following questions:
- Were you able to locate both sources? If not, what specifically was missing or did not match when you searched?
- What does it mean for a CRAAP evaluation if the source itself cannot be verified or located?
- If a source cannot be found, does that automatically mean the information in it is wrong? What other explanations might there be?
- How does this exercise change how you think about using a completed checklist someone else wrote as evidence that a source is good?
- What step would you add to any source evaluation process based on what you found, or did not find, in this activity?
Related modules you might find useful:
Would you like to see some more How tos? See more How tos