Program / Course 23: Research Methodology / Lesson 2.4
What you will learn
- Apply a short set of questions about authority, evidence, purpose and currency to any source.
- Tell scholarly, popular, devotional, polemical and commercial writing apart.
- Evaluate a website using signs that can be checked, not appearance alone.
- Use a trustworthy source with care and recognize when to set one aside.
The big idea
Reliability is not a yes-or-no label on a source. It is a judgment about how far a particular source can be trusted for a particular claim. Useful questions concern the author’s expertise, the evidence offered, the purpose of the work, who published it, how recent it is, and whether other sources agree. A polished website can be unreliable, and an unattractive page can be excellent. The skill is to look past appearance to features that can be checked.
Key terms
- Reliability
- The degree to which a source can be trusted to give accurate information on a given point.
- Authority
- The expertise and standing of a writer on the subject, shown by training, publications and the quality of past work.
- Scholarly work
- Writing by specialists for other specialists, with documented evidence, usually reviewed before publication.
- Popular work
- Writing for general readers, which may be accurate and well researched but usually has fewer notes and simplifies.
- Devotional work
- Writing meant to nourish faith and prayer. It is valuable for that purpose but is not meant as historical evidence.
- Polemic
- Writing meant mainly to attack an opponent or defend a position. It may contain truth, but it selects and shapes evidence for its aim.
- Currency
- How up to date a source is, and whether later discoveries or scholarship have changed the picture.
- Corroboration
- Independent support for a claim from other reliable sources.
Background
Everyone must trust sources they cannot check themselves. The task is to place trust wisely. Scripture itself urges testing: the Bereans searched the Scriptures daily to see whether what they heard was so (Acts 17:11), and John tells his readers to test the spirits (1 John 4:1). In research the same prudence applies to books, articles and websites.
The internet has made information abundant and has made judgment more necessary. Anyone can publish with a professional appearance. Pages that rank high in a search are not thereby accurate, and some websites exist to sell something, to attract clicks or to promote a cause. At the same time there is excellent material online, including digitized primary sources, free scholarly journals, and encyclopedias written by specialists. The question is never whether a source is online or in print. It is what kind of source it is and how it fits your claim.
Kinds of writing differ in how much they can bear. A peer-reviewed article or a monograph from a university press is meant to withstand expert checking. A popular book by a good scholar can be accurate while simplifying. A sermon or devotional book aims to build up faith, not to settle a historical question. A polemical tract aims to win. A website selling a product aims to sell. None of these are dishonest by nature, but each must be read according to its purpose.
Step-by-step explanation
Step 1: Check the author
Who is the author, and what qualifies him or her to write on this subject? Look for training, a position at a university or seminary, other publications, and the way other scholars refer to the work. Unsigned articles and anonymous pages give you nothing to check.
Expertise is specific. A distinguished scientist or a famous preacher may not be an expert on the Council of Chalcedon. Equally, an unknown writer may know a great deal. Judge by the work.
Step 2: Check the evidence
Does the source name its evidence? Does it give notes, a bibliography or references that you can follow? Do the references actually say what the source claims? Spot-check two or three by looking them up.
A source that makes large claims with no citations asks for blind trust. A source that quotes selectively, or that reports one side only, deserves extra care.
Step 3: Check the purpose and the publisher
Why does this source exist? To inform, to persuade, to entertain, to sell, to edify, to recruit? Who published it: a university press, a church body, a commercial publisher, a political group, a person with a blog? Publishers are not neutral, and their aims shape what they print.
Confessional sources, which come from a particular church or tradition, can be well researched and still shaped by commitments. That applies to Catholic, Orthodox, Protestant and secular sources alike.
Step 4: Check currency and corroboration
Is the source recent enough? In some fields, new manuscripts, editions and studies change the picture. An old book can still be a classic, but you should check whether later scholarship has revised it.
Does the claim appear in other reliable sources, written independently? A claim found only in one popular book, repeated by many websites that copy each other, is not corroborated. Trace repeated claims back to the beginning.
Step 5: Judge it for this use, and record your judgment
A source can be reliable for one purpose and not for another. A devotional book is good evidence of how a community prays, though not of what happened in the fourth century. A polemical work is good evidence of what its author argued. Note on your source sheet how far you trust each source and why.
When in doubt, find a better source. If a claim is important, it should rest on the best evidence you can find.
- Who wrote this, and how do I know they know the subject?
- What evidence is given, and can I follow it to its source?
- Why was this written, and who paid for or published it?
- How recent is it, and has later work changed the picture?
- Do independent reliable sources agree?
- For what purpose am I using it, and does it suit that purpose?
Examples
A scholarly article
An article in a journal published by a university or learned society, with notes, a bibliography and a named author at an institution. It has been reviewed before publication. It is a strong source, though still open to critical reading.
A popular book
A well-written general history by a trained historian that gives few footnotes. It may be accurate, but you should find the scholarly sources it relies on before using it for a detail.
A website with no author
A page that claims ‘the early church taught X’ with no named author, no references and an advertisement for a product. A reader has nothing to check, so the page cannot be used as evidence. It might, at most, suggest a question to research elsewhere. (A constructed example.)
A confessional source
A catechism or a denominational history is a reliable source for what that church teaches or says about itself. It is less reliable as a neutral account of disputes with other churches. A scholar from another tradition may supply a corrective.
Different perspectives
Catholic
Catholic institutions publish serious scholarship, and the Church also issues authoritative teaching documents. A Catholic researcher distinguishes the two: a papal encyclical is authoritative for what the Church teaches, but is a different kind of source from a historian’s study of the same subject. Both can be used for their proper purposes.
Orthodox
Orthodox sources include scholarly work, the writings of the Fathers, liturgical texts and popular and devotional works. Orthodox readers often take care to read popular summaries against the patristic and liturgical sources they rely on, since simplified accounts can distort the tradition.
Protestant
Protestant writers include scholars at universities and seminaries and a very large popular and online literature. Some Protestant traditions place great trust in the pastor and the commentary, and others in the individual reader’s study. Both groups are advised to check popular claims against scholarly sources, especially about church history.
Academic and library professionals
Librarians and scholars teach students to evaluate sources by questions such as authority, accuracy, purpose, currency and coverage. They emphasize lateral reading: instead of staying on a page, open other tabs and see what independent sources say about the page and its author. This is a quick and effective way to test an unfamiliar website.
Why it matters
The quality of a paper cannot be higher than the quality of its sources. A student who learns to judge sources saves the paper from embarrassing errors, and also gains protection from misinformation in daily life. For believers, careful judgment of sources is part of loving the truth and of the call to test what is heard (1 Thessalonians 5:21).
Common misunderstandings
Misunderstanding: If a page ranks high in a search, it is reliable.
Correction: Ranking reflects many things, including popularity and advertising. It does not measure accuracy. Always check the author, evidence and purpose.
Misunderstanding: Only print sources can be trusted.
Correction: Reliability depends on the source, not on the medium. Many printed books are poor, and many online sources, such as digitized manuscripts and journals, are excellent.
Misunderstanding: A source from my own tradition is automatically reliable, or one from another tradition automatically biased.
Correction: Every tradition produces strong and weak work, and every source has commitments. Apply the same questions to all.
Misunderstanding: A peer-reviewed source must be correct.
Correction: Peer review is a good sign of quality, not a guarantee. The next lesson explains what it does and does not do.
Knowledge check
What are five questions to ask of any source?
Who is the author and what is their expertise? What evidence is given, and can it be followed? Why was it written and who published it? How recent is it? Do other independent sources agree? Others include whether the source has been reviewed and how it treats opponents. Any five of these are correct.
What is lateral reading?
Lateral reading means leaving a page to see what other sources say about it and its author, instead of judging from the page alone. You open a new tab, search for the author or organization, and look at independent descriptions. It quickly shows whether a site is trusted, commercial or a front for a cause.
Why is a polemical work not always useless as a source?
Polemic is written to attack or defend, so it selects and shapes evidence. But it is good evidence of what its author argued and what the dispute was about, and it often quotes or describes opponents’ views that are otherwise lost. The researcher uses it for those purposes and checks its claims about opponents against other sources.
Why does repetition across many websites not prove a claim?
Websites often copy each other. Many pages repeating one claim may all depend on a single source, which may be wrong. A claim is corroborated only when independent reliable sources support it. Trace the claim back to where it began and judge that source.
Think it through
- Think of a claim about church history that you believe. Where did you first meet it, and what would it take to check it?
- Which kinds of sources do you trust too easily, and which do you distrust too quickly?
- How do 1 John 4:1 and Acts 17:11 fit with trusting a pastor or a teacher?
- What is the difference between a source being biased and a source being unreliable?
Independent assignment
Evaluating three sources
In your own notebook, find three sources on the same church history question: one scholarly, one popular and one website. Write about 400 words in total evaluating them by the five steps of the lesson. Say how far you would trust each and for what purpose. Check one claim from the website against a scholarly source and report what you found. Do not send this to anyone.
Further study
- Stanford Encyclopedia of Philosophy, Fallacies Read the sections on appeals to authority and on genetic fallacies, two common errors in judging sources.
- F. F. Bruce, The New Testament Documents: Are They Reliable? Read critically as an apologetic work. Compare its date and purpose with more recent scholarship.
- Richard Bauckham, Jesus and the Eyewitnesses Read selectively and judge it by the criteria of this lesson.
- Justo L. Gonzalez, The Story of Christianity A well-regarded survey. Note how it lists further reading.
- 1 John 4:1 Test the spirits.
- Acts 17:11 The Bereans examined the Scriptures.
- Proverbs 14:15 The prudent consider their steps.