Evaluating Information
In this session you will learn how to critically evaluate the information that you use for your assignments and academic work. Evaluation involves, among other things, the ability to decide which information sources are worth pursuing, and determining whether the information that you have found is appropriate for your work.
Why evaluate?
Not all information sources are equally useful or reliable, or likely to lead you to the truth of the matter on particular topics, and a source that may be appropriate in one case may not be suitable in another. To be successful at CDU you need to be able to substantiate your research with acceptable sources. Evaluating also saves time so why look for or read through sources that are not going to be of use to you?
When to evaluate?
You should evaluate continuously, for example, when you first look for information in a catalogue, database or search engine and also when you have the information in your hand or on your screen and are deciding whether it is usable, reliable, in part on in whole.
Tips for evaluation
Evaluating information is very important and needs to be done thoughtfully. Academic and scholarly works should be subject to evaluation, just like other sources. It is even more important to evaluate information found on the web.
Books, journal articles, websites etc can be evaluated to determine their quality to some extent - and with certain limitations - according to some commonly used criteria. Evaluating requires you to ask yourself questions about the information.
Scope
You are looking for information in books, journals and on the Web for a particular reason. The type of information you are seeking must fulfil your purpose and cover what you need to know.
When you find an information source (books, journal article, website) ask yourself these questions:
- What Is the purpose of the information provider? (Is it scholarly or informational?)
- What topics are covered?
- Discipline specific or interdisciplinary?
- How in-depth is the material covered?
- Who is the target audience? (eg. Is the level appropriate for your purpose?)
- For websites are the links provided relevant?
Activity
Scope
The scope of a book, website, etc is the range and extent of matters dealt with - regarding its subject area. To evaluate scope we consider questions like: - What sort of information is covered and on what subjects? At what level; whom is it intended for? Is it brief or detailed?
You have found a catalogue record below while looking for books on a controversial issue in medical ethics
Author: Watt, Helen 1962-
Title: Life and death in health care ethics: a short introduction
Publisher: London, Routledge, 2000
Collation: vii, 97p, 21cm
Judging by this record how far is the scope of the book likely to be appropriate for your search?
Type your answer in the box below
Now have a look at the Choice Magazine Website and look at the About Us Section
Briefly characterise its scope and the main subjects does it deal with?
Timeliness
As with any information, its usefulness depends on whether it is timely. You are probably using the Internet to find current information on your topic.
When you find a web site, ask yourself these questions:
- Are there dates on the site?
- Is the information provided on the page up-to-date?
- When was the site last revised?
- If links are provided, are they active and do they reflect current information?
- Is the information provided on the website the type of information that dates quickly (eg. statistics, technological information, etc)?
Activity
Timeliness
The timeliness of a work means appropriateness to an information need - in so far as the time of writing or publication of the work is concerned. Evaluating timeliness includes looking at:
* What point in time something was written or published, and
* Whether or not the passage of time has affected its value or appropriateness - in general or for your particular information need.
You are wondering how to look at the question of timeliness with regard to the materials relevant to your assignment topic. Just how important is it to find material as recently published as possible? And are books and articles published, say, 20 years ago of little use?
It entirely depends on the topic. For some topics it is essential to take recent developments into consideration, while for others, works written a long time ago may contain as much wisdom as the works of today, or more!
Give examples of subject areas for which date of publication is of determining importance
Type your answer in the box below
and for which it is generally not of determining importance
Type your answer in the box below
Further example: Constitution of Australia was written in 1900 and has changed very little. As such the website is fairly static - http://www.aph.gov.au/senate/general/constitution/
or the basic links to Australian Government do not change frequently http://www.gov.au/
Authority
Authority involves authorship, the question being how much weight should be given to the works of a particular author (or organisation) on a particular subject? In other words, how authoritative is the work, in whole or in its parts. Some factors to consider are:
- Reputation
- Achievements
- Qualifications
- Expertise in particular areas
- Affiliations
- Positions held
- Previous publications, etc.
Activity
Authority
Authority involves authorship, the question being: how much weight should be given to the works of a particular author on a particular subject? (Or, of a group or organization, if it isn't an individual author.) In other words, how authoritative is the work, in whole and in its parts? Some of the factors to consider are: reputation, achievements, qualifications, expertise in particular areas, affiliations, position held, previous publications, etc.
You have read two journal articles on a particular social issue with legal and political implications. At the end of both articles, you find brief information about the respective authors, along the lines of:
...Jean Bethke Elshtain is Professor of Social and Political Ethics at the University of Chicago. She has an impressive list of publications, including...
...Robert P. George is Professor of Jurisprudence at Princeton University. His latest book is In Defence of Natural Law (Oxford University Press).
Which of the two could possibly be an authoritative source on your topic? (To know for certain, you would need to know more about them, as what is listed above doesn't actually guarantee authoritiveness.)
Type your answer in the box below
Usually you can find some similar information about the authors of books too, on book covers, in prefaces, etc. In general it should be understood, though, that appearing authoritative is not yet the same thing as actually being accurate in a work, another factor when evaluating information.
Further Example
Read the first 2 paragraphs of the following article by David A Pentice's Adult stem cells
Scroll to the end of the article and note the credentials of the author. Also note the original publisher of the article.
Now go to website www.stemcellresearch.org
Note the sections of the homepage you can find David A Prentice’s name
As you can see this site is of reasonably high reliability. David A. Prentice is part of a cross disciplinary group of ethicists, doctors and scientists who, as a coalition, take on more than just specialist expertise.
In your own assignment you will need to evaluate websites of a professional association in your own area of study or interest. Authority will be something you need to take into account when assessing these sites.
Accuracy and point of view
The quality of information found is often directly related to who is putting it up on the Web. Consider whether you would ask a vet for advice about a toothache or would you prefer a dentist's opinion?
When you find a web site, ask yourself these questions:
- Who wrote the Web page?
- What are their qualifications and affiliations?
- Are they an expert in their field?
- Is there a link to the author's home page or curriculum vitae?
- Is the information reliable? (eg. Is it controversial, misleading etc.?)
It is important to be aware of possible bias that can compromise the accuracy of the information provided by the web site.
When you find a web site, ask yourself these questions:
- Is there any bias on the page (eg. racial, political etc.)?
- Is there a commercial sponsor who is using the page to promote their product over others?
- Which country/region does the information cover?
Activity
Accuracy and Point of View
The accuracy and truthfulness of a work is the most important attribute of all. Often it is also the most difficult to determine. Jumping to conclusions on little evidence is not the way to go, but mature consideration and good reasons can support definite conclusions regarding accuracy. In some cases reserving your judgment might be needed.
You are trying to estimate the accuracy of the contents of a documentary on video relevant to your essay topic. Much of the documentary is taken up by a well-known personality, with proper credentials, expounding his views. You have heard some of those views already as the personality in question is a favourite of much of the media and is often interviewed or featured by them. In the video he certainly puts his views persuasively, and the accompanying footage appears to support him well; but you are a bit surprised by the fact that only a minimum, token space is given to views opposed to his. In the end you wonder if you should, or should not, examine the accuracy of the video more closely.
Type your answer in the box below
There are some simple questions you can ask yourself to help identify whether the information is accurate or not. Such questions include:
- are statements supported, especially with references or footnotes?
- are the sources of all statistics given?
- is it peer-reviewed?
- are all the academic qualifications of the author listed, etc?
Useful as such points are in helping to estimate certain aspects of accuracy, they might oversimplify the issues and fail to make proper distinctions. Such questions need to be considered thoughtfully and not applied automatically.
For example, consider the following:
You are trying to estimate and compare the value of two particular articles you have found on your topic.
Article A is by an author known by you for the general soundness and reliability of her work, previous publications, affiliations, etc. The particular article here is in a current affairs type of publication aimed at the general public, where the author seriously but informally summarises her understanding of, and insights into, the topic. There are no references or statistical tables given.
Article B is in a formally academic publication, where a different author, whose reputation is not well known by Luis, follows all rules and conventions of academic argumentation and referencing. Several statistical tables are provided.
Can you just assume that Article B is the more valuable and accurate of the two?.
Type your answer in the box below
Point of view
What especially complicates evaluating for accuracy, is the question of point of view or "bias".
What constitutes "bias" and in what sense, and how it all effects accuracy, is a difficult question. We should certainly try to recognize support for a particular side of a controversy, for instance, by the author(s) of a work whether it's plainly or subtly expressed. Now this may - or may not - effect the accuracy or truthfulness of the material. Objectivity in a work does not mean neutrality at all costs, or "fence-sitting".
Often a particular stance is supported in a book, article or website in the tacit understanding that there are plenty of available materials supporting the other side, too, elsewhere, for anyone to check. Therefore, when evaluating, for instance a website of that kind, it would not be realistic to fault the authors merely because they do not rehash the arguments of the other side as well for the sake of "balance". The truth of the matter is what really matters.
The discussion above identifies some very important features of evaluating information:
* It must be done carefully, in a mature way, considering the circumstances and various evaluation criteria together, with a good deal of common sense.
* Recognising the limitations of the relevance and applicability of some specific criteria is important, too.
Further Example
In the article you read in the section on authority by David A. Prentice he cites over 192 scientific references. Then on the website he has a powerpoint of his original report, but cites no references. Do you think this makes the information less accurate?
Again on the website www.stemcellresearch.org click on the founding statement and read the first paragraph there. What is the coalition's point of view on stem cell research?
Summary
The Evaluating Information module in InfoSmart discusses the uses of some basic, initial criteria.
- Scope
- Timeliness
- Authority
- Accuracy and point of view
Be aware, however, that these criteria are merely a starting point. You will still need to use your knowledge of a particular area, as well as your basic ability to think logically when evaluating an information source. For example, timeliness might be more relevant in some areas than others; there are many permanent principles or concepts in various fields, such as philosophy, that do not change with time.
The information below highlights some frequent pitfalls to be aware of when evaluating websites.
Evaluating websites - Some common pitfalls to avoid
Evaluation criteria need to be applied with a great deal of common sense. There are several pitfalls to avoid. For example:
Faulty evaluation (at least in some cases):
“Some of the writings on the site are several years old. In a rapidly changing area, this makes them out of date…”
Comment: Even in a rapidly changing area, that’s not necessarily true. For example, there can be underlying principles that don’t change with time.
Faulty evaluation:
“The authors are highly qualified experts, academics, etc. That by itself is sufficient reason to assume that what they write is accurate…”
Comment: By itself, it is not sufficient. For example, being a Nobel laureate in science doesn’t make one an authority on ethics, philosophy or public affairs. Another point to note: on many issues experts can, and do, disagree with one another – so on a particular issue some of them can be right, and others can be wrong.
Faulty evaluation (at least in some cases):
“The site contains only subjective opinion, there are no statistics or measurements provided to back it up…”
Comment: Statistics, measurements and the scientific method are only relevant to the quantitative aspects of reality – and even there they are often used and quoted in a misleading way. In other aspects of reality, such as the ethical, it is good principles and good reasoning rather than numbers that lead to the truth of the matter.
Faulty evaluation:
“The site is not balanced. It doesn’t contain any views opposed to its own position. So it cannot be really accurate…”
Comment: That does not necessarily follow at all. On most of the controversial issues there are different websites arguing for different sides of the argument – and it’s understood people are free to check out opposing websites, if they wish. So the authors of a particular website should not be faulted merely for not sitting on the fence.
The truth of the matter is what really matters – not whether the presentation is “balanced” or not.
Activity
The Australian Dental Association Website is an example of good website for a professional association. Take a little time to evaluate this site in terms of scope, timeliness, authority and accuracy and point of view.
Evaluation of websites does not have to be undertaken to this extent for each site but if you keep these questions in mind when reviewing a site it should help you to eliminate sites that are not relevant to your academic study.