Ewa Dobrogowska-Schlebusch, Barbara Niedźwiedzka

Department of Information Studies. Institute of Public Health, Jagiellonian

University, Kraków. Poland

Nearly every second person

participating in the study was

shown to have limited

(inadequate or problematic

health literacy)

It is estimated, that almost

40% of Polish citizens have

poor health literacy level

To create evidence-based,

comprehensive, easy to find

source of health information

for Polish citizens

Department of Information Studies of

the Institute of Public Health

(Jagiellonian University, Kraków) is

currently conducting the study:

„Assessment of the quality of online

health resources in order to identify

the examples of best practises in

creating portals for patients”.

Main aim of the study:

To identify health information websites

of best quality created for patients/

healthcare consumers which could

serve as models for future Polish

website(s)

How to evaluate existing

portals to find the best ones?

How many evaluation tools

exist? Which is the best?

Something that an Internet

user could use to assess the

quality of website containing

health information (Breckons et

al. 2008)

A literature review done prior to conducting

the research showed that although many

instruments used to rate the quality of health

information websites had been developed (in

2005 Bernstam et al. Identified 273

evaluation instruments) none of these seems

to have been validated

The question was: should we

validate one of the existing tools

or is it better to develop on their

base a new one, and validate it?

Source: pixbay.com

We decided to to the last step…..

What is actually the

aim of this

presentation?

Source: http://openclipart.org

source: frickr.com

To show the preliminary phase of

the project:

The development and validation of a

new tool for evaluation of health

information websites

(Narzędzie Ewaluacji Stron

Internetowych o Zdrowiu – NESIOZ).

source: http://pixabay.com

* I phase of the study: to compile set of quality

criteria most commonly used in instruments

developed to assess the quality of health

information websites

* II phase of the study: pilot test of NESIOZ,

version I with a group of 31 public health

students

* III phase of the study: pilot test of NESIOZ,

version II with a group of 22 dietetics students

* IV phase of the study: pilot test of NESIOZ,

version III with a group of 14 dietetics students

and a group of experts

Systematic review of available health information websites’ quality

assessment instruments was performed (PubMed Medline,

Scopus, Embase, ISI Web of Science, Academic Search Premiere)

from January 2008 through August 2013

Instruments were eligible for review if they had been used at least

3 times in the research in the past five years to evaluate the

quality of health information websites with their rating criteria

available

Each rating instrument included to the study was examined on

quality criteria applied

Most frequently used quality criteria were extracted, analyzed and

later rewritten as questions

I version of NESIOZ tool was created

A total of 132 studies published in the last five

years, were found where the quality of health

information websites was assessed with quality

rating instrument

8 instruments were found, which had been used at

least 3 times to evaluate the quality of health

information website in the last 5 years

Rating instruments: How many times it was used to assess

the quality of websites in the last 5

years?

1. DISCERN 92

2. HON Code of Conduct 31

3. JAMA benchmark 23

4. LIDA tool (Minervation Validation

Instrument for Health Care Web Sites)

22

5. Khazaal scale 8

6. University of Michigan Website Evaluation

Checklist

5

7. Sandvik Score (general griteria) 4

8. EQIP 4

DISCERN

–a questionnaire developed

by Charnock et al. (1999)

which provides users with a

valid and reliable way of

assessing the quality of

written information on

treatment choices for a

health problem.

Currency of information (the original posting date is given; date of last update, frequency of update).

Attribution and documentation (presentation of clear references, details about additional sources of information, balanced/unbiased evidence, evidence-based

Authority of source (authorship, credentials/ qualifications of authors)

Disclosure of developers, sponsors (ownership): (identification of purpose, funding source/ sponsorship, advertising policy)

Contact addresses or feedback mechanism (availability of contact information/ possibilies of sending queries to a postmaster, availability of support/ help section, interactivity/ chat/ feedback forms/ supporting bodies)

Appropriateness for intended users

Quality/ relevancy of links

Design and aestetic

Ease of use (including navigation)

Accessibility and availability

Source: http://pixabay.com/

Content of site (currency of information, attribution and

documentation)

Disclosure (authority of source, funding source including

advertising policy)

Design and aestetics (layout, navigation)

Availability and accessibility (fee for access/ registration,

interactivity, disclaimer)

User support (availability of contact information, site map)

Appropriateness for intended audience

The NESIOZ tool, version I

was pilot tested with a

group of 31 public health

students (Jagiellonian

University – Medical

College) in order to

determine:

the usability of the tool

the clarity of its question

to measure how much time is

needed to asses the webpage

with a tool

http://portal.abczdrowie.pl

Students taking part in pilot test mostly expressed positive opinions about utility of the tool

Pilot test group reported it took about 15 minutes to complete the evaluation process

The largest differences in students’ evaluation emerged in regard to questions pertaining to: the date of last update, authority of source, disclosure of financial support, advertising policy and ease of use

These questions after pilot test I

were rewritten in order to increase

their clarity.

The NESIOZ tool, version II

was tested with a group of 22

Dietetics students to further

determine the clarity of its

questions and instructions.

The students were required

to evaluate the quality of 5

selected nutrition websites:

www.apetyt-na-zdrowie.pl,

www.eufic.org, www.mojacukrzyca.org,

www.dieta.mp.pl,

www.sztukaodzywiania.org

The greatest discrepancies between

students’ evaluations were found in

questions pertaining to design and

web site aesthetics (Is the website

layout clear?), ease of use (Is the

website easy to navigate?) and

advertising policy (Is advertising

distinct from content?)

terms and condition of website

Main changes:

* One question (concerning terms & conditions of website use)

was deleted from a tool

* Some revisions were made in order to further increase the clarity of questions (e.g.

guidelines/ principles of good website layout and navigation techniques were

provided)

NESIOZ tool, version III is currently being tested with

a group of students and experts (medical librarians

or information science scientists) to establish its

reliability and face validity.

Especially experts’ opinion will be taken into

consideration to further improve the utility of the tool

It is impossible to achieve consensus over website’s quality assessment as assessing some aspects of website (the aesthetic visual quality of website, ease of navigation, advertising distinct from content) is a highly subjective task and determined by personal characteristic of the evaluator.

Despite the presence of relatively precise definitions some criteria remained difficult for evaluators to assess (Sagaram et al. 2004)

The criteria used must be unambiguous so that the different people applying them will agree upon their usage (Sagaram, et al.)

The problem is probably to large extent not the tool itself, but the websites which were the subject of assessment in the pilot tests

The assessment of health information website even with a very short and precise evaluation tool might pose a problem for the untrained user

Thank you very

much for your

attention!

Source: http://en.wikipedia.org/wiki/Bowing