Page 1

Development of User Warrant Ontology for Improving Online Health Information Provision Arya Prakasa & Dr. Daswin De Silva Centre for Organisational and Social Informatics, Monash University, Australia

Overview • • • • • • • • • • • • • ACIS 2013

Definitions Research Context Case Study Research Question Research Plan Research Design Challenges Foundation Theory Research Design Implementation Evaluation Significance Conclusions 2

Definitions Ontology “Ontologies provide controlled, consistent vocabularies to describe concepts and relationships, thereby enabling knowledge sharing� (Gruber, 1993) Concept of Warrant Fundamental principles that guide the process of selecting and expressing terminology (Wallace, 2007) User Warrant User warrants relies primarily on the language of people who are expected to use the system, which include both people who construct the systems and people who use the controlled vocabulary (Wallace, 2007). Literary Warrant Method of constructing and capturing ontology terms solely based on the text existing in the literatures. principle of literary warrant relies heavily on the use of terminology and concepts captured in the documents (Miwa and Kando, 2007). ACIS 2013


Research Context Online Health Care Information •Health care information is one of the most browsed materials on the web •62% of Australians have used the internet to search for health information, 50% have used that information to diagnose themselves

 Quality of the information  Accuracy  Completeness  Information update  User’s information searching skill (Hopkins and Fogg, 2002; van den Haak and van Hooijdonk, 2010; Bundorf et al., 2006)

(Healthdirect Australia, 2010)

ACIS 2013


Research Context (2) Health Information Portal (HIP) Internet based system which aims to assist users to find relevant internet based health information (Fisher et al., 2009). Common Features: catalogue of health information, search engine, personalization system, and network of communities (Luo and Najdawi, 2004).

ACIS 2013


Case Study Breast Cancer Knowledge Online (BCKOnline) An Australian user-centred personalised HIP for breast cancer. Hybrid system of controlled vocabularies which combines the use of:  Medical Subject Heading (MeSH)  BreastCare Victoria Glossary (BVG)  BCKO Glossary ACIS 2013

Figure 1. Domain expert dashboard


Research Problems Recent study on BCKO usage data analysis highlights several portal problems, some of the most important problems are: • misclassification/miss-indexing of information content • user searching behaviour (short and simple strategies and hard low persistence) (Nguyen et al., 2011)

The study recommends to accommodate user generated terms and common query fault for indexing The study reveals a potential of developing ontology based on the combination of existing vocabularies and user generated terms to overcome the mismatch between users and systems terminology

ACIS 2013


Research Question

“How to integrate the existing controlled vocabularies and user queries to construct ontology based on users warrants principles?�

ACIS 2013


Research Plan User Warrant

Standard Vocabulary

BCKOnline BCKOnline Controlled Controlled Vocabulary Vocabulary


Other Authority

User UserTerms Termsfrom from data datausage usage Proposed Proposed Ontology Ontology

Subject/Glossary Subject/Glossary Resources Resources

Breast BreastCare Care Victoria Victoria Glossary Glossary(BVG) (BVG)

Figure 2 Personalize search on BCKOnline portal

ACIS 2013


Challenges • •

• • • •

Breast Cancer Knowledge: Terms in BCKOnline is specific to a certain domain of knowledge. Need to gain sufficient knowledge in breast cancer. Incompleteness: Terms in BCKOnline does not actually complement each other. There is a potential need to introduce new terms to transform the existing terms into ontology. Duplication: There is a need to clean the data due to duplication of terms found in each vocabulary. User Terms: Huge amount of User-terms data on the portal, need some method to filter and select the terms. Validation: There is a need to validate the selection of terms and its relationships to subject expert. Evaluation: There is a need to evaluate the ontology to the user of the system.

ACIS 2013


Foundation Theory ••Health HealthInformatics Informatics ••Information InformationRetrieval Retrieval ••Health HealthInformation InformationPortal Portal ••Quality QualityAssured AssuredHIP HIP ••User UserSensitive SensitiveHIP HIP ••Breast BreastCancer CancerTerms Terms

IEEE 1074-2006 standard ACIS 2013

••Ontology OntologyEngineering Engineering ••Ontology OntologyDefinitions Definitions ••Ontology OntologyProjects Projects ••Ontology OntologyDevelopments Developments ••Ontology OntologyApplications Applications ••Ontology OntologyLanguage Language ••Ontology OntologyEvaluation Evaluation

Figure 3 Foundation Theories


Foundation Theory (2)

Figure 4 User Warrant Elements ACIS 2013


Research Design Phase 1

Identify the purpose and requirements specification Define concepts and concepts relations

User Terms from search engine query log


ACIS 2013

Adopt Existing Terms / Controlled vocabularies

Ontology Implementation

Phase 2

Phase 3

Enumerate important concepts and terms


Terms Acquisitions

Evaluation Y Confirmed ConfirmedFormal FormalOntology Ontology


Figure 5 Research Design

Research Design – Phase 1 The pre-development phase follows Fernández-López, Gómez-Pérez and Juristo’s (1997) Methontology development framework Phase Pre-Development

ACIS 2013

Step i) Define intended uses, scenarios of use, and end-users, ii) Level of formality of (informal, semi-informal, semi-formal or rigorously formal), and iii) Scope, which consist of a set of terms to be represented, its characteristics and granularity


Research Design – Phase 2 Phase Development

Step Term Acquisitions: I. Enumerating Important Concepts and Terms II. Adopt Existing Terms from the Portal III. User Terms from Search Engine Query Log Define Concepts and the Relationships between Terms Ontology Implementation (Cmap)

ACIS 2013


Enumerate Important Concepts 1st Class Symptoms & Diagnosis

Figure 6. BCKOnline topic search page

2nd Class Diagnosis, Screening & Testing Types of Breast Cancer Treatment & Side Effects Surgery Chemotherapy Radiation Therapy Hormonal Therapy Targeted Theraphies Alternative/Complementary & Holistic Medicine Drugs for Treatment Treatment for Pain Treatment Side Effects Clinical Trials Day-to-Day Matters Psychosocial Exercise Palliative Care Bone Health Menopousal Symptoms Sex and Intimacy Fertility and Pregnancy Non-Medical Research & Statistical Government Support Name

Basic classification from


Adopt Existing Terms The BCKOnline systems consist of three controlled vocabularies: 1)BCKOnline glossary which specifically developed by the portal’s domain experts, 2)BreastCare Victoria which adopted from BreastCare Vicoria glossary, and 3)Terms adopted by Medical Subject Heading (MeSH). The terms were gathered from each vocabulary then sorted based on its original sources and then cleaned from duplication

ACIS 2013


Correlations between terms Genus-Species Relationships CARCINOMA

ductal carcinoma in situ (DCIS) lobular carcinoma in situ (LCIS)

Carcinoma, Intraductal, Noninfiltrating

Elaborate Variance of Terms/Synonym ADJUVANT THERAPY SIDE EFFECTS

adjuvant treatment adverse reaction

Chemotherapy, Adjuvant adverse effects

Elaborate Terms with different concepts antimetabolites

5FU fluorouracil

Adrucil Carac Efudix


ACIS 2013

magnetic resonance imaging (MRI) Magnetic resonance imaging Mastectomy mastectomy


Harvest user terms from search engine query log User Query Analysis (Jansen, Spink and Saracevic, 2000) Transaction logs containing 1,942 queries posted by 548 users. The data sample taken from period: 6th of March, 2008 to 27th of August, 2009. The data analysis and manipulation was conducted in MySQL server. Term Chemotherapy Complementary Therapy Surgery Breast Reconstruction Advanced Breast Cancer Diagnosis Lymphedema Risk factor Early breast cancer Hormonal therapy ACIS 2013

Frequency 64

Term Recurrent breast cancer













Breast Cancer












36 35 30

Pregnancy Palliative Care Liver

13 12 11

Ductal carcinoma in situ Hair Recurrence BRCA




Liver function tests





Breast Reduction


Table 2. BCKOnline terms frequency

7 7 7 6


From 1,942 queries, only 450 are actually unique. The result then further compared to the portal’s existing vocabularies gathered from the previous step. The result shows only 138 terms from 450 terms are unique. Thus, the unique terms that can be utilize from user queries are 138 from the total original 1,942 queries (7%). Measure Total Terms Unique Terms Terms occurring once Terms occurring more than 50

Original 1,942 450 436

Cleaned 450 138 138

Percentage 23,17% 30,6% 31,65%




Table 3. BCKOnline unique terms


Define Concepts and the Relationships between Terms The BCKOnline ontology is implemented in Concept Map (Cmap) which supports OWL. Terms from BCKOnline glossary choosed as classes to define and describe the concepts, subclasses, properties, and associated relationships of the domain interest, while the other terms such from BreastCare Victoria and MeSH and the extraction of data usage analysis will act as a complementary of BCKOnline glossary.


Implementation – Ontology Preview

ACIS 2013

Figure 5 Proposed User-Warrant Ontology for BCKOnline portal


Criteria-Based Evaluation (Haghighi et al., 2012) Evaluation Criteria



Definition of terms Documentation Clarity – Input from subject expert

Consistency/ Coherence

General terms is preferred Cross-check with the definition of terms


Redundant Term

Expendability/ Extendibility

Re-use and extension


Evaluation from subject expert

Minimal Ontological Commitment

Multiple view support


Competency question: What are the relative risks of breast cancer? Answers: Age, Lifestyle, Family history

Coverage ACIS 2013

Terms represents the information content of 23the portal


Significance and Contribution • The central contribution of this research lays on the demonstration of the potential of user warrant ontology to resolve information retrieval problem in HIP. • The research covers the implementation of user warrant principles for domain ontology construction, which is not highly covered in the existing literature. • Providing an evidence of how ontology engineering can be understood from DSR in information systems.

ACIS 2013


Conclusions •

• • •

The research aims to increase the portal information retrieval capability by transforming the existing vocabularies and usage data into an ontology based on user-warrants principles. Terms in BCKO glossary are fit to be the main source for the proposed ontology because the terms were constructed based on user-centric method. User warrant accommodate the important role of users in ontology development processes. In this research, User warrant consist of three elements: portal terms, user terms, and evaluation by subject expert. Ontology consists of four main classes which further breakdown into 27 instances The techniques and resulting implementation system will be suitable as a robust platform for further analysis and refinement. Further study is needed to test the implementation of the ontology to the portal

ACIS 2013



ACIS 2013


Guidelines Design as an Artifact Problem Relevance Design Evaluation Research Contributions Research Rigor

Design as a search Process Communication of Research ACIS 2013

Description Ontology and methodology developed in this study is the artifacts that is created to improve online health infromation provision The information retrieval problems in HIP is extremely relevant. User warrant ontology is needed to empower users to find relevant information in HIP. Ontology developed in this study is evaluated through several evaluation procedures (criteria-based evaluation). The evaluation process ensures the quality and efficacy of ontology as designed artefact. The development of method contributes to the development of domain ontology based on the user warrant principles. The proposed ontology also contributes to the information retrieval in semantic web. The development of research design was based on the previously identified problems in HIP’s information retrieval systems. The idea for the artifact was grounded based on the finding on the literature regarding the capability of ontology to improve information retrieval in HIP, relationships between ontology and warrant principles, and the selection of ontology development method based on IEEE 1047-2006 to support iterative processes. The research design was articulated based on the literature finding, covering on an overall phase, and on a detailed level. The ontology development method consists of a set of phases, and the concepts of domain ontology was developed in the first phase, then implemented in the second phase. The last phase consists of validation and evaluation which enables continuous evaluation process and redesign The study present the successful implementation of domain ontology construction within the BCKOnline protal as the case study to communicate the relevance and importance of this study for academic-oriented audience.


Design Science Cycles (Hevner, 2007) Relevant cycle. This research is motivated by the desire to improve HIP information retrieval systems by constructing user-warrant ontology. The proposed artifact is constructed based on the problem found on the environment, and being evaluated by the user of the application domain to ensure its relevancy. Rigor cycle. This research analyses the existing researches from two major disciplines, namely, consumers health informatics and ontology engineering. The sub-topics of both disciplines were analysed to find the possible relationships, and identify knowledge gaps within the body of knowledge. Based on that, this research is able to propose appropriate theories and methods to construct and evaluate the artifact. Design cycle. The proposed ontology development methodology was built based on the adoption of IEEE 1074-2006 due to its iterative framework. The artifact was evaluated according to a set of criteria-based evaluation to measure the ability of the artifact to fulfil the design requirements.


Adaptation of Hevner's (2007) Design Science Research Cycles 29

Development of user warrant ontology for improving online health information provision  

Australasia Conference in Information Systems (ACIS) 2013, Melbourne, Australia.

Development of user warrant ontology for improving online health information provision  

Australasia Conference in Information Systems (ACIS) 2013, Melbourne, Australia.