Commencing in 2002, the first stage of the project is now complete and the resulting portal is available at:
                 http://www.bckonline.monash.edu.au

In line with the original objectives of the project, the portal represents a workable model or prototype. As such, it requires further development and funding. Some funding has been secured in order to develop Phase 2 of the project – entitled BCKOnline2 – which is currently underway.

BCKOnline - Stage Two (BCKO2)    [2005 Onwards]

Project Description
This project extends the first stage of BCKO, ARC-Linkage 2001-2003 funded research, which has resulted in a design and a prototype BCKOnline portal which provides differentiated access to breast cancer knowledge resources. This portal contains user aware resource descriptions, which it is matching with user needs information profiles. We plan to further extend research portal's metadata repository to move toward the concept of Smart Information Portals, which aims at meeting knowledge and decision support needs of health care consumers for quality online information. We also explore to what extent these approach can be generalised to other areas within the health domain and to online government information provision.

This project seeks to further research in the following ways:

• Extend the ‘intelligence' of the portal by incorporating some adaptive system behaviour.
  Current portal does not support intelligence features, eg. “sound like” search, learning and reasoning. We plan to extend the current functionality exploiting a metadata-driven resource management approach.
• Extend the user-centric resource description focus of the portal.
  Technical and time factors have limited the prototype's ability to automate some of the aspects of knowledge repository maintenance. We are exploring the use of intelligent technologies to address these deficiencies .

Research Team

• Frada Burstein
• Julie Fisher
• Sue McKemmish
• June Anderson
• Rosetta Manaszewicz
• Shonali Krishnaswamy
• Sergio Viademonte da Rosa
• Andiwijaya Sumartono

Outcomes
The BCKO2 outcomes will have direct application to the breast cancer community.

BCKOnline - Stage One Completion - Final Report to the A.R.C.

Final Report [June 2004]
The research has produced a user-centred portal to breast cancer information resources. Although some further refinement is desirable, and planned, the portal is now available to the breast cancer community. The project has demonstrated that by using a metadata schema to support the matching of users with resources, it is possible to tailor information provision to the particular needs of an individual user. This demonstartes a new approach to online information provision.

Project Outcomes

• The quality analysis rationale developed for the project will particularly equip women to make better informed health and lifestyle decisions. Information access and patient empowerment are not only important patient rights, but there is a growing body of evidence that they contribute to better clinical outcomes - clearly a significant national benefit;
• These aspects of the portal are of particular benefit to population sectors, such as remote and rural communities;
• There is strong potential for the broader application of the concept to other health areas and other disciplines. This has raised prospects for a number of further research directions and partnerships;
• Using portal technology and metadata schema, the portal addresses the major barriers to successful internet resource discovery by the individual :-
  ie. Information overload and relevance.
  The portal interface empowers the user to specifically tailor information (discovery and retrieval) according to her specific needs, circumstances and personal values.
• Quality reporting – provides the user with a ‘checklist’ of specific criteria which enables the user to identify and prioritise criteria which are of immediate and specific relevance to her need, and to the type of information sought.
• The research has shown that it is possible to provide information tailored specifically for women, at the time they need it, via a portal which gives them access to worldwide information without overloading them with too much information.
• It has also demonstrated the value of advocacy groups working with research groups to guide the research process. This collaboration has been particularly successful and has meant satisfactory outcomes for all concerned.
• The research has highlighted the challenges of designing a medical search engine that allows information pertinent to the needs of consumers, to be accessed easily. Our organisation has also been exposed to the implications of this type of technology, such as maintenance, quality control, metadata, keeping information up to date, and marketing.
• This program provides the opportunity to strengthen links with research for the overall benefit of the cancer community.
• The project's goals and progress have been reported within the breast cancer community via newsletters and more broadly via print and electronic media. Whenever the research has been reported, to a range of audiences it has attracted much interest. International researchers in ICT, patient information providers, and e-health advocates have recognised its significance and potential adaptability to other information contexts.

One overarching characteristic of the entire project has been the enthusiasm and support for the concept, and for the actual prototype. This has been evidenced in all of the research teams' conference presentations; and certainly demonstrated in the usability testing, and the responses gleaned from the target community – ie. Women and their relatives who have breast cancer, and from breastcare nurses who deal with patients on a daily basis. Further ‘evidence’ would be that, since December 2004, and without a major 'marketing campaign', the number of hits to the portal prototype has exceeded all expectations. At the time of writing, the prototype has received over 3000 hits in the space of 5 months. It is anticipated that with further development and marketing of the service this rate of usewill increase - indicating the benefits to, and the response from, the general community.

BCKOnline - Portal Usability Report [Nov. 2004]

Portal Usability / Evaluation Results
The following section briefly outlines the highlights of the results of the usability testing, which was completed in July 2004. Since this testing, the portal interface has been converted into a html format, and the process of adding another 600 records to the database has begun.
The major findings from the evaluation and usability testing were:

• Ease of use and navigability - user's overwhelmingly reported their success in locating and retrieving materials which catered to their specific
  circumstances and needs. This was largely attributed to the 'personalised search page' and its intuitive nature.
• Users reported most favourably on the 'value-added' feature of the quality report. This enabled them to quickly and easily sort retrieved records
  according to individual priorities and values
• Users were enthusiastic as to the amount of Australian content available and also the inclusion of materials which were often difficult to locate and
  access, such as facilitative and experiential information.

The full "Portal Usability Report" is available here.

Progress Report [June 2003]

Work in Progress
1.      Resource Selection and ‘Quality’
2.      Metadata Schema Design
3.      Portal Design and Software Selection

1.      Resource Selection and ‘Quality’

In developing a framework for resource selection the Breast Cancer Knowledge Online team has drawn on James Gardner’s ‘personal outcomes’ perspective in which the success, or outcome, of a medical service or system is measured according to the extent to which an individual user’s expectations of that service or system are met. Gardner identifies three categories of outcomes:

• Clinical Outcomes: which might focus on cure and symptom reduction
• Functional Outcomes: which concentrate on increasing functional status in specific areas
• Personal Outcomes: which focus on items and issues that matter most to people in their lives.
  (Gardner, James (1997): Challenging Tadition: Measuring Quality through Personal Outcomes, http://www.ncor.org/article1.htm)

In the context of the BCKOnline portal, such a schema is particularly relevant for the identification, description and final selection of resources. It allows scope for the interplay of individual preferences and values within the processes of indexing, cataloguing etc. For example, the user may simply be interested in only one aspect of a particular topic, yet for another subject area she may require the entire spectrum of materials.

Gardner’s 3 outcome areas may be modified to deal directly with the needs of the breast cancer population:

• Medical outcomes are those related to the treatment and management of disease, such as various treatment options, clinical trial reports, drug news.
• Supportive outcomes may be regarded as psychosocial outcomes; the effect of the disease on the woman and her family; aspects of social and psychosocial functioning either on a temporary basis or as the long-term consequences of the disease; facilitative information  which may include addresses of support groups or government assistance information.
• Personal outcomes are those areas which matter most to the woman and/or her family. This may overlap with the psychosocial outcomes, but may be different and generally embody the value system in the life of the individual woman. These outcomes are often based on reflection of the experience of the disease, including its treatment rigours, and incorporate the stories of other women. The ‘voice’ of the health professional would also be included in this category.

In addition to developing the medical/supportive/personal taxonomy the research team is also proposing to organise resources included in the portal according to a number of subject categories reflecting different types of breast cancer; various prevention and risk factors; a range of surgical, medical and radiation treatments; aspects of recurrence and advanced disease; palliative care and complementary/alternative medicine.

The research team is also developing resource selection criteria. Although the criteria are not finalised, the following indicates the team’s current approach to resource selection.

Resources categorised as ‘medical’ must meet criteria for authorship (credentials of creator and publisher must be clear); currency (dates of creation, posting, amendment must be stated); evidence (users will be alerted to the evidential nature of the resource, for example whether it is consensus opinion or personal opinion, based on a randomised clinical trial, case/cohort study etc.; and references (evidence must be cited). Information about the editorial policy and/or review process is also regarded as desirable. The comprehensiveness of a resource, in particular whether it acknowledges areas of disagreement or controversy, is also relevant.

Resources categorised as ‘supportive’ must also meet criteria for authorship and currency. Their purpose (for example educational, promotional, commercial) must also be evident. It is preferred that these resources are also referenced.

Resources categorised as ‘personal’ must establish the ‘credentials’ of the author and publisher. They must encompass a range of views on a particular issues and should have relevance to an Australian context. A panel of experts reflecting a holistic approach to the care of women with breast cancer will also contribute to the selection of resources in this category. Terms of reference for this panel are currently being developed.

The issue of ‘quality assessment’ has been, and continues to be, considered at length by the research team. The project recognises:

• The importance of personal meaning and individual values of the potential user.
• The ‘value’ of diversity in the sense of not privileging information according to source or type or content – the ‘decisions’ are to be based on user predilections and choice.
• The often ambivalent nature of the information seeking process for the individual with a serious illness.

The team’s current thinking is that value-added information about resource ‘quality’ will in part be drawn from the metadata relating to authorship, currency and evidence. They will also draw on models currently used elsewhere (ie HonCode) and the principles contained within these.

2.      Metadata Schema Design

Current approaches to web searching generally do not support the delivery of timely information tailored to individual needs to the extent needed for informed decisionmaking. This project proposes a portal which harnesses metadata – structured resource descriptions – to deliver relevant, user specific information. The project has adopted AS5044, the Australian Government Locator Service (AGLS) Metadata Standard, as the starting point for developing a metadata schema which enables user-aware resource description. Several extensions and modifications to the AGLS nineteen element metadata set are proposed, including:

• Extension of the Audience element to attach user profiling information to a resource, including age group, disease stage, information preferences, user type (family situation), locality/care proximity
• Introduction of a Quality element to describe resources in terms of the review process, whether references are provided, evidence type, purpose and balance.
• The AGLS elements Function, Coverage and Mandate will not be used.
• The proposed typology of medical, supportive and personal will be captured using the AGLS Type element.

Research is now underway to confirm the encoding schemes which will be used. A combination of AGLS schema, BCKOnline specific encoding values and tools such as Medical Subject Headings and/or Breast Cancer Victoria Glossary will be used.

More information about AGLS is available at http://www.naa.gov.au/recordkeeping/gov_online/agls/summary.html.

3.      Portal Design and Software Selection

Research is now underway to develop the BCKOnline prototype portal as an intelligent decision support system. The team is currently investigating how the following portal functionalities should operate in the BCKOnline framework:

Information retrieval – the use of user profiles and user-sensitive resource descriptions is planned as the means supporting user-aware retrieval. The user’s first interaction with the system will be to select a profile that best matches her characteristics and circumstances (age, disease stage etc). This input will facilitate a metadata driven search of resource descriptions in the portal repository, which in turn will point to the user to relevant online or offline resources.

Adaptivity – an adaptive system is continually trying to configure itself so as to match the input data; in a sense, to ‘learn’ from its use. It is not planned for the BCKOnline portal to have advanced reasoning and adaptivity mechanisms, however it will have the potential to monitor requests submitted and thus to guide the future growth of the repository of resource descriptions, and refinement of the user interface.

Prioritising information – matching of user profiles and user-sensitive resource descriptions will provide the user with listings of the most relevant resources.

Explanation facility – an intelligent system should be able to explain to its users both the knowledge it contains and the reasoning processes it applies. In the BCKOnline context, users will be provided with value-added information about resources, including their authority, provenance, currenty and quality rating.

The research team is investigating ‘HotMeta’ – the metadata repository and search engine developed by the Distributed Systems Technology Centre as the probable prototype engine.

Summary Report - User Needs Analysis: June 2003

This phase of the project was completed earlier this year. Fifty-nine women, eleven breast care nurses, and seven partners of other family members participated in focus groups or interviews.

The study used the technique of purposive sampling, often used in qualitative research, to ensure that the major characteristics considered to be relevant were included in the sample and examined in the study. Data collection and analysis took place simultaneously, and new insights were incorporated into subsequent data collection. Categories and themes emerged from the data – a grounded theory approach.

Understanding the Sample

The following figures relate to the 59 women with breast cancer who participated in the user needs analysis.

1.      Sample Participants by Age

The following table presents the age distribution in the sample, at time of the study as well as at the time of diagnosis. 

2.      Disease Stage

88.13% of women participating in the study were in an early stage of the disease and 11.86% had advanced breast cancer.

3.      Internet Usage

*% of women in the study who reported that they have used in internet to access information about breast cancer.
** % of women in the study who reported that friends/family used the internet and other sources on their behalf.

Key Issues to Emerge from the Study

Timing of Information Delivery

“now I'm so much further down the track, the information is much more readable and acceptable whereas earlier on it was too confronting. I think now I'm quite comfortable reading things and taking them in, whereas before I couldn't.”

Information Overload

“the main thing is too many booklets, too much stuff to read… I had too much stuff to read. And to find out the information, the specific information that you need to have, you have to read, you have to read through too many things. …. You think ‘Oh boy – there’s too much, too much information here’, but you want to find it, to know which way to find it. Have I got to read through all that to find what I want?”

Relevance of Information

“but I really found that a lot of the time the information that you get, until it suddenly is part of what you’re going through, it’s not relevant and therefore perhaps you have read a lot of information that you’re not taking in. It’s a process.”

Quality of Information

“how do you know which one has credence and which one doesn’t? I mean if you’re going to go into the internet and try to find a site that’s going to tell you something, how do you know that that’s not valid information? Just by reading it? I think that’s a lot of hogwash!”

Format of Information

“That’s what I wanted, I wanted diagrams. For the first time last week I actually saw what invasive (lobular) carcinoma looks like as opposed to a ductal or different type of cancer. I wanted to see exactly what was really going on.”

“all my information is still sitting in the bottom of the drawer and I was diagnosed nearly nine years ago.. I have not opened any of it. My information basically came from talking to other people. I did not want to read because I didn't think I could take it in. It was all too daunting.”

Gaps in Information and its Communication

“My biggest gap is fertility. There’s not enough known about that and I spoke to my surgeon last Monday and I have one child and I was just told blanket, ‘you should wait two years before you have another child’. And my question was ‘what if I do get pregnant?’… And her answer to me…was, ‘well, we don’t know’”.

Accessibility of Information

“I’ve spent ages on the internet wandering all around, looking for things that I don’t really want to know, but I find it interesting, until I think very clearly what is the keyword that I really want to know. And that’s the same with asking questions, that sometimes we just don’t know. We know we want to know something, but we don’t know how to get there.”

“All the terms. All the jargon and the drug names….  It’s not that hard. It’s fairly formulaic. So how come it’s not accessible? There’s stacks of information out there…. I should simply be able to go and find that this is the research, this is the incidence of success. It should have been reasonably easy to do.”

Issues Emerging from Family/Friends Interviews

• Although medical information is important for families and friends at initial diagnosis, partners in particular report a strong need to know other people’s personal stories. This needs seems to be greater, and is unmet to a greater extent, than it is for the women themselves.
• There are substantial gaps in both the facilitative and supportive information available to men.
• Succinct, and/or visual, information is important.

Some Observations about the Findings/Themes

In many areas the research reiterates findings from elsewhere. For example, it is not new to observe that information needs varying according to factors such as age, location or education, or to note the importance of timeliness or the phenomenon of information overload. However, through our qualitative method we have moved beyond a simple understanding that these characteristics have an effect - to obtain in-depth insights about their importance and role.  We have also explored how generalisations, such as those regarding age and education, do not take into account people who may not fit the norm.

The truly innovative feature of this research is that it focuses specifically on how available information is best utilised: what type of information is most important at a particular time in the disease trajectory? How is this information best presented at this time? Many of the above observations are in fact expressions of ‘relevance’: information has quality and utility when women perceive it to be relevant.