Within the scientific domain, one frequently finds that paper forms are heavily used as a means of both
collecting data and also reporting on the analysis of data. For example, in breast screening clinics,
mammograms are analysed by experts referee to as readers to determine whether or not the patient should be
recalled for further tests. For each patient, four mammograms are taken — two views of each breast — and
the resulting film is analysed by a reader who reports the findings on a paper form. The form contains
breast outlines corresponding to the four mammogram views and the reader will annotate these to indicate
any abnormalities or special features and, generally, any reasons for their decision whether or not to
recall the patient.
To investigate the approach of annotation and link authoring on paper, as well as the needs and possible
benefits of cross-media annotation in scientific environments in general, the Global Information Systems (GlobIS)
research group at ETH Zurich developed a prototype
system to support the work of the mammography screening process. It uses an Anoto-based solution
for the authoring of both links and annotation content.
The X-ray images used in mammography are available on film and the readers use a paper form to
report on the results. Some breast screening centres have already switched to digital technologies
or are considering it. The X-ray images can be displayed on large computer screens and computers
can be used to report on the X-ray images. However, a digitalisation of the entire process brings
some problems. First, the clinics currently see the generation of digital versions of the mammograms
as simply creating an extra step in the process, resulting in extra delays and costs. Second, it
is much more comfortable and quicker for a reader to enter data on a paper form, especially the
annotations. It is also interesting to see that readers will often process mammograms in batches,
analysing a number of mammogram sets until a significant feature is detected and then filling in the
forms for the whole batch.
The main advantage of a purely digital solution is that data is available in a digital form for
querying, ubiquitous access and for archiving. Researchers are also working on automated analysis
of the images. However, switching to digital media results in a significant change of the working
process and it appears that it also brings disadvantages. Therefore, in our mammography screening
prototype, we use our interactive paper platform to digitally store and retrieve reports while
interfering as little as possible with current work practices. It is common in most clinics to have
double readings to reduce the chances of missing a possible tumour. In many cases, the second reader
will first analyse the mammograms and annotate the form before checking the analysis of the first
reader. This gives them the chance to double check whether or not they have overlooked something.
The ability to read the annotations of other readers also plays an important role in helping less
experienced readers learn from more experienced ones. To better understand the needs and possible
benefits that could be afforded by cross-media annotation in scientific environments, we have had
many discussions with researchers who have studied the practices of various breast screening clinics,
inclusive of their training processes.
The mammography reading workplace based on our interactive paper technology is shown in Fig. 1.
A special application database has been designed for the mammography screening prototype
based on an OMS Java database. This database contains information about patients and screenings
as well as the annotation forms. Furthermore, the database provides rich functionality and
querying facilities for the stored data.
It is not convenient to print the Anoto dot pattern on each of the X-ray images for position detection.
However, for the mammograms the mimio Xi, which captures handwriting based on high-resolution ultrasonic
positioning, could be used as an input device without requiring changes to the X-ray film. Since we only
have access to a limited number of X-ray images on film, we also used paper copies as substitutes in
our prototype, applying Anoto technology for both the mammograms and the report forms. For each patient,
the reader has multiple sheets of paper, covered with the Anoto pattern—the mammograms of the patient
and a screening report form. The report form contains an active area linking to a patient’s medical record.
To fill in the report form, the reader marks checkboxes and writes free text in the corresponding areas.
The content of the checkboxes is analysed and stored in the corresponding database objects. The free-text
annotations are stored as images and added to the database as an annotation of the mammography. The free-text
annotations will only be interpreted by the other human readers of the mammogram. Therefore, there is no
need to store the annotations in a machine-readable form.
If a doctor wants to annotate a specific area of a mammogram, they can use the pen to draw a shape around
it before entering the annotation text on the report form. The system creates an active area on the
mammogram in the form of a polygon and links this area to the digitised annotation text, thus creating
a digital annotation to the paper document. This gives the readers the possibility of creating annotations
on different levels of granularity. They can annotate the whole mammogram as well as very specific
details referring to small parts of the X-ray images.
Existing annotations are available to the current users of the paper documents. The images
corresponding to the annotations can be displayed on the computer screen along with additional
information such as the creator of the annotation or the date. A smaller version of the mammogram
is also displayed together with the active areas of the paper documents. However, in order to
support blind double reading, i.e. independent analysis of the mammograms by different readers,
annotations of other users are not visible by default. The readers perform their analysis and first
create their annotations before consolidating their findings with those of the other readers. For
controlling access to different annotations and comments based on a user’s profile, the mammography
screening prototype accesses the user management functionality provided by the iServer platform.
Since iServer defines access rights at the entity level, we have full control if access should be
restricted to specific resources or single links and further can use the group concept for classifying
different users.
The context-dependent visualisation of the annotations based on contextual information about the
user is one of the biggest advantages of our prototype over the traditional non-digital solution.
In current work practices, multiple versions of the paper form have to be accessed to support blind
double reading. Another advantage is the instant availability of the annotations in a digital form.
There is no need to scan or transcribe the report forms.
The mammography screening process is just one example of many where users are very reluctant to
move away from paper. Sometimes it may be part of general resistance to change, but studies of
the working environment and work practices often reveal that there are very good reasons for
remaining with paper and other forms of non-digital media. On the other hand, if we consider
the breast screening example, there are certain activities such as the reviewing of previous
reports and annotations that can be improved with digital support. Summarising, we think it
is important to investigate not only which parts of a work process can be digitised, but also
which parts should be digitised in order to increase the effectiveness of work practices.
In the end, an integrated solution where parts of the mammography screening application still
remain paper-based but become tightly interweaved with a digital information system might
be the appropriate approach.
Digital Annotation of Printed Documents,
Corsin Decurtins, Moira C. Norrie and Beat Signer,
Proceedings of CIKM 2003, 12th International Conference on Information and Knowledge Management,
New Orleans, USA, November 2003