'Some users will need to be able to track the changes that have been made to a document': ReScript editing issue 8

Page section
Article editor

Heuristic
Memorability

Description
Some users (for instance editors) will need to assess other users' work and will need to compare different versions of the same document.

Impact severity
High

Recommendation
Insert links to the entire version history of a document, in reverse chronological order.

Examples
The transcription of a document will lead to certain decisions being made by the user on structure as well as mark-up. A version feature would enable editors to see how users have interpreted guidance, and provide practical feedback for future work.

Figure 8—1: we learn from our mistakes, and currently there is no way for a user to see the exact changes which an editor has made on one of their documents.

Quantitative measure

Click where you would expect to access and view a previous version of this document

Actual question

Click where you would expect to access and view a previous version of this document

Initial click test result ('before')

November 2011: 120 responses.

Figure 8—2: before

Development change
Inclusion of a folder tab entitled 'version history', appearing third in the folder structure, and giving links to previous versions.

Figure 8—3: new folder tab

Follow-up click test result ('after')

December 2011: 90 responses.

Figure 8—4: after

Reflections

Clear grouping of clicks on the appropriate folder tab, objective has been met. The alignment of question wording with interface seems to improve the chance of click test success - however, this wording is generated from qualitative interviews therefore it is fair to assume that by institutionalising it at an early stage, we have made the process more robust.

'Users like to navigate documents quickly using search': ReScript editing issue 7

Page section
Article editor

Heuristic
Learnability, memorability

Description
At times, users need to locate specific parts of sometimes quite lengthy and heavily marked-up documents, either to check the style of mark-up employed, or to look for a particular phrase which might be a candidate for mark-up. They may also wish to have a quick way of comparing the mark-up of matching items. Without a search function, this is problematic.
Impact severity
High

Recommendation
The inclusion of a search function in the control pane, with matches highlighted.

Examples
Parliamentary texts may continually refer to the same individuals, and users may choose to find all instances and mark them up consecutively.

Figure 7—1: the more heavily the text is marked up, the less easy it becomes to scan for key words.

Quantitative measure

You are looking for the name “Wallworth”. Click where you would expect to find a function which allowed you to search for this name within the current document.

Actual question

You are looking for the name “Wallworth”. Click where you would expect to find a function which allowed you to search for this name within the current document.

Initial click test result ('before')

November 2011: 120 responses.

Figure 7—2: before

Development change

Inclusion of a search function limited to the editing window, appearing second in the new toolbar.

Figure 7—3: matches within the text are highlighted and the document position is automatically moved to the first match

Follow-up click test result ('after')

December 2011: 90 responses.

Figure 7—4: after

Reflections

Using search is a necessary skill to add sohphistication into web browsing; users recognise and understand it as a function and the click groups above show that the revised interface is resoundingly clear in meeting this objective.

'There is a need to provide guidance that is tailored both to the source, and to the role of the current user': ReScript editing issue 6

Page section
Article editor

Heuristic
Memorability

Description
Guidance about the source itself will be essential for all users. In addition, different levels of guidance will be needed depending on whether the user is editing or transcribing. This guidance should extend into practical advice about mark-up, preferably including worked examples.

Impact severity
High

Recommendation
Creation of a persistent page object into which guidance may be tailored to both the source and role of the current user.

Examples
The names of some items are likely to be somewhat generic or esoteric depending on the source, e.g. event in the Alumni Oxonienses draws together related details within education and occupation.


Figure 6—1: there appears to be no linear flow for the editing process meaning it is difficult for people to know how and where to start


Quantitative measure

Click where you would expect to find guidance on how to mark up the entity "East india Company"

Actual question

Click where you would expect to find guidance on how to mark up the organisation "East india Company"

Initial click test result ('before')
November 2011: 120 responses.

Figure 6—2: before

Development change
Creation of a tab folder, entitled 'About the source' for editorial guidance. Easy to switch between guidance and editing.

Figure 6—3: new folder tab structure enables guidance to appear alongside editing function without changing pages

Follow-up click test result ('after')

December 2011: 90 responses.

Figure 6—4: after

Reflections

This issue remains unmet as there is no particularly strong concentration of clicks on the 'About the Source' folder tab, and the generic Help link in the top right attracts a similar amount of attention in both versions.

On reflection, this may be a simple labelling issue. 'About the source' is a general term but inappropriate in this context. Results would likely have been better if it were titled 'Mark-up guidelines', 'Editing guidelines' or even 'How to edit this'. This is a frustratingly small error at the final execution phase which has led to the issue being unmet and should have been resolved by QA.

'Users need to be able to specify whether their comments are intended for publication': ReScript editing issue 5

Page section
Article editor

Heuristic
Memorability

Description
A formal distinction exists between phases of editorial work – the initial transcription and mark-up procedure may uncover ambiguities in the original of which the reader must be made aware. In addition, further secondary or explanatory material (for instance, footnotes) may be added by another user in a subsequent phase.

Impact severity
High

Recommendation
Although the semantic structure of the underlying content mark-up is flexible enough to support the presence of notes, users conceive of marking-up and editing as two separate functions which, in the course of a project, may be carried out by separate users. Support these functions distinctly and enable editors to specify whether their comments are intended for publication (Typical categories include: Added, Deleted, Gap, Supplied, Critical apparatus (e.g. footnote)). This solution needs to be extensible, will rely on editorial style and is expected to differ between sources.

Examples
During the editorial process, a user may comment on difficulties encountered during the transcription of the text, e.g. "This appears to read Botolph, but could possibly read Butolph." They may also add explanatory material, e.g. "The Botolph referred to is the parish of St Botolph Aldgate". At present, they cannot specify which of their comments are intended for publication, e.g. as footnotes, and which are not.

Figure 5—1: at times, the transcriber may need to advise the reader that the originals were unclear

Quantitative measure

Where would you expect to click in order to add a footnote to this paragraph?

Actual question

Where would you expect to click in order to add a footnote to this paragraph?

Initial click test result ('before')
November 2011: 120 responses.

Figure 5—2: before

Development change
Creation of dedicated toolbar function, entitled 'Commentary' for editorial notes. Works in the same material way that the ready mark-up function does.

Figure 5—3: commentary device added to toolbar and separated from mark-up device by search function

Follow-up click test result ('after')
December 2011: 90 responses.

Figure 5—4: after

Reflections
This objective was only partially met in spite of a new dedicated device being created for this purpose. The mark-up function to the left attracted a substantial number of clicks. It should have been clearer that the applying critical apparatus was distinct from the more formal marking up of content and so a change in labelling may be useful, from 'Commentary' to 'Footnotes, comments'.

There seems to be a risk here that, because the actual enhancement suggested is altering a few words, that that would undermine the substantial investment in time and resources in usability analysis. By its nature, usability reviews pick up on aspects of the interface which are likely to have a negative effect on the user experience so care needs to be taken to prevent the only output from this process appearing slight or subjective.

It feels frustrating that the recommendation here is to change a few words and that that is the difference between assigning a Met and a Partially Met classification to the work. It's almost as if we need a pre-usability-test usability test.

'Users need standardised visual cues to use devices which move or transform': ReScript editing issue 4

Page section
Article editor

Heuristic
Memorability

Description
Users were unanimous in liking the manuscript image viewer but none of them understood its potential for movement/resizing without interviewer intervention.

Impact severity
High

Recommendation
That design conventions for resizing objects be researched and employed on the manuscript image viewer, with some accompanying help text.

Examples
The user engaged with the controls inside the image viewer (zoom in/out, normal size and shrink to fit, but failed to resize the whole box, meaning their image viewer stayed the same size throughout their session, unnecessarily restricting its utility.

Figure 4—1: users controlled the size of the manuscript within the box - it wasn't apparent that they could extend the viewer's size as well by dragging the box border.

Quantitative measure

Click where you would expect to be able to expand the size of the manuscript images

 

Actual question

Click where you would expect to be able to expand the size of the manuscript images

Initial click test result ('before')
November 2011: 120 responses.

Figure 4—2: Before

Development change
Consolidate manuscript images into one 'viewer' with a menu to select different images. Note that the design of the extensible border was not changed.

Figure 4—3: single image viewer concentrates attention on one spot and eliminates the possibility of the image appearing below the fold and so being missed

Follow-up click test result ('after')
December 2011: 90 responses.

Figure 4—4: After

Reflections
The objective was met through the convincing grouping of clicks; however, it is not possible to discern the effectiveness of the extensible border because this type of test is static, i.e. users interact with an image of the page, not the page itself, so they cannot hover over the border, see the mouse cursor change to a resizable icon. Although impossible to test in this way, it should still be possible to include this functionality in a demonstration video, or a prior training course.

The objective of the question is so clear that it reveals a certain number of clicks from users who are simply not engaged with the test - this helps to give the analyst a sense of the amount of disinterested or wasted replies and think about whether the people approached to take the survey were properly identified and communicated with. In addition, the sequencing of difficult questions at the start may have led some users to think that they could not manage the rest of the survey.

'Users do not check work by looking at mark-up': ReScript editing issue 3

Page section
Article editor

Heuristic
Learnability

 

Description

Many users will be unused to editing XML marked-up text, and may find this daunting. When checking the document, users will likely read it through using the preview pane rather than the XML editing window, and will expect to be able to select parts of the text that they wish to edit from within the preview pane.

Impact severity
High

Recommendation
Extend the functionality of the preview pane so that double-clicking on text there simultaneously selects the same text in the XML editing window as well.

Examples
The user looks through and selects text in the preview pane, and then uses the toolbar to insert the relevant attribute. As the cursor position in the XML editing window is not in sync with the cursor position in the preview pane, the attribute code is inserted in the text in the XML editing window in the wrong position, as illustrated below.

Figure 3—1: the real-time preview pane allows text selection as does the main editing window. If the text selection is different, the user could assume they are inserting the relevant mark-up given the preview pane position but the system would  proceed using the text window.

 

Quantitative measure

Click where you would expect to add an attribute to the name "Evance Jones".

Actual question

Click where you would expect to add an attribute to the name "Evance Jones".

 

Initial click test result ('before')
November 2011: 120 responses.

Figure 3—2: Before

Development change

Removing the preview pane from simultaneous viewing alongside the text editing window, and limiting the toolbar to only appear alongside the text window.

Figure 3—3: de-coupling the preview pane leaves more room for the editing window and new toolbar, presented within a tab folder structure

 

Follow-up click test result ('after')
December 2011: 90 responses.

 

Figure 3—4: After

 

Reflections
The wording of the question could have been better: the term 'attribute' is quite generic and would normally be replaced with a worked example, e.g. "where would you expect to mark-up Evance Jones' surname". This had the effect of confusing users of the test, leading to a group of clicks on the third toolbar option, titled 'Commentary'. This would be mitigated if this toolbar option were to be renamed along the lines of 'Footnotes, Comments'.

Overall, there has been a movement of clicks away from the text and into the toolbar but some still remain. By this point in the sequence of tests, I am beginning to think that a further context menu might be necessary which pops up at the cursor position and also permits access to the 'insert mark-up' functions. This would be a different means to support people but it would not be easy to investigate using this type of click test.

'Some parts of articles are repetitively structured and would suit some automation': ReScript editing issue 2

Page section
Article editor

Heuristic
Efficiency

Description
Articles within a specific source typically have structures which are repeated several times. Users often mark up structures by hand and/or use 'copy and paste'. This sometimes leads to incomplete and/or inaccurate mark-up, slowing the editorial process down and making additional editorial intervention necessary.

Impact severity
High

Recommendation
Part of the set-up process for a project should be the definition of repetitive code structures. It should then be possible for users simply to drop copies of these accurate and complete structures into the article wherever they are needed.

Examples
When attempting to copy the mark-up structure used for one instance of the item person to another instance, users sometimes accidentally omitted parts of the tagging code.

Figure 2—1: the name is repeated in the text with a slight variation in spelling. The risk of transcription inaccuracy  is increased with the temptation to 'cut and paste' and move on.

Quantitative measure

You want to mark-up the text "Evance Jones" as a person. Rather than marking-up by hand, click where you would expect to instruct the system to insert some ready-made mark-up for a person.

Actual question

You want to mark-up the text "Evance Jones" as a person. Rather than marking-up by hand, click where you would expect to instruct the system to insert some ready-made mark-up for a person.

Initial click test result ('before')
November 2011: 120 responses.

Figure 2—2: Before

Development change
Toolbar now includes a single menu for inserting mark-up with elements arranged hierarchically.

Figure 2—3: prominent toolbar with markup device first

Follow-up click test result ('after')
December 2011: 90 responses.

Figure 2—4: After

Reflections
This issue has been met, as shown by the movement of large click groupings from the text and preview 'contexts' to the toolbar function. The extended manuscript image was a red herring, designed to confuse those people who have very little grounding in this type of work - the amount still clicking on the image suggests that there is a need for an initial period of training for quite a number of potential contributors.

Regarding the efficacy of the test, the revised function itself uses a turn of phrase from the test itself ('...ready made mark-up'). That may to a degree, invalidate the findings, if users are mentally 'matching' the question to a part of the page. However, by using as little jargon as possible in this type of test, the software benefits far more from having labelling which is easy to understand, than the test loses in fairness. After all, the questions emanate from qualitative research where the analyst is able to reflect language which they think will be well understood

'Users need constant feedback when using something new': ReScript editing issue 1

Page section
Article editor

Heuristic
Learnability

Description
Users often mark up data using attributes without identifying the item to which they belong. This makes it impossible to extract certain types of meanings from the article and so making additional editorial intervention necessary.

Impact severity
High

Recommendation
When applying an attribute, visually emphasise the item to which it relates, and report if the appropriate logical parent is not marked up.

Examples
Users often marked up attributes, such as forename or surname, but omitted to nest these within the relevant item, e.g. person, which would correctly limit the scope/context of the attributes.

Figure 1—1: here the properties forename, surname and occupation appear without a parent element to delimit their scope to the individual Richard Casye

Quantitative measure

You have just marked up "Richard Casye". If there was a problem with your mark-up, where would you expect to see a warning message?

Actual question

You have just marked up "Richard Casye". If there was a problem with your mark-up, where would you expect to see a warning message?

Initial click test result ('before')
November 2011: 120 responses.

Figure 1—2: Before


Development change
Toolbar has been reconfigured to arrange attributes directly next to its correct parent item, in a single drop down menu. In addition, a status bar at the bottom warns of any incorrect XML.

Figure 8—3: the toolbar is much more prominent, and the preview device has been moved to a different folder tab

Follow-up click test result ('after')
December 2011: 90 responses.

Figure 1—4: After

Reflections
This was quite an involved question to begin the test with, and hopefully set the tone for the remainder of the survey, giving users an idea of what level of question to expect. The correct answer for the follow-up was the status bar message which appears beneath the text editor which did attract some clicks; the code around the problem mark-up also turns red when incorrect so those that clicked on the text itself are also correct.

There are other incorrect groups of clicks though, the right sidebar and top toolbar most prominent. IN hindsight, this particular test may have been better positioned later on in the test, when users had had a chance to better acquaint themselves with the system. However, this is still a difficult question to answer as it asks to draw on an idea of interactivity for a software programme of which they probably have no experience.

'Index filters sometimes produce prohibitively long lists': ReScript querying issue 9

Page section
Index

Heuristic
Efficiency

Description
Index filters sometimes produce very long lists, which are unwieldy to navigate.

Impact severity
Medium

Recommend
Remove the Index section. The new design will incorporate a search function with either predictive text, or alphabetical sections.

Examples
There are currently 4,000 surnames listed in the Parish Clerks’ Memoranda Books.

Figure 9—1: the length of site menus is subject to every variation within the text

Quantitative measure

Click where you would expect to search for the surname “Gaveston".

Reflections
This issue was not tested because of it was covered within the tests for the other search-based issues. It reminds us of issue #1, in which historians did not adopt the functionality which comes from seeing text as a 'database of words'. For many users, free-text search is an essential (non-negotiable?) part of the research process.

Bruce Tate, Project Manager
Donna Baillie, Project Officer

'There is currently no “keyword search” function': ReScript querying issue 8

Page section
Index; Search legend

Heuristic
Efficiency

Description
It is currently not possible for users to conduct full text searches using keywords.

Impact severity
High

Recommendation
Replace the current Search legend with a “Super Search” form which includes a simple keyword search function here.

Examples
It is currently not possible for users to search on terms that are not included in the Index, e.g. scripture references such as “1 Peter”.

Figure 8—1: no free-text search feature

Quantitative measure

Where would you click to search on all inclusions of scripture verses from 1 Peter?

Actual question

Where would you click to search for all records with scripture verses from 1 Peter ?

Initial click test result ('before')
September 2011: 89 responses.

Figure 8—2: Before

Development change
The publication's home page is now quite configurable, and contains a box to search on article titles (with a jQuery powered auto-complete function), as well as a conventional free-text search.

Figure 8—3: not one but two search boxes (three if you include the full site search)

Follow-up click test result ('after')
November 2011: 330 responses.

Figure 8—4: After

Reflections
The article title search box on the main left panel attracted some clicks (erroneously) but the clear majority plumped for the correct sidebar option, with a few going for the wider site search at the head of the page. Search is so important that having a couple of options is a strength, but there must be swift feedback if a user is entering the wrong kind of information; for instance, using the article title search to look for a keyword.

Bruce Tate, Project Manager
Donna Baillie, Project Officer

'Variant spellings currently have to be searched on separately': ReScript querying issue 7

Page section
Index

Heuristic
Efficiency

Description
All spelling variants currently have to be searched on separately.

Impact severity
High

Recommend
In the redesigned search form, provide two options: one for searching individual spelling variants, and one for searching all variants together.

Examples
There are currently several spellings for many search items, e.g. agewe.

Figure 7—1: each user must combine different spellings by hand

Quantitative measure

Where would you click to search on all variant spellings of “agewe”?

Actual question

Where would you click to search on variant spellings of the word 'widow' ?

Initial click test result ('before')
September 2011: 89 responses.

Figure 7—2: Before

Development change
Creation of a thesaurus model for each publication which would permit suggested expansions for queries. These will also be filterable for particular predicates, e.g. suggest farmer only when marked up as an occupation, not a surname.

Figure 7—3: a search for widow here has returned a couple of editor-controlled expansions

Follow-up click test result ('after')
November 2011: 330 responses.

Figure 7—4: After

Reflections
These will be opt-in; the user must choose to add them to their query, forcing them to think about how the source is written. It also indicates where the source has particular areas of strength which if simply narrated in the 'About the source' section could be missed by the majority of users. The objective has been pretty clearly met.

Bruce Tate, Project Manager
Donna Baillie, Project Officer

Tales from the Parish Clerks’ Memoranda No.13: The Final Tale

For this the thirteenth and final Tale from the Parish Clerks’ Memoranda for the ReScript blog, I wanted to take a moment to reflect on the processes of record-keeping, stretching from the painstakingly hand-written Memorandum Book entries of the 16^th century, to the painstakingly key-tagged digital renditions of the same for the very modern ReScript editing interface, that have combined to help bring the parishioners of St Botolphs without Aldgate vividly to life for 21^st century historians.

Because the memorandum books were written freehand, with no standardised spelling, and without a terribly strict format, the amount of automated key-tagging that we were able to do was very limited. This meant that I had to check every single entry to try to ensure that everything a historian might feasibly want to search for using ReScript could be easily found. I confess that sometimes, especially in the early stages of the work, this could feel quite tedious.

But an interesting thing happened as the work progressed: as I ploughed through innumerable notices of births, engagements, weddings, and funerals, I started to get a sense of the rhythm of life in the parish. I found myself picturing dancing and celebration at the weddings, and feeling fearful for the fate of infants who were in such poor health that they had to be baptised at home instead of at the church. I grew to recognise some of the parishioners and church officials who appeared in the records again and again, like the wonderfully named Balthazar Piggin. And I felt a sense of dread as the first mentions of plague began to appear in the records.

I also began to feel a strange synchronicity with the parish clerks. They and I both tended to stumble and make errors when we had to undertake particularly long and repetitive entries of events (their errors remain visible for all time while mine, mercifully, could be corrected out of existence). And I could practically hear their heavy sighs of resignation when yet another funeral service had to be marked “unpaid”. The role of the church in marking every milestone of the lives of the parishioners – and of many non-parishioners – of their little area of London meant that the ministers and parish clerks probably knew more about the goings-on of the neighbourhood than anyone else.

Having key-tagged countless instances of Mr Hayes the minister flawlessly fulfilling his duties, I found that the following entry rendered him endearingly more human:

14 Iune Ano 1589

Judith Hall the dawghter of Edmund Hall a Surngionn Dwellinge in mother Figgis Alye beinge in the High Streete was cristned the xiiijth Day of June anno 1589 By the minester of Creechurche mr Cowse For that mr Hayse was not well or a Slepe

P69/BOT2/A/019/MS09234/001 fol. 68r

It was the “…or asleep” that tickled me. To conclude this series, I leave you with Mr Hayse’s heroic attempt five years earlier at forcibly establishing brotherly love amongst the parishioners:

24 Maye 1584

Coone clarke was put by the communion

Memerandumm that mr Hayze wt the consent of Bothe the church wardens ded putt Coone clarke and his Wyfe From Receyvinge the comunion the xxiiijth Daye of maye in ano 1584 For that theye Weare not in love and charetie with their neyghbores Hentell Suche tyme as theie Do reconsyle them selves

P69/BOT2/A/019/MS09234/001 fol. 55r

Thank you to everyone who has been following Tales from the Parish Clerks’ Memoranda. I have thoroughly enjoyed writing it, and hope that you will keep checking in to the blog to chart the progress of ReScript as Bruce Tate and the team continue their excellent and innovative work. And be sure to stay in love and charity with your neighbours. It’s what Mr Hayes would have wanted.

Many thanks to St Botolph without Aldgate and London Metropolitan Archives for permission to reprint extracts from the Parish Clerks' Memoranda.

The Parish Clerks' Memoranda transcripts were prepared by the Centre for Metropolitan History team as part of their Economic and Social Research Council-funded Life in the Suburbs project (Grant Reference: RES-062-23-1260; http://www.history.ac.uk/projects/life-in-the-suburbs).

'There is currently not enough information about the source being searched in the results pages': ReScript querying issue 6

Page
Source page

Heuristic
Learnability, Memorability

Description
There is currently no information, apart from its title, about the source being searched.

Impact severity
Medium

Recommend

1. Include a brief description of the source directly underneath the name; and
2. create a clickable image that takes users to a stand-alone page with comprehensive information about the source.

Examples
Users reported that it would be useful to have information about the source documents being searched on the individual source search pages.

Figure 6—1: the scope and provenance of this source is absent

Quantitative measure

Where on this page would you expect to find background information about the Alumni Oxonienses?

Actual question

Click where you would expect to find background information about the 'Alumni Oxonienses'.

Initial click test result ('before')
September 2011: 89 responses.

Figure 6—2: Before

Development change
Inserted a new link to the publication in the title bar, a short bibliographic description in the first sidebar, and a link to create a citation.

Figure 6—3: bibliographic information appears in two locations, the title and sidebar

Follow-up click test result ('after')
November 2011: 330 responses.

Figure 6—4: After

Reflections
The response was split between the title and sidebar and was very successful overall. The sidebar has the further attribute of leading into the contextually relevant calendar function which permits highlighting of matching attributes in the text. By associating the publication with these functions, users should have an accurate understanding of the kinds of queries which will be successful with this source.

Bruce Tate, Project Manager
Donna Baillie, Project Officer

'Users have no way of creating a citation, nor any help for saving their search': ReScript querying issue 5

Page
Source page

Heuristic
Efficiency

Description
Once users have conducted a search, there is no way for them to save either the search path or the results.

Impact severity
Medium

Recommend
Create an option for saving searches.

Examples
If users create a search and later want to re-run it either as-is, or with amendments, they have to start from scratch.

Figure 5—1: complicated queries cannot be saved when viewing

Quantitative measure

Where would you expect to click in order to save the results of this search?

Actual question

Where would you expect to click to save the results of this search?

Initial click test result ('before')
September 2011: 89 responses.

Figure 5—2: Before

Development change
Search results pages now have a Remember this search facility, which will be accompanied by a drop down list of those recently saved.

Figure 5—3: New 'My Links' section in title bar will also link to saved articles

Follow-up click test result ('after')
November 2011: 330 responses.

Figure 5—4: After

Reflections
The grid layout of the new design is no less complex compared to the previous version, but the Remember function seems to be well positioned given the clear grouping of clicks. It could have been named 'Save' but that has connotations of downloading / exporting to files which isn't what was meant. 'Remember' is a more informal word, and the function itself uses the HTML5 localStorage feature, which ought to make it compatible with mobile devices.

Bruce Tate, Project Manager
Donna Baillie, Project Officer