Timeline and work packages

Timeline

Work package 1: project management
Ensure the timely delivery of all project outputs to the required standard, and keep stakeholders informed throughout the project.

Outputs

• Project plan
• Project blog
• Dissemination of outputs across IHR communication channels
• Final budget and completion report

Work package 2: analysis of public interface
The analysis work package will encompass the first quantitative and qualitative evaluation from the usability / learnability testing exercises and deliver a prioritised list of recommendations to be fed into the build phase.

Outputs

• Task definitions
• Individual user interviews and task observation
• 4 focus groups from workshop held at the IHR
• System usability scale (SUS)
• Remote testing using VerifyApp
• Case study report of usability issues identified

Work package 3: build phase A
Using report from WP2, extend and build new functions which better support users accessing the public query interface

Outputs

• Build prototype solutions to usability issues, which may include
• Modifications to nomenclature and navigation
• Enhancements to statistical tools
• Creation of graphing tool
• Contextual help
• Addition of devices to better reflect temporal or spatial relationships

Work package 4: analysis of administration interface
Focus within this work package will shift towards identifying scholarly editing requirements using quantitative and qualitative techniques and delivering a prioritised list of recommendations to be fed into the subsequent build phase.

Outputs

• Task definitions
• Individual user interviews and task observation
• Focus group held at the IHR
• Remote testing using VerifyApp
• Case study report of usability issues identified

Work package 5: build phase B
Using report from WP4, extend and build new functions which better support editors creating new content and marking up existing data

Outputs

• Build prototype solutions to usability issues, which may include
• Modifications to nomenclature and navigation
• Tools which look at existing patterns of mark up to identify variance
• Permissions and workflow tools

Work package 6: evaluation
Undertake follow-on research to uncover satisfaction with actual / proposed revisions to services.

Outputs

• (Shorter) individual user interviews and task observation
• Interview with selected user from Focus group
• Continued monitoring of SUS
• Repeat remote testing using VerifyApp
• Report drawing together issues and feedback for remedial solutions

Meet the team

Donna Baillie recently joined the Institute of Historical Research as Project Officer for ReScript. With an interdisciplinary background encompassing qualitative and quantitative research methods in the social sciences as well as experience in media production, Donna is involved in incorporating feedback from user groups and key stakeholders into the design and functionality of ReScript, creating usability tests for the iterative designs, and producing instructional screencasts for the website. She is also responsible for reading through the approximately 1.4 million words of the Parish Clerks' Memoranda from St Botolph’s Church in Aldgate, and key tagging the content for inclusion on the ReScript website.

Jonathan Blaney joined the Institute of Historical Research in 2007 as Project Editor for an AHRC-funded project to complete the digitisation of the National Archives’ Calendars of State Papers via British History Online (BHO). He now continues to work for part of the time on BHO, as well as contributing to a range of other IHR Digital projects. Jonathan has worked as lexicographer for Oxford University Press and as an editor on the Oxford Dictionary of National Biography. He subsequently worked for the Oxford Digital Library, where he was a Text Encoding Reviewer on the Early English Books Online Text Creation Partnership and Eighteenth Century Collections Online Text Creation Patnership, and also advised on a number of digitisation projects.

Jonathan will be involved with the analysis phase of the project, conducting user testing and evaluating the findings. Like all members of the team he will also have some responsibility for dissemination of results.

Bruce Tate has been Project Manager at the Institute of Historical Research working on British History Online since 2002 and has overseen the site since its launch in June 2003 through to the present day. He is accountable for all aspects of planning, budget, timeframe and quality, and is PRINCE2 (Practitioner) certified. He also developed the front end of the JISC-funded Connected Histories website. Previously, he worked for the Audit Commission and commercial publishers including Macmillan and Wilmington.

Bruce is responsible for project managing ReScript as well as for the development work required to incorporate user feedback (two iterations). He also plays a key role in the dissemination of the project findings. Bruce will be very busy over the next few months as he is also leading a Strand B Usability project for British History Online.

Peter Webster is Editorial Controller of British History Online and Manager of SAS-Space, the digital repository for the School of Advanced Study. He also contributes to a range of other IHR Digital projects, and has particular responsibility for the analysis and benchmarking of user behaviour. Before joining the IHR, Peter completed his PhD at the University of Sheffield, and was Technical Assistant to the AHRB Russian Visual Arts Project in the Sheffield Humanities Research Institute. He also taught in the University's Department of History.

Peter will be involved with planning and running remote and in-house usability/learnability testing and will also contribute to dissemination activities.

Jane Winters has been Head of Publications at the Institute of Historical Research since 1999, and of the new IHR Digital since the autumn of 2010. She is responsible for the IHR's publishing and scholarly communications strategy, including the management of a range of research projects focusing on the provision of digital resources for historians. Currently, she is Co-Director of the JISC-funded Connected Histories project; Principal Investigator of the AHRC-funded Early English Laws project to digitise Anglo-Saxon legal texts; and Publishing Editor of the Bibliography of British and Irish History. She is also Executive Editor of the IHR's journal, Historical Research. Before joining the IHR, Jane completed her PhD at King’s College London.

Jane is the Director of the project, and will be particularly involved in the dissemination and writing up of our findings.

Aims and objectives: ReScript usability / learnability enhancement

Goal

To enhance the ReScript interface to match the expectations of a variety of researchers working with very different texts, and with differing levels of expertise. This will make the project more attractive to a range of third-party projects looking for a collaborative editing solution, strengthening its case for sustainability.

Our primary outcome for this project is to learn how both editors and readers will interact with the source data and to produce recommendations for ways in which the identification of issues can be built into the ongoing managerial process. We will also build two iterations of the service, with improving quantitative success metrics, thus demonstrating the project's ability to learn from user insight.

The choice of goal was dictated by the Institute of Historical Research's commitment to supporting the study of innovation in humanities research and the study of history more generally. This means looking at the current level of technical skills/proficiencies among researchers, as well as the range of content types which projects are currently producing. Only by using an adaptive and learnable model will the ReScript project be able to match researchers’ expectations as it is likely that each build phase will, at least in some small way, advance their understanding of both what is required and what is possible.

Success measures

Produce evidence of improved quantitative ratings and qualitative feedback on revised designs in each of the areas under review, in two iterations enabling longitudinal analysis. Second, reflect on the specific conditions under which the tools and techniques used generate the most value.

All of the usability components are employed to baseline performance during the initial analysis phase. After the first build phase, remote testing will be used, but will include both quantitative and qualitative strands, with the intention of comparing the two sets of results. Some tests, such as the SUS, will be used through all consultation phases, enabling a complete longitudinal analysis to be produced.

The measures are clear enough to be understood by different roles within the organisation, i.e., they can be used equally to justify change to business managers and to indicate development areas to the information architect or developer.

Conducting the research within the project means that the ambition of the changes proposed is realistically linked to the amount of resource which the project has its disposal, leading to recommendations that are practicable to implement.

A simpler approach would be to identify and invite a small set of researchers to join a closed list in order to discuss proposed changes. However, it becomes impossible to judge where resources should be assigned to the maximum effect (are two medieval historians better than one early modern?). It would also become difficult to decide on action based on quantitative data if it meant overruling the research group.

The project could also have focused wholly on canvassing either qualitative or quantitative feedback and extended the depth of consultation. However, that would be to assume that what people say and what people do is materially equivalent, which would not necessarily be true.

Approach

The following techniques will be used throughout the project: interviews, remote testing (e.g. click, annotation, labelling of system designs), user groups, and the system usability scale (SUS).

	What people say	What people do
Initial analysis	Individual Interviews User group SUS	Click tests
Between build phases	Annotation tests SUS	Click tests A/B tests
Final analysis	Individual interviews Annotation tests SUS	Click tests A/B tests

The initial analysis phase will result in a number of identified usability issues which will be presented as report cards. The report card device is easily understood and may not only be re-used by other projects, but also serve as a starting point for discussion around usability issues. This may be critical to the widespread recognition of usability as a core component of academic information service provision.