Guideline

When should you conduct user research?

Web site projects should be based on information derived from user research. Critical points at which to conduct user research are:

1. Gathering initial requirements.When the Web site:

• has no precedent;
• requires a considerable investment in resources; or
• is a key output of a programme entity.

2. Prototyping. If you are releasing a prototype of a system, testing with a small sample of prospective users can give insight into the interface and the logical flow of information. This information is then included in the development or finalization stages. Consider the pre-launch as a prototype and avoid any kind of user test close to this date (except, of course, for quality control of content and functionality). When you are focusing on core content and functionality it is too late to discover key features to be changed or corrected. If you need to do a user test before launching a Web site, plan it well in advance.

3. Monitoring the success of a Web site. There is no need to wait for a formal evaluation to verify the success of your Web site. Adjustments or improvements can be done in small steps, driven by user feedback or unusual patterns in the traffic statistics (for example).

4. Validating the introduction of a key feature or area. If you receive requests for a new feature that will substantially change the site without going through a site redesign.

5. Evaluation. User research is an evaluation technique in itself - its purpose and application changing according to its specific goals at a certain point of a Web site's lifecycle. Sometimes, formal evaluations (external or auto-evaluations) are required. In this case, aim your user research at gathering feedback based on the specific objectives of the Web site or the programme entity that it is part of.

How to conduct user research

Following is an overview of the main steps to plan and implement a user research project. Specific skills are required to implement these methods and analyse the results.

1. Contact OCP for advice, guidance and coordination. OCP conducts user research and needs-assessment. Data is available from previous research and OCP staff can help you with project definition and choice of methodology.

2. Define a user research strategy. Define why you need to conduct user research: is it to evaluate usability or general user satisfaction? Is it about general user characteristics, or what kind of information users look for on the site? Decide what kind of data you want to obtain from your research and why it is needed to solve specific problems.

3. Look for existing sources of information. Sometimes it may not be necessary to start from scratch. Talk with different stakeholders: has anything similar been done before? Who is the person in contact with users (e.g. the Webmaster)? Is the site or system used in the field? Talk with OCP and/or OSP to find out whether other divisions have done something similar (e.g. profiling surveys): is their data applicable to your case as well?

4. Select and recruit users. Where to find them: mailing lists, lists of workshop participants, through the Regional Offices, partners, committee meetings (although not always helpful because people may be too busy), announcements on newsletters and discussion forums. OCP has a list of people who have volunteered to participate in user research.

5. Choose the right method (or the right mix). Each set of issues and research questions requires a suitable method. Also, consider the current stage of the lifecycle of your site and the availability of resources. Here is a list of common, easily adoptable methods to collect user information (Explore other user research methods by stage/resources/access to users at UsabilityNet.):

• Gathering initial requirements

Surveys: good for collecting information about potential users and their needs, especially if you need some quantitative data. It requires good sample identification, selection, proper questionnaire design and statistical analysis (more on surveys at UsabilityNet).

Group brainstorming: the right mix of stakeholders is important to produce as many creative ideas as possible. Brainstorming can be used with stakeholders and users together. Remember to abide by the rules of group brainstorming: everybody suggests ideas building on top of the others, not criticising or discussing details (more on group brainstorming at UsabilityNet).

Interviews: good for yielding large amounts of insight into real-life experience, but also opinions and perceptions. The analysis of the results can be lengthy (more on interviews at UsabilityNet).

Focus groups: a group discussion to elicit opinions and perceptions. A professional moderator is crucial (more on focus groups at UsabilityNet).

Card sorting: a method for discovering the latent structure in an unsorted list of statements or ideas. The investigator writes each statement on a small index card and requests six or more informants to sort these cards into groups or clusters (working alone). The results of the individual sorts are then combined and if necessary analysed statistically (more on card sorting at Usability.Net).

Paper prototyping: high-level schematics of the screens are produced to test proposed tasks, information architecture and design solutions. This is good for interface and interaction requirements gathering before site development (more on paper prototyping at UsabilityNet).

• Prototyping

Surveys: designed to gather feedback on specific features of the prototype;

Usability testing/Observations: good for testing the implementation of the initial concept of the site and the interface design. This yields great insight into how the design will scale to the final site. At this stage, testing with a small number of relevant users (five to ten) is enough: if there are any major problems they will emerge consistently even with small numbers of users.

• Validation/Evaluation

Surveys

Usability testing/Observations

• Monitoring/Gathering user feedback

Webmaster e-mail/form: a well-established basic channel of communication that allows the gathering of user feedback over time. Make sure you have the inbox regularly monitored and respond to inquiries in reasonable time. Collect and classify them by theme or topic in order to identify any problem area.

E-mail updates/newsletters: useful for sending out updates and keeps users returning to the site. Use if the Web site is large or rich in information. You can establish a simple mailing list or use a specific management tool (look for customizable tools where you can introduce profiling questions to be filled out by newsletter subscribers).

Feedback forms: instead of a generic e-mail address, you can set up a feedback form with basic profiling/satisfaction-related questions so that you may gather specific information from your users.

• Evaluate results and store them for future reference.

Consider using user research experts and a statistician if you use a method that yields quantitative data, like surveys.