Website Usability Testing

BMO Usability Testing

UX Research| User Testing| Data Analysis |Project Management

This is a study organized by Sheridan College in collaboration with the Bank of Montreal

All insights are unavailable due to NDA

Fall 2019

Challenge

How might we improve the online chequing account opening process for low-tech, BMO users.

How might we improve the online chequing account opening process for low-tech, BMO users?
My Role
Project Manager
User Test Facilitator
Data Analyst
Tools Used
Zoom
Miro
InDesign

Goals

Ensure the user path to open an account through BMO is clear and usable for low-tech users.

Provide BMO with usable suggestions to improve their online user experience.

Evaluate how BMO’s chequing account opening experience compares to their competitors’.

Conduct user testing with emphasis on the ease of use and clarity of information provided by bmo.com

14 Weeks
Timeline
Research
Interviews, User journey map
Design
Defining scope, Site map
Mockup
Wireframing , UI design and Style Guide
Prototype
Visual design, Animations
Research Plan
Purpose
Methodology
Recruitment Procedure
Scenarios and task list
Data Collection
Note taking
Task success
Analysis
Graphing quantitative data
Affinity diagram
Findings
Extracting insights
Sorting insights
Revisiting raw data
Research Plan

Methodology

We defined a low-tech user as someone who
had never sent or received an E-Transfer

Research Questions

1.  What are low-tech users’ pain-points related to the online BMO account opening experience?

2. Is it clear to brand new users of the BMO website where they begin the process of opening a chequing account?

3. What information from banks do users most rely on when deciding which chequing account best fits their needs?

4. How do the usability metrics of effectiveness, efficiency, and satisfaction for the BMO’s account opening experience compare to the metrics for experiences offered by Scotiabank and TD Canada Trust?

Scenarios and Task Lists

We used the counterbalancing technique when testing the 3 scenarios to avoid introducing confounding variables
This helped us gather quantitative data
that was easily comparable in all scenarios.

Data Collection

Note Taking

We kept our notes from each test in individual excel sheets, organized by participant, scenario and task, and then combined our excel sheets to create a master sheet of notes. This made them easy to compare and help gain insights.

Analysis

Quantitative Data

We collected Quantitative data through 3 means:
1. A Post-Scenario Questionnaire
2. A Post-Test Questionnaire
3. Task Success Rate

We used these to compare each scenario and extract insights. We also selected specific information to graph to better show the comparison or to emphasize a point. For task success we converted all the results into binary data to determine success rates

   

   

Blurred due to confidential material
Analysis

Affinity Diagram

We converted our session notes from our excel document to Miro -- a collaborative digital white board tool.

The data put into the Miro board were raw notes from our participants. This would allow us to organize the participant notes by category so that we could more easily see where the patterns emerge. Before doing the card sorting, we organized them by participant for ease of reference.

We sorted the collected data into:
1. Primary Themes
2. Sub - Themes

Extracting Insights

Step 1: Go through each sub - category and develop statements that represent the core meaning of the data

Step 2: Compare all insights and highlight the most important points to be considered and further supported by data to aid in the process of presenting findings and design recommendations.

Revisiting the Raw Data

After pulling out the key insights we all re-watched the user sessions to pull more quotes to support our insights. This second round helped us look at the interviews with a different lens; ensuring that we have more than one perspective on the data.

Blurred due to confidential material

Qualitative Insights

We supported each finding with a screenshot, highlighting an example on the website, a relevant quote from a user and a design recommendation to combat the problem. This ensured that each finding was backed up by substantial data and that we outlined each opportunity for improvement.

Learnings and Reflections

Research Plan

Learned that spending more time creating a thorough and well though out research plan results in smoother testing and more insightful results

Research Synthesis & Analysis

Used multiple methods of collecting and processing raw data to gain accurate and meaningsul insights

Actionable Insights

Learned how to connect research insights to opportunities in order to inform future design decisions

Facilitating Testing

Learned how to facilitate testing by leading users through the process while remaining neutral

Other Work

Bird
UX/UI | Motion Design | Brand Analysis
Onboarding Volunteers
UX Design  |  Visual Design
Understanding Barriers to Personal Finance
UX Research | UX Design| Strategic Planning