Usability Testing of RIT's Website

Project Overview

During the project-based HCIN 6301 Usability Testing course in Fall 2022, we worked in groups to plan, design, and conduct a formal evaluation of a user interface. Throughout the course, we collaborated with a client under the guidance of our instructor, Dr. Charles Schneider.

My group worked with Raman Bhalla, Associate Vice President of Web Services at the Rochester Institute of Technology, as our client. The task was to conduct a comprehensive usability evaluation of the website, particularly for content targeted at prospective college students. Under Dr. Schneider’s guidance, we began with a market analysis and heuristic evaluation using Jakob Nielsen’s 10 usability heuristics for interaction design. We then used these findings to plan and design a usability test that included pretest and posttest surveys, SUS questionnaire, as well as virtual contextual inquiries, with the objective of evaluating the overall user experience, including wayfinding, and identifying pain points with completing tasks relevant to the identified user group.

Timeline

Research Questions

These are the main questions we aimed to answer:

  • Task completion: Can participants find and request additional information about the university, schools, and programs? Can participants submit requests and schedule visits? How satisfied are they after each task?
  • Wayfinding: Do participants use the search function or the navigation bar when browsing for information? How satisfied were they with the option they chose?
  • Satisfaction: Are they satisfied with the website overall? Are there any main frustrations?

Methodology

To answer these questions, we conducted the test with six participants (high school students and college sophomores). While this sample size provided valuable qualitative insights, all participants’ affiliation with RIT may have influenced their satisfaction ratings. We observed them complete the following five tasks while measuring the number of clicks, qualitative data from think-aloud protocols, and task success rates.

Tasks and Scenarios

Task 1: Find the Mechanical Engineering BS degree page.

  • Scenario: Imagine you are a high school senior with a passion for race cars and physics. You even spent time helping a friend of your father’s at their auto repair shop last summer. It is your dream to one day work as a mechanical engineer for a company like Toyota. You have heard RIT offers a mechanical engineering degree, but you are not sure what classes are needed to fulfill the degree requirements, so you head to rit.edu to find out more.
  • Success criteria: The participant lands on the Mechanical Engineering webpage.
  • Success rate: 100% completed, 6 of 6 participants

Task 2: Find the Engineering Co-op webpage on the RIT website.

  • Scenario: Imagine an admissions counselor told you that all engineering undergraduate programs require at least one co-op. You have never heard of a co-op before. So you head to rit.edu to find out more.
  • Success criteria: The participant lands on the Engineering Co-op webpage.
  • Success rate: 0% completed, 0 of 6 participants

Task 3: Find and identify the Electronic Gaming Society (EGS) club on the RIT campus groups page.

  • Scenario: Imagine you enjoy playing Minecraft on Xbox with your friends at home, where you grew up. You hope to find a similar group of friends if you attend RIT. You want to confirm a specific club on campus that shares the same interests as you, so you head to rit.edu.
  • Success criteria: The participant identifies the moderator and the club’s name, Electronic Gaming Society, while on the campus groups webpage.
  • Success rate: 50% completed, 3 of 6 participants

Task 4: Book a campus tour for December 27, 2022, starting at 11 am on RIT’s campus.

  • Scenario: Imagine over your winter break, your parents have scheduled a trip to visit Western New York so you can see the surrounding colleges. During this trip, you want the first stop to be RIT, so you head to rit.edu to schedule a guided tour for December 27th.
  • Success criteria: The participant fills in the entire form of Undergraduate Campus Visit for December 27th, 2022, at 11 am, and presses the submit button.
  • Success rate: 66% completed, 4 of 6 participants

Task 5: Find a way to book a virtual Q&A session with an admissions counselor.

  • Scenario: After you visit the campus, you realize you still have questions about the admission process. In an information session you attended, you remember the presenter saying to speak with an admissions counselor virtually. So you head to rit.edu to find out how to schedule an appointment with an admissions counselor.
  • Success criteria: The participant arrives at the form that will allow users to book a virtual appointment with an admissions counselor.
  • Success rate: 0% completed, 0 of 6 participants

Key Findings

Wayfinding Challenges

The low success rates (0-66%) for three of five tasks in the test indicate the need for improving the user interface design of the website. Especially around wayfinding, participants had difficulty locating information and forms to complete tasks.

Search Widget Usage

Participants relied on the navigation menu and neglected the global search widget for all tasks. This might be due to the lack of visibility of the global search widget and confusion between the global search and the degree search widget, which are both located close to each other on the home page. For task 1, most participants relied on the degree search widget to find the academic program and also reported high perceived ease of use and satisfaction. For tasks 2-5, students relied heavily on the navigation widget, which made task completion cumbersome and prone to finding roadblocks. Participants weren’t familiar with the sitemap, so they had to do trial and error to find the information path that would take them to the desired page where they needed to complete the task or find some information. All of this indicates that making the global search widget more prominent and optimizing it could enhance the usability of the website.

Satisfaction vs. Performance Gap

Despite the difficulties observed during the tests, participants reported a high level of satisfaction with an 80.42 SUS score, which is slightly above the industry benchmark of 80. The high satisfaction rate reported by participants could be due to favorable biases toward RIT, given that all participants are part of the RIT community. Additionally, the SUS questionnaire was administered live right after the contextual inquiries; participants may be more hesitant to rate their satisfaction in front of a group honestly. A potential solution to the latter is to distribute a link to the SUS questionnaire to be filled out by the participants anonymously and online.

Recommendations

Additionally, we identified two main elements for further exploration and testing:

  • Make the global search widget more prominent on the home page and optimize it
  • Increase the visibility of important links within user flows

Future Research

Future research should explore why the majority of tasks had a low success rate and test if improvements to the global search widget design by making it more prominent on the home page and optimizing the search engine contribute to higher success rates. This can be done through an A/B test with a larger pool of participants and expanding reach to those outside the RIT community. The complete process and results are documented in HCIN6301 Usability Evaluation of the RIT Website Report.

Last updated: September 29, 2025