Collecting real user research for product success, justifying design updates

Minute Read Time
Will Breen
Back To All Posts
North Left Arrow Icon

Summary: In this article, you'll get an exclusive glimpse into our data-driven approach that swiftly empowers startups to turn their ideas into reality. I'll delve into the proven process I've implemented for previous clients and present a completed case study where we prioritized quality and speed, guided by valuable user feedback. Read on as I unveil the transformative strategy...

I used to think that only designers could create app experiences. After all, how could someone without any UI or graphic design knowledge possibly know how to design an app effectively? However, I now realize that my belief was only partially correct. While many people may not have the technical skills to design an interface, they clearly understand what they want from the overall experience.

The only valid justification for updating a design is through user research. It's important not to make hasty changes based solely on the opinion of a designer like M, who may not represent the ideal audience. However, suppose genuine users like J and their extensive network of friends consistently complain about the color of a button. In that case, it becomes necessary to consider an update to address their feedback.

Here's a definition from ChatGTP about what "user-based data" is:

"User-based data" refers to information or feedback gathered directly from real users of a product or service. It includes insights, opinions, and observations provided by individuals who belong to the target audience or user group. This data is valuable for making informed design changes because it reflects the actual experiences and preferences of the people who interact with the design, ensuring that updates are driven by real user needs and feedback rather than assumptions or opinions from non-users.

I collaborated closely with the RockShox team to compile and gather user research. Our goal was to evaluate the company's app and propose ideal next steps. Although the current app had approximately 3k active daily users, the experience was subpar and did not align with the business objectives. Some users mentioned the lack of simplicity, while others indicated a lack of awareness about the app.

To summarize, I successfully collaborated with the team to conduct extensive user research and gain a deeper understanding of their needs. Throughout this process, I conducted multiple rounds of research using various methodologies. This involved engaging with hundreds of individuals in person and an additional three hundred through online surveys. The insights obtained from this UX research were instrumental in justifying the client's next steps to company stakeholders and informing project planning.

So, how can you justify implementing new design updates? One way is to examine user feedback more closely. Let's dive deeper into this topic.


I began my collaboration with the RockShox team, delving into the purpose of their existing app and exploring why it didn't quite meet the user's needs as effectively as possible. Over 48 hours, we worked closely with the client in person, conducting a thorough discovery process to uncover the current state of affairs and pave the way for our next steps.

Once I better understood the product, audience, use cases, and pain points, it was time to identify the real issue. While we could explore new solutions and brainstorm fresh features endlessly, it would be ineffective if the user did not require these revisions. RockShox found themselves in an intriguing position. They recognized that the app was "working" for their audience but fell short of meeting everyone's expectations. The team and stakeholders acknowledged that the app's experience did not align with the audience's needs, yet struggled to justify the cost of an extensive update without solid user feedback to provide evidence for the findings. How can we effectively showcase the need for an upgrade in a compelling manner?

We outlined specific areas of the experience where we sought more user insights. This focus allowed us to ask precise questions that would contribute to addressing the pain points faced by ideal community members. To commence the research process, I recognized the need to gather qualitative (descriptive) and quantitative (numeric) insights.

Qualitative Research Process:

To initiate the research, I identified local events that would attract the ideal audience. As we conducted this research towards the end of the year, a popular time for the RockShox audience, several local events were available for us to attend and gather valuable insights. Despite the limited timeline of less than four weeks, I collected insights from two major audience events, resulting in over 25 in-person interviews about the app. The beauty of this research method lies in the opportunity to engage in one-on-one conversations with audience members, each offering a unique perspective about how they use the app. While there were overlapping issues, I recorded distinct viewpoints that provided fresh ideas and insights into the app's realistic usage.

The primary objective of this research process was to immerse myself in the perspectives of ideal audience members within a realistic environment where they could articulate their experiences clearly. During the debrief conversation, a significant observation was made regarding discussions on inclusivity within the mountain biking community. This included concerns expressed by a blind rider about inclusivity and the potential for app features catering to disabled riders. Positive feedback was received regarding the app's ability to impact the riding experience and suspension. Altogether, these findings underscored the importance of inclusivity and the app's potential to enhance the mountain biking experience for a diverse range of riders.

Quantitative Research Process:

To complement our in-field research, I conducted a broader survey requiring us to ask specific yet inclusive questions that anyone could answer, regardless of their background. While it may not sound like we were setting ourselves up for success, we had confidence that the target audience had a basic understanding of RockShox's offerings. Our objective was to collect responses that quantitatively justify the design updates rather than relying solely on extensive insights or our initial "new feature" ideas. In fact, we further included these "new" features in the survey to justify the project's optimal next steps.

Fortunately, we achieved our desired outcome. Within just five days, we received roughly 300 responses, providing us with a comprehensive perspective on the community's outlook regarding the future of the application's service. To ensure diverse responses, we engaged with specific audience groups. The findings revealed significant interest from individuals who wanted a feature showcasing their previous interest in products. Additionally, almost half of the respondents hesitated to manually input numbers, even if it would enhance their overall ride experience. These specific insights and outcomes formed the basis for justifying the implementation of new features.

Apply To Your Business:

Developing a well-defined approach for engaging with the audience, such as the process above, is crucial to achieving a successful research outcome. To begin with, I have identified key objectives we aim to accomplish. In this case, I have outlined high-level questions, including:

  1. How can we encourage non-customer audience members to utilize the app?
  2. What strategies can we employ to transition larger persona groups (not yet users) into the next stage and convert them into buyers?
  3. How can we increase the number of users within our target audience who choose our brand?

We can streamline our efforts and optimize results effectively by starting with design-thinking questions like these. Next, I provided a list of specific questions to ask the audience. This process involved using a Likert-type scale survey, allowing respondents to complete it quickly. At the end, we also included an option for open-ended feedback to gather additional insights. These questions included:

  1. How frequently do you have your product serviced?
  2. When I use my product, it perfectly aligns with the experience I desire.
  3. I am fully aware that upgrade options are available for many products without purchasing a new one.

While these questions may appear broad, I tailored them by replacing keywords such as "product" or "service" to align with the client's specific product area, most commonly related to suspensions. However, you can modify these keywords to suit your desired outcome and direct the survey's questions accordingly.


After completing qualitative and quantitative research processes, I outlined the findings and presented them to the team. I designed visualizations with figures and bold numbers for the quantitative research to highlight the most popular outcomes. As for the qualitative findings, I summarized my notes and conversations, including any overlapping perspectives that aligned with our desired outcomes.

Next, it was time to establish the link between the research findings and the reasoning for the upcoming app updates. It's important to note that this was a crucial part of the stakeholder presentation, so the information needed to seamlessly connect during my delivery. I curated this information to highlight the priority of each feature based on its potential value to the user, incorporating insights gleaned from qualitative and quantitative research. Additionally, I included a visual "scale" to visually demonstrate the range of value the feature could bring to the business and user, as well as the complexity involved in implementing the update.

I created an example design to outline the presentation I mentioned earlier. I utilized this design to deliver a presentation to RockShox and received positive feedback regarding the outline.

Presenting the priority of app updates or feature requests in this manner offers stakeholders a comprehensive view of the necessary information to make informed decisions. They can easily grasp the priority level, understand the data-driven justifications, and evaluate the value versus cost required to bring the feature to fruition.

By incorporating justification into design updates, businesses can avoid developing new features without first justifying their value. For instance, companies like RockShox can prioritize creating features that matter most to their audience. This approach ensures that time and resources are invested wisely in designing and developing features people want to use while avoiding unnecessary costs. No more designing "in the dark"!

In the past, I used to believe that by simply using the right colors and fonts, people would love using the app as long as the core features were present. However, I have come to realize that this perspective is incorrect. As a designer, my role goes beyond providing execution; it is crucial to prioritize the humans' requirements and design in a creative and unique manner that is accessible to the audience.

My goal is to back my design updates with data from ideal community members within an audience. We can only rely on assumptions about our apps up to a certain point within a project's lifecycle. We must listen to what our users want regarding features, interface, and user experience design. My primary focus is to help my clients launch their apps quickly. The conversations we had before the launch were essential, but the real test came after the launch when I could design the app's experience based on data-backed audience research. This helps justify the need for features, design enhancements, and technical updates.

Attempting to complete this process without testing with real people does not set my product's design up for success. To ensure a successful outcome, involving real users in the testing and improvement process is vital.

Key Takeaways:

  1. Utilize Data-Driven Design: By integrating realistic user feedback and making data-backed decisions, product owners can enhance feature relevance, design, and functionality, ensuring the product truly resonates with its intended users.
  2. Embrace Testing and Iteration: Continuous user testing post-launch is indispensable. Product owners can gain critical insights and make iterative improvements, driving the success of their product by genuinely addressing the users' needs.
  3. Speed to Market Matters: Launching quickly gives you a competitive advantage. However, this shouldn't compromise the quality or user experience. Use the initial deployment to gather user data for further refinements. Product owners should consider spending the least amount of time with the first version and more time understanding user feedback.
  4. Quality without Compromise: While efficiency and prompt delivery are valued, maintaining high standards in product quality is crucial. Avoid the pitfalls of rushing development at the cost of user satisfaction by integrating internal or external feedback loops into every stage of the build process.