A/B Testing + Experimentation: Rapidly Design UI to Gather Feedback and Avoid Designing in the Dark

February 15, 2024
7
Minute Read Time
Will Breen
Placeholder
Back To All Posts
North Left Arrow Icon

Summary: Discover how A/B testing and rapid experimentation form the cornerstone of effective UI design. By swiftly building and iterating interfaces, we gather crucial user feedback from the outset, ensuring that we never design in the dark. Lean into our latest article to understand the power of data-driven strategies that fuel scalable and efficient solutions for renewable energy startups.

In today's startup landscape, time is crucial to get your product into the hands of users. Many startups face the challenge of moving forward on multiple fronts simultaneously, often overlooking potential solutions right before them. Regarding software, the importance of validating user feedback and insights has never been greater. While designing software may take years and significant financial investment, this drawn-out process is typically reserved for corporations with ample resources. That being said, I firmly believe in taking the time to understand user needs. However, is it justifiable to spend all that time up front when we can achieve the same outcome by putting user data first? Product-led companies should prioritize their customers and users by making data-based decisions rather than getting caught up in semantics.

Consider a scenario where a startup's development cycles or product launches are halted due to UI/UX semantics. In such cases, they are essentially designing blindly, without the validation from actual users that the design time is well-spent. By adopting a more data-driven approach, businesses can propel themselves forward, with users guiding the way toward the ideal next steps. This approach leads to higher product adoption rates, a thriving community of customers, and an exceptional brand experience overall.

The Importance of Speed in UI Design

Companies must move quickly to achieve impactful success in the startup business world. The rapid pace of new technologies and evolving industries necessitates getting your product out the door as quickly as possible. I recently completed a project with a startup called CN-Seamless, which produces state-of-the-art semi-robotic CNC machines. With a perfect product ready to launch and eager customers waiting, the experience design process needed to be expedited. Within a mere two weeks, we developed a fully designed HMI for the machine, enabling the development team to forge ahead with product development.

Once a prototype was ready for use, we initiated the user feedback process by seeking input from individuals outside the company. This allowed us to identify crucial experience improvements we had not considered. By continuing to iterate on the design and test new features before delivering the software to customers, we rapidly adapted based on user feedback.

Fast forward three months after the development process, and the startup secured over $100k in funding within just one month of launching. This injection of funds empowered them to fulfill orders more effectively while continuing to collect user feedback and provide updates.

The key to moving swiftly in your design process lies in your ability to adapt based on user experience and feedback. When reviewing an experience, I always recommend taking a moment to imagine yourself as an ideal user. Ask yourself, "Would I have asked for this feature?" This thought experiment allows you to consider your ideal user's perspective as if they were using your product for the first time rather than someone who is already familiar with the design.

A/B Testing: A Tool for Immediate Insights

The process I completed with CN-Seamless sounds promising, but how can you apply it to your business today? This is a question I often encounter, and my initial response is always "A/B testing." We all understand the significance of A/B testing so that I won't delve into too many details. However, the ultimate objective of successful A/B testing is to compare your current design or build (sample A) with a refreshed approach (sample B) that aligns with your customers' insights.

Now, let's connect A/B testing with its application in your company today. Firstly, collaborate internally with your team. If you have multiple designers or developers, they may already have ideas for improvements. By involving them in this process, with a leadership member as the facilitator, you can foster a more positive attitude towards testing and gradually expand the user base. Furthermore, keeping the initial testing phase within your company will help you identify crucial considerations that may have been overlooked before conducting tests. Select the ideal testing group and request an audit of the current plan. There's a specific approach to this, but I'll save that for another post. Once the audit is complete, share the findings with your design team and refine the concepts for implementation.

After development, compare the refreshed option with your current interface using a select group of customers. Allow customer feedback to validate the need to proceed with the new concepts, explore a different direction, or refine the existing design. If you already have a product design, continuing with the same interviewees as in the previous audit is not recommended. Keep your customers and community-engaged by actively involving them in the feedback collection process to propel your software forward.

Designing with User Feedback: Beyond Assumptions

Some argue that processes like AB testing can be time-consuming, suggesting that designing the product they believe is ideal for users would be more efficient. However, when designing, it is crucial to go beyond our assumptions. How can we create an impactful product if we are unaware of our customer's wants and needs? Many startups need to deliver the product quickly, leaving little room for extensive user testing or design revisions. As a result, they often rely on assumptions about user preferences. Without actively collecting UX feedback, there is a risk of designing in isolation, focusing solely on internal ideas and features rather than listening to the voice of the customers.

Nevertheless, conducting AB testing does not have to be lengthy, especially when there are time constraints. Depending on the scope of the testing, meaningful comparisons can be made within a few hours. When time is limited, it is important not to overcomplicate the testing process. Instead, focus on specific areas of the app that require more insights. Seeking input from teammates who are familiar with the project's goals can provide valuable perspectives. Once the initial version is refined, external feedback becomes essential. There are two main approaches to collecting feedback: quantitative feedback through online surveys and qualitative feedback from unbiased individuals unfamiliar with the project. While online surveys can provide quick insights from a specific audience, they may not yield the most useful information due to their limited nature. Personally, I prefer the qualitative approach, reaching out to friends or colleagues who can offer a fresh perspective on the user experience. Valuable feedback can be gathered within a few hours, allowing for necessary revisions before engaging with the target audience. Design updates can be justified by comparing the original "A" concept with the updated "B" concept based on general feedback.

In my experience working with startups, I have encountered resource constraints that hindered the ability to test different designs. Resource allocation can be a significant challenge for any company. However, collecting feedback does not have to be a stressful process that jeopardizes the project's outcome. As outlined below, starting with a simple "friends and family" approach is a practical starting point when resources are limited. I have often found that team members who are deeply involved in the design or development tend to get caught up in specific details of the solution. Therefore, sharing progress with individuals outside the project for a few hours has proven beneficial in identifying missing elements or unanswered questions that may have been overlooked during the design phase. Even minor user feedback is more valuable than no feedback at all.

By focusing intensely on advancing my HMI design for launch, I can achieve better outcomes right from the start by completing the initial round of design quickly. The first round is not intended to be the final version; it serves as a proof of concept. My aim is to offer innovative UI solutions that are technically feasible at the beginning of the project, motivating the team to progress towards the project's objectives. Subsequently, I dedicate more time to listening to user feedback and considering external perspectives that can validate the next steps for an optimal UX. This iterative design process empowers me to create superior experiences for ideal users, leveraging their insights. If you feel that your HMI or app experience is lacking something, consider seeking a fresh perspective on the issue and be open to surprising ideas that may arise.


Key Takeaways

  1. Start Testing Today: Engaging with a "friends and family" circle can provide crucial early feedback that identifies gaps and enriches the HMI design process, avoiding a narrow focus on specific details by internal teams.
  2. The First Version Isn't The Last: Rapidly completing the first design iteration allows for a viable proof of concept that propels the team toward project goals and motivates further development.
  3. Consistently Test Designs: An iterative design approach, prioritizing user feedback and external perspectives, is critical to validating subsequent steps and optimizing the user experience.
  4. Move Fast With Your Team: Innovation in UI solutions requires openness to fresh ideas and unexpected insights that can resolve deficiencies and enhance the overall app or HMI experience.