Why do we design with assumptions rather than data?

Minute Read Time
Will Breen
Back To All Posts
North Left Arrow Icon

Summary: In the constantly evolving landscape of design, the battle between data-driven decisions and intuitive assumption-based design is ongoing. In this article, I will delve into why numerous companies, stakeholders, and "designers" frequently become enamored with their initial product vision, often at the expense of user needs. Additionally, I will explore strategies to remain attuned to user needs and the correct approach to introducing novel concepts. It offers insights into why assumptions, guided by experience and intuition, often lay the groundwork for innovation and exploration in design, leading to fresh, uncharted solutions. I will unfold the rationale behind this strategy, examining its impact and the balance required to harness its full potential while mitigating inherent risks.

The Pitfall of Assumption-Based Design

Startups must operate at a rapid pace to keep pace with evolving industry trends and achieve significant business results. However, fast-moving companies often compromise on quality, overlook processes, and focus solely on innovation. When collaborating with new startups lacking a Minimum Viable Product (MVP) or initial app design, the first set of Human-Machine Interface (HMI) designs heavily relies on assumptions made by me and the team regarding the target demographic's needs and challenges. While the initial design helps introduce the idea to the market and validate the business concept, the process can go awry if subsequent versions of the product are mere iterations of the original HMI. While making assumptions about user requirements during the initial HMI design phase is common, solely depending on these assumptions without market validation can lead to high costs in designing, developing, managing, and marketing an unproven application. There is a concerning trend where companies develop software without seeking input from real users. Imagine a scenario where customer feedback precedes product development – quite revolutionary!

It seems like I must be typing this from another planet.

Let's delve into why designers and stakeholders often become attached to their initial vision of the Human-Machine Interface (HMI). Picture a scenario where startup founders invest significant time and resources to bring their first HMI version to life. Unfortunately, this scenario is common among startups that secure substantial funding to maintain their initial app version. It can be challenging to differentiate the product or HMI from the business and refine the HMI continuously until audience feedback is near-perfect. Even then, audience testing should not halt. When a significant investment is made in a "version zero" concept, it becomes even more challenging to admit that the initial concept is far from ideal, considering the costs incurred. This process often leads stakeholders to prioritize advancing their first concept over heeding user insights for business growth. Cutting costs does not always equate to improved business outcomes, especially in software or HMI development.

The challenge arises when the team, which dedicated time and resources to the project, contributes to exacerbating the issue. "Confirmation bias," the inclination to favor information that aligns with one's beliefs, can be stubborn once established. In working with cross-functional teams, reluctance may arise to repeat similar processes if the team feels they invested significantly in the initial version, only to face additional work rectifying issues revealed during user testing. "Confirmation bias" also plays a role when teams share insights with stakeholders to avoid a comprehensive redesign, even if it is the project's optimal solution. When team members see their role as merely a "task," they may perceive the product as an ongoing project with no clear deadline, leading to a lack of motivation for creating a high-quality initial version. This mindset can result in delays and complications. Continuous product improvement and maintenance are essential, particularly with a dedicated team, especially the founding team. Prioritizing the audience's experience over personal interests is crucial, ensuring decisions align with audience feedback rather than solely stakeholder input. Ideally, both software teams and stakeholders should prioritize customer feedback to advance the project effectively.

The Dangers of Designing in a Vacuum

Starting a business involves numerous risks, but the riskiest move is neglecting the customer. In the past, I used to believe that less experienced designers or product founders had no right to dictate what should be included in a design when collaborating with thought leaders and experts, myself included. This proved to be a quick lesson for me. Working in a team that relies on assumption-led design decisions feels like creating a vacuum where only internal opinions are considered, leading to limited exploration. I tend to exit such roles swiftly because designing based on assumptions can result in significant Human-Machine Interface (HMI) issues and product rejection.

As noted at the beginning of this article and in various other pieces, I frequently emphasize the significance of speed and urgency in the HMI design process. Nonetheless, it is crucial to differentiate between hastening the design of an HMI for audience approval and rushing the design of a "final" product offering. This parallels the examples mentioned earlier; a design process devoid of user input can result in flawed assumptions and product failure. The necessity for swift design iterations and prompt output is not the issue. The crux of the matter lies in a design process that neglects user input during development. Swiftly generating design outcomes is not the hurdle, particularly when my client already possesses a well-established brand or design system. Additionally, speed in design should not compromise quality if I strive for an optimal solution for the audience. When crafting and presenting concepts expeditiously, I aim to refine my designs repeatedly until they resonate with the ideal customer and the primary role the HMI or product could fulfill in the person's desired outcome. Essentially, how rapidly can I devise an optimal solution that aligns with the user's challenges. Let's employ an example to elucidate this subject.

Company A embarked on developing an HMI within a month to complement their web-based controls for existing customers. Despite having an internal design and development team ready, uncertainty loomed over the platform's utility to the majority of its clientele. With a lull in the team's workload, the company designed and developed the web controls. Three months and significant resources later, Company A unveiled its web control, backed by vigorous marketing efforts and customer incentives. However, the returns on this investment were delayed as customer adoption of the tool fell short of expectations due to limited demand for the revised platform among the majority of their clients.

Let's contrast this scenario with Company B, which also develops an HMI within a month to align with web-based controls as an additional service for their existing customers. Company B opts to engage an external firm to ensure swift delivery, though the firm's process differs from what the team previously followed. Nevertheless, they are receptive to new ideas for this offering. Company B has not yet confirmed if the majority of its customers would benefit from an updated platform. They communicate this to the firm, which initiates the design phase. During this stage, the firm requests specific customer information from Company B to present four distinct design concepts they are contemplating before proceeding with development. Following the feedback, the firm discovered that two concepts outperformed the others. As the deadline nears, the firm conducts another round of evaluation, now comparing the top two concepts with a third "combination" concept. Once again, one of the original concepts emerges as the preferred choice during testing. Although the company deems the design less than ideal, they prioritize the customer data and proceed with platform development. Company B introduces the tool within its set timeline, enabling it to extend the new offering to a wider customer base. With the platform now launched, more customers explore it, providing additional insights to enhance the new offering. Company B remains committed to refining the new offering by leveraging customer feedback and fostering ongoing collaboration within its community.

Let's analyze these scenarios. Both Company A and B were highly focused on introducing their new tool to customers, whether or not there was a demand for it. Despite uncertainties, both companies proceeded to invest in the new platform. However, this is where their similarities end, and the distinction between a "successful" and "unsuccessful" launch becomes apparent.

Company A decided to keep the development process internal, relying solely on their team's insights for the new web-based platform. In the end, Company A created a tool that suited some customers but not all. On the other hand, Company B opted to collaborate with an external provider to guide and support their team in implementing the new idea. Despite lacking sufficient customer validation, Company B expressed concerns about the return on their investment. Like Company A, Company B did not verify whether the majority of their customers could benefit from an updated platform. Nonetheless, Company B proceeded with the design phase.

Here is where the significant difference between Company A and B emerges. Company B took the initiative to consult its existing customers about its ideal concept before diving into development. Following the development phase, it "launched" the product with an agile approach, enabling its team to refine and innovate based on customer feedback continuously. This iterative process aimed to create an optimal tool that effectively addresses the customers' needs.

Company B started collecting audience insights as soon as possible, while Company A chose to wait until they had their "perfect" vision ready to launch. Company B was successful with the launch because they found out the customer's preferences early in the process and prioritized those needs over the ideas the internal team held true.

The process described is more challenging in practice than in theory. It requires a strong dedication to customer success and ensuring positive user experiences by initiating early and frequent sharing of work with existing or potential customers. The key concept I aim to emphasize here is the rapid development of a Human-Machine Interface (HMI) to validate the product with users. While I avoid presenting customers with mere pen and paper sketches, I can still stimulate their ideas by introducing low-fidelity concepts, which clients typically prefer reviewing before transitioning to high-fidelity designs. My objective is to solicit feedback from the audience as soon as possible and as frequently as possible to continually validate the investment in the optimal solution for the customer rather than solely for the business.

Agile and Iterative Design: A Solution to Assumption Bias

Design is far from a straightforward, step-by-step process. It's a messy endeavor that demands getting your hands dirty to achieve exceptional results. Embracing an agile design methodology is key to delivering a positive customer experience. Agile Human-Machine Interface (HMI) design offers a flexible, iterative approach to crafting interfaces that align seamlessly with the agile nature of software development. Within the realm of HMI design, this strategy encourages continuous improvement and fine-tuning of design elements based on user input and iterative adjustments throughout the development process. The term "agile" is often thrown around liberally in the industry, with many claiming adherence to the approach without truly embodying it, instead relying on traditional methods steeped in theory rather than proven practice.

From Jeremy Bird on LinkedIn

To set my product up for success, a key aspect is an agile approach to its delivery. As I mentioned earlier, the aim is not to delay the initial design version's release to the public but to ensure it's not the final iteration.

Recently, I came across an intriguing point in LinkedIn comments discussing the distinction between "user testing" and "product testing." While the debate was mostly about semantics, it raised a valid point. When I refer to "user testing" or "user feedback," my focus isn't solely on users understanding how to use my design but rather on the design's impact on their workflow, outcomes, and overall productivity. This process aligns more closely with "product testing" as it aims to gather insights on how the product performs within a user's workflow, not the other way around.

I've collaborated with hundreds of startups over the past five years, and the key difference between successful and unsuccessful ventures lies in their approach to design. The unsuccessful ones tend to operate in isolation rather than engaging with their audience. Recently, I caught up with founders from past projects, and one particular success story stood out. This startup merged with another, doubling its initial valuation and securing millions in new funding for a redesign and redevelopment. When I inquired about audience insights, the founder shared their own experiences and suggestions from the other business leaders involved in the merger. Unfortunately, relying solely on internal, secretive designs will hinder progress. It's crucial for startups to prioritize user feedback post-launch, moving away from internal stakeholder ideas. While a rapid initial product release may align with internal perspectives, ongoing success hinges on user-centric design decisions.

I think I've explained enough about how important user feedback is, but how can the process not slow a startup down? I'm glad you asked!

The solution is straightforward, yet the day-to-day implementation poses a real challenge. While I won't delve into specific methods for gathering user feedback today, I will share how I streamline the process for my team. I emphasize the importance of integrating the collection of audience insights seamlessly into the design process itself. Many startups have systematized their design and development workflows to optimize every moment and avoid delays in decision-making. However, the flaw lies in the internal nature of this process; stakeholders propose new features, which are then handed over to the engineering team, triggering a back-and-forth with the design team for updates. This cycle perpetuates a system. In a nimble startup environment, I advocate prioritizing the integration of a system or "systemizing" UX feedback within the existing design workflow. Rather than keeping design and feedback collection separate, I aim for a cohesive design process that incorporates feedback loops organically.

NN Group on Design Thinking 101

Referencing the image from Nielsen Norman Group, amidst its complexity, my focus is drawn to the arrows positioned at the top. These arrows, all leading back to the concept of "empathize," as defined by NN Group, involve the act of directly observing user behavior, thoughts, and desires. Delving into questions like ‘What motivates or discourages users?’ or ‘Where do they encounter frustration?’ is key. The essence lies in accumulating ample observations to connect with users and comprehend their viewpoints genuinely. Notably, the graphic underscores empathy's significance, a fundamental aspect that persists even post-launch.

When audience feedback becomes a collaborative process within the team, the task integrates more effectively into the overall outcome with validation from the most suitable individuals. As previously mentioned, the method of gathering insights is not critical, but one significant aspect needs highlighting. All the feedback I receive from the audience is purposeful, whether direct or indirect. This means I aim to gather insights directly and indirectly from product users. I often encounter numerous "survey" requests in a day from companies seeking indirect feedback on their products, offers, or processes. These responses are considered indirect as users provide insights based on a standard set of questions. While these insights are valuable and contribute to positive outcomes, they represent only part of the diligence. Direct feedback, on the other hand, involves seeking insights on specific product areas through interview-style data capture. By combining both indirect and direct feedback approaches, I ensure that a comprehensive mix of qualitative and quantitative insights guides the team's next steps.

The "Empathy Gap"

I'm sharing the NN Group graphic above because I appreciate their use of "Empathize" as a crucial part of the design process. In many of my projects, stakeholders often focus on designing features they want in an app rather than what the audience desires. A collaborative team should engage with designers and developers who not only execute tasks but also explore new ideas for improved business and audience outcomes. It's common for stakeholders or founding teams to get overly excited about progress or design and request more outcomes, features, and ideas.

The most significant mistake I observe among stakeholders arises from their belief that they know everything about a market due to their "expertise" or tenure in the field. Despite being a "design expert," I seek feedback from individuals unfamiliar with design to evaluate my work. I consider this feedback to determine what to retain, revise, or remove to enhance the audience experience and business outcomes. I wouldn't consider myself a "design expert" if I disregarded feedback on my designs and didn't strive for a more positive outcome on subsequent attempts. I recall a project where a client engaged me to assess their product and provide actionable insights. After completing the process and sharing the insights, the client instructed me on the design, their preferences, and my perceived errors. This is not to criticize the client; rather, it highlights that the client or stakeholder failed to acknowledge an expert's insights, let alone seek input from actual users regarding the initial product features.

Audience testing goes beyond post-launch and regular product users. For optimal results, I aim to gather specific insights through user interviews and numerous quick insights that pinpoint the core problem/solution. Teams can narrow their empathy gap by engaging in simple, focused conversations with product users. The objective is not to overwhelm the team with extra work but to encourage open discussions with active users. The approach doesn't need to be overly complex; simplicity with swift action is key. Acquire insights and implement them for ultimate success.

Before I end things, I'd like to share one last interesting point someone made when I asked them about this topic. It's the "Fear of the Unknown." Explained as the fear of user research potentially revealing negative feedback or challenging previous assumptions. And I have a simple answer to this limiting belief.

As a founder, I ventured into a risky business, and the ongoing journey poses continued risks. Gathering customer feedback isn't a gamble; it serves as a safety net. While it might bring forth negative outcomes, a single misstep doesn't spell doom for the business. The perceived risk associated with "building in public" seems outdated unless handling top-secret government projects, where consulting other officials remains prudent. Take, for instance, the recent case of Artificial Intelligence and Google's Gemini mishap, which involved replacing historical photos with "culturally appropriate" AI-generated images. Despite the initial backlash, Google rectified Gemini based on feedback, and users continue to benefit from the tool. The concept of getting "canceled" only materializes when one halts sharing and seeking improvement. If a stakeholder craves control, pivot discussions towards justifying the business's continuity instead of relying on familiar assumptions. It seems implausible that any stakeholder would dismiss insights from individuals willing to invest in ideal solutions. Remember, there exist numerous ways to gather insights, but choosing not to capture any isn't a viable option. It's time to dismantle the notion of operating in isolation and embrace innovative methods to elevate the business experience.

Conclusion: Cultivating a Culture of User-Centric Design

In a rapidly evolving business landscape, the significance of customer feedback in shaping products and services cannot be overstated. Embracing user-centric design principles is a strategy and a necessity for survival and growth. This reduces risks and strengthens the company's market position, ensuring products stay relevant and valuable to users. It's evident that sustained innovation and business success are achieved through the insights and experiences of those we serve. Stakeholders must promote a culture that values user engagement and fosters an environment where feedback drives development and success.

Key Takeaways

  1. Prioritize Continuous Feedback: Establish mechanisms to gather constant feedback from users. This ensures that products and services are continuously refined to meet the evolving needs of customers, thereby staying relevant in the market.
  2. Foster a User-Centric Culture: Integrate user-centric design principles into the company's culture. Encourage all stakeholders to value and seek out user feedback, making it the cornerstone of development strategies to drive innovation.
  3. Leverage Insights for Agility: Use customer insights to enhance business agility. The ability to quickly adapt to consumer needs, as demonstrated in the narrative of Google's Gemini, is crucial for staying competitive and fostering growth.
  4. Break Down Isolation Barriers: Move beyond operating in isolation by engaging with a wide array of stakeholders for diverse perspectives. This approach bolsters the business's ability to identify more innovative solutions and elevates the overall business experience.