Heard us on
The AI Daily Brief Podcast?

Move from AI ambition to coordinated execution in 30–45 days.

Product, not PowerPoint: How to Evaluate Enterprise AI Partners 

A practical framework for enterprise AI vendor selection that prioritizes functional product. 

There is a simple truth in basketball: when someone claims they can dunk, you do not want their biography. You want to see them take off, rise above the rim, and throw it down. Until the ball goes through the hoop, everything else is just pregame chatter. 

Traditional business pitches are no different. Slide after slide explaining talent, process, and commitment to excellence. Everyone insists they are fast, strategic, and powered by artificial intelligence. It all blends together. 

And just as in basketball, none of it matters until you see the dunk. 

Why Enterprise AI Partner Evaluation Has Changed 

I have spent the last year watching something shift in how enterprise buyers evaluate technology partners. The change is not subtle. AI collapsed the timeline for what is possible. Engineers use artificial intelligence to automate repetitive tasks, reveal gaps, and support rapid iteration. User experience teams model real behavior and refine interactions in a fraction of the usual time. Designers explore and adapt visual directions quickly while matching a client’s brand and needs. At the strategy level, artificial intelligence helps teams explore concepts, identify edge cases, and clarify problems before anyone designs anything or writes code. 

Teams can now build first versions far earlier than they once could. It is now possible to walk into a meeting with something real, rather than something hypothetical. 

Traditional Evaluation Arrives Too Late  

Yet enterprise evaluation still moves as if early builds take months. Teams can create quickly, but organizations are asked to decide slowly. Forrester’s 2024 Buyers’ Journey Survey reveals the scale of this shift: 92% of B2B buyers now start with at least one vendor in mind, and 41% have already selected their preferred vendor before formal evaluation even begins. Traditional vendor selection leans on slides that outline intent, case studies that point backward, and demos that highlight features. These keep judgment at arm’s length and often arrive too late to matter. 

An early milestone changes that dynamic. A deck explains. A first version proves. 

What Functional Products Reveal About AI Vendors 

A healthcare technology company came to us through a partner referral. They needed to modernize their pharmacy network’s web presence, which included hundreds of independent pharmacy websites, each with unique branding and content, all needing migration into a modern, SEO-optimized content management system. They had already sat through multiple vendor presentations that week. Each promised speed, AI capabilities, and transformation. 

At Robots & Pencils, we stopped presenting what we could do and started showing what we already built. 

Building the Functional Product in 10 Days 

Our team had a week and a half. Our engineers used AI agents to automate content scraping and migration. Our UX team modeled user flows and tested assumptions in days instead of weeks. Our designers explored visual directions that preserved each pharmacy’s brand identity while modernizing the experience. Our strategy team identified edge cases and clarified requirements before a single line of production code was written. 

We walked into the meeting with a functional product. 

The Client Demo: Testing Real Data in Real Time 

The client entered one of their pharmacy’s existing URLs into our interface. They selected brand colors. They watched our AI agents scrape content, preserve branding structure, and generate a modern, mobile-responsive website in real time. Within minutes, they were clicking through an actual functioning site built on a production-grade CMS with an administrative backend. This was not a mockup or a demo, but a working system processing their real data. 

The entire conversation shifted. They immediately started testing edge cases. What about mobile responsiveness? We showed them the mobile view that we had already built based on pre-meeting feedback. What about the administrative interface? We walked them through the CMS backend where content could be updated. They stopped asking, “Can you do this?” and started asking “What else can we build together?” and “How quickly can we expand this?” 

After the meeting, their feedback was direct: “I appreciate the way you guys approached us. Going through the demo, it wasn’t just this nebulous idea anymore. It was impressive from a build standpoint and from an administration standpoint.” 

Why Early Functional Products Prevent Partnership Failures 

When clients see a working product, even in its earliest form, they lean forward. They explore. They ask questions. They do not want to return to a deck once they have interacted with actual software. And this is precisely why the approach works. 

Most enterprise partnerships that fail do not fail because of weak engineering or design. They fail because teams hold different pictures of the same future, and those differences stay hidden until it is too late to course correct easily. A shared early version fixes that. Everyone reacts to the same thing. Misalignments surface when stakes are low. You learn how a partner listens, how they adjust, and how you work through ambiguity together. No deck presentation can show these things. 

How Early Functional Delivery Transforms Vendor Selection 

The Baseline Iteration Before Contract Signing 

At Robots & Pencils, we think of this functional product as more than a prototype. It is the baseline iteration delivered before contract signing. It shapes how the partnership forms. The client comes into the work from the start. Their data, ideas, and context shape what gets built. 

Why This Approach Stays Selective 

Because this early delivery takes real effort and investment on our behalf, we keep the process selective. We reserve early functional product development for organizations that show clear intent and strong alignment. The early artifact becomes the first shared step forward, rather than the first sales step. 

The Lasting Impact on Partnership Formation 

When you start by delivering something meaningful, you set the tone for everything that follows. The moment that first version hits the court, the moment you see the lift, the rim, and the finish, the entire relationship changes. 

In the end, the same lesson from basketball holds true. People do not remember the talk. They remember the dunk. And we would rather spend our time building something real than explaining why we could. 

If you want to explore what it looks like to begin with real work instead of a pitch, we would love to continue the conversation. Let’s talk. 


Key Takeaways 


FAQs

How long does early functional delivery take to create? 

Early functional product delivery typically takes 5-10 days, depending on complexity and data availability. At Robots & Pencils, we focus on demonstrating how we interpret requirements, handle real constraints, and collaborate under actual conditions rather than achieving feature completeness. 

What makes this approach different from a proof of concept? 

Unlike traditional proofs of concept, our baseline iteration is built with the client’s actual data and reflects real-world constraints from day one. It demonstrates partnership dynamics and problem-solving approach, not just technical capability. 

Which types of organizations are best suited for this approach? 

Organizations that show clear intent, strong alignment on objectives, and readiness to engage collaboratively benefit most from early functional delivery. This approach works best when both parties are committed to testing the partnership through real work rather than presentations. 

Can this approach work for regulated industries like healthcare or financial services? 

Yes. We’ve successfully delivered early functional products for healthcare technology companies and financial services organizations. The approach adapts to industry-specific requirements while maintaining rapid delivery timelines. 

Robots & Pencils Opens Studio for Generative and Agentic AI in Bellevue

The Seattle-area AI Studio is live, growing, and hiring engineers and builders ready to deliver impact at velocity. 

Robots & Pencils, an applied AI engineering partner known for high-velocity delivery and measurable business outcomes, today announced the opening of its Studio for Generative and Agentic AI in Bellevue.  

Candidates seeking high-impact engineering, data, and design roles can learn more at robotsandpencils.com/careers. 

A Strategic Expansion to Meet Demand for Rapid Enterprise AI 

The Studio in downtown Bellevue is fully operational and actively building its founding team as enterprise demand accelerates for AI systems that move from experimentation to production with speed, precision, and accountability. 

The Studio expands Robots & Pencils’ AI-native delivery model and represents a significant step in the company’s U.S. growth, supported by global operations in Cleveland, Calgary, Toronto, Bogota, and Lviv. It adds meaningful capacity to support organizations launching AI-enabled products, platforms, and agentic systems at scale. 

Strong Leadership Driving Focus and Velocity 

The Studio in Bellevue operates under the leadership of Jeff Kirk, Executive Vice President of Applied AI at Robots & Pencils, and reinforces the company’s growing presence in the Pacific Northwest while serving global clients pursuing ambitious AI initiatives. 

“This Studio is designed for builders who want real ownership and real impact,” said Kirk. “We are bringing together experienced teams who move quickly, think clearly, and take responsibility for outcomes. Our Studio model gives people the trust and focus to make strong decisions and deliver AI systems that translate directly into business value.” 

Working with AWS to Accelerate Enterprise AI Delivery 

As an Amazon Web Services Partner located near Amazon headquarters, the Studio in Bellevue supports clients building and scaling AI solutions on Amazon Bedrock, Amazon SageMaker, Amazon Bedrock AgentCore, Amazon Quick Suite, and related AWS services. This proximity strengthens collaboration and supports faster experimentation and production-ready delivery for complex enterprise environments. 

Robots & Pencils was recently selected as one of 11 inaugural partners in the invite-only AWS Pattern Partners program. The program works with a select group of consulting partners to define how enterprises adopt next-generation AI and emerging technologies on AWS through validated, repeatable patterns. 

This recognition acknowledges Robots & Pencils’ experience delivering production-grade AI architectures for enterprise customers. Working with AWS, the company supports secure and scalable AI delivery across regulated and high-impact industries while enabling teams to move with clarity and confidence from design through deployment. 

A Destination for Elite AI Builders 

The Studio for Generative and Agentic AI reflects Robots & Pencils’ long-standing commitment to talent density and engineering craft. Employees average fifteen years of experience and contribute patents, published research, and category-defining products across industries. The Studio in Bellevue offers engineers, applied AI specialists, product leaders, and user experience innovators the opportunity to shape a new hub while influencing high-stakes client work from the ground up. 

“To support our substantial client demand, we need incredible GenAI talent and are significantly investing in how we work with AWS. Our Bellevue AI Studio places our teams in close proximity to AWS, creating an environment that supports knowledge sharing and enables us to tap into the Seattle-area hot bed of incredible, wicked-smart talent,” said Len Pagon Jr., CEO of Robots & Pencils. “The Bellevue location expands our ability to deliver applied AI outcomes at scale while creating an environment where experienced builders can do the most meaningful work of their careers. This expansion reflects confidence in our teams and the direction we are taking the company.” 

Velocity Pods Deliver AI Products in Weeks 

Teams in the Studio operate in industry-focused Velocity Pods supporting Education, Energy, Financial Services, Healthcare, Manufacturing, Transportation, and Retail and CPG. These pods launch AI generative and agentic products to market in 30-to-45-day cycles while addressing complex modernization and intelligent automation programs across the enterprise. 

Now Hiring for AI Engineering Jobs in Bellevue 

Robots & Pencils is actively staffing the Studio for Generative and Agentic AI in Bellevue and invites experienced engineers and builders to apply. Open roles span engineering, applied AI, product, and design. 

Interested candidates can explore opportunities and submit applications at robotsandpencils.com/careers. 

The Studio in Bellevue opens with momentum, leadership, and a clear mandate to build AI solutions that matter.  

The pace of AI change can feel relentless with tools, processes, and practices evolving almost weekly. We help organizations navigate this landscape with clarity, balancing experimentation with governance, and turning AI’s potential into practical, measurable outcomes. If you’re looking to explore how AI can work inside your organization—not just in theory, but in practice—we’d love to be a partner in that journey. Request an AI briefing

How I Learned to Stop Worrying and Love AI Code: A Designer’s Journey 

How Designers Are Using AI Code Tools: From Figma to Functional Prototypes 


The team Zoom call felt like an intervention. “Just try it,” they said. “Everyone open Claude Code.” My palms were sweating. Twenty years of visual design, and I’d only clumsily played around with code. I was the furthest thing from a developer you could imagine. 

Two hours later, I couldn’t stop. I’d built three working prototypes. My ideas, the ones that lived and died in Figma for years, were suddenly real. Interactive. Alive. 

This is the story of how I went from code-phobic to code-addicted in a single afternoon. And why every designer reading this is about to follow the same path. 

The Designer–Developer Divide 

For decades, we’ve accepted a fundamental lie: Designers design, developers develop. The gap between these worlds felt like a chasm requiring years of computer science education to cross. HTML, CSS, JavaScript were foreign languages spoken in basement servers and terminal windows. 

I believed this myth completely. My job was making things beautiful. Someone else’s job was making them work. This division of labor felt natural, inevitable, and even efficient. Why would I learn to code when developers already did it so well? 

That myth cost me every idea I couldn’t prototype myself, every interaction I couldn’t test, and every vision that got lost in translation. Twenty years of creative constipation, waiting for someone else to birth my ideas. 

Five Minutes to AI-Powered Prototyping 

“Open your terminal,” they said. Haha. I’d only ever really seen it used in The Matrix. The black window appeared. The cursor blinked in judgment. Type ‘claude’ and tell it what you want to build. 

My first prompt was embarrassingly simple: “Make me a color palette generator.” I expected nothing. Error messages, maybe. Definitely not working code. 

But there it was. A functioning app. My app. Built with my words. 

The next prompt came faster: “Add a feature that saves palettes locally.” Done. “Make the colors animate when they change.” Done. Each success made me braver. Each response made me hungrier. 

By the end of that call, I wasn’t just using AI to code. I was thinking in code. The barrier I’d spent two decades accepting had evaporated in minutes. 

The New Addiction: Vibe Coding 

They call it “vibe coding,” this conversational dance with AI. You describe what you want. The AI builds it. You refine. It rebuilds. No syntax to memorize. No documentation to parse. Just pure creative expression flowing directly into functional reality. 

I became obsessed. That first night, I built seven prototypes. Not because anyone asked. Not because I needed them. Because I could. Every design idea I’d shelved, every interaction I’d dreamed about was suddenly possible. 

The feeling was intoxicating. After years of creating static mockups, watching my designs move and respond felt like gaining a superpower. Click this button, trigger that animation. Hover here, reveal that detail. My aesthetic decisions instantly became experiential. 

When Designers Start Coding 

Something profound happens when the person with design taste controls implementation. The endless back-and-forth disappears. The “that’s not quite what I meant” conversations vanish. The design is the product is the code. 

UXPin’s research shows designers can now “generate fully functional components with just a few inputs.” But that clinical description misses the emotional reality. It’s not about generating components. It’s about giving creative vision direct access to digital reality. 

I started noticing details I’d never considered before. The precise timing of transitions. The subtle response to user actions. The difference between functional and delightful. When you control every aspect of implementation, you start designing differently. You start designing more ambitiously, more precisely, and with more courage.  

AI Code Tools That Make It Possible 

The technology enabling this transformation is staggering. Visual Copilot converts Figma designs directly to React code. Codia processes designs 100x faster than manual coding. These aren’t incremental improvements. They’re paradigm shifts disguised as product features. 

But the tools are just enablers. The real revolution happens in your mind. That moment when you realize the prison was self-imposed. The guards were imaginary. The key was always in your pocket. 

Natural language is the new programming language. If you can describe what you want, you can build it. If you can envision it, you can ship it. The only barrier left is imagination. 

The Future of Designer-Coders 

Organizations clinging to traditional designer-developer divisions are about to face a reckoning. While they coordinate handoffs and manage miscommunications, designers who code are shipping. Iterating. Learning. Building. 

This shift amplifies designers. Developers can focus on complex systems and architecture. Designers can implement their vision directly. Everyone works at a higher level of abstraction and impact. 

The competitive advantage is obvious. Teams with designer-coders ship better products faster. Not because they’re more efficient, but because they’re more effective. Vision and execution unified in a single mind. 

Your First Steps with AI Coding 

I know what you’re thinking. “But I’m not technical.” Neither was I. “But I don’t understand programming.” You don’t need to. “But I’m just a designer.” That’s exactly why you’re perfect for this. 

The same skills that make you a great designer, understanding users, crafting experiences, and obsessing over details, make you a natural at AI-powered development. You already think in systems and interactions. Now you can build them. 

Start small. Open a terminal and type a prompt. Build something stupid. Then build something slightly less stupid. Within hours, you’ll be building things that matter. Within days, you’ll wonder how you ever worked without this power. 

The Designer You’ll Become with AI 

Six months later, I barely recognize my old workflow. Static mockups feel like cave paintings. Design documentation seems like elaborate fiction. The idea of handing off my vision for someone else to interpret? Unthinkable. 

My role hasn’t changed. I’m still a visual designer. But my capability has transformed. I create experiences versus just imagining them. I propose ideas and prove them. I don’t just design products and ship them. 

The code anxiety is gone. Every limitation that once constrained me now seems artificial. The only question left is what to build next. 

Your journey starts with a single prompt. What will yours be? 

The pace of AI change can feel relentless with tools, processes, and practices evolving almost weekly. We help organizations navigate this landscape with clarity, balancing experimentation with governance, and turning AI’s potential into practical, measurable outcomes. If you’re looking to explore how AI can work inside your organization—not just in theory, but in practice—we’d love to be a partner in that journey. Request an AI briefing. 


Key Takeaways 


FAQs 

Q: What are AI code tools? 
AI code tools (like Claude Code, GitHub Copilot, or Visual Copilot) let you describe what you want in natural language, then generate working code automatically. 

Q: How can designers use AI code tools? 
Designers can turn Figma mockups or written prompts into functional prototypes, animations, and interactions—without learning traditional programming. 

Q: Does this replace developers? 
No. Developers focus on complex architecture, scaling, and systems. AI coding empowers designers to own interaction and experience details, speeding collaboration. 

Q: Why does this matter for organizations? 
Teams that adopt AI prototyping iterate faster, align design and development more tightly, and ship higher-quality products with fewer miscommunications. 

Q: What skills do designers need to start? 
Curiosity and creativity. If you can describe an idea clearly, you can build it with AI code tools. 

The Death of the Design Handoff: How AI Turns Tastemakers into Makers 

Every designer knows the ritual. You pour weeks into pixel-perfect mockups. You document every interaction, annotate every state, and build out comprehensive design systems. Then you hand it all to development and pray. 

Three sprints later, what comes back looks… different. Not wrong exactly, but not right either. The spacing feels off. The animations lack finesse. That subtle gradient you agonized over? Gone. The developer followed your specs perfectly, yet somehow the soul got lost in translation. 

Designers have always accepted this degradation as the cost of building digital products. We tried creating processes to minimize it, like design tokens, component libraries, and endless documentation, but we never stopped to question the handoff itself. 

Until now. AI just made the entire ritual obsolete. 

AI Ends the Design Handoff 

The design-to-development pipeline has always been messy, more like a game of telephone in a storm than a straight line. A designer’s vision turns into static mockups, those mockups get turned into specs, and then the specs are coded by someone who wasn’t there when the creative calls were made. 

Every step adds noise. Every handoff blurs the details. By the time the design reaches a user, the intent has been watered down through too many layers of translation. To manage the loss, we added layers. Product managers translate between teams, QA engineers catch mistakes, and design systems impose order. But taste cannot be standardized. 

AI design-to-code tools eliminate this process entirely. When a designer can move directly from Figma to functional code, the telephone line disappears. One vision, one implementation, and zero interpretation. 

Developers Spend Half Their Time on UI 

Here’s a truth we rarely say out loud. Developers spend 30–50% of their time on UI implementation. They’re not solving tough algorithms or designing big system architectures. They’re taking what’s already laid out in Figma and turning it into code. It takes skill and attention, but it’s work that repeats more than it invents. 

I’m not criticizing developers. I’m criticizing this process. We’ve asked our most technical team members to spend a third of their time as human transpilers, converting one formal language (design) into another (code). The real tragedy? They’re good at it. So good that we never stopped to ask if they should be doing it at all. 

When Airbnb started generating production code from hand-drawn sketches, they weren’t just saving time. They were liberating their engineers to focus on problems that actually require engineering. 

The Rise of the Tastemaker-Maker 

Something big shifts when designers can bring their own vision to life. The feedback loop shrinks from weeks to minutes. When something doesn’t look right, you can fix it immediately. If inspiration strikes, you can send it to staging and get real reactions in hours instead of weeks. What used to take whole sprints now fits inside a single coffee break. 

It’s tempting to frame this as designers turning into developers, but that misses the point. What’s really happening is that taste itself can now be put into action. The person who knows why a button feels right at 48 pixels, or why an animation needs a certain ease, or why an error state demands a particular shade of red, can actually make those choices real. 

That shift is giving rise to a new kind of role: the tastemaker-maker. They’re not confined to design or development but move fluidly between both. They hold the vision and the skills to bring it to life. They think in experiences and build in code. 

What Happens When Handoffs Disappear 

The implications ripple outward. When handoffs disappear, so do the roles built around managing them. The product manager who translates between design and development. The QA engineer who catches implementation mismatches. The technical lead who estimates UI development time. 

Teams start reorganizing around vision rather than function. Instead of design teams and development teams, you get product teams led by tastemaker-makers who can move from concept to code without translation. Supporting them are engineers focused on what AI can’t do: solving novel technical challenges, building robust architectures, optimizing performance at scale. 

This is job elevation. Developers stop being expensive markup translators and become true engineers. Designers stop being documentation machines and become product builders. Everyone moves up the value chain. 

AI Design to Code Speeds Shipping 

Companies using AI design-to-code tools report shipping features 3x faster with pixel-perfect accuracy. That’s a step function change in capability. While your team is still playing telephone, your competitors are shipping experiences that feel inevitable because they were never compromised by translation. 

The gap compounds daily. Each handoff you eliminate saves time on that project and builds institutional knowledge about what becomes possible when vision and execution converge. Your competitors are shipping faster and learning faster. 

How to Reorganize Without Handoffs 

Adopting AI design-to-code tools is the easy part. The hard part is reimagining your organization without handoffs. Start here: 

Identify your tastemaker-makers. They already exist in your organization. These are the designers who code on the side with strong aesthetic sense. Give them AI tools and watch them soar. 

Reorganize around products, not functions. Small teams with end-to-end ownership beat large teams with perfect handoffs every time. 

Measure differently. Stop counting tickets closed and start counting experiences shipped. Quality and velocity aren’t tradeoffs when the same person owns both. 

The End of the Design Handoff Era 

The design handoff was a bug in digital product development. A workaround for the technological limitation that the person who could envision the experience couldn’t build it. That limitation just died, and with it, an entire way of working that we tolerated for so long we forgot it was broken. 

The future belongs to those who can both dream and deliver. The handoff is dead. Long live the makers. 

The pace of AI change can feel relentless with tools, processes, and practices evolving almost weekly. We help organizations navigate this landscape with clarity, balancing experimentation with governance, and turning AI’s potential into practical, measurable outcomes. If you’re looking to explore how AI can work inside your organization—not just in theory, but in practice—we’d love to be a partner in that journey. Request an AI briefing. 


Key Takeaways 


FAQs 

What is a design handoff? 
The process where designers deliver mockups and specifications to developers, who then translate them into code. 

Why is the handoff inefficient? Each translation from design to documentation to implementation introduces information loss, slowing delivery and compromising quality. 

How do AI design-to-code tools change the process? 
They allow direct conversion from design tools like Figma into functional code, eliminating the translation step. 

What is a tastemaker-maker? 
A hybrid role that combines a designer’s vision with the ability to implement in code, collapsing feedback loops and accelerating iteration. 

Does this replace developers? 
No. It elevates developers to focus on complex engineering challenges, while routine UI translation is handled by AI. 

What’s the business impact? 
Companies using these tools report shipping 3x faster with higher fidelity—creating both a speed and learning advantage. 

AI Agents Are Users Too: Rethinking Research for Multi-Actor Systems 

The first time an AI assistant rescheduled a meeting without human input, it felt like a novelty. Now it happens daily. Agents draft documents, route tickets, manage workflows, and interact on our behalf. They are no longer hidden in the background. They have stepped into the front lines, shaping experiences as actively as the people they serve. 

For research leaders, that changes the question. We have always studied humans. But when an agent performs half the task, who is the user? 

Agents on the Front Lines of Experience 

AI agents reveal truths that interviews cannot. Their activity logs expose where systems succeed and where they stumble. A rerouted request highlights a friction point. A repeated error marks a design flaw. Escalations and overrides surface moments where human judgment still needs to intervene. These are not anecdotes filtered through memory. They are live records of system behavior. 

And that’s why we need to treat agents as participants in their own right. 

A New Kind of Participant 

Treating agents as research participants reframes what discovery looks like. Interaction data becomes a continuous feed, showing failure rates, repeated queries, and usage patterns at scale. Humans remain the primary source of insight: the frustrations, the context, and the emotional weight. Agent activity adds another layer, highlighting recurring points of friction within the workflow and offering evidence that supports and extends what people share. Together, they create a more complete picture than either could alone. 

Methodology That Respects the Signal 

Of course, agent data is not self-explanatory. Logs are noisy. Bias can creep in if models were trained on narrow datasets. Privacy concerns must be addressed with care. The job of the researcher remains critical: separating signal from noise, validating patterns, and weaving human context into machine traces. Instead of replacing human perspective, agent data can enrich and ground it, adding evidence that makes qualitative insight even stronger. This reframing doesn’t just affect research practice, it also changes how we think about design. 

Designing for Multi-Actor Systems 

Products are no longer built for humans alone. They must work for the people who use them and the agents that increasingly mediate their experience. A customer may never touch a form field if their AI assistant fills it in. An employee may never interact directly with a dashboard if their agent retrieves the results. Design must account for both participants. 

Organizations that learn to research this new ecosystem will see problems sooner, adapt faster, and scale more effectively. Those that continue to study humans alone risk optimizing for only half the journey. 

The New Research Frontier 

Research has always been about listening closely. Today, listening means more than interviews and surveys. It means learning from the digital actors working beside us, the agents carrying out tasks, flagging failures, and amplifying our actions. 

The user is no longer singular. It is human and machine together. Understanding both is the only way to design systems that reflect the reality of work today. 

This piece expands the very definition of the user. For the other shifts redefining research, see our earlier explorations on format, how to move beyond static deliverables, and scope, how AI dissolves the depth vs. breadth tradeoff. 

The pace of AI change can feel relentless with tools, processes, and practices evolving almost weekly. We help organizations navigate this landscape with clarity, balancing experimentation with governance, and turning AI’s potential into practical, measurable outcomes. If you’re looking to explore how AI can work inside your organization—not just in theory, but in practice—we’d love to be a partner in that journey. Request an AI briefing.   


Key Takeaways


FAQs

Why consider AI agents as research participants?
AI agents actively shape workflows and user experiences. Their activity logs reveal friction points, errors, and escalations that human feedback alone may miss. Including them as research participants offers a more complete picture of how systems actually perform.

Do AI agents replace human participants in research?
No. Humans remain the primary source of context, emotion, and motivation. Agent data adds a complementary layer of evidence, enriching and grounding what people already share.

What types of insight can AI agents provide?
Agents surface recurring points of friction, repeated errors, and escalation patterns. These signals highlight where workflows break down, offering evidence to support and extend human feedback.

What role do researchers play when analyzing agent data?
Researchers remain critical. They filter noise, validate patterns, address bias, and ensure agent activity is interpreted with proper human context. The shift broadens qualitative practice rather than replacing it.

What is a multi-actor system in research?
A multi-actor system is one where both humans and AI agents interact to complete tasks. Designing for these systems means studying the interplay between people and machines, ensuring both participants are accounted for.

How does including agents in research improve design?
By listening to both humans and agents, organizations can spot problems sooner, adapt faster, and create systems that reflect the true complexity of modern workflows.

How AI Ends the Depth vs. Breadth Research Tradeoff 

The transcripts pile up fast. Ten conversations yield sticky notes that cover the wall, each quote circled, each theme debated. By twenty, the clusters blur. At thirty, the team is saturated, sifting through repetition in search of clarity. The insights are still valuable, but the effort to make sense of them begins to outweigh the return. 

This has always been the tradeoff, go deep with a few voices or broaden the scope and risk losing nuance. Leaders accepted that limitation as the cost of qualitative research. 

That ceiling is gone. 

The Ceiling Was Human Labor 

Generative research has always promised what numbers cannot capture, the story beneath the metric. But human synthesis is slow. Each new transcript multiplies complexity, until the process itself becomes the limiter. Teams stopped at 20 or 30 conversations not because curiosity ended, but because the hours to make sense of them did. Nuance gave way to saturation. 

Executives signed off on smaller studies and called it pragmatism. In truth, it was constraint. 

AI Opens the Door to Scale 

Large language models change the equation. Instead of weeks of sticky notes and clustering, AI can surface themes in hours. It highlights recurring ideas, connects outliers, and organizes insights without exhausting the team. The researcher’s role remains. Judgment still matters, but the ceiling imposed by human-only synthesis disappears. 

Instead of losing clarity as the number grows, each additional conversation now sharpens the signal, strengthening patterns, surfacing weak signals earlier, and giving leaders the confidence to act with richer evidence. 

Discovery Becomes Active 

The real breakthrough is not only scale, but also timing. With AI-enabled synthesis, insights emerge as the study unfolds. After the first dozen conversations, early themes are visible. Gaps in demographics or use cases show up while there is still time to adjust. By week two, the research is already feeding product decisions. 

Instead of waiting for a final report, teams get a living stream of discovery. Research shifts from retrospective artifact to active driver of strategy. 

Nuance at Speed 

For organizations, this ends the false binary. Depth and breadth no longer compete. A bank exploring new digital features can capture voices across demographics in weeks, not months. A health-tech team can fold dozens of patient experiences into the design cycle in real time. A software platform can test adoption signals across continents without sacrificing cultural nuance. 

The payoff is more than efficiency. It is confidence. When executives see both scale and nuance in the evidence, they act faster and with greater conviction. 

The New Standard 

The era of choosing between depth or breadth is behind us. AI frees research leaders from the constraints of small samples or limited perspectives. With AI as a synthesis partner, the standard shifts: hundreds of voices, interpreted with clarity, delivered at speed. 

For teams still focused on fixing the format problem, our previous piece, The $150K PDF That Nobody Reads, explores how static reports constrain research. Our next article examines an even bigger shift: what happens when your users are no longer only people.

The pace of AI change can feel relentless with tools, processes, and practices evolving almost weekly. We help organizations navigate this landscape with clarity, balancing experimentation with governance, and turning AI’s potential into practical, measurable outcomes. If you’re looking to explore how AI can work inside your organization—not just in theory, but in practice—we’d love to be a partner in that journey. Request an AI briefing.   


Key Takeaways


FAQS

What is the depth vs. breadth tradeoff in qualitative research?
The depth vs. breadth tradeoff refers to the long-standing belief that teams must choose between conducting a small number of interviews with rich nuance (depth) or a larger sample with less detail (breadth). Human synthesis struggles to handle both simultaneously, forcing this choice.

How does AI change the depth vs. breadth tradeoff?
AI dissolves the tradeoff by enabling researchers to process hundreds of conversations quickly while still preserving nuance. Instead of diluting insight, scale strengthens pattern recognition and surfaces weak signals earlier.

Why has qualitative research been constrained to small sample sizes?
Human synthesis is time-consuming. After 20–30 interviews, transcripts become overwhelming, and important signals get lost in the noise. This labor bottleneck led leaders to view small samples as “pragmatic,” even though it was really a constraint of capacity.

Does AI replace the role of the researcher?
No. AI accelerates synthesis, but the researcher remains critical for judgment, interpretation, and ensuring context and nuance are applied correctly. AI acts as a partner that expands capacity rather than a replacement.

What is the impact of AI-enabled synthesis on decision-making?
With faster synthesis and preserved nuance, research insights emerge in real time rather than only in final reports. Leaders gain richer evidence earlier, which supports faster, more confident decisions.

What does this mean for the future of qualitative research?
The old tradeoff between depth and breadth is over. AI makes it possible to achieve both simultaneously, shifting the standard for research to hundreds of voices interpreted with clarity and delivered at speed.

The $150K PDF That Nobody Reads: From Research Deliverables to Living Systems 

A product executive slides open her desk drawer. Tucked between old cables and outdated business cards is a thick, glossy report. The binding is pristine, the typography immaculate, the insights meticulously crafted. Six figures well spent, at least according to the invoice. Dust motes catch the light as she lifts it out: a monument to research that shaped… nothing, influenced… no one, and expired the day it was delivered. 

It’s every researcher’s quiet fear. The initiative they poured months of work, a chunk of their sanity, and about a thousand sticky notes into becomes shelf-ware. Just another artifact joining strategy decks and persona posters that never found their way into real decisions. 

This is the way research has been delivered for decades, by global consultancies, boutique agencies, and yes, even by me. At $150K a report, it sounds extravagant. But when you consider the sheer effort, the rarity of the talent involved, and the stakes of anchoring business decisions in real customer insight, it’s not hard to see why leaders sign the check. 

The issue isn’t the value of the research. It’s the belief that insights should live in documents at all. 

Research as a Living System 

Now picture a different moment. The same executive doesn’t reach for a drawer. She opens her laptop and types: “What causes the most friction when ordering internationally?” 

Within seconds she’s reviewing tagged quotes from dozens of interviews, seeing patterns of friction emerge, even testing new messaging against synthesized persona responses. The research isn’t locked in a PDF. It’s alive, queryable, and in motion. 

This isn’t a fantasy. It’s the natural evolution of how research should work: not as one-time deliverables, but as a living system

The numbers show why change is overdue. Eighty percent of Research Ops & UX professionals use some form of research repository, but over half reported fair or poor adoption. The tools are frustrating, time consuming to maintain, and lack ownership. Instead of mining the insights they already have, teams commission new studies, resulting in an expensive cycle of creating artifacts that sit idle, while decisions move on without them. 

It’s a Usability Problem 

Research hasn’t failed because of weak insights. It’s been constrained by the static format of reports. Once findings are bound in a PDF or slide deck, the deliverable has to serve multiple audiences at once, and it starts to bend under its own weight. 

For executives, the executive summary provides a clean snapshot of findings. But when the time comes to make a concrete decision, the summary isn’t enough. They have to dive into the hundred-page appendix to trace back the evidence, which slows down the moment of action. 

On the other hand, product teams don’t need summaries, they need detailed insights for the feature they’re building right now. In long static reports, those details are often buried or disconnected from their workflow. Sometimes they don’t even realize the answer exists at all, so the research goes unused, or even gets repeated. An insight that can’t be surfaced when it’s needed might as well not exist. 

The constraint isn’t the quality of the research. It’s the format. Static deliverables fracture usability across audiences and leave each group working harder than they should to put insights into play. 

Research as a Product 

While we usually view research as an input into products, research itself is a product too. And with a product mindset, there is no “final deliverable,” only an evolving body of user knowledge that grows in value over time. 

In this model, the researcher acts as a knowledge steward of the user insight “product,” curating, refining, and continuously delivering customer insights to their users: the executives, product managers, designers, and engineers who need insights in different forms and at different moments. 

Like any product, research needs a roadmap. It has gaps to fill, like user groups not yet heard from, or behaviors not yet explored. It has features to maintain like transcripts, coded data, and tagged insights. And it has adoption goals, because insights only create value when people use them. 

This approach transforms reports too. A static deck becomes just a temporary framing of the knowledge that already exists in the system. With AI, you can auto-generate the right “version” of research for the right audience, such as an executive summary for the C-suite, annotations on backlog items for product teams, or a user-centered evaluation for design reviews. 

Treating research as a product also opens the door to continuous improvement. A research backlog can track unanswered questions, emerging themes, and opportunities for deeper exploration. Researchers can measure not just delivery (“did we produce quality insights?”) but usage (“did the insights influence a decision?”). Over time, the research “product” compounds in value, becoming a living, evolving system rather than a series of static outputs. 

This new model requires a new generation of tools. AI can now cluster themes, surface patterns, simulate persona responses, and expose insights through natural Q&A. AI makes the recomposition of insights into deliverables cheap. That allows us to focus on how our users get the insights they need in the way they need them. 

From Deliverable to Product 

Treating research as a product changes the central question. It’s no longer, “What should this report contain?” but “What questions might stakeholders need to answer, and how do we make those answers immediately accessible?” 

When research is built for inquiry, every transcript, survey, and usability session becomes part of a living knowledge base that compounds in value over time. Success shifts too: not in the number of reports delivered, but in how often insights are pulled into decisions. A six-figure investment should inform hundreds of critical choices, not one presentation that fades into archives. 

And here’s the irony: the product mindset actually produces better reports as well. When purpose-built reports focus as much on their usage as the information they contain, they become invaluable components of the software production machine. 

Research itself isn’t broken. It just needs a product mindset and AI-based qualitative analysis tools that turns insights into a living system, not a slide deck. 

Next in the series, we look at two more shifts: AI removing the depth vs. breadth constraint, and the rise of agents as research participants.

The pace of AI change can feel relentless with tools, processes, and practices evolving almost weekly. We help organizations navigate this landscape with clarity, balancing experimentation with governance, and turning AI’s potential into practical, measurable outcomes. If you’re looking to explore how AI can work inside your organization—not just in theory, but in practice—we’d love to be a partner in that journey. Request a strategy session.  


Key Takeaways


FAQs

What is the problem with traditional research reports?
Traditional reports often serve as static artifacts. Once published, they struggle to meet the needs of multiple audiences and quickly become outdated, limiting their impact on real decisions.

Why is research often underutilized in organizations?
Research is underutilized because its insights are locked in formats like PDFs or decks. Executives, product teams, and designers often cannot access the right detail at the right time, so findings go unused or studies are repeated.

What does it mean to treat research as a product?
Treating research as a product means building a continuously evolving knowledge base rather than one-time deliverables. Insights are curated, updated, and delivered in forms that align with the needs of different stakeholders.

How does AI support this new model?
AI makes it possible to cluster themes, surface weak signals, and generate audience-specific deliverables on demand. This reduces maintenance overhead and ensures insights are always accessible when needed.

What role do researchers play in this model?
Researchers become knowledge stewards, ensuring the insight “product” is accurate, relevant, and continuously improved. Their work shifts from producing final reports to curating and delivering insights that compound in value over time.

How does this benefit organizations?
Organizations gain faster, more confident decision-making. A six-figure research investment can inform hundreds of decisions, rather than fading after a single presentation.

Designing for the Unpredictable: An Introduction to Emergent Experience Design 

Why Generative AI Requires Us to Rethink the Foundations of User-Centered Design 

User-centered design has long been our north star—grounded in research, journey mapping, and interfaces built around stable, observable tasks. It has been methodical, human-centered, and incredibly effective—until now. 

LLM-based Generative AI and Agentic Experiences, have upended this entire paradigm. These technologies don’t follow predefined scripts. Their interfaces aren’t fixed, their user journeys can’t be mapped, and their purpose unfolds as interaction happens. The experience doesn’t precede the user—it emerges from the LLM’s interaction with the user. 

This shift demands a new design framework—one that embraces unpredictability and builds adaptive systems capable of responding to fluid goals. One that doesn’t deliver rigid interfaces, but scaffolds flexible environments for creativity, productivity, and collaboration. At Robots & Pencils, we call this approach Emergent Experience Design. 

The Limits of Task-Based UX 

Traditional UX design starts with research that discovers jobs to be done. We uncover user goals, design supporting interfaces, and optimize them for clarity and speed. When the job is known and stable, this approach excels.  

But LLM-based systems like ChatGPT aren’t built for one job. They serve any purpose that can be expressed in language at run-time. The interface isn’t static. It adapts in real time. And the “job” often isn’t clear until the user acts. 

If the experience is emergent, our designs need to be as well. 

Emergent Experience Design: A UX Framework for Generative AI 

Emergent Experience Design is a conceptual design framework for building systems that stay flexible without losing focus. These systems don’t follow scripts—they respond. 

To do that, they’re built on three types of components: 

1. Open Worlds 

Open worlds are digital environments intentionally designed to invite exploration, expression, and improvisation. Unlike traditional interfaces that guide users down linear paths, open worlds provide open-ended sandboxes for users to work freely—adapting to user behavior, not constraining it. They empower users to bring their own goals, define their own workflows, and even invent new use cases that a designer could never anticipate. 

To define these worlds, we begin by choosing the physical or virtual space—a watch, a phone, a desktop computer, or even smart glasses. Then, we can choose one or more interaction design metaphors for that space—a 3D world, a spreadsheet grid, a voice interface, etc. A design vocabulary then defines what elements can exist within that world—from atomic design elements like buttons, widgets, cells, images, or custom inputs, to more expressive functionality like drag-and-drop layouts, formula editors, or a dialogue system. 

Finally, open worlds are governed by a set of rules that control how objects interact. These can be strict (like physics constraints or permission layers) or soft (like design affordances and layout behaviors), but they give the world its internal logic. The more elemental and expressive the vocabulary and rules are, the more varied and creative the user behavior becomes. 

Different environments will necessitate different component vocabularies—what elements can be placed, modified, or triggered within the world. By exposing this vocabulary via a structured interface protocol (similar to Model-Context-Protocol, or MCP), LLM agents can purpose-build new interfaces in the world responsively based on the medium. A smartwatch might expose a limited set of compact controls, a desktop app might expose modal overlays, windows or toolbars, and a terminal interface might offer only text-based interactions. Yet from the agent’s perspective, these are just different dialects of the same design language—enabling the same user goal to be rendered differently across modalities. 

Open worlds don’t prescribe a journey—they provide a landscape. And when these environments are paired with agents, they evolve into living systems that scaffold emergent experiences rather than dictate static ones. 

2. Assistive Agents 

Assistive agents are the visible, intelligent entities that inhabit open worlds and respond to user behavior in real time. Powered by large language models or other generative systems, these agents act as collaborators—interpreting context, responding to inputs, and acting inside (and sometimes outside) the digital environment. Rather than relying on hardcoded flows or fixed logic, assistive agents adapt dynamically, crafting interactions based on historical patterns and real-time cues. 

Each assistive agent can be shaped by two key ingredients: 

These two ingredients work together to shape agent behavior: instinct governs what the model can do, while identity defines what it should do in a given context. Instinct is durable—coded in the model’s training and architecture—while identity is flexible, applied at runtime through prompts and context. This separation allows us to reuse the same foundation across wildly different roles and experiences, simply by redefining the agent’s identity. 

Agents can perceive a wide variety of inputs from typed prompts or voice commands to UI events and changes in application state—even external signals from APIs and sensors. Increasingly, these agents are also gaining access to formalized interfaces—structured protocols that define what actions can be taken in a system, and what components are available for composition. One emerging standard, the Model-Context-Protocol (MCP) pattern introduced by Anthropic, provides a glimpse of this future: an AI agent can query a system to discover its capabilities, understand the input schema for a given tool or interface, and generate the appropriate response. In the context of UI, this approach should also open the door to agents that can dynamically compose interfaces based on user intent and a declarative understanding of the available design language. 

Importantly, while designers shape an agent’s perception and capabilities, they don’t script exact outcomes. This allows the agent to remain flexible and resilient, and able to improvise intelligently in response to emergent user behavior. In this way, assistive agents move beyond simple automation and become adaptive collaborators inside the experience. 

The designer’s job is not to control every move the agent makes, but to equip it with the right inputs, mental models, and capabilities to succeed. 

3. Moderating Agents 

Moderating agents are the invisible orchestration layer of an emergent system. While assistive agents respond in real time to user input, moderating agents maintain focus on long-term goals. They ensure that the emergent experience remains aligned with desired outcomes like user satisfaction, data completeness, business objectives, and safety constraints. 

These agents function by constantly evaluating the state of the world: the current conversation, the user’s actions, the trajectory of the interaction, and any external signals or thresholds. They compare that state to a defined ideal or target condition, and when gaps appear, they nudge the system toward correction. This could take the form of suggesting a follow-up question to an assistant, prompting clarification, or halting actions that risk ethical violations or user dissatisfaction. 

Moderating agents are not rule-based validators. They are adaptive, context-aware entities that operate with soft influence rather than hard enforcement. They may use scoring systems, natural language evaluations, or AI-generated reasoning to assess how well a system is performing against its goals. These agents often manifest through lightweight interventions—such as adjusting the context window of an assistive agent, inserting clarifying background information, reframing a prompt, or suggesting a next step. In some cases, they may even take subtle, direct actions in the environment—but always in ways that feel like a nudge rather than a command. This balance allows moderating agents to shape behavior without disrupting the open-ended, user-driven nature of the experience. 

Designers configure moderating agents through clear articulation of intent. This can include writing prompts that define goals, thresholds for action, and strategies for response. These prompts serve as the conscience of the experience—guiding assistants subtly and meaningfully, especially in open-ended contexts where ambiguity is the norm. 

Moderating agents are how we bring intentionality into systems that we don’t fully control. They make emergent experiences accountable, responsible, and productive without sacrificing their openness or creativity. 

From Intent to Interface: The Role of Protocols 

The promise of Emergent Experience Design doesn’t stop at agent behavior—it extends to how the experience itself is constructed. If we treat user goals as structured intent and treat our UI vocabulary as a query-able language, then the interface becomes the result of a real-time negotiation between those two forces. 

This is where the concept of Model-Context-Protocol becomes especially relevant. Originally defined as a mechanism for AI agents to discover and interact with external tools, MCP also offers a compelling lens for interface design. Imagine every environment—from mobile phones to smartwatches to voice UIs—offering a structured “design language” via an MCP server. Agents could then query that server to discover what UI components are supported, how they behave, and how they can be composed. 

A single requirement—say, “allow user to log in”—could be expressed through entirely different interfaces across devices, yet generated from the same underlying intent. The system adapts not by guessing what to show, but by asking what’s possible, and then composing the interface from the capabilities exposed. This transforms the role of design systems from static libraries to living protocols, and makes real-time, device-aware interface generation not just feasible, but scalable. 

A Mindset Shift for Designers 

In this new paradigm, interfaces are no longer fixed blueprints. They are assembled at runtime based on emerging needs. Outcomes are not guaranteed—they are negotiated through interaction. And user journeys are not mapped—they are discovered as they unfold. This dynamic, improvisational structure demands a design framework that embraces fluidity without abandoning intention. 

As designers, we have to move from architects of static interfaces to cultivators of digital ecosystems. Emergent Experience Design is the framework that lets us shape the tools and environments where humans co-create with intelligent assistants. Instead of predicting behavior, we guide it. Instead of controlling the path, we shape the world. 

Why It Matters 

Traditional UX assumes we can observe and anticipate user goals, define the right interface, and guide people efficiently from point A to B. That worked—until GenAI changed the rules. 

In agentic systems, intent is fluid. Interfaces are built on the fly. Outcomes aren’t hard-coded—they unfold in the moment. That makes our current design models brittle. They break under uncertainty. 

Emergent Experience Design gives us a new toolkit. It helps us move from building interfaces for predefined jobs to crafting systems that automate discovery, collaboration, and adaptation in real time. 

With this framework, we can: 

In short: it lets us design with the user, not just for them. And in doing so, it unlocks entirely new categories of experience—ones too dynamic to script, and too valuable to ignore.