What it’s like to be part of a research operational team

Case study

The Challenge

The need to have a research operational team was triggered in the context of a digital transformation project, where 23 agile pods were working simultaneously to redesign the experience and the architecture of an airline company. (during pandemic times, hello remote research environment)

Our mission as a research team is to ensure continuous user research for 23 teams, creating the guidelines, platforms, and processes to plan and execute research activities in an agile way, and to measure the impact of these activities on the overall user experience.

The main KPI was to validate at least 40% of the user-stories that had front-end. (screens, flows, components, storytelling) Because we assume that user flows that were validated with users, reduce risks, and experience pitfalls before the development of the product. (Before more time, effort, and money were invested)

I like to imagine that we are the power rangers of user research ops⚡️

The framework of experimentation and user research

Everything looks so good on paper (or digital paper nowadays), and maybe that’s why new processes are born this way.

We create an operative model that allows teams to plan and execute moderated and unmoderated user testing in 1 sprint (2 weeks)

How we make this happened?

1. We provide teams with templates and guidelines (that they can adapt to their particular situation) to standardize and speed up the delivery of new insights about their products.
2. We implement guidelines according to different user research methods. Creating briefs, scripts, and documentation templates for moderated tests and unmoderated tests.
3. We are always looking and testing new research software to provide better research resources to user experience teams. (Optimal workshop, usability hub, maze, Typeform, etc)

⚠️ None of these tools are perfect, so as designers and researchers we have to adapt and use the tool in our favor. If the tool is not working, we change the software and try a different one. #experimentmindset

Recruitment platform

This is one of the main foundations of agile and efficient user research.

How do we systematize user recruitment?

1. To provide teams with the right users for their experimentation, we create a testers program with clients of the company.
2. Implementing a widget on the website and asking for volunteers to sign up to participate in new initiatives and product testing. After they register in our testers program, we send a profiling form, so we can better understand their demographics and some of their behaviors and preferences.
3. Also, we had to be aligned with the legal team, to communicate with clients adequately according to regulations. For every activity with users, we create templates and NDA forms, to keep everything in legal terms.
4. For user management, we use Mailchimp as the principal tool to send communications, invitations, and links to testing. Mailchimp allows us to create email templates, manage audiences, and tag users according to different characteristics. This is a very good asset to recruit in an organized way. (avoiding to contact the same person as spam)

The result is that with processes of recruitment inside the company, we can provide real users to the teams in a systematic, agile, and continuous way. Increasing the number of users signed in the program and diversifying the samples to make research activities in all markets where the company operates.

0800 (Research hour) or, in other words, the coaching and assistance platform for UX teams.

To support UX teams with their research (goal and hypothesis definition, selection of right tools, recruitment, facilitation, and co-analysis of results) we create a format of coaching on-demand called “research hours”.

In that period of time, UX and Visual designers could have the assistance of any researcher and find support for planning and building their research instruments.

We use a calendar system (with calendly app) to schedule meets 1:1 with user experience teams, which allows us to streamline the process and organize better our team capacity.

This coaching platform ensures knowledge sharing, growth of the research capacities in the teams, and also, guarantee quality and well definition of experiments and research activities.

Monitoring research implementation

As a team we had a challenge, and was to ensure that the product teams were testing their designs as fast as possible, so, how can we visualize if that is happening? (In a certain way we have to be able to quantify the impact of the team and justify the investment that comes with it)

The process of tracking research activities is a trial and error history

After various intents of trackers, we build a data studio dashboard, that connects user stories (from Jira) that could require some kind of user validation, versus user stories that pass through some research. (thanks to the PM of our team 😍) Having an indicator allows us to monitor the progress and create initiatives to help teams reach the goal.

Also, we correlated the indicator of research developed with client’s feedback in certain processes where we measure CSAT and CES. (Still, we don't have the proper quantity of data to make conclusions, but we will #commingsoon)

The Impact

For me, is very rewarding to be part of this operational research team. It’s a great space to propose, experiment, and talk with so talented people around the project.

These are some of our achievements 💖

• 23 product teams (agile cells) making user testing in a continuous and agile way 🙌🏻
• Creation of guidelines and toolkits for each methodology of research
• Creation of a library of research examples
• Creation of a testers program with on-demand users to deliver user testing (2.000 users signed up)
• The company it’s on its way understand how the research activities had an impact on the overall indicator such as CSAT and CES
• After each testing, UX and visual designers propose to their team improvements on the experience, iterating their designs with user data, and building better bridges with business and technology roles. 🤟🏼

Some learnings of being part of a research operational team

• All these operational research initiatives were co-created with design teams. And along with the implementation, we must always gather continuous feedback to provide the best guideline and tools for teams. So, building trust with teams ensures transparent communication to improve our research operational processes. In the end, our mission is to serve other teams and not to become blockers.
• Agile research comes hand in hand with agile delivery. If inside the team there is no alignment, research efforts can become a huge list of improvements that never pass to production environments.
• Standardized until a certain point: Each person has different skills and different ways to create and communicate. That’s why a lot of standards can become a force shirt that limits the creativity and comfort of design teams. After we realize that, we pivot to create a library of examples and each designer could submit their research processes, and make them available for other teams to inspire and use as references.
• Being in a transversal research team requires to be in touch with many teams, and to understand the product at least at a certain level. Is not possible for the researcher in an operational team to know exactly every screen, every user flow, and every decision the team is making. So we must assist with an open mind, avoiding design critiques if the context is not well known.

Peace out ✌🏼⚡️