EHRma

Enabling research study teams to visualize their recruitment efforts and understand their audience.

DESIGN LEAD (September 2023-Present)

Challenge

How can research study teams better understand where, how, and who they are recruiting for research studies?

One of many solutions to address the challenge

Provide study teams a visual representation of their recruitment efforts and integrate these visuals in their recruitment process.

Our impact

Our project allowed research teams to get a snapshot of their recruitment efforts. We visualized data of potential participants by demographics, location (zip), while still keeping patient data de-identified. The integration of the analytics with their current recruitment process made recruitment that much more powerful.

The Messy

DESIGN PROCESS

CLEARING THE FOG

Understanding priorities and goals

We knew from the MVP launch that our clients needed data visualizations that include participant demographics to enhance their recruitment efforts. They also expressed interest in SMS recruitment. I mocked up two different possible solutions to help them prioritize the most valuable things to build next.

Evaluative research on data analytics in Salesforce

Once the clients decided to prioritize the analytics dashboard, my next goal was to understand what data to visualize and how to properly visualize the data we are collecting so that our clients can better understand how they are currently recruiting.

I set up a few rounds of interviews with our clients and created a wireframe to help them visualize their goal. The charts had different displays, filters, and a few demographics to start, including language, race, ethnicity, and age.

From these interviews, I was able to understand their priorities and goals:

  • Ability to understand % of eligible participants by contact status, race, ethnicity, age, location, sex, and language.

  • Allow research teams to recruit from the data presented in the visualizations within the system.

Initial ideas for evaluative research / text recruitment / EHRma v2.0

Initial ideas for evaluative research / analytics dashboard / EHRma v2.0

“We want to add a dashboard related to equity for the study team to view, so they can easily see the race, ethnicity, language, sex, age, of their overall cohort versus those responding, so they can have better visibility into who they’re recruiting.”
— EHRma client
SETTING THE RIGHT DIRECTION

Rapid prototyping to guide our next build

With the initial information we have, I worked with our lead engineer and solutions architect to create a prototype. The goals for this prototype is to:

  • understand the feasibility of the technology particularly working with our current MVP and integrating with EHR.

  • understand the viability of the product given the clients’ budget, and

  • know the design limitations from customizing our existing MVP

Initial prototyping with engineer and solutions architect

Understanding limitations and getting buy-in

We were able to come up with an initial working dashboard with data coming from the EHR and expanded filters. We tested the prototype with the product owner to get initial feedback, communicate limitations right away, and get relatively accurate estimates for the build. 

UNANTICIPATED BARRIERS

Uncovering more challenges

  • Challenge 1: Our prototype did not reflect real-world data.

    Our sample size was too small to be insightful. Meaning it did not accurately visualize the large amount of data we have to get insightful feedback about the usefulness of the graphs in the analytics dashboard.

    • Solution: I worked with the engineer, mule developer, and solutions architect who specialize in data integration with Epic to mimic the actual size of the data in order to understand the challenges that lie ahead, including chart display, limitations with the types of charts and things you can do with them, and lag time.

  • Challenge 2: Integrating new features with the existing tool.

    • Solution: I brainstormed with the team and mocked potential options for different workflows ranging from low to high complexity to understand the implications for integrating a new capability with the existing tool and communicate tradeoffs to the client.

    • I also worked with our client to architect the information so the most useful elements are at the top.

  • Challenge 3: Getting lost in translation

    Our clients were getting caught up on the various filters within the analytics dashboard. Presenting the information was also getting a bit too complicated.

    • We went back to wireframing and focused on formulating the questions they want the data to answer.

Back to the drawing board. Sketching with our clients to create a better information architecture when integrated with the current workflow.

Going back to wireframing to understand the questions they want the data to answer.

Second prototype that reflects real data. I also used the standard colors that passed the WCAG 2.0 guidelines.

SPEEDING AHEAD

Building iteratively with the team

Using my mockups and design specifications, we prioritized a set of user stories.

Articulating tradeoffs

We wanted the clients to understand that importing a large amount of data sets for the analytics dashboard and integrating a new feature with the existing product can have tradeoffs including:

  • Delays caused by working with a large data set (millions of records).

  • Layout and types of charts are limited to what the Saas product offers.

  • Integrating existing MVP with the dynamic charts may disrupt the current recruitment workflow.

Usability and accessibility testing

We tested at the end of every sprint cycle with different 3-4 users and asked for feedback. It was also our first time testing with users with disabilities to get a sense of how accessible our product is.

Deploying a solution within reasonable timeframe and budget

We iterated for eight weeks until we got to a product that seamlessly displayed the right demographic information and gave the research team the ability to recruit participants using the same data displayed in the charts.

New prototype that includes real-world data, integration from existing tool, and improved information architecture.

Usability tests / scripts and results.

Accessibility tests / short clip

LOOKING BACK... TO LOOK AHEAD

Key takeaways from this iteration:

  • Start small, but test with realistic data. Starting small was a good way to test if the technology works. However, visualizations were hard to envision without data that reflects the right size in the real world. Hence, it was hard for our clients to visualize the right way to display the charts because three vs one million records can look a lot differently.

  • Do the right thing, even if it’s hard. We initially wanted to build two separate features that were not integrated with each other— one was to visualize the data and the other was to recruit (MVP). Though easier, it wouldn’t make much sense to have our users recruit based on data that was not real-time. We knew that their recruitment metrics mattered, so we had to find ways to build one system that integrated the two functionalities.

  • Test as often as time and money will allow. Learning as much as you can about how the users will interact with your product at each stage of the development cycle is extremely valuable. Both usability and accessibility tests were of utmost importance, and it paid off.

Final product / EHRma 2.0