Understanding physician performance
By using predictive analytics and statistical methods, Elsevier's Physician Performance tool is able to provide both the Medical Director and Physicians a comparative view of their performance and how it compares to their peers.
To comply with my non-disclosure agreement, I have omitted and / or concealed confidential information in this case study. All information in this case study is my own and does not necessarily reflect the views of Elsevier.
Project Summary
To meet the needs of healthcare providers, the team at Elsevier’s Clinical Analytics division was tasked with leading a market viability study for a new tool aimed at assessing Physician performance. This was a large project team– composed of subject matter experts (Physicians, Nurses…), Business Analysts, and Designers. The goal was to identify which proposed features users needed most, how they needed to use it, and whether the functions/features of the product matched the needs and expectations of healthcare providers in the field.
Deliverable
Assess the viability of a new Physician Performance Assessment tool against proposed designs, using UX methods such as persona development, heuristics evaluation and usability testing, followed by design revisions.
My Role
This was an interesting situation, where the product team in-house at Elsevier's Clinical Analytics division had already completed a functional prototype based off of a similar legacy product and internal expertise. The prototype did not do well in early market testing, and Elsevier decided to hire an outside UX Consultant who could conduct an impartial review. I was tasked initially with UX research and analysis, but ultimately remained through the product design phase and beyond.
WHY does this product matter?
Everyone is a patient at some time or another, and we all want good medical care. We assume that doctors are all medical experts and that there is good research behind all their decisions, but that's not always the case. In order to implement changes to improve health care for patients, it is imperative that appropriate assessments are completed to determine baseline satisfaction of patients and improve patient outcomes. The objective of Medical Directors and administrative staff is to ensure the highest possible quality of care. As such they conduct regular physician evaluations. Physicians on the other hand want fair reviews based on accurate data. By using predictive analytics and statistical methods, the Physician Performance tool is able to provide both the Medical Director and Physicians a comparative view of their performance and how it compares to their peers.
HOW does the product address the why?
The Physician Performance tool uses visually rich data analytics to generate meaningful insights about physician performance to determine what areas need to be fine tuned (if any) and how.
Achieving User Understanding
Understanding current product and users
Persona Development
I conducted a heuristics evaluation of the current prototype, identifying potential usability concerns and areas for improvement and documenting the primary user's journey through the product. This helped the team identify the product's workflow across multiple users with varying objectives. In this we were able to identify the Provider Relations Manager as the more of a heavy-lifter doing the research and analysis legwork for hand-off to a Medical Director. The Director consumes this data and builds a case for quality improvement measures that are ultimately served up to the Physicians on staff.
Challenges
The product team made the seemingly obvious assumption that the primary user would be the Medical Director. However, despite the fact that reports were ultimately reviewed and approved by the Medical Director, the primary user was the Provider Relations Manager.
Usability
The heuristics evaluation was followed by formal usability testing at on-site client locations, where we could observe the tool being used within their practice.
What we discovered
1) Product not aligned with user's workflow
Although the product DID in fact provide all of the data necessary to appropriately conduct a Physician’s quarterly evaluation, the layout and availability of information did not align with the user’s workflow. Hence, leaving the impression that the product lacked accuracy.
“Allow the information to tell you how it wants to be displayed. As architecture is ‘frozen music’, information architecture is ‘frozen conversation’. Any good conversation is based on understanding.” — Richard Saul Wurman
2) Visual design lacked focus
The ad hoc use of color and placement of screen elements made key evaluation criteria difficult to find.
3) Primary user misunderstood
Knowing the true primary user was the Provider Relations Manager, we honed-in on their objectives vs those of the medical Director after hand-off.
Provider Relations
Top 3 goals observed
- Identify the best opportunities to improve our practice's performance
- Providing accurate and actionable data to Physicians
- Serving as a liaison between the provider and health plan
Medical Directors
Top 3 goals observed
- Visually rich, easy to read, and relevant graphs of performance metrics
- Easily creating reports or views of with quality insights our Physicians trust
- Connecting performance data with proven recommendations
Aligning insights with design
Revised prototype
The revised prototype re-focused the experience and information architecture based on the needs and evaluation style of the Provider Relations Manager. The system then generated reports targeted to the needs of the Medical Director. Aligning the new version directly with the office workflow for performance assessments.
Results and Impact
Three months post-release a survey of the product's early-adopters was conducted to gauge overall satisfaction and outcomes. Practices reported a 56% increase in overall satisfaction with the product, a 33% higher trust in data accuracy (despite the underlying data remaining unchanged), and a three point improvement in the level of effort required to complete tasks and achieve goals. Additionally, their patient satisfaction went up almost 10%.
Be sure to continue to the next project story - JPMorgan's mobile app for investment clients.
