Understanding False Positives
My role: project planning, planning and conducting research, experience mapping
​​
Background
​
Compliance officers in financial institutions use Global Relay’s surveillance software to monitor communications for risks such as bribery, money laundering and market abuse.
This monitoring process can surface a lot of false positives, messages that can be immediately dismissed as not relevant or suspicious. A high number of false positives can slow down compliance teams, creating a lot of unnecessary work.
As the UX and product teams were beginning to look into solutions to reduce these false positives for customers, a decision was made to do some foundational research on the topic. The brief was to conduct a broad study with customers to better understand the sources of false positives, how they manage this issue and the priority of it in their workflows.
My role in this project was as primary UX researcher. A technical product manager and UI designer acted as support, participating in workshops and note-taking in interviews. ​
​​Research plan
​​
Phase 1: Internal & desk research
I began by gathering existing knowledge from internal stakeholders and customer-facing teams, and reviewing previous feedback from customers. I also conducted a competitor analysis to understand how other companies were approaching the issue. This data was used to inform the research plan and discussion guide for talking to customers.
Phase 2: Customer interviews
6 semi-structured interviews were conducted with the target customer persona. I led the discussion for all interviews with a technical product manager and UI designer sitting in to observe and take notes.
​
Phase 3: Analysis and presentation
After reviewing the interview recordings and notes, I conducted an analysis workshop with the core team where we extracted themes and began to form an experience map. After the workshop, I refined the map and created a research report of the findings, with support and feedback from the core team. This was then presented to the wider product and design teams.
Findings
1. Definition of a false positive
A key finding was that there was no clear consensus amongst participants (or the field at large) on the definition of a false positive. However two distinct examples emerged:
-
An unintended hit that is clearly not a violation (e.g., a keyword hit in an email disclaimer)
-
Anything that is not a violation (anything that is not a true positive).
When reviewing these findings with the product team, we chose to use the first definition within the organisation moving forward, in order to create a shared understanding when we discussed this topic.
2. Sources of false positives
There were no surprises in the sources of false positives, with the two most common (mentioned by all participants) being keyword matches in email disclaimers and duplicated messages
3. Process for dealing with false positives
The process of identifying, investigating and dealing with false positives was very manual and often done by one person on the team (the compliance administrator persona). Additional team members' role in the process was sporadic and disorganised.
Although no participants had a standardised process defined within their team, we found the high-level workflow was similar, allowing us to create an experience map of the process. This map became the biggest takeaway of the study, as it created a clear map of how to begin addressing this issue.
Current features to reduce false positives (such as a safe senders list) were well used and liked by customers.
3. Priority
Reducing false positives was a top priority for customers overall. They saw it as the main way to reduce their workload while still meeting regulatory requirements.
Experience map V1
Outcomes
Workshop with the product team
Following the presentation of the research findings and experience map to the broader product and design team, I facilitated a focused workshop with key product team members to identify gaps and opportunities within the experience map and evaluate them against the product roadmap. This session was crucial in identifying which roadmap initiatives directly addressed the challenge of false positives, while also mapping out and prioritising other unmet needs.
By bringing the team together in this way, the workshop fostered strong alignment around the product roadmap and created a unified vision for tackling false positives through targeted initiatives over the coming year—ensuring the team moved forward with clarity and shared purpose.
Wider team education
As false positives were one of the key challenges determining upcoming feature development, I also presented the study and findings to a broad range of teams—including developers, customer-facing teams, and key stakeholders. This ensured everyone gained a shared understanding of the problem and a clear, unified view of the issues that needed solving. Throughout the following year, this research became a foundational reference point for multiple projects, guiding teams as they aligned feature development with the insights and ensuring each step in the workflow effectively addressed the core challenges.
False positives experience map