Why Ghost Networks Are Destroying Trust in Healthcare, and How Health Plans Can Solve Them

The Human Toll of Bad Data

After a traumatic miscarriage, Val Calderon, a special education teacher in NYC, began having suicidal thoughts. As these feelings rapidly intensified, she started calling the mental health providers listed by her insurer.

She went down the list, making call after call, but no one would take her on. Why? Because they were all either out of network or not accepting new patients.

And it’s not just patients who are affected. This is a major problem for providers, too. Doctors have vented about struggling to be removed from these lists in KevinMD articles, STAT News reports, subreddits, and medical society meetings.

In 2023, the president of the American Medical Association testified before the U.S. Senate about the problem.

NPR then did a feature on psychiatrists being forced to turn away vulnerable patients that were wrongly sent to them by ghost networks.

And now, in 2026, professional medical associations like the American Psychiatric Association are taking the unprecedented step of suing insurers over it, and EmblemHealth has paid a $2.5 million fine.

How Did the Ghost Network Problem Get This Bad?

The responsibility to provide accurate information on providers goes beyond CMS requirements and regulations. It’s a moral obligation.

Online provider directories are meant to do this, but they come with significant challenges and inefficiencies in our industry:

This messy arrangement inherently creates significant errors and inaccuracies. CMS reports provider data accuracy is commonly lower than 50%, which means cases like Val’s are frighteningly common.

Not only does this leave untold numbers of people without the help they desperately need. It also drives exorbitant costs to providers (burdened with labyrinths of requirements), the payers themselves, and the healthcare system as a whole.

A Post-Agile Solution to the Provider Directory Management Problem

The Challenge and Charter

Maintaining provider directories is expensive and complex. Every payer has their own contract nuances and suppression logic.

For example, a Medicare Advantage plan like ours with no members under the age of 18 has no need to list pediatricians.

As another example, sometimes all the data, APIs, and business rules say a provider SHOULD show as available, yet somehow they don’t.

If a payer has tens of thousands of providers and hundreds of thousands of members, how big a team do you need to hire to monitor the website to get ahead of all these problems? Too big for us!

Some organizations have tried to outsource provider directory management to a third-party vendor, but we’ve seen this model break down before, too. Culture and expertise need to be compatible here, and it’s difficult to find a right fit.

So, to provide the most accurate data possible for our health plan members at our own organization, it became clear we needed to build our own tools. In 2025, we set out to engineer a tool that would provide roster transparency through every step of our process:

• Receiving raw files
• Validation
• Data loading
• Data transformation
• Suppression logic
• Display on our website

Eliminating Time Consuming Provider Data Issues

When a member calls to say their in-network provider isn’t showing up, the answer isn’t always obvious. It could be a data error, a contract exclusion, or missing information from a medical group.

Tracing the cause requires digging through raw files, multiple systems, and business logic. This can involve 5 to 10 hours of work per ticket, spread across several teams. With hundreds of tickets a year, the drain on our team during open enrollment was unsustainable.

We needed a tool that would automate that research, catch problems before members encountered them, and free our team for higher-value work. The barriers were real: three months until open enrollment, no off-the-shelf solution, and institutional knowledge scattered across the organization.

For this situation, agile development, is far too slow. A fully staffed agile product design would have added too much red tape before we could conduct vital analytics. What we ended up doing closely resembled what’s now called “Forward Deployed Engineering,” which we’ll explain in the sections ahead.

What Is a Forward Deployed Engineer, and Why Did We Need One?

Traditional waterfall and agile approaches have consistently failed to deliver reliable provider directories. Both just move far too slowly. Custom enterprise healthcare software typically takes 12–18 months to build.

We didn’t have that. We had three months before open enrollment, and not one second more.

Forward Deployed Engineering (FDE) offered a different path. An FDE is an engineer embedded directly within a client’s team, making complex software work in the real world rather than in a vacuum.

In this article, we cover how that played out for us: fixing thousands of provider directory problems before members encountered them, building tools that saved significant research time, and proving that healthcare doesn’t have to move slowly.

Interestingly, we didn’t start by looking for an FDE. We just knew we needed three things in one person:

• Health plan and payer expertise to quickly grasp our business context, member lifecycles, and broker relationships
• Data science skills to measure, visualize, and communicate findings to C-Suite leadership
• AI engineering capability to build maintainable systems and APIs alongside our IT team

We worked with STLTH CNSLTNG, a health tech product and strategy firm, who connected us with Modular Feedback. Once we had our FDE (my co-author, Chris), it was time to get to work.

Launching the Provider Directory Initiative with the FDE

Launching a traditional agile project requires defining the product vision, creating the product roadmap, developing personas, gathering user stories, building the backlog, planning releases, etc.

How did we do this with so little time? We didn’t.

The approach was simple: Learn by doing.

We embedded the FDE directly into our team of analysts, strategists, and business leaders involved in growing our network and addressing member pain points. The approach was simple: Learn by doing.

Building Relationships Instead of Requirements

The FDE’s first task was to interview people within analyst, growth, strategy, and IT teams to understand our systems, databases, and where things can go wrong. Yes, there were existing diagrams and documents, but we all know systems and processes change faster than documentation does. Real-time knowledge lives with people on the front lines.

Those conversations surfaced data silos and processes that had gone unexamined for over a decade. Perhaps even more importantly, though, they built trust. Without a top-down mandate or pre-approved plan, the FDE needed relationships to get the access and infrastructure support the project would eventually require.

And relationships are more than just Zoom calls. They’re collaborative action. The FDE had to show the team value by actually getting things done.

Starting with the Right Projects

With the FDE, we had to resist two temptations:

1. Handing the FDE a vague mandate to “build a platform”
2. Simply using them as extra hands on existing tickets.

The first offers no clear starting point; the second produces no lasting change. Instead, we gave them a focused charter:

Build a system that tracks every provider end-to-end—across every database and the live website—and explains exactly why a provider isn’t showing up when they should be. Help with incoming tickets on the side.

While we knew the FDE had AI and ML expertise, we didn’t explicitly mandate using the technology. That judgment belonged to them. If they couldn’t make those calls independently, the development model wouldn’t work.

It was a deep-end-of-the-pool assignment, but the relationships the FDE built were lifeguards. I’ll share a couple of examples of small “deep end” missions that grew into big results: 1, State Data Auditing Tool and 2, Simulating the Online Experience.

State Data Auditing Tool

It appeared that our in-network endocrinologists in one state were being wrongly suppressed, hurting member access to care. For members with mobility issues, being forced to drive 30+ miles to see their providers was more than an inconvenience. In many scenarios, it could be a death sentence.

This was a five alarm fire, and our network relations and growth teams were visceral and heated because we needed to address it fast. Our FDE was less than 2 weeks in and barely had system access, but we brought them onto the call anyway.

Tech teams mostly receive requests as bullet points in a ticket. Being in the room for a live crisis gave the FDE something far more valuable: context.

This urgency inspired the FDE to build a small database auditing tool that would later become the backbone of how we’d find operational issues.

Takeaway: Don’t shy away from high pressure scenarios. This intense, high stakes scenario was more impactful for the FDE than any training module we could have developed.

Simulate the Online Experience + Collaborate with IT

Passing every data and API check doesn’t guarantee the website is showing the right information.

For a group of, say, 342 providers across 27 locations, a manual quality check means searching through over 9,000 records. Each provider could take 5-25 minutes to research. You do the math on that.

So, this is where the conversation got a little more sci-fi. We wanted a bot that could simulate what a real member or broker sees when they search the directory. The conversation went something like this:

Heather: Is there a way we could have an AI pretend to be a member and tell us what it’s seeing on the website?

Chris: That’s possible, but do we really need to do that? The API can tell us exactly what’s supposed to show on the website.

Heather: What’s in the API isn’t always what’s on the screen. We need an outside-in view to catch what the API misses.

Collaborating with IT to Break Barriers

This sent the FDE on another sleuthing mission with our web, API, and analytics teams.

This was no small feat. Having bots browse our website at scale was unprecedented, so we had lots of hurdles to overcome with IT to even test the system.

By linking up with a separate IT initiative already stress-testing the site, the FDE secured the access needed.

Simulated Online Experience Results

• With the tool, provider search time dropped from 5-25 minutes to 5-15 seconds
• The tool found more than 8,000 records wrongly hidden from our website, prompting infrastructure changes
• After seeing the risk the system uncovered, the FDE apologized (lovingly) for doubting the vision!

Making it Real: Getting Our Team on the Tools

The small database tool from Mission 1 grew to incorporate ALL databases in the provider lineage, from raw roster loads all the way to the API load. The website tool plugged into this, completing the picture by reporting what was missing from the website and why.

The FDE created additional tools that gave sales teams and brokers visibility into network coverage, availability by region, and business rule compliance, among other things.

These were all great backend tools to help answer questions FAST, but there was one catch: You had to know Python to run them.

By now, we’re in the thick of open enrollment, and our ability to jump on problems quickly made the process much smoother than the prior year. However, we didn’t want that value to vanish once the FDE was gone.

Training an analyst to use the system was the initial plan, but that would just plug a better tool into the same bottleneck. The real unlock would be to give the entire team a user-friendly web app. This shifted our goal from faster ticket resolution to eliminating tickets altogether.

The problem? End user software wasn’t part of the original scope! Leadership and IT weren’t prepared for it, and this meant we now had big political and technical barriers to clear. We were going to have to break some rules.

Selling the Initiative to Operations Leadership

Preparing Leadership for Big Transformations

The real pitch to leadership happened long before any formal meeting. By running targeted provider data audits, resolving tickets quickly, and regularly sharing growth reports, visualizations, and data field guides, we stayed on leadership’s radar and built credibility over time.

That groundwork made the eventual ask much easier.

What would you rather ask Sr. Leadership?

If there’s a big ask you’re thinking about at your organization, finding smaller wins first really helps. It also helped establish guardrails. Other leaders helped steer us away from redundant or unnecessary work.

These relationships helped address potential friction before the demo.

Demoing the Solution

Our COO, company president, and senior leaders from sales, network support, and IT agreed to meet. We had one week to prepare.

The temptation was to build a polished, interactive mockup, but we saw a trap there. A flashy UX demo would derail the conversation into design opinions before we’d even tested the fundamentals. Right now, we needed to demonstrate accuracy and reliability, not a cool UX. That would come later.

So we made a couple rules:

• No PowerPoint
• Ground the audience on data quality pains
• Be adaptable

Our resulting agenda for the demo call was deliberately simple:

• Intro
• Manual vs Bot: Scanning Our Website Fast
• Visualizing provider dropoff risk
• Innovation Discussion

This approach allowed us to frame up the business need, show some ‘shock and awe’ by showing the manual research process vs. the automated one, and then have a business conversation on data quality and end user research.

Manual vs Bot Visualization

We kicked off the conversation by going through a manual research process. This meant clumsily clicking through various menus, cross-referencing sources, and writing through spreadsheets.

We stopped at 5 minutes in, then we demonstrated an automation that could do those same steps + data reconciliation in 5 seconds. Showing the time savings was powerful, and showing the results was as well.

An Actionable View for Provider Data Management

We presented a tool that analyzed & validated physician data lineage, from roster to website. This could be generated for any provider, physician group, or market in our directory.

It showed clearly where urgent action was needed, such as dropoff occurring for unknown reasons.

This data simulation is based on that experience. When you hover and refresh the simulation, notice the dropoff reasons and mismatched counts between steps. These are errors and ghost networks.

The reaction from leadership was immediate: “How do we get this in our teams’ hands?”

Prepping for the Bigger Build

This is where “Be Adaptable” paid off. Leadership’s imagination was now engaged on features and views that would make it effective: spreadsheet readouts, growth mapping, network adequacy reporting, market filters, etc.

We now had the greenlight to build and test with users, and our CIO publicly backed the effort.

Collaborating with IT for Proper Test & Deploy

Shadow IT: The Elephant in the Room

Raise your hand if you’ve had a project nuked out of orbit because it’s shadow IT (risky software implemented without IT approval).

We’d sidestepped agile to move fast, and that created a real tension:

• The agile path would have developed a predictable set of requirements that would already be on IT’s radar to build and deploy
• Our FDE path prioritized starting small, iterating with users, and scaling only after proving value

This risked the shadow IT label, even though we had support from the top.

How to get the FDE model to work in a largely agile world?

Switching to agile wasn’t an option for us. It would sacrifice too much of the speed and adoption we’d gotten by folding data science and engineering into our team.

Instead, we paired our FDE with an IT architect who had deep internal relationships and could translate our needs into terms the agile workflow could absorb.

The FDE handled the business and engineering conversations. The architect handled infrastructure and IT specifics. And I kept leadership aligned. It was a small team, but the right one.

The biggest challenge was that, until this project, IT focused on infrastructure for large-scale deployments with top-down backing. So, we proposed the opposite: start lean, run user acceptance testing on lighter architecture, and invest in scale only once value was proven.

Framing it this way actually removed a lot of the resistance, because it kept IT from feeling like we were adding yet another massive project to their backlog.

This type of empathetic approach was crucial, because IT could very well have nuked the entire project, since it was outside the agile model.

If you have a new project that risks the shadow IT label, the game below helps reframe thinking for the type of empathy and due diligence needed to better work with IT.

To get off the ground, this project sidestepped agile and decades of established processes to take on a new approach.

Of course there was friction and debate between business and IT, but there was a mutual empathy:

• We empathized with IT over their backlog and mountains of new requests
• IT empathized with the business case and the mission for our members

Despite the friction, mutual empathy helped us approach the FDE path together as a team. Yes, we had to move quickly and break some rules, but we all agreed that these were rules that needed to be broken.

Conclusion

When agile isn’t fast enough, the playbook has to change.

Our members and brokers didn’t have time to wait on the old playbooks to get accurate data on provider coverage.

Still, we would have gotten nowhere if the approach were simply to dump tech and tools onto IT while moving at breakneck speed.

Our next post will take a first-person perspective on co-building with users, incorporating their feedback, and delivering impact with the new tooling.

Guest Author:

Heather Nairn - a healthcare executive & economist with nearly 20 years of payer and provider operations experience.

Editors:

Michael Barrett - whose em dashes are handmade with love.

Dr. Kay Nikiforova - who believes networks should strive to be more than just “adequate.”