How to Go from Pilot to Full-Scale Adoption

Hi friend,

Poorly executed pilots are a leading cause of death for mental health solutions. Many mental health businesses develop great products and work tirelessly to sign a deal with a health system. But getting your intervention used and proving it works is the hardest step. Do it right, and you’ll renew bigger, better contracts. Do it wrong and you’ll join the poor victims of pilotitis. 

Trina Histon, PhD has decades of experience driving the adoption of mental health interventions. During their time at Kaiser Permanente, they deployed over 20 digital mental health apps within primary and speciality care services. Through this work, Trina learned firsthand where companies struggle to move from pilot to full-scale adoption, and what it takes to overcome these challenges. 

Drawing on Trina's experience, this guide takes you through the practical steps to evaluate pilots, plan for them and execute them in a way that gets your product adopted and your contracts renewed.

Securing a pilot with a healthcare system is an important milestone, but not every pilot offers a genuine opportunity to demonstrate value. You may be desperate to sign an agreement, but before you do, you must consider whether the pilot is designed to succeed. 

An ‘un-winnable’ pilot is one that;

Resist pressure to test your intervention outside its intended use or promise clinical outcomes that cannot realistically be demonstrated within the pilot timeframe. You risk producing evidence that your product is ineffective in a population it was never designed to serve.

There is nothing wrong with declining a pilot if its design makes it impossible for you to succeed.

The second type of pilot to avoid is one where there is no clear definition of success, no agreed threshold for expansion, or no decision-maker responsible for progressing the intervention beyond the initial deployment.

Some pilots can demonstrate promising results but stall with no clear route to scale. Founders can become trapped in a cycle of continuous pilots.

Before signing, establish what happens if the pilot meets its targets. Who will review the evidence? Who can authorise expansion? What further approvals or resources will be required? Without answers to these questions, a successful pilot may still lead nowhere.

Different health systems have different levels of experience in deploying mental health interventions. Before you do anything, you need to know how ready the health system is to partner with you. This will affect the amount of support and resources you dedicate to the partnership and how you approach it.

Trina asks four questions to evaluate the readiness of a health system:

To answer these, start with publicly available information, including previous partnerships and published case studies. You can also use your conversations with the partner to probe for information that might help you answer these questions, such as challenges they encountered with previous deployments.

With these answers, you should classify the health system as ‘Clear, Complicated or Chaotic’.  

Working in the ‘Clear’ space means there are processes in place for deploying products similar to yours. You should therefore focus on understanding and following the existing processes of your client and not waste time building new ones — which the client may not accept or will demand you adapt to their existing pathways.  

In a ‘Complicated’ environment, the healthcare partner might have deployed digital interventions but lack the infrastructure to scale them. For example, they might have experience running pilots but no established processes for integrating interventions into clinical workflows or sharing patient outcomes with clinicians. Or they might have deployed and scaled a lower-risk wellness tool but haven’t managed the clinical risk associated with a mental health tool. In these situations, you need to identify these implementation gaps early. Then, work with your partner to test solutions to these gaps on a smaller scale, before replicating what works.

Entering a ‘chaotic’ environment will be harder. These organisations have limited experience adopting solutions like yours and could also have low overall digital maturity. While you need to be aware of how difficult this will be, don’t let it deter you. Healthcare systems with less established infrastructure may be those that need your product most. Achieving full-scale adoption in these settings will demand more time, effort and resilience, but can also provide some of the richest learnings. You just need to set realistic expectations internally about the time and effort it will take. 

In general, don't underestimate the time needed for deployment in any of these systems. As Histon puts it, “Deployment is a marathon made up of many sprints.” Based on Histon's experience, expect around 18 months from first conversation to deployment in an integrated health system, or three to five years when partnering directly with individual providers. If your partner falls towards the ‘chaotic’ end of the spectrum, plan for the longer end of these timelines.

You should also consider your own experience when applying this framework. Even in the ‘clearest’ healthcare system, deploying an intervention for the first time may feel chaotic. Your first deployment will be harder and take more time than your tenth.

Once you’ve defined your partner's readiness, you can move on to defining what you can hope to achieve with your pilot.

Before you do anything, you need to know what you’re aiming for. Agree with your health system partner upfront what success looks like and what evidence would justify scaling the intervention. Different stakeholders at the health system will have different priorities, so co-design each pilot around the metrics that matter most to your buyer.

These should include:

Clinical outcomes and financial returns may take longer to demonstrate, so agree with your healthcare partner on how and when you will measure them from the outset. If your buyer prioritises reach and health equity, track whether your intervention reaches the populations it aims to serve.

What health systems want has changed. In the 2010s, digital health products typically relied on 'social media' style engagement metrics, tracking monthly and daily active users. There is now a growing awareness that these metrics can translate poorly to behavioural health and don’t address the things health systems care about.

Yes, health systems still want to see engagement metrics, but averages can conceal important differences in how patients use an intervention. Look for engagement clusters: subgroups of patients with similar patterns of use. For example, in an eight-week trial of Woebot, three different types of users were identified based on their engagement patterns. Segment users by engagement patterns to understand whether different groups are deriving value from the intervention. 

Engagement alone, however, does not demonstrate clinical value. Health systems increasingly expect digital interventions to demonstrate improvements in patient outcomes.

Typical outcome measures for mental health apps are the PHQ-9 for depression and GAD-7 for anxiety. Everyone recognises these measures have limitations. But in the earlier stages of deployment, it’s best to just use the measures that the health system is comfortable with. You may want to collect data on other measures as well, but advocate for this alongside collecting your validated outcomes. Just be prepared to justify the measures you are using.

Quality is another series of metrics you must be focused on. Healthcare systems are held to quality standards that increasingly move beyond process adherence (e.g., is the PHQ assessment completed for each patient?) towards evaluating outcomes (e.g., are patients actually responding to treatment?). They may expect a digital tool to help them meet these standards, which is why mature systems often want outcome measures collected by digital tools to be written back into electronic health records.

Early on, focus on whether patients are engaging with the intervention and whether it is reaching the intended populations. As the deployment progresses, assess whether validated outcomes are being collected at appropriate intervals and whether those outcomes are improving.

To deliver a successful pilot, you will need strong relationships with multiple stakeholders, not just clinical champions. It’s important to map these stakeholders on day one of deployment and understand their needs. Each one could halt your pilot if their needs are not met. 

This is the person or organisation responsible for the budget, purchasing decision and renewal. They are looking for evidence that the intervention delivers value for money, rather than simply demonstrating that the technology works. Proving financial value normally takes longer than the initial pilot. You should find out what the buyer considers valuable and develop a plan for demonstrating that ROI, even if the answer won’t be immediately available. What signals will a healthcare leader need to see to continue to expand before scale?

This is an influential clinician who advocates internally for the intervention and connects you with decision-makers. They are often the person who has brought you into the organisation in the first place. Leverage their knowledge of the organisational culture during decision-making. They want to improve outcomes for patients, and they need to have confidence in the intervention’s efficacy. Give them clear explanations of the intervention’s purpose and equip them with any early evidence you may have that it works. Do not blindside them. Keep them informed about delays or problems. They will expend political capital on their relationship with you, so you don’t want to put them in a position where they must defend something they weren't told about.  

These are the clinicians you’ll expect to refer patients to your intervention. They want a tool that helps their patients without adding to their workload or threatening their professional role. Invite clinicians to shape how the intervention is introduced, so that it complements existing care. At Kaiser Permanente, Histon involved therapists from day one, gaining valuable information about the patients they were seeing and how the tool could help. For example, therapists identified patients' difficulty completing therapeutic homework, highlighting an opportunity for the intervention to reinforce treatment between sessions. It’s also important to address any concerns about additional workload or whether the tool will replace the clinician. 

Information security and technical teams are responsible for assessing the intervention's security and integration risks. They need evidence that the product meets applicable security requirements and handles patient data appropriately. Information security reviews can take longer than the sales cycle. Make sure to build in enough time for these reviews and run them in parallel with legal/compliance reviews where possible. 

Examples of requirements the client might expect, particularly when working with US healthcare systems, are: 

The legal, privacy and compliance teams are responsible for determining whether the partnership and its data-handling arrangements are acceptable. They need clarity on contractual obligations, patient privacy, data flows, clinical responsibilities and liability. Identify legal and compliance requirements early; in the US, this often involves Business Associate Agreements and HIPAA compliance. Run these in parallel with the security reviews mentioned above. Keep in mind that legal teams own a hard no. Unresolved compliance issues can stop deployment altogether, so maintain open channels of communication with legal counsel, whether in-house or external. 

The end users of the intervention. They might have minimal involvement in the purchasing decision, but if they don’t find the intervention useful and don’t actually use it, everything else becomes irrelevant. Involve patients in testing the product's usability, rather than assuming that an intervention endorsed by clinicians will automatically appeal to those receiving it. Like most of healthcare, your core user and your customer may be different people. Make sure you recognise that and the different needs of both parties.

Clinicians are stretched for time. Even an effective intervention is unlikely to be adopted if it creates significant additional work. 

To embed your tool into clinical workflows in a way that drives adoption, you need to know: 

Patients may access an intervention through several referral pathways, including primary care physicians, behavioural health consultants, therapists or health coaches. 

Rather than attempting to establish all these pathways simultaneously, demonstrate value through one before expanding into others.

Work with clinicians in your chosen pathway to streamline referral. A referral process requiring ten clicks is unlikely to be used. But even a simple referral pathway needs clinicians to remember to use it. Provide regular refreshers to help clinicians recognise suitable patients and keep the intervention front of mind. It’s a mistake to assume that initial training alone will translate into referrals. 

Even when clinicians are making referrals, this doesn't mean patients are using the intervention. 

Track where patients drop out between referral, enrolment and first use, and identify opportunities to reduce friction. Warm handoffs from trusted clinicians, simple onboarding that helps patients experience value quickly, and personalised messages (with human support where needed) can all help turn referrals into meaningful engagement.

Great implementation is one of the best levers for growth. The vast majority of revenue to be earned from a health system will come after the pilot. You’ve spent all the time and effort to get a contract signed and have started implementation. Incremental effort to ensure this implementation goes well has a high ROI. Don’t just move on to trying to sign the next pilot. 

Instead, treat deployment as the start of a long and important partnership with the health system. 

Healthcare systems often sign three-year contracts, but renewal is often won in the first three months. Ensure you allocate enough resources to this point of your partnership. Founders often get this wrong, allocating a lot of resources towards identifying partners and negotiating contracts and much less to partnership building and integration. 

In these early months, establish a regular dialogue with your health system partner, meeting weekly if possible. Assign a consistent core team to manage the relationship, rather than sending different representatives to every meeting. Specialists, such as your Chief Medical Officer or scientific lead, can join when their expertise is needed. Use these meetings to demonstrate early value, identify problems and respond quickly to feedback. 

The first three months can also reveal opportunities you hadn't scoped. Clinicians may bring up additional populations who could benefit from the intervention, or suggest new ways it could support existing services. Be receptive to these ideas, but assess them collaboratively rather than rushing into expansion.

If things go wrong, which they will, you must respond in a way that maintains and builds trust with your partner. Good, honest responses to setbacks can strengthen the partnership. Bad responses will undermine the trust you’ve worked so hard to build.

Your aim is to become a partner in solving the health system's problems. You can use quarterly business reviews to demonstrate value beyond the original agreement. This can help you to break out of the ‘vendor’ box towards a useful collaborator. Come to these meetings with insights from other deployments, share lessons learned and explore how these might help address your partner's evolving priorities. 

Successful deployments can do more than secure renewals. They can also help you win new healthcare partners. Histon recalls taking calls from other health systems interested in companies Kaiser Permanente had deployed. Having seen these companies in action, Histon could share firsthand experience of the partnership and its impact on patients.

Healthcare systems operate within closely connected networks; people are much more trusting of their peers' recommendations than a stranger’s pitch. A successful deployment can become a powerful reference for future partnerships, making disciplined implementation a growth strategy in its own right.

Choose the right partners, define shared success and drive adoption by embedding your solution in clinician workflows. Prove that you can deliver on the outcomes your health system cares about and build relationships with a coalition of stakeholders. Do all this while prioritising trust, and you’ll be a partner, not just a vendor. Implementation may not be glamorous, but it is one of the most powerful drivers of growth.

Many thanks to Trina for sharing their insights and experience for this article. If you want to get more tactical information on how to build an impactful mental health organisation, consider becoming a Hemingway Pro Member.

That’s all for this week.

Keep fighting the good fight!

Steve Duke

Founder of Hemingway