AI Case Study: Walk Before You Run to Unlock the Value of AI
If you’re like us—and many of our clients—you’re still cutting through the noise to determine how and where to expand the use of artificial intelligence (AI) in your team’s workflow. One way to reduce risk or exposure is to run an AI pilot project to quickly test a tool’s functionality and learn about its real-world application to your business.
Recently, we partnered with a team to take a custom-built AI tool from idea inception through implementation. Our learnings should help you focus your efforts and make progress on your team’s AI initiatives. If you’re on the cusp of expanding your team’s AI literacy, or if you’re exploring ways to integrate AI more prominently in your team’s daily work, our experience may provide inspiration to nudge you forward into action.
It’s no secret that the AI marketplace is overwhelming, with constant announcements of new features and use cases (read more on the state of enterprise AI here). While many AI tools have a “wow” factor, not all are approachable or immediately applicable to certain businesses or teams. Leaders we engage with across various types of industries and company sizes want to be seen as cutting edge and want to leverage the productivity gains promised by AI tools. However, today’s leaders feel constrained by obstacles like data privacy concerns, budget limitations, or organizational culture barriers.
These constraints were fully present during a recent AI pilot project.
Where Do We Begin
When we started, the requesting team had already completed an important step: identifying potential AI use cases relevant to their business. If you’re at this stage in your process, refer to these practical AI use cases—countless variations exist as the technology continues to evolve. We recommend tapping into the wisdom of your team to brainstorm current pain points that may have the opportunity to be alleviated with AI, and then prioritize those based on true business value (e.g., time saved, manual processes removed, document reviews reduced).
A critical phase of the requesting team’s workflow involved analyzing large amounts of unstructured data, a repetitive and time-intensive process. They saw AI’s potential to streamline their workflow, and save time for their employees, by using an AI tool to take the first pass at analyzing data and generating insights. While they were excited for the possibilities of using AI to solve this problem, they still hadn’t solved for their primary concern: data privacy.
The Holy Grail: Securing Your Data
Perhaps like you, this team was extremely wary about not wanting their confidential data to be leaked or ingested into public AI models. Early on, data security concerns delayed their ability to innovate and leverage the efficiency gains that AI tools offer. When we started working together, we recommended two initial next steps in order to move past those concerns and get them feeling more confident with using AI: first, revisit company AI policies, and second, consider the development of a custom AI tool built within a closed loop system.
Revisit Company AI Policies
By now, most companies have established policies on AI usage to provide guidance to employees and protection to the business. We see a range of policies across our clients on a spectrum of very restrictive to wide open. Typically, the style of AI usage policy matches the company or brand culture and level of risk-aversion—with the more open policies belonging to companies seen more as early adopters of new technology, compared with more restrictive policies showing up at companies who typically are more risk-averse and slower to jump on board with new trends. Factors impacting a company’s approach to using AI include the level of sensitivity of their data, regulatory data compliance requirements that vary by industry, as well as customer/supplier contract requirements.
Given the pace of innovation in AI, your company’s AI policies can’t be on a “set it and forget it” type of schedule. AI policies need to be reevaluated multiple times throughout the year to adapt to the changing landscape and shifting acceptance of certain AI tools that push the boundaries of new norms. Policies must also consider both internal (closed loop) systems vs. external (more open) solutions. Our client modified their AI policy to provide additional clarity to employees on how and when to safely use AI, and in their revised policy, they communicated more explicitly the business risks associated with the misuse of AI.
Stay in the Loop
In coordination with refreshing their AI policy, the requesting team decided to protect their data by building the AI tool in a closed-loop system. A closed-loop system is a self-contained feedback system where data is collected, processed, and utilized within a secure, isolated environment (more often known as a virtual private cloud). This ensures sensitive data remains private (in the “closed loop”) and is not exposed to external entities. For this team, building the AI tool in a closed-loop system was the final unlock needed for them to be ready to go!
Alongside our AI partner Hyacinth, we worked with the requesting team to run a low fidelity pilot, train the model by ingesting sample data, and perform rigorous testing on data security mechanisms. With the assurance that their data was secure, the client was thrilled to see the results of putting the AI tool through multiple tests with various data sets and by various users, and maturing the development of the AI tool and the quality of its output along the way.
What Success Looks Like
The success of our client’s AI pilot can be traced to several key factors:
A Clear Vision and Well-Defined Goals: Establishing a solid foundation began with a clearly defined vision for the tool. For this launch, the team identified specific outcomes, such as reducing time spent processing unstructured data to generate initial insights. Clear goals ensured alignment among stakeholders for the best (i.e. most effective and value-add) path forward.
User-Centric Design: Incorporating employee feedback at every stage of development ensured the tool addressed real-world needs. Early versions were tested to gather insights on functionality, usability, and areas for improvement. This iterative approach allowed for adjustments before a broader rollout, increasing tool relevance and effectiveness.
Demonstrating Practical Value: An AI tool is as much about education as it is about practicality. Our client prioritized clear communication about the tool’s capabilities, limitations, and expected benefits through the development of job aids and Q&A documentation. By demonstrating how the AI tool could reduce time spent on manual repetitive tasks—achieving 60% time savings on average in the early stages of adoption—they built trust among their employees (the end-users). Hands-on training sessions further enabled adoption and opportunities to provide real-time feedback.
Strong Governance Framework: Given the client’s primary concern about data privacy, a robust governance framework was crucial. We created policies to ensure compliance with sensitive data handling, and implemented safety measures within the tool to keep confidential information secure.
These lessons underscore the importance of balancing technical innovation in the new world of AI, with practical applications to gain user trust. By starting small, gathering feedback from the end-user, and planning an intentionally structured roll-out, the requesting team successfully introduced AI into their workflows to increase efficiency and add business value to their processes while avoiding their primary data privacy concerns.
Sustaining Success Post-Launch
For the client we worked with, launching an AI tool through a small-scale pilot project was just the beginning. Their approach to sustaining success centers on a commitment to continuous improvement and proactive adaptation. Early on, the establishment of regular performance monitoring and encouraging employees to provide feedback through a short 3-question digital survey (using Microsoft Forms) allowed for quick refinements to meet evolving needs and address unanticipated issues that arose.
The team recognizes that as their workforce and priorities shift, the AI tool must evolve too. They are aligning the tool’s development roadmap with broader organizational objectives, ensuring it stays relevant and impactful. Maintaining a strong security posture is also a core element of their strategy. With data privacy being a major concern from the outset, they continue to conduct regular policy audits to keep sensitive information secure, which not only protects their data but reinforces trust among employees and stakeholders.
Looking ahead, the team plans to scale the tool’s impact across other workflows and departments. They are building a collaborative culture by inviting employees to identify enhancement requests and additional use cases for AI tools, while encouraging team members to embrace ways AI might enhance (not replace) their work. This emphasis on scalability ensures the tool’s benefits extend beyond its initial scope, driving even greater value for the organization. Ultimately, sustained success is rooted in a forward-thinking mindset, investment into ongoing education and training, which will foster an empowered workforce embracing the powers of AI.
What Are You Waiting For?
The requesting team received a valuable new AI-powered solution to increase their team’s efficiency when processing large quantities of data without compromising data security. Wherever you are in your AI journey, apply some of our AI tool launch learnings and let us know how it unlocks your team’s momentum.
—Amy Stencel & Justin Braun