Running an 8-week generative AI pilot

Weeks 1 and 2: Ambition and foundation

In the initial phase of the 8-week pilot, you focus on building a strong foundation.

In the first two weeks, you concentrate on defining the project team, creating a detailed project plan and conducting a formal kickoff with an introduction to generative AI. This phase is crucial to align all stakeholders with the project goals and ensure that the necessary IT and security requirements are met.

You start by forming the project team and establishing a clear working structure. The project plan is then detailed, outlining the steps, timelines and resources needed. During the formal kickoff, you must introduce the team to generative AI concepts and set the stage for the pilot’s objectives. This is where you define the ambition and desired outcomes, ensuring that everyone understands the project’s goals.

It is also essential to verify that all necessary IT security measures are in place and that you have the right people, resources and data access. If any of these critical elements are missing, it is better to pause and address these gaps before proceeding.

The measures you take in this preparation phase ensure that you can smoothly move forward with the pilot without unnecessary interruptions.

Weeks 3 to 6: Pilot development and testing

The core of the pilot revolves around 2-week sprint cycles dedicated to iterative development and testing. Each sprint cycle includes planning, feature identification and hands-on testing with target user groups. This phase typically involves two to three sprints, depending on the complexity of the solution.

In the first week of each sprint, you conduct a pilot design workshop to scope the features and success criteria. This involves identifying the necessary data, preparing it and setting up the initial technical environment. By the end of the first week, your team should have a preliminary version of the solution ready for internal testing.

The second week of the sprint focuses on refining and testing these preliminary solutions. You conduct tests with target user groups to gather feedback and make necessary adjustments. The outcome of this week is a more polished solution that is ready for demonstration. You repeat this cycle of identification, development and testing to ensure comprehensive coverage and improvement of the pilot solution.

It is important to note that this 8-week format is flexible. Sometimes, additional sprint cycles might be necessary, extending the pilot to 10 or 12 weeks, especially for more complex solutions. However, the core of these middle sprint cycles remains consistent, focusing on iterative development and user feedback.

Weeks 7 and 8: Scaling and the road ahead

In the final two weeks, you shift the focus to assessing the success of the pilot and planning for scaling. This phase involves a thorough evaluation of the pilot outcomes, documentation of solutions and development of a comprehensive scaling plan.

By the end of the sprint cycles, you gather all the learnings from the project. The findings and the code base are reviewed to ensure that they support a scaling path. You also document the requirements for deploying the solution, including specific user needs and IT infrastructure considerations. Often, this phase includes developing a business case to present to management, outlining the potential impact and benefits of scaling the solution.

During this phase, you also create a benefits realisation plan detailing the expected value of the pilot when fully scaled. This plan includes the necessary architectural and capability boosts required to support broader implementation. The final step involves agreeing on the next steps and planning for future use cases, ensuring a smooth transition from pilot to production.

Conclusion

Running a generative AI pilot within an 8-week framework provides a structured yet flexible approach to testing and validating AI solutions. By breaking down the process into clear phases and sprints, businesses can efficiently manage resources, gather valuable insights and develop a solid foundation for future AI initiatives. This framework not only helps identify immediate value but also sets the stage for scalable and impactful AI solutions.

Before embarking on the pilot, it is essential to mobilise the team and ensure that proper IT security and infrastructure are in place. This typically involves setting up a sandbox environment with a frontier-class LLM model like GPT-4, Gemini 1.5, Claude 3.5 or Llama 3. A well-structured team usually includes one or two tech specialists and one business resource with light support from IT and Legal and substantial ad hoc support from the line of business or subject matter experts.

This pilot is not a typical machine learning project. Instead of feature identification, we focus on scoping the solution’s functionality. By the end of the pilot, tools like an impact case or benefits realisation plan can provide valuable insights and a clear path forward, ensuring the pilot’s success and scalability.