insights

Preparing for an AI Proof of Concept

We explain what is a Proof of Concept, and how can it be leveraged in order to minimize risk and maximize ROI

It is always safer to test the water before jumping into it. It is no different in Artificial Intelligence. In AI, testing the water usually goes through a Proof of Concept (PoC). A PoC gives an opportunity to "kick the tires" on a new AI technology before dedicating to fully implement it. We are continuing our series on how to start your AI project by explaining the measures which can guarantee the success of your Proof of Conceptor.

What is a Proof of Concept

A Proof of Concept (PoC) is the initial testing phase of an idea in order to obtain a confirmation that the idea is feasible and applicable in practice. A Proof of Concept consists of two major pillars: Concept and Proof.

  • The Concept pillar aims at showing how the planned AI solution will work, which features it will offer and more.
  • The Proof pillar aims at evaluating whether your AI solution can reach a specific target.

What are the key steps in planning a Proof of Concept

Identify objectives

The first step to guarantee a successful Proof of Concept is to identify what are the objectives that the PoC is supposed to serve. In other words: what are the organizations’ needs that the solution, if deployed, will be used for. From one organization to another, there can be a multiplicity of objectives. Our recommendation would be to spend sufficient time understanding where you want to go, so the PoC helps you go into this direction.

Define a time limit

Different use cases demand different time limits for the Proof of Concept. Generally, the PoC process can last from a couple of days to a couple of weeks. It is important to define a roadmap for the project, giving a time limit for each step. However, you should remember that a PoC should be limited in time.

Define metrics

It is important to define tangible, measurable and meaningful metrics when starting with a Proof of Concept. Make sure these targets are understood and accepted by everyone involved as they will be your guiding principles during the project. We take a look below at what these metrics - or KPIs - should be.

Assess your data

Data is of paramount importance in an AI project. Before starting a PoC, you should always ensure you have access to the necessary data. Check this article to understand what are the data sources that can be used in an AI project.

What are the KPIs of a Proof of Concept

Here is an indicative list of KPI, to be adapted depending on the very focus of your PoC. Every PoC is unique, so you should be able to tune your list of KPIs depending on the focus of your PoC.

  • Data’s predictive value - whether the quantity of the collected data is enough, and in sufficient quality
  • Technical performance - what is the accuracy of the model, the speed of the AI solution, and its ability to scale
  • Future value - what will be the added value or the Return on Investment (ROI) of the AI solution once deployed, both in the short run and the long run
  • Cost - what would be the forecasted cost of deploying the solution into production
  • User acceptance - if some early users are involved, what is their feedback and level of acceptance of the tool
  • Transparency - whether the results from the model can be explained and understood

These KPIs will be used throughout the PoC and enable you to evaluate its impact.

Evaluate results

At the end of the PoC’s testing period, your team should have a thorough understanding of the pilot’s results and assess whether the solution should be deployed into production. If the KPIs are reached, they should be considered as direct evidence of the effectiveness of the AI model in improving the targeted outcomes; If the KPIs are not reached, you should understand why, and what should enable you to reach these KPIs (and at which cost). This analysis should be part of your final recommendations.

After all data and feedback have been evaluated from the PoC’s testing, a full deployment - if the initiative is still deemed worthy - should be possible. Enjoy the journey!

In our previous article, "What is a Proof of Concept and how to make the most of it", we explain what is a Proof of Concept (PoC), which value can be delivered by a PoC, the methodology behind a PoC, and why is it important.

If you are interested to talk through ideas for your new software solution and explore a proof of concept, book a 30min slot and have a first exchange on English, French or Dutch. Or simply get in touch with our team, we’d love to hear from you!

Insights

Artificial Intelligence and the COVID-19 Black Swan
COVID-19 has been named a Black Swan, but it was predictable. Here is why and how.Read More
How much data is needed to build an AI solution
We explain different AI use cases and the amount of data each of these AI algorithms needs in order to be trained and work properly.Read More
Preparing for an AI Proof of Concept
We explain what is a Proof of Concept, and how can it be leveraged in order to minimize risk and maximize ROIRead More
Who should be your AI project manager
We share some lessons we learned from running AI projects, on how to appoint the best person in charge of your new AI project managementRead More
Artificial Intelligence for price prediction
We explain what is price forecasting, how AI can help and what are the business benefits of AI for price predictionRead More