Evaluating Tasks for AI Implemenation
This blog post is meant to inform semi- or non-technical managers about AI possibilities.
Between omniscient and useless
People who have no or little experience with AI tend to hold one of two extreme opinions: either they expect an “AI something” to autonomously connect to all of their data and immediately offer opinions on how to optimize their business, or they have decided AI is useless.
The truth is, as always, somewhere in the middle and is without any doubt moving towards the more valuable end of the spectrum.
Let’s start by looking at an analogy between an AI component or agent and a new human employee to get a better understanding of what is involved in an AI project compared to a conventional software project.
A New Employee
Our current single-sentence test for feasibility is “If you were able to hire a very smart and efficient human employee, would you expect them to be able to perform the task at an acceptable level?”
For the purposes of this blog post, let’s treat the AI implementation as a new human employee. If you could complete the required “onboarding”, the task might be feasible for AI.
First, the new employee will need access to your data, and you need to explain the data or information to be used. This explanation is called metadata. It is given as natural language without any specific format. More detail is better, especially if something has a name which is a term specific to your industry.
Your company will have rules and processes for employees to follow. The new employee also needs to have these explained for them. The explanation is called a system prompt. It is also given as natural language.
The individual tasks the new employee should perform are described using prompts, which are given in a natural language. When the new “employee” can complete the given task successfully, the task can be automated by writing simple software to prompt the finished AI implementation and integrate with whatever system is to receive the results.
Conventional Software Components in AI Projects
Some of your data processing or reporting must be done in a very specific way due to laws, regulations or convention. This data processing can be done using deterministic software components written to your specifications. This is one of the non-AI parts of your AI implementation, and it works similarly to any system integration project.
The other major conventional software part of an AI project is integrating with your ERP systems and databases. There is no magic bullet, but we implement our AI projects so that the system integration components have an internal HTTP API (or equivalent) so that other AI projects can reuse them.
Some Specific Questions We Usually Ask
Our offer process has a wide range of concerns we like to discuss. Here are some of the fundamental questions and why we ask them:
Is the information required for the task contained in a confusingly large amount of documents of different types?
It is quite common that your documents are stored somewhere on your network without much organization. Retrieval-augmented generation tools will ingest your documents and search them semantically to provide the AI implementation with relevant data. Most of the time they can replace what is usually known as “training” or “fine-tuning” the model.
Is handling data a significant part of the solution? Are there systems involved that are more feasible for software to access than a human to use?
A very common candidate for AI implementation is augmenting or replacing human work that is currently done using multiple separate systems with user interfaces that are not appropriate for the task at hand, and has not been implemented as a conventional system integration project because the process is not really suitable for conventional software.
Do you have a method of determining if the results of the process are correct? Failing that, do you have a method for determining that the results are incorrect?
Can the AI implementation’s output be validated by conventional software? The other option is to use your domain experts to do the validation, which can be a significant amount of work in the long run. Of course, you could use another LLM to validate the results.
Conclusion
If you have any questions or ideas for AI projects, don’t hesitate to contact us. We will discuss AI possibilities with you using your language. As part of our sales process, we usually create a demonstration or simple proof of concept of AI capabilities using your data at no cost to you. We can do this because we have implemented our own AIVO metadata+vector database solution and know how to use it quite efficiently. Give us a call!
