Table of Content
All began when GenAI emerged
Back in the days at the end of 2022 when ChatGPT got out and completely disrupted the whole world, we realized this new technology had the potential to bring great value to our Panaya ForeSight customers.
We all knew that this technology would be disruptive and at the same time, could lead to great achievements, but the question was where to focus? As a customer-centric product, our journey began by listening to our users, to hear directly from them which use cases would offer them most value. And this is how ForeSight ExplAIn was first suggested.
The Journey to Foresight ExplAIn
Speaking to our clients, two main issues came to light:
- Understanding complex components like Flows and Apex classes can take a lot of time and a context switch in order to deep dive into them. During the analysis process of the org this can cause a bottleneck.
- Despite the fact that Salesforce professionals are technical, they face difficulties understanding code. Therefore, in some cases, during the investigation process, they need to reach out to developers to understand the functionality of a specific piece of code.
It’s about more than just Gen AI
One of the key benefits of ForeSight ExplAIn that accelerates its adoption as part of the existing user flow is that it provides additional value, which complements Panaya’s Impact Analysis.
Panaya Impact Analysis provides a comprehensive dependency map of the different components in the org, which details the overall inter-connectivity of the components. Foresight ExplAIn then comes in to offer knowledge on the logic of each individual component. The composition of these two capabilities together leads to easy navigation within the implementation logic.
ForeSight ExplAIn, when used in conjunction with Panaya ForeSight Impact Analysis, provides a comprehensive view of the Salesforce Org. It actually supports our unique value proposition, which GenAI drives forward.
ForeSight ExplAIn was integrated into the common user UX flow, allowing its value to be immediately felt, without introducing new methods of working.
Proof of Concept (POC) and Agile Validation
You have definitely heard this before: even if you run as fast as you can, you will not be able to keep up with the pace of the AI revolution. The market waits for no one.
With this in mind, we quickly started on a Proof of Concept (POC) for ForeSight ExplAIn. By collaborating with design partners, we validated its value proposition, while simultaneously setting the architectural foundation for future enhancements.
But we came across a few barriers.
When we started working with GenAI, we quickly realized that it works differently to anything we are used to. There are tasks that are still more suited to standard development. These are the tasks based on concrete logic and clear rules.
GenAI really works its magic with tasks that are difficult to complete using concrete logic like summarization. Basically, those tasks that are more unclear and fuzzier. At first it seems like all you have to do is clearly state what you want.
While in many cases GenAI looks great after the first attempt, in order to get a more robust product, it will require more attention, qualification and fine tuning. This is the reason we see many demos and early pics with GenAI based products.
To achieve a robust product, it is important to have a framework that allows a quick feedback loop, so you can deliver changes faster. Along with that, businesses should setup quality rails to make sure the results are sufficient.
The quality process for LLM based solutions works differently to traditional QA. LLMs are not deterministic in their answers, so standard assertion will not work. Instead, you need to take new quality measures that will validate the value for the user. In some cases, it is common to use LLM as a QA agent to validate your LLM solution and in other cases embed a user feedback loop to improve the quality like in RLHF (Reinforcement Learning from Human Feedback).
Keep Your Eyes on the ball
In the hyper fast ecosystem of LLMs models and application, it is very challenging to maintain your focus. Every week a new model comes out and it is smarter, faster or cheaper.
It is very tempting to check out all the new possibilities and pursue the most up to date models. But when you are developing a product you need to keep your eyes on the ball and in this case the ball is your users, and the use cases you deliver. The adjustment or replacement of a model should be done mainly if it solves a business need or provides an architectural edge in cost, privacy or quality.
An important take is that if you have a good QA and validation process of your product in place, this can reduce the cost of replacing a model. At the same time, it allows you greater flexibility in your GenAI implementation.
From Ideation to Production
Seeing our ideas come to life is exhilarating. With a lot of hard work, development, innovation and creativity, ForeSight ExplAIn transitioned from concept to reality. It now operates in production, assisting users in unraveling complex Salesforce components. Seeing the smiles and responses from our users validates all our efforts.