Evaluate AI agent response correctness with OpenAI and RAGAS methodology
This n8n template demonstrates how to calculate the evaluation metric "Correctness" which in this scenario, measures the compares and classifies the agent's response against a set of ground truths. The scoring approach is adapted from the open-source evaluations project RAGAS and you can see the source here How it works This evaluation works best where the agent's response is allowed to be more verbose and conversational. For our scoring, we classify the agent's response into 3 buckets: True
At a glance
Evaluate AI agent response correctness with OpenAI and RAGAS methodology is a ready-made n8n workflow you import as a workflow JSON file — no build required. It connects OpenAI. It's free to download. Follow the 5-step import below to go live in minutes.
- Platform
- n8n
- Connects
- OpenAI
- Modules
- 24
- Price
- Free
- Version
- v1.0

About this workflow
This n8n template demonstrates how to calculate the evaluation metric "Correctness" which in this scenario, measures the compares and classifies the agent's response against a set of ground truths. The scoring approach is adapted from the open-source evaluations project RAGAS and you can see the source here How it works This evaluation works best where the agent's response is allowed to be more verbose and conversational. For our scoring, we classify the agent's response into 3 buckets: True Positive (in answer and ground truth), False Positive (in answer but not ground truth) and False Negative (not in answer but in ground truth). We also calculate an average similarity score on the agent's response against all ground truths. The classification and the similarity score is then averaged to give the final score. A high score indicates the agent is accurate whereas a low score could indicate the agent has incorrect training data or is not providing a comprehensive enough answer. Requirements n8n version 1.94+ Check out this Google Sheet for a sample data
How to import this n8n workflow
- 1
Download the workflow JSON file after purchase.
- 2
Open n8n → click the menu → Import from File.
- 3
Select the downloaded JSON and import.
- 4
Set up credentials for each node that requires them.
- 5
Click Execute Workflow to test, then activate.
Setup guide
Setup guide included
Purchase to unlock the full step-by-step guide
Related N8n workflows
Automatically classify and label Gmail emails with Google Gemini AI
Send an email notification for every new article from RSS feed using
Multilingual voice & text Telegram bot with ElevenLabs TTS and LangChain agents
Find YouTube videos by keywords, summarize with ChatGPT, and email results
Automate financial operations with O3 CFO & GPT-4.1-mini finance team
Automate Pinterest analysis & AI-powered content suggestions with Pinterest API
Discover hidden website API endpoints using regex and AI
Call center analytics with dual-AI verification using DeepSeek models
Reviews
No reviews yet
Be the first to buy and share your experience.
Leave a review
Sign in to share your experience with this workflow.
Create a free account to purchase workflows.
- JSON blueprint — instant download
- Setup guide PDF included
- 5 downloads · valid 30 days
- Works with n8n
Need help setting this up?
Book a 3-hour live setup session with an Agility consultant.
- Configure live on Google Meet / Zoom
- Free follow-up if workflow has defects
- Platform expert assigned to you