AutomationMart
Home/Browse/Compare GPT-4, Claude & Gemini Responses with Contextual AI's LMUnit Evaluation
n8n

Compare GPT-4, Claude & Gemini Responses with Contextual AI's LMUnit Evaluation

n8nn8n16 modulesv1.0
OpenAIGemini

PROBLEM Evaluating and comparing responses from multiple LLMs (OpenAI, Claude, Gemini) can be challenging when done manually. - Each model produces outputs that differ in clarity, tone, and reasoning structure. - Traditional evaluation metrics like ROUGE or BLEU fail to capture nuanced quality differences. - Human evaluations are inconsistent, slow, and difficult to scale. This workflow automates LLM response quality evaluation using Contextual AI’s LMUnit, a natural la

At a glance

Compare GPT-4, Claude & Gemini Responses with Contextual AI's LMUnit Evaluation is a ready-made n8n workflow you import as a workflow JSON file — no build required. It connects OpenAI, Gemini. It's free to download. Follow the 5-step import below to go live in minutes.

Platform
n8n
Connects
OpenAI, Gemini
Modules
16
Price
Free
Version
v1.0
Compare GPT-4, Claude & Gemini Responses with Contextual AI's LMUnit Evaluation workflow diagram

About this workflow

PROBLEM Evaluating and comparing responses from multiple LLMs (OpenAI, Claude, Gemini) can be challenging when done manually. - Each model produces outputs that differ in clarity, tone, and reasoning structure. - Traditional evaluation metrics like ROUGE or BLEU fail to capture nuanced quality differences. - Human evaluations are inconsistent, slow, and difficult to scale. This workflow automates LLM response quality evaluation using Contextual AI’s LMUnit, a natural language unit testing framework that provides systematic, fine-grained feedback on response clarity and conciseness. > Note: LMUnit offers natural language-based evaluation with a 1–5 scoring scale, enabling consistent and interpretable results across different model outputs. How it works - A chat trigger node collects responses from multiple LLMs such as OpenAI GPT-4.1, Claude 4.5 Sonnet, and Gemini 2.5 Flash. - Each model receives the same input prompt to ensure fair comparison, which is then aggregated and associated with each test cases - We use Contextual AI's LMUnit node to evaluate each response using predefined quality criteria: - “Is the response clear and easy to understand?” - Clarity - “Is the response concise and free from redundancy?” - Conciseness - LMUnit then produces evaluation scores (1–5) for each test - Results are aggregated and formatted into a structured summary showing model-wise performance and overall averages. How to set up - Create a free Contextual AI account and obtain your CONTEXTUALAIAPIKEY. - In your n8n instance, add this key as a credential under “Contextual AI.” - Obtain and add credentials for each model provider you wish to test: - OpenAI API Key: platform.openai.com/account/api-keys - Anthropic API Key: console.anthropic.com/settings/keys - Gemini API Key: ai.google.dev/gemini-api/docs/api-key - Start sending prompts using chat interface to automatically generate model outputs and evaluations. How to customize the workflow - Add more evaluation criteria (e.g., factual accuracy, tone, completeness) in the LMUnit test configuration. - Include additional LLM providers by duplicating the response generation nodes. - Adjust thresholds and aggregation logic to suit your evaluation goals. - Enhance the final summary formatting for dashboards, tables, or JSON exports. - For detailed API parameters, refer to the LMUnit API reference. - If you have feedback or need support, please email feedback@contextual.ai.

n8n

How to import this n8n workflow

  1. 1

    Download the workflow JSON file after purchase.

  2. 2

    Open n8n → click the menu → Import from File.

  3. 3

    Select the downloaded JSON and import.

  4. 4

    Set up credentials for each node that requires them.

  5. 5

    Click Execute Workflow to test, then activate.

Setup guide

Setup guide included

Purchase to unlock the full step-by-step guide

Related N8n workflows

Flight deal analyzer with weather data using GPT, Google Flights & WordPress

Introduction Automates flight deal discovery and intelligent analysis for travel bloggers and deal hunters. Scrapes live pricing, enriches with weather data, applies AI evaluation, and auto-publishes to WordPress—eliminating manual research and accelerating content delivery. How It Works User submits route via form, scrapes real-time flight prices and weather data, AI analyzes deal quality considering weather conditions, formats results, publishes to WordPress, sends Slack notification—fu

Free

Generate horror faceless shorts with OpenAI TTS, Replicate Video, and YouTube upload

Who’s it for Creators who want to create faceless videos automatically, while keeping human oversight and quality control. How it works / What it does - AI generates 8 story beats, which can be reviewed, edited, or re-ordered by a human. - Each beat is converted into narration (audio), imagery, and short clips. - Final video is assembled and stored in Google Drive, ready for review and regeneration if needed. - Chat commands trigger each step, giving full human control. How to set u

Free

Auto create & publish X-threads (Twitter) with GPT via Telegram & approval loop

AI X (twitter) Threads Bot with Approval Loop This n8n workflow transforms your Telegram messenger into a personal assistant for creating and publishing X-Threads. You can simply send an idea as a text or voice message, collaboratively edit the AI’s suggestion in a chat, and then publish the finished thread directly to X just by saying “Okay.” What You’ll Need to Get Started Before you can use this workflow, you’ll need a few prerequisites set up. This workflow connects three different services,

Free

Send automated replies on Telegram using ChatGPT assistant messages

Automatically respond to Telegram messages using the ChatGPT assistant. This template watches updates, generates replies, and sends them back on Telegram.

Free

Clone LinkedIn writing styles with AI analysis & prompt generation to Airtable

Boost Your Content Speed & Grow Authority in Your LinkedIn Space Are you tired of spending HOURS crafting the perfect LinkedIn posts? This revolutionary n8n workflow is changing the game for content creators, marketing agencies, and business leaders worldwide. WHAT MAKES THIS TOOL A GAME-CHANGER The LinkedIn Style Extractor & Prompt Generator uses cutting-edge AI technology to analyze ANY LinkedIn profile, extract their unique writing fingerprint, and create custom AI prompts that perfectly re

Free

Create a Notion AI assistant with Google Gemini for managing tasks & content

This n8n workflow template uses community nodes and is only compatible with the self-hosted version of n8n. Build an AI agent for Notion (with Notion official MCP server) Use case This template empowers Notion power-users to build their own AI assistant, deeply integrated with their workspace. It solves the constant problem of copy-pasting and context-switching between a separate AI chat and Notion by creating a direct, conversational bridge. Now you can interact with an intelligent agent that

Free

Auto-generate blog & AI image from YouTube videos with Dumpling AI & GPT-4o

Who is this for? This template is designed for content creators, marketing teams, educators, or media managers who want to repurpose video content into written blog posts with visuals. It's ideal for anyone looking to automate the process of transforming YouTube videos into professional blog articles and custom images. --- What problem is this workflow solving? Creating written content from video material is time-consuming and manual. This workflow solves that by automating the entire pipeline:

Free

Convert YouTube videos into SEO blog posts with GPT-4o, Dumpling AI, and Flux

Workflow Description This workflow helps content creators automatically repurpose YouTube videos into SEO-friendly blog posts. It extracts the video transcript, uses AI to generate a full blog post with a relevant image, and sends the complete package via email, ready for publication. Prerequisites/Requirements This workflow relies on external AI services. You will need: 1. OpenAI Account: Used for generating the blog post text (specifically mentioned using GPT-4o in the workflow notes). Creden

Free

Reviews

No reviews yet

Be the first to buy and share your experience.

Leave a review

Sign in to share your experience with this workflow.

Log in to review
Free
No ratings yet

Create a free account to purchase workflows.

  • JSON blueprint — instant download
  • Setup guide PDF included
  • 5 downloads · valid 30 days
  • Works with n8n

Need help setting this up?

Book a 3-hour live setup session with an Agility consultant.

₹2,499/ session
3 hrs · video call
  • Configure live on Google Meet / Zoom
  • Free follow-up if workflow has defects
  • Platform expert assigned to you
Book installation session
Free