AutomationMart
Home/Browse/Autonomous AI crawler
n8n

Autonomous AI crawler

n8nn8n28 modulesv1.0
OpenAI

This workflow with AI agent is designed to navigate through the page to retrieve specific type of information (in this example: social media profile links). The agent is equipped with 2 tools: - text tool: to retrieve all the text from the page, - URLs tool: to extract all possible links from the page. πŸ’‘ You can edit prompt and JSON schema connected to the agent in order to return other data then social media profile links. πŸ‘‰ This workflow uses Supabase as storage (input/output). Feel free to

At a glance

Autonomous AI crawler is a ready-made n8n workflow you import as a workflow JSON file β€” no build required. It connects OpenAI. It's free to download. Follow the 5-step import below to go live in minutes.

Platform
n8n
Connects
OpenAI
Modules
28
Price
Free
Version
v1.0
Autonomous AI crawler workflow diagram

About this workflow

This workflow with AI agent is designed to navigate through the page to retrieve specific type of information (in this example: social media profile links). The agent is equipped with 2 tools: - text tool: to retrieve all the text from the page, - URLs tool: to extract all possible links from the page. πŸ’‘ You can edit prompt and JSON schema connected to the agent in order to return other data then social media profile links. πŸ‘‰ This workflow uses Supabase as storage (input/output). Feel free to change it to any other database of your choice. 🎬 See this workflow in action in my YouTube video. How it works? The workflow uses the input URL (website) as a starting point to retrieve the data (e.g. example.com). Using the "URLs tool", the agent is able to retrieve all links from the page and navigate to them. For example, if you want to retrieve contact information, agent will try to find a subpage that might contain this information (e.g. example.com/contact) and extract the information using the text tool. Set up steps 1. Connect database with input data (website addresses) or pin sample data to trigger node. 2. Configure the crawling agent to retrieve the desired data (e.g. modify prompt and/or parsing schema). 3. Set credentials for OpenAI. 4. Optionally: split agent tools to separate workflows. If you like this workflow, please subscribe to my YouTube channel and/or my newsletter.

n8n

How to import this n8n workflow

  1. 1

    Download the workflow JSON file after purchase.

  2. 2

    Open n8n β†’ click the menu β†’ Import from File.

  3. 3

    Select the downloaded JSON and import.

  4. 4

    Set up credentials for each node that requires them.

  5. 5

    Click Execute Workflow to test, then activate.

Setup guide

Setup guide included

Purchase to unlock the full step-by-step guide

Related N8n workflows

Detect financial risk and orchestrate compliance with GPT‑4o, Slack and email

How It Works This workflow automates comprehensive risk signal detection and regulatory compliance management across financial and claims data sources. Designed for risk management teams, compliance officers, and financial auditors, it solves the critical challenge of identifying potential risks while ensuring timely regulatory reporting and stakeholder notifications. The system operates on scheduled intervals, fetching data from multiple sources including financial APIs and claims databases, th

Free

πŸ”„οΈ AI warehouse inventory cycle count bot using GPT, Telegram and Google Sheets

Tags: Logistics, Supply Chain, Warehouse Operations, Paperless Processes, Inventory Management Context Hi! I’m Samir, Supply Chain Engineer, Data Scientist based in Paris, and founder of LogiGreen. > Let's use AI with n8n to help SMEs digitalise their logistics operations! Traditional inventory cycle counts often require clipboards, scanners, and manual reconciliation. With this workflow, the operator walks through the warehouse, sends voice messages, and the bot automatically updates the in

Free

Send Telegram messages for new articles using RSS feeds and ChatGPT text generation

Automatically send Telegram messages for new articles by converting RSS feed content to text, generating summaries with OpenAI, and sharing updates via Telegram.

Free

Manage finances, tasks, tweets and Gmail with GPT‑4.1 on WhatsApp

n8n Wizard πŸͺ„ – Your personal AI assistant inside WhatsApp This workflow transforms WhatsApp into a powerful personal AI using n8n + Green-API. Send text or voice messages β€” the assistant understands intent and handles daily tasks automatically. Key features - πŸ’° Expense & income tracking β€” record spending, view summaries & category breakdowns (Google Sheets, append-only) - βœ“ Google Tasks management β€” create, list, update, delete tasks & reminders - 🐦 Post to X/Twitter β€” write and publish single t

Free

Telegram AI bot with LangChain nodes

This workflow connects Telegram bots with LangChain nodes in n8n. The main AI Agent Node is configured as a Conversation Agent. It has a custom System Prompt which explains the reply formatting and provides some additional instructions. The AI Agent has several connections: - OpenAI GPT-4 model is called to generate the replies - Window Buffer Memory stores the history of conversation with each user separately - There is an additional Custom n8n Workflow tool (Dall-E 3 Tool). AI Agent uses this

Free

Index Documents from Google Drive to Pinecone with OpenAI Embeddings for RAG

Google Drive Upload Trigger β†’ Pinecone Vector Upsert for Document Indexing Category: AI & LLM / Document Indexing Level: Intermediate Tags: Google Drive, Pinecone, OpenAI, Embeddings, Vector Store, LangChain, RAG πŸ“„ What This Workflow Does This workflow watches a specific Google Drive folder and automatically uploads any newly added document to a Pinecone vector database β€” complete with OpenAI-generated embeddings. Perfect for setting up retrieval-augmented generation (RAG) pipelines, semantic se

Free

Generate AI videos with Sora2 Pro, save to Google Drive and upload to YouTube

This workflow automates the entire process of creating, managing, and publishing AI-generated videos using OpenAI Sora2 Pro, Google Sheets, Google Drive, and YouTube. --- Advantages βœ… Fully Automated Video Pipeline From idea to YouTube publication with zero manual intervention after setup. βœ… Centralized Control via Google Sheets Simple spreadsheet interface β€” no need to use APIs or dashboards directly. βœ… AI-Powered Video Creation Uses OpenAI

Free

Summarize Microsoft 365 outage alerts with ChatGPT and send to Slack

Built this for a dedicated Slack outage-notifications channel β€” works well on both desktop and mobile. This is for: IT Administrators & small MSPs looking to streamline M365 alerts from one or multiple mailboxes into a single or specific Slack channels IT Admins who prefer ChatOps over management-by-email What does it do Scans for M365 outage alerts emails (every 1 min) Checks if it impacts a specific user region (if the alert calls it out, countries have to be manually set) Summarizes the

Free

Reviews

No reviews yet

Be the first to buy and share your experience.

Leave a review

Sign in to share your experience with this workflow.

Log in to review
Free
No ratings yet

Create a free account to purchase workflows.

  • JSON blueprint β€” instant download
  • Setup guide PDF included
  • 5 downloads Β· valid 30 days
  • Works with n8n

Need help setting this up?

Book a 3-hour live setup session with an Agility consultant.

β‚Ή2,499/ session
3 hrs Β· video call
  • Configure live on Google Meet / Zoom
  • Free follow-up if workflow has defects
  • Platform expert assigned to you
Book installation session
Free