Autonomous AI crawler
This workflow with AI agent is designed to navigate through the page to retrieve specific type of information (in this example: social media profile links). The agent is equipped with 2 tools: - text tool: to retrieve all the text from the page, - URLs tool: to extract all possible links from the page. π‘ You can edit prompt and JSON schema connected to the agent in order to return other data then social media profile links. π This workflow uses Supabase as storage (input/output). Feel free to
At a glance
Autonomous AI crawler is a ready-made n8n workflow you import as a workflow JSON file β no build required. It connects OpenAI. It's free to download. Follow the 5-step import below to go live in minutes.
- Platform
- n8n
- Connects
- OpenAI
- Modules
- 28
- Price
- Free
- Version
- v1.0

About this workflow
This workflow with AI agent is designed to navigate through the page to retrieve specific type of information (in this example: social media profile links). The agent is equipped with 2 tools: - text tool: to retrieve all the text from the page, - URLs tool: to extract all possible links from the page. π‘ You can edit prompt and JSON schema connected to the agent in order to return other data then social media profile links. π This workflow uses Supabase as storage (input/output). Feel free to change it to any other database of your choice. π¬ See this workflow in action in my YouTube video. How it works? The workflow uses the input URL (website) as a starting point to retrieve the data (e.g. example.com). Using the "URLs tool", the agent is able to retrieve all links from the page and navigate to them. For example, if you want to retrieve contact information, agent will try to find a subpage that might contain this information (e.g. example.com/contact) and extract the information using the text tool. Set up steps 1. Connect database with input data (website addresses) or pin sample data to trigger node. 2. Configure the crawling agent to retrieve the desired data (e.g. modify prompt and/or parsing schema). 3. Set credentials for OpenAI. 4. Optionally: split agent tools to separate workflows. If you like this workflow, please subscribe to my YouTube channel and/or my newsletter.
How to import this n8n workflow
- 1
Download the workflow JSON file after purchase.
- 2
Open n8n β click the menu β Import from File.
- 3
Select the downloaded JSON and import.
- 4
Set up credentials for each node that requires them.
- 5
Click Execute Workflow to test, then activate.
Setup guide
Setup guide included
Purchase to unlock the full step-by-step guide
Related N8n workflows
Detect financial risk and orchestrate compliance with GPTβ4o, Slack and email
ποΈ AI warehouse inventory cycle count bot using GPT, Telegram and Google Sheets
Send Telegram messages for new articles using RSS feeds and ChatGPT text generation
Manage finances, tasks, tweets and Gmail with GPTβ4.1 on WhatsApp
Telegram AI bot with LangChain nodes
Index Documents from Google Drive to Pinecone with OpenAI Embeddings for RAG
Generate AI videos with Sora2 Pro, save to Google Drive and upload to YouTube
Summarize Microsoft 365 outage alerts with ChatGPT and send to Slack
Reviews
No reviews yet
Be the first to buy and share your experience.
Leave a review
Sign in to share your experience with this workflow.
Create a free account to purchase workflows.
- JSON blueprint β instant download
- Setup guide PDF included
- 5 downloads Β· valid 30 days
- Works with n8n
Need help setting this up?
Book a 3-hour live setup session with an Agility consultant.
- Configure live on Google Meet / Zoom
- Free follow-up if workflow has defects
- Platform expert assigned to you