Turn long web pages into clean markdown with links, ready for AI and research. Teams that audit content, build knowledge bases, or study competitors can load a list of URLs and get structured results without manual copy and paste.

The flow starts when you click run. A sample list or your database feeds a Page column. The list is split so each URL becomes one item. A limit of 40 items controls server memory, and a batch size of 10 respects the Firecrawl rate of 10 requests per minute. A wait of 45 seconds spaces each round. Each URL is sent to Firecrawl, which returns title, description, markdown, and links. A mapping step formats these fields for storage, and the loop repeats until the list is done.

To set it up, add your Firecrawl API key, keep your URL column named Page, and connect your output store. Expect faster content prep and fewer errors. You can process large lists in cycles that fit your server and the API limits. Use it for SEO audits, internal link maps, and AI data pipelines that need clean, consistent text.

• Manual run to kick off on demand processing
• Split a Page array so each URL becomes its own workflow item
• Limit processing to 40 items at a time to control server memory
• Batch 10 requests to align with the Firecrawl rate limit
• Wait 45 seconds between rounds to avoid rate limit errors
• Send each URL to Firecrawl and receive markdown, links, and metadata
• Map title, description, markdown, and links into clean fields
• Loop continues until all URLs are processed
• Placeholders to connect your own input and output data sources

• Reduce manual copy and paste from hours to minutes for large URL lists
• Respect 10 requests per minute automatically with a timed pause
• Improve data quality by pulling titles, descriptions, links, and markdown from the source
• Handle large volumes by processing 40 items per memory cycle and 10 per batch
• Prepare AI ready text that is clean and consistent for analysis

1. Import the template into n8n: Create a new workflow in n8n > Click the three dots menu > Select 'Import from File' > Choose the downloaded JSON file.
2. You'll need accounts with Firecrawl. See the Tools Required section above for links to create accounts with these services.
3. Open the node named Retrieve Page Markdown and Links. In the Authentication section choose HTTP Header Auth, then click Create new credential. Name it Firecrawl Bearer.
4. In the credential set Header Name to Authorization and Header Value to Bearer followed by your Firecrawl API key. Save the credential.
5. In the same HTTP node confirm the method is POST, the endpoint is the Firecrawl scrape path, and the JSON body includes url and formats with markdown and html.
6. Open Example fields from data source and add a few test URLs in the Page array. Keep one URL per entry.
7. Run nodes up to Split out page URLs and check that each URL appears as a separate item with the Page field.
8. Verify 40 items at a time matches your server memory. Lower this value if you see memory errors.
9. Check 10 at a time and the Wait amount of 45 seconds. If you hit rate limit errors, increase the wait time or reduce batch size.
10. Execute the HTTP node on one test item and confirm you see data.metadata.title, data.markdown, and data.links in the response.
11. Review the Markdown data and Links node to confirm fields are mapped correctly for your storage plan.
12. Replace the placeholder nodes for input or output with your own database connectors when ready, then run the whole flow and confirm all batches complete without errors.