Collect key facts from website pages at scale and return clean results to your app. Great for market research, price tracking, and competitor checks. Teams send one request and get structured answers back fast.

Here is how it works. A webhook receives your JSON with a subject or a direct page link and up to five data points to extract. If you only send a subject and a domain, the flow searches the web and finds the best page match. It then spins up a browser session, can route traffic through a proxy, and tweaks the browser to avoid simple bot checks. If you pass cookies, it injects them to access logged in pages. The flow captures one or more screenshots, converts them to files, and uses an OpenAI vision model to read the page and pull the values you asked for. Branches handle blocked pages, wrong cookies, or missing URLs and always close sessions cleanly.

Setup needs a running browser container, a residential proxy if you scale, and an OpenAI API key. Expect manual checks to drop from hours to minutes while handling many more pages with the same team. Use it to track followers, star counts, pricing tables, and other public or session based info. Send a POST to the webhook and receive structured JSON with the results.

• Webhook intake that accepts subject, target URL, cookie list, and up to five data fields.
• Smart page discovery using a search step and an HTML link match to find the right page when only a subject is given.
• Browser automation with session creation, window resize, and optional proxy support.
• Anti bot tweaks that remove common browser flags to reduce simple detection.
• Cookie injection path for logged in pages and a no cookie path for public pages.
• Multiple screenshots converted to files and passed to an OpenAI vision model for extraction.
• Structured JSON responses with branches for success, blocked pages, or cookie mismatch.
• Reliable cleanup with session delete calls to free resources after every run.

• Reduce manual page checks from hours to minutes by automating data capture end to end.
• Handle up to 10 times more pages with proxy routing and session cleanup that keeps runs stable.
• Improve data accuracy by using screenshots and AI reading to avoid copy paste errors.
• Cut troubleshooting time with clear webhook responses for blocked sites, bad cookies, or missing URLs.
• Use one endpoint for both public and logged in pages by injecting session cookies when needed.

1. Import the template into n8n: Create a new workflow in n8n > Click the three dots menu > Select 'Import from File' > Choose the downloaded JSON file.
2. You'll need accounts with OpenAI. See the Tools Required section above for links to create accounts with these services.
3. In your OpenAI account, create an API key. In the n8n credentials manager, create a new OpenAI credential and paste the API key. Select this credential in each OpenAI node.
4. Deploy a Selenium browser container that is reachable from n8n. If the service URL differs, update the session and navigation HTTP Request nodes to point to your host and port.
5. If you plan to scale, set up a residential proxy and whitelist your server IP. In the Create Selenium Session node, add the proxy argument to route traffic.
6. Open the Webhook node and copy the production webhook URL. This is the endpoint you will call from your app or from curl.
7. Confirm the anti bot script is enabled in the Clean Webdriver node. Keep the sessionId expression intact so it runs on the active session.
8. For private pages, pass session cookies as an array in the request body. The flow will inject them before loading the page.
9. Send a test request with a subject and a domain, or a direct target URL, plus up to five data fields you want to extract.
10. Check the response. If you see a blocked message, switch proxy IPs or reduce request frequency. If you see a cookie error, verify the cookies match the target site.
11. Review the extraction prompts in the OpenAI nodes and adjust field names so they match the values you expect to receive.
12. Run another test and confirm sessions are deleted at the end of the run. Check the execution log for the session delete calls.
13. Set the Limit node if you plan to process many items and want to cap throughput for safety.
14. Secure the webhook by adding a secret header and checking it in the workflow before processing.