Turn any web page into clean, structured records that your team can search later. This setup is great for research teams, marketing ops, and product managers who need fresh insights from the web and a fast way to send results to other tools.

Here is how it runs. You start it by clicking Test workflow. A Set node takes your target URL and a webhook URL. The flow calls Bright Data Web Unlocker to fetch the page. Google Gemini then formats the raw HTML into a neat JSON shape with fields like id, title, summary, keywords, and topics. The flow posts this structured data to your webhook. Next, an AI agent extracts key facts, splits the text into chunks, creates embeddings with Google Gemini, and writes them to your Pinecone index for semantic search.

To use it, you need accounts for Bright Data, Google Gemini, and Pinecone, plus a webhook receiver. Enter your URL and webhook in the Set node, add your credentials, and pick your Pinecone index. Expect faster research, fewer copy paste tasks, and a searchable knowledge base. Common uses include news tracking, competitor monitoring, and content research for blogs and briefs.

• Manual start with a Test workflow button for safe runs
• Set node to enter the target URL and your webhook endpoint
• Bright Data Web Unlocker HTTP request pulls raw page content
• Google Gemini formats results into a clear JSON structure
• Structured Output Parser enforces valid fields like title and keywords
• AI Agent extracts key facts and cleans the text
• Recursive text splitter creates search friendly chunks
• Google Gemini embeddings turn text into vectors
• Pinecone insert writes vectors to your selected index
• Two webhook sends notify your system with structured data and agent output

• Reduce manual research from 2 hours to 10 minutes per page
• Streamline web data processing by 70 percent with one click
• Improve data quality by 60 percent using a fixed JSON schema
• Handle 10 times more pages with Pinecone vector indexing
• Connect Bright Data, Google Gemini and Pinecone in one flow
• Send near real time alerts to your app through a webhook

1. Import the template into n8n: Create a new workflow in n8n > Click the three dots menu > Select 'Import from File' > Choose the downloaded JSON file.
2. You'll need accounts with Bright Data, Google Gemini and Pinecone. See the Tools Required section above for links to create accounts with these services.
3. In n8n, open the Set node and update the url with the page you want to crawl and the webhook_url with a URL that can receive POST requests. You can use your app endpoint or a test tool like a temporary webhook receiver.
4. Open the HTTP Request node that calls Bright Data. In the credentials dropdown, click Create new credential and choose HTTP Header Auth. Add an Authorization header with your Bright Data API token from the Bright Data API page, then save.
5. Confirm the Bright Data body parameters include your zone name and format set to raw. Replace the zone value with a valid Web Unlocker zone from your Bright Data dashboard.
6. For Google Gemini nodes, open any Gemini node, choose Create new credential, select the Google Gemini or Google PaLM API type, and paste the API key from Google AI Studio. Save and test the connection.
7. Open the Pinecone Vector Store node. Click Create new credential, choose Pinecone API Key, and paste your API key and environment from the Pinecone console. Select or enter your index name.
8. Check the Embeddings node is set to models/text-embedding-004 and linked to the same Gemini credential.
9. Review the Structured Output Parser and JSON formatter prompts. Keep the schema fields you need such as id, title, summary, keywords, and topics.
10. Click Test workflow. Verify you receive a POST on your webhook with the formatted data. In Pinecone, confirm new vectors appear in the chosen index.
11. If you see a 401 or 403 error, recheck API keys, the Bright Data zone, and header names. If no vectors appear, confirm the index name and that embeddings are being created.
12. Once validated, duplicate the workflow and change the Set node url for each new source you want to index.