Run fair split tests on your chat assistant prompts. Each chat session is assigned to either a baseline prompt or an alternative prompt and stays on that path for the whole conversation. This helps product and marketing teams compare tone, instructions, and style with clean data.

A chat message triggers the flow. A Set node stores both prompt versions. Supabase checks a table called split_test_sessions for the session id. If the session is new, it gets a random assignment. Another Set node picks the correct prompt for that session. The AI Agent then answers with the OpenAI Chat Model and saves conversation history in Postgres so the bot remembers context. This design keeps each session stable, which makes results easier to measure.

To set it up, connect Supabase, OpenAI, and your Postgres database. Update the baseline and alternative prompts in the Define Path Values node, activate the workflow, and test in the n8n chat. Expect faster experiments, less manual work, and clearer insight into which prompt leads to better results like higher engagement or better answer quality.

• Chat trigger listens for new messages and starts the flow automatically
• Set node defines baseline and alternative prompts in one place for easy edits
• Supabase check gets the session record by session id to see if it already exists
• Random assignment writes a new record with a true or false flag for the alternative prompt
• Conditional Set picks the correct prompt based on the stored flag
• AI Agent uses the chosen prompt as a system message to guide responses
• OpenAI Chat Model powers fast and smart replies for each message
• Postgres Chat Memory saves history by session so the bot remembers context

• Reduce manual variant assignment from hours to minutes by auto storing session choices in Supabase
• Keep tests clean by fixing each session to one prompt, improving result accuracy by removing cross mixing
• Support many chat sessions at the same time with session based memory in Postgres
• Speed up experiment cycles by letting teams swap prompts without code changes
• Connect OpenAI and Supabase in one flow for easier test management and data checks

1. Import the template into n8n: Create a new workflow in n8n > Click the three dots menu > Select 'Import from File' > Choose the downloaded JSON file.
2. You'll need accounts with Supabase, OpenAI and PostgreSQL. See the Tools Required section above for links to create accounts with these services.
3. In your Supabase project, create a table named split_test_sessions with columns session_id as text and show_alternative as boolean. Make sure you have insert and select permissions for your service key.
4. Open the Supabase nodes in n8n and connect credentials. If unsure, double click the node, choose the 'Credential to connect with' dropdown, click 'Create new credential', then follow the on screen instructions to integrate that service.
5. Open the OpenAI node and connect your OpenAI API key. If unsure, create a new credential from the dropdown and paste your API key from the OpenAI API page.
6. Open the Postgres Chat Memory node and connect to your Postgres database. If using Supabase, use your connection string. If unsure, create a new credential in the dropdown and follow the prompts.
7. In the Define Path Values node, enter your baseline and alternative prompt text. Keep both clear and different enough to measure impact.
8. Confirm the Get Correct Prompt node maps the chosen prompt to the field named prompt. This is used as the system message for the AI Agent.
9. Enable the workflow and open the n8n chat. Send several messages with different sessions to ensure both prompt paths are being assigned.
10. In Supabase, check the split_test_sessions table to see new rows with session ids and the show_alternative flag. Verify you see a mix of true and false assignments.
11. If no rows appear, recheck the Supabase credential and table name. If replies do not remember context, verify the Postgres Chat Memory session key is the same session id from the trigger.
12. Optionally switch the model in the OpenAI node if you need a different model. Test again and compare response quality across sessions.