n8n

How to Compare OpenRouter in Google Sheets Evaluation?

Send one chat message and see two model answers side by side. The flow also logs both replies, your prompt, and the chat context to Google Sheets for simple scoring. It helps teams choose the best model for an assistant, bot, or content tool before going live.

A chat trigger starts the run when a message arrives. The message is split so each model in your list receives the same input. Memory is isolated per model, so history does not mix and your comparison stays fair. An AI Agent calls the selected OpenRouter model, then the results are prepared, grouped, and pushed to your sheet. The interface shows both answers together by combining the outputs, so you can compare quickly in one place.

Setup is simple. Connect OpenRouter and Google Sheets, then enter the model IDs you want to test. Add a system prompt and tools in the AI Agent if you need task specific behavior. Expect faster decisions, less copy and paste, and a clean record for every test. This is well suited for IT and product teams that want proof before picking a model for production.

What are the key features?

  • Chat message trigger starts a run the moment a user sends a message
  • Define a list of model IDs and split them into separate test items
  • Isolated memory per model using Simple Memory and a Memory Manager
  • AI Agent calls the chosen OpenRouter model for each item
  • Loop sends the same prompt to each model for a clean comparison
  • Prepare Data node maps fields for chat display and Google Sheets
  • Group results for evaluation and write rows to Google Sheets automatically
  • Concatenate answers and set output so both replies appear together in the chat

What are the benefits?

  • Reduce manual copy and paste from 30 minutes to 2 minutes per comparison
  • Automate 100 percent of logging and tracking in Google Sheets
  • Improve evaluation fairness by keeping model histories separate
  • Handle up to five times more model tests per hour using the loop
  • Connect OpenRouter and Google Sheets in one repeatable workflow

How do you set it up?

  1. Import the template into n8n: Create a new workflow in n8n > Click the three dots menu > Select 'Import from File' > Choose the downloaded JSON file.
  2. You'll need accounts with Google Sheets and OpenRouter. See the Tools Required section above for links to create accounts with these services.
  3. Create an OpenRouter API key in your OpenRouter account settings. Keep the key safe.
  4. In n8n, open the Credentials section. Create a new OpenRouter credential. Paste your API key and save. If unsure, double click the OpenRouter node, choose 'Create new credential', and follow the on screen steps.
  5. Prepare your spreadsheet. Copy the provided template or create a new Google Sheet with columns for sessionIdBase, chatInput, model, model_answer, and context.
  6. In n8n, open the Google Sheets node named Add Model Results to Google Sheet. In the credential dropdown, click Create new credential and sign in to your Google account to grant access.
  7. Select your Spreadsheet and Sheet tab in the Google Sheets node. Confirm column mapping matches your headers.
  8. Open the node Define Models to Compare and enter the full model IDs you want to test, such as openai/gpt-4.1 and mistralai/mistral-large.
  9. Optional but recommended: Open the AI Agent node and add a system prompt and tools that reflect the task you want to evaluate.
  10. Start a session from the n8n Chat interface. Send a message. Confirm you see two answers in the chat and a new row in Google Sheets.
  11. If no row appears, recheck the Google Sheets credential, the selected spreadsheet and sheet, and that headers match the fields written by the workflow.
  12. If both answers look the same, verify the model IDs differ and the OpenRouter credential is active. If context seems shared, confirm the sessionId field contains the model name.

Tools Required

$24 / mo or $20 / mo billed annually to use n8n in the cloud. However, the local or self-hosted n8n Community Edition is free.

Google Sheets

Sign up

Free: $0 (Google Sheets API usage has no additional cost; quota limits apply)

OpenRouter

Sign up

Free models: $0 via API, 20 requests/min; 50/day or 1000/day with ≥10 credits

Similar Templates

Join Futurise to access 1,200+ automation templates

Get instant access to ready-made automation workflows for n8n, Make.com, AI agents, and more. Download, customise, and deploy in minutes.