How to Automate Conversation QA Using n8n & AI

Customers are advised to review AI-handled conversations after enabling AI Agent, but manual QA becomes too time-consuming at scale. This guide shows how to automate conversation QA using n8n + an AI model, so every closed conversation can be reviewed automatically and logged into Google Sheets for reporting.

In summary, this n8n workflow is triggered every time a conversation is closed. Once triggered, it calls the List Messages API to fetch the last 50 messages (with an option to fetch up to 100 using pagination), then passes the transcript to an AI model for QA processing — such as sentiment analysis, engagement scoring, and resolution checks. The results are tracked in a Google Sheet so you can easily look up any conversation by contact ID.

What You'll Build

An automation that:

• Triggers when a conversation is closed

• Retrieves the last 50 messages using the List Messages API (optionally up to 100 with pagination)

• Cleans and formats messages into a transcript

• Sends the transcript to an AI model for QA evaluation

• Parses AI output safely (JSON-only contract)

• Appends results into Google Sheets (one row per conversation)

Requirements

Tools

• Automation platform: n8n

• Respond.io Developer API — List Messages endpoint

• AI API — OpenAI, Anthropic, Gemini, or any provider supported by n8n

• Reporting / storage — Google Sheets (or your preferred storage tool)

Credentials

• Respond.io Developer API key

• AI provider API key (OpenAI, Anthropic, Gemini, etc.)

• Google Sheets credential in n8n (OAuth or service account)

Before You Start

1) Create a Google Sheet for QA results

Create a spreadsheet and add a sheet (e.g. QA_Results) with these headers:

• timestamp

• contact_id

• opened_at

• channel

• overall_score

• resolved

• customer_sentiment

• reasoning

• engagement_score

2) Confirm your trigger payload fields

From the Conversation closed trigger output, identify the exact field names for:

• Conversation ID

• Contact ID

• Conversation opened timestamp

• Conversation closed timestamp (optional)

Field names vary by implementation. In the code nodes below, you'll see placeholders like trigger.conversationId. Update those mappings to match your trigger output.

Step-by-step Guide

1. Trigger: Conversation Closed

When a conversation is closed in respond.io, we want to automatically trigger the n8n workflow. This ensures every completed conversation goes through QA — no manual review needed.

n8n node:Conversation closed trigger

1. In n8n, add the respond.io trigger node.

2. Choose Conversation Closed.

3. Connect your respond.io API key credentials. Learn how to set this up in n8n with our integration guide.

This trigger makes sure you only review conversations after they are finished.

2. Fetch Conversation Messages

In this step, you'll fetch the last 50 inbound and outbound messages using the List Messages API. This is usually enough for a thorough QA review. If you need more context, you can optionally fetch another 50 messages (up to 100 total) using pagination.

n8n nodes:

• get 1st 50 messages (HTTP Request)

• Is there a second page? (IF) — optional

• get 2nd 50 messages (HTTP Request) — optional

2.1 Get first 50 messages

Node:get 1st 50 messages

1. Select Core > HTTP Request

2. Method: GET

3. URL: your List Messages endpoint

Example URL (replace with your actual API base URL + endpoint format):

https://api.respond.io/v2/contact/{{identifier}}/message/list

Where identifier needs to be replaced with one of the following formats: id:<contactID>, phone:+<countryCodeAndPhone>, or email:<contactEmail>

1. Turn on Send Query Parameters > Using Fields Below, then add:

   • Name: limit

   • Value: 50

If your API supports it, set sorting oldest → newest. Example: sort=asc.

1. Enable Send Headers > Using Fields Below, then add:

   • Name: Accept

     • Value: application/json

   • Name: Authorization

     • Value: Bearer <your_respond_api_key>

If your List Messages response includes pagination.next, you can use it directly to fetch the second page.

2.2 Check and get more than 50 messages (Optional)

If 50 messages isn't enough for your QA needs, you can fetch a second page of 50 messages. Add an IF node to check whether there are more messages available.

Node:Is there a second page? (IF)

• Left value (Expression): {{ $json.pagination.next }}

• Operator: is not empty

If true, it means there are more messages to fetch. If false, the workflow continues without fetching more — this prevents the workflow from failing when there's no second page. The Merge node downstream will wait for both paths to complete, so the workflow still runs smoothly either way.

2.3 Get the next 50 messages (Optional)

If the IF node passes (a second page exists), fetch the next batch.

Node:get 2nd 50 messages

1. Add another HTTP Request node.

2. Set:

   • Method: GET

   • URL: {{ $json.pagination.next }}

3. Add the same headers:

   • Accept: application/json

   • Authorization: Bearer <your_respond_api_key>

3. Merge Message Pages (Optional)

This step is only needed if you're fetching more than 50 messages. If you chose not to fetch a second page, you can skip this node and connect get 1st 50 messages directly to the next step (Clean + keep messages since last open).

The Merge node combines the first 50 messages with the second 50 messages into a single list. Without it, the workflow can't process two separate API responses together.

n8n node:Merge

1. Add Flow > Merge.

2. Set Mode to Append.

3. Set Number of Inputs: 2

4. Connect:

   • Input 1: get 1st 50 messages

   • Input 2: get 2nd 50 messages

4. Clean and Filter Messages

The API response is still raw and contains a lot of extra information the AI doesn't need. This step cleans it up — removing old messages, normalizing sender labels, and structuring everything into a simple list that's ready for AI processing. All you need to do is add the Code node and copy-paste the JavaScript below.

n8n node:Clean + keep messages since last convo open

To do this, Add node → Select Core → Code → Code in JavaScript.

This code will:

1. Combine messages from both pages

2. Filter out messages sent before the conversation opened

3. Sort messages from oldest → newest

4. Normalize senders into consistent labels: Contact, AI Agent, Human Agent, Workflow

5. Add message index numbers

6. Return a clean, structured list for the transcript step

Notes: This example filters using messageId compared to the opened timestamp converted to microseconds. If your API provides createdAt timestamps instead, filter by createdAt rather than messageId. Ensure your conversation_open_timestamp includes a timezone. If it does not, set it to your workspace timezone before parsing.

Paste this into the Code node:

Make sure to set Mode = Run Once for All Items and Language = JavaScript.

The script above is a reference. Your workspace may return different fields or structures. To build the right JSON parsing logic for your setup, copy the output from the previous node, paste it into an AI tool (e.g., ChatGPT or Claude) along with the reference script above, and describe the output format you need. The AI can then adapt the script to match your actual data.

5. Build Transcript

This step converts the cleaned message list into a Markdown-style transcript for AI processing. Structured transcripts improve AI understanding and reduce hallucinations.

n8n node:Build markdown transcript (Core → Code → JavaScript)

Paste this into the Code node:

Make sure to set Mode = Run Once for All Items and Language = JavaScript.

6. AI-based conversation reviews

This node sends the transcript to an AI model (e.g., OpenAI 5.4) and returns a structured JSON response.

How to set up

1. Select your AI provider credentials. In this example, we will use OpenAI.

2. Resource: Message a Model

3. Operation: Message an Assistant

4. Messages:

   1. Type: Text

   2. Role: User

   3. Prompt: {{ $json.transcript }}

5. Simplify Output: Toggle On

6. Add Option:

   1. Instructions - This is just an example template, but you can input this to another AI (i.e. ChatGPT, Claude, etc) to edit it to your needs:

Required: strict JSON output

Your assistant instructions should require a JSON Schema.

Recommended Output Format:

1. Type: JSON Schema (recommended)

2. Name: Conversation QA Schema

3. Strict: Toggle On

4. Schema: This is also just an example schema to output over quality score of the conversation, status of the issue, customer sentiment, engagement score, reasoning behind its result, and failure flags of the conversation. Remember to edit it so it fits your needs.

After this node, connect it to a Map Output node and continue to step 7.

7. Parse and normalize AI output

This step prevents malformed or partial AI responses from breaking your workflow.

n8n nodes:

• Map Output (Data Transformation → Edit Fields / Set)

• Parse Output (Core → Code → JavaScript)

7.1 Map Output

Node:Map Output

• Mode: Manual mapping

• Create a field called Output and map it to the AI response.

This makes the next node consistent (it can always read $json.Output).

7.2 Parse Output safely

Node:Parse Output

Paste this into the Code node:

Flattening note: arrays and nested objects will become columns like issues_0_type, issues_0_severity, etc. This makes it easier to store in Google Sheets.

8. Store Results

The final step stores the QA results into Google Sheets for tracking and reporting.

n8n node: Google Sheets (append)

• Credential to connect with: Connect your Google Sheet account

• Resource: Sheet Within Document

• Operation: Append Row

• Select document: From list → Select the spreadsheet name

• Select sheet: From list → Worksheet name

• Mapping Column Mode: Map each column manually

Then, map each value to the column name you've set up in your sheet, ideally following the column names described in Section 1.

Where each field comes from

From the webhook trigger (conversation metadata):

Map these using the Webhook node reference, e.g.:

{{ $('Webhook').first().json.body.contact_id }}

• contact_id

• conversation_opened_timestamp

• conversation_closed_timestamp

From the parsed AI output (Section 7):

Map these using the parsed output fields, e.g.:

{{ $json.overall_score }}

• overall_score

• resolved

• sentiment

• reasoning

• engagement_score

For debugging (recommended during rollout):

• raw_ai_output — store the full AI response so you can spot-check results and refine your prompt

💡 Tip: Check the actual output of your Webhook node to confirm the exact field names — they may vary depending on your workspace's trigger configuration.

Optional Enhancements

• Mask PII before AI review (emails, phone numbers, order IDs)

• Process more than 100 messages by looping through pagination until pagination.next is empty

• Sample conversations (e.g., only review 10% of closed conversations to control costs)

• Route high severity issues to Slack/Teams for faster follow-up