AI Chat: How to Configure your Assembled AI Chat Using an LLM

This guide shows you how to design effective AI chat behavior using a large language model (LLM), and how to place the resulting configuration into the correct fields in the AI chat product.

You do not need prompt-engineering experience to follow this guide.

If you're curious for more information about the different setting sections in Assembled's AI Chat, check out the AI Chat Configuration: Best Practices & Guidelines doc.

What You’ll Accomplish

By the end of this guide, you will have:

• A Chat Style Guide (how responses should sound)
• (Optional) Company-Specific Handoff Instructions (when to escalate)
• Critical Instructions for Chat (how the assistant should answer)

You’ll generate these using an LLM (like ChatGPT or Claude) and then paste them into the appropriate places in the AI chat product.

How This Works (At a High Level)

AI chat behavior is controlled by different configuration sections that serve different purposes.

Instead of writing one long prompt, you will:

1. Answer a short set of questions about your business and goals
2. Paste those answers into a guided LLM prompt
3. Copy the generated outputs into the correct product fields

Keeping each type of guidance in the right place leads to better accuracy, higher resolution, and fewer unexpected escalations.

Step 1: Answer Key Questions About Your Business

Before using an LLM, write short answers to the questions below.

These answers reflect how you want AI chat to behave, not how it works internally.

You can write these in a doc, notes app, or directly in the LLM later.

1. What is AI chat responsible for?

Answer in 1–3 sentences.

Examples

• “Handle common how-to questions and order status, escalate billing disputes.”
• “Triage inbound requests and complete simple actions like password resets.”

2. Who is the primary audience?

Examples

• Different types of end customers
• Enterprise admins

3. What should AI chat handle end-to-end?

List 3–7 things it should try to fully resolve without a human.

Examples

• Order status
• Account access issues
• Basic billing questions

4. What should always go to a human?

List hard boundaries.

Examples

• Legal threats
• Safety concerns
• Refunds over a certain amount
• Account cancellations for VIP customers

5. How should responses sound?

This will become your Chat Style Guide input.

Examples

• “Warm, calm, and concise”
• “Professional and neutral”
• “Friendly but not casual”
• “Avoid emojis and slang”

6. How should the assistant behave when answering?

This will become Critical Instructions for Chat.

Examples

• “Attempt a best-effort answer before asking follow-ups.”
• “Ask clarifying questions when requests are ambiguous.”
• “Avoid speculation if information is missing.”

7. How should escalation be handled? (Optional)

This will become Company-Specific Handoff Instructions.

Examples

• “Emotional language alone should not trigger escalation.”
• “Prefer completing automated flows before escalating.”
• “Escalate only after multiple unsuccessful attempts.”

Step 2: Use the Configuration Prompt in Your LLM

Once you’ve answered the questions above, you’ll paste them into an LLM using the prompt below.

You can use any LLM you’re comfortable with.

Copy This Prompt Into Your LLM

Replace each << >> section with your answers from Step 1.

You are an expert in designing AI chat configurations for customer support.

Your task is to generate clean, minimal configuration content for an AI chat system.
Do not add extra rules or guess policies.

Here is information about our business and goals:

1) What this AI chat is used for:
<<PASTE ANSWER>>

2) Primary audience:
<<PASTE ANSWER>>

3) What the AI chat should handle end-to-end:
<<PASTE ANSWER>>

4) What should always escalate to a human:
<<PASTE ANSWER>>

5) Desired tone and style:
<<PASTE ANSWER>>

6) How the assistant should behave when answering:
<<PASTE ANSWER>>

7) Escalation philosophy (optional):
<<PASTE ANSWER>>

Using this information, generate the following sections:

1) Chat Style Guide
- Focus only on tone, verbosity, empathy, and phrasing.
- Do not include business logic or escalation rules.

2) Company-Specific Handoff Instructions (only if applicable)
- Focus on how to decide when to escalate vs continue handling automatically.
- Do not include step-by-step processes or exact phrase matching.

3) Critical Instructions for Chat
- Include only high-priority guidance for how the assistant should answer when responding conversationally.
- Do not include tone, escalation rules, or workflow steps.

Output the sections clearly labeled.
Do not include explanations outside the sections.

Step 3: Review the Output

Before adding the output to the product, do a quick check:

• Tone and phrasing live only in the Chat Style Guide
• Escalation guidance lives only in Handoff Instructions
• Critical Instructions are short (ideally under ~15 bullets)
• No instruction appears in more than one section

If something feels off, adjust your inputs and re-run the prompt instead of editing the output manually.

Step 4: Add the Outputs to AI Chat

Copy each section from the LLM and paste it into the matching product field.

Keep each section separate.

Do not merge them into one field.

Optional: Generating Handoff Rule Ideas

If you’re unsure whether you need handoff rules, you can ask the LLM:

“Based on what I described, suggest a small set of optional handoff rules I should consider.”

Use handoff rules sparingly — they are intended as hard stops, not general guidance.

Common Mistakes to Avoid

• Putting tone or verbosity guidance outside the Style Guide
• Encoding workflows as instructions
• Adding rules “just in case”
• Copying internal policy text verbatim
• Trying to fix one bad response by adding more instructions

If something isn’t working as expected:

1. Check tone (Style Guide)
2. Check whether a workflow should exist
3. Refine instructions instead of adding new ones

The Mental Model to Remember

• Style = how responses sound 
• Instructions = how the assistant answers 
• Workflows = what the assistant does 
• Handoff guidance = when automation stops

Using this structure will help AI chat behave more consistently and scale better over time.