Skip to content
Implementation Guides

How to Choose an AI Automation Agency: 12 Questions and Red Flags

Written by
CEO of Appyweb
7 min read Updated November 6, 2026

The short answer

Pick the AI automation agency that shows you a working system close to yours, agrees on how success gets measured before you sign, leaves you owning the code, prompts and accounts, and tells you what it wouldn’t automate. Walk away from guaranteed results, numbers with no source and fuzzy answers on ownership.

To choose an AI automation agency, look for one that can show you a working system similar to what you need, agrees with you on how success will be measured before any work starts, leaves you owning the code, prompts, workflows and accounts, and tells you straight out what it wouldn’t automate. Twelve questions, asked over one or two calls, are enough to tell an AI automation agency worth hiring from one that will leave you paying for a rebuild.

How should you run the selection process?

Treat it like hiring a contractor for a kitchen remodel: the same brief for everyone, a short list, a look at work they’ve done, written quotes and a small first job. In practice, that’s five steps.

The selection process

  1. Step 1

    Write the brief

    One page: the process, volume, systems, what success means and your constraints.

  2. Step 2

    Shortlist three agencies

    Look for ones that have built something close to your process, rather than “AI” in general.

  3. Step 3

    See a live system

    Ask each one to open a working system and let you try it with your own questions.

  4. Step 4

    Compare written proposals

    The same rows for everyone: scope, acceptance criteria, ownership, third-party costs, support, exit.

  5. Step 5

    Run a paid pilot

    One process, a fixed price, a few weeks, and tests the system has to pass before you commit to more.

What goes in the brief?

The brief makes every agency answer the same question, and it exposes the ones that quote without asking you anything.

  • The process, step by step, and who handles it today. “Two office staff re-key supplier invoices from email into QuickBooks” is a process. “We want AI” isn’t.
  • Volume: calls, leads, documents or tickets per week, as best you know.
  • Systems: CRM, calendar, phone system, ERP, help desk, and whether each one has an API.
  • Success: one or two outcomes, like hours saved, faster response times or fewer errors.
  • Constraints: sensitive data (health, financial, minors), budget range, deadline, languages.

If you can’t fill in the process line yet, start with our AI readiness checklist. It’s the homework that keeps vendor calls short.

The 12 questions to ask an AI automation agency

The questions fall into five areas: proof, scope, ownership, operations and money. The table shows what a good answer sounds like, and what should send you looking elsewhere.

What good and bad answers sound like

A good answer

1. Can you show me a live system similar to this?
They open something that runs every day and let you try it
2. Who exactly will build and maintain it?
Names, roles, where they’re based, what gets subcontracted
3. What would you not automate in our process?
Specific steps that should stay with people, or aren’t worth it at your volume
4. How will we measure success, and what’s the baseline?
One metric, and how it’s measured today and after launch
5. What happens when the AI gets something wrong?
Guardrails, human review for risky actions, error alerts, a handoff path
6. Which of our systems will you connect, and how?
System by system: API, webhook, a connector in n8n or Make, or an MCP server
7. Who owns the code, prompts and workflows?
You do, in writing, and everything can be exported
8. Whose name is on the third-party accounts?
Yours, with access granted to the agency
9. How will you handle our data?
Which vendors process it, retention, training opt-outs, which agreements they sign
10. How will you test it before launch?
Real examples from your business, written test cases, a limited rollout
11. What does support look like after launch?
Monitoring, response times, how changes are priced, plans for model retirements
12. How is it priced, and what makes the price go up?
The pricing model, the cost drivers, and usage fees listed separately

Walk away if you hear

1. Can you show me a live system similar to this?
Only slides, or a recording you can’t interrupt
2. Who exactly will build and maintain it?
“Our team” and nothing else
3. What would you not automate in our process?
“We can automate all of it”
4. How will we measure success, and what’s the baseline?
“You’ll see huge efficiency gains”
5. What happens when the AI gets something wrong?
“It doesn’t make mistakes”
6. Which of our systems will you connect, and how?
“It integrates with everything”
7. Who owns the code, prompts and workflows?
A license you lose when you leave
8. Whose name is on the third-party accounts?
Everything resold through their accounts
9. How will you handle our data?
“We’re fully compliant”
10. How will you test it before launch?
“We’ll tweak it after go-live”
11. What does support look like after launch?
Nothing defined past the launch date
12. How is it priced, and what makes the price go up?
One number with nothing behind it

Three of these are worth a closer look.

Question 3 tells you the most. An agency that understands your process will name steps that should stay human: complaints, judgment calls, anything regulated, or tasks too rare to be worth building. An agency that says yes to everything hasn’t understood the work.

Question 6 tests real engineering. Ask what they’ll do if one of your systems has no API. Good answers range from a small API in front of the database to a nightly export. Screen scraping should be the last resort, and they should say so. Our guide to integrating AI into existing software covers these options if you want to walk into the call prepared.

Question 12 should come with a cost breakdown. A fixed-scope project, setup plus a monthly fee, or a retainer are all legitimate. What matters is knowing what moves the number: extra integrations, call minutes, document volume, languages, change requests. Our article on what AI automation costs explains each factor.

Red flags that should end the conversation

Some patterns keep showing up in AI agency marketing, and regulators have noticed.

Red flag Why it’s a problem Ask instead
Guaranteed ROI or “replace your staff” Results depend on your process, volume and data; nobody knows before discovery “What would have to be true for this to pay off for us?”
Statistics with no source If there’s no organization and year behind a number, you can’t check it “Where does that number come from?”
Case studies with no details you can verify Anonymous clients and round numbers are easy to make up “Can I see the system, or talk to someone who uses it?”
“HIPAA compliant” or “TCPA-safe” with no paperwork Compliance depends on contracts, configuration and how you use the system “Which agreements will you and your vendors sign?”
A price before any discovery They haven’t understood the process, so the scope will change later “What do you need to know before you quote?”
One tool for every problem Good teams choose n8n, custom code, a voice platform or nothing, depending on the job “Why this tool for this process?”
No human handoff in the design Customers will get stuck with an AI that can’t help them “Show me how a caller reaches a person.”
Pressure to sign this week Good partners don’t need a countdown clock Take the time you need

What should the contract include?

Put every proposal into the same structure, and the blanks will tell you as much as the numbers. Before anyone signs, make sure the agreement covers all of this.

Before you sign

Have your attorney review it like any other service agreement. If governance questions come up along the way, our guide on writing an AI usage policy shows what your side of the rules should look like.

How do you run a pilot that actually tests the agency?

If two agencies look equally good, a pilot settles it. Keep it to one process, a few weeks, a fixed price and acceptance criteria written before work starts, like: “the agent books correctly into our calendar in all twenty test scenarios, and transfers every caller who asks for a person.”

You’ll learn three things no sales call will tell you: how the agency communicates when something goes wrong, how it deals with your messy real-world data, and whether the system holds up at your volume. If you’d rather get an outside view before talking to any vendor, an AI readiness assessment gives you a ranked list of processes worth piloting.

How we answer these questions

Appyweb is an AI automation agency headquartered in Austin, Texas, with an engineering team in Spain and more than 20 in-house specialists building software since 2012. When you ask question 1, we open systems that run every day. Scope and price are agreed in a written proposal before work starts, and accounts are set up in your name.

We’d rather you ask us all twelve questions than none. Bring your process and book a free consultation, and we’ll answer every one with your system on the table.

Frequently asked questions

Should I hire an AI automation agency or a freelancer?

A freelancer can be the right choice for a single, well-defined workflow in a tool like n8n or Zapier. An agency makes more sense when the project touches several systems, needs ongoing support, or involves phone lines, customer data or custom code. Ask both the same twelve questions.

Is it a red flag if an agency builds on n8n, Make or Zapier?

No. Workflow tools are often the right choice for simple, linear automations. The red flag is an agency that uses the same tool for everything, or can’t explain when it would switch to custom code.

How long should choosing an agency take?

For a small or midsize business, a brief, two or three calls and a written proposal from each agency on your shortlist usually fit into a few weeks. A short paid pilot adds a few more weeks but takes most of the risk off the table.

What is a fair way to structure payment?

Tie payments to milestones you can see: discovery and design, a working version on test data, and final payment once the acceptance criteria are met. Usage costs from AI, telephony and automation providers should be billed separately, ideally to your own accounts.

What should I do if an agency won’t show a live system?

Ask why. A new agency may not have one yet, and that’s not a dealbreaker if it offers a small paid pilot instead. An agency that claims lots of clients but can’t show you anything working is a different story.

More on this topic

Keep reading

All topics
Implementation Guides

AI Readiness Checklist: 25 Questions to Answer Before You Automate

An AI readiness checklist tells you whether a specific process is ready to automate before you spend money on it. These 25 questions cover five areas: process, data, systems, people and risk. Every “no” points to something to fix first, and most of those fixes cost far less than an AI project that fails.

Raúl Aránega Segura
· 7 min read
AI Automation

n8n vs Make vs Zapier for AI Workflows: Which One Fits Your Business?

Pick Zapier when non-technical teams need to hook up popular SaaS apps fast, Make when your workflows branch a lot and reshape data, and n8n when a technical team wants complex AI agent logic, custom code or self-hosting. For high volume or unusual systems, custom code can beat all three.

Germán Gutiérrez
· 8 min read
AI Automation

AI PPC Management: How AI Agents Automate Paid Media in 2026

AI PPC management means connecting AI agents to Google Ads, Microsoft Ads, Meta, GA4 and your CRM so they watch the accounts, investigate what changed and draft fixes, while people approve anything that moves money. Automate monitoring, search term triage, pacing checks and reporting first; keep strategy, budgets, conversion goals and brand claims human.

Raúl Aránega Segura
· 21 min read

Want help putting this into practice?

Tell us what you want to automate. We’ll give you a straight answer on what’s possible and what it would take.

  1. 01 We read what you sent before we talk.
  2. 02 A free call to map the workflows worth automating.
  3. 03 A written proposal with scope and a fixed price.

We need an email or a phone number to get back to you.