Skip to content

LLM & RAG DEVELOPMENT

LLM and RAG development for assistants that answer from your documents

We build retrieval-augmented generation (RAG) systems and LLM applications that search your policies, contracts, manuals and records, answer in plain English and show the passage behind every claim. When the documents don’t cover a question, the assistant says so and passes it to a person.

  • 20+ in-house specialists
  • MCP servers and AI agents in production
  • Answers with sources, by design

Free consultation

Get a proposal for your project

Tell us what you want to automate. An engineer will read it and reach out to set up a free consultation.

We need an email or a phone number to get back to you.

RAG IN ACTION

One question, two documents, a cited answer

A RAG system answers in two moves. First it searches your content for the passages that matter, then the language model writes the answer from those passages only. In this illustrative example, a tenant of a Denver property management company asks about the pet policy, and every sentence of the reply points to its source.

Illustrative example of an AI assistant that answers from company documents for a fictional Denver property management company. A resident asks whether dogs are allowed in the Elm Street units and whether there is a pet deposit. The assistant searches the knowledge base, picks two relevant passages, from the pet policy and the Elm Street lease addendum, and writes an answer where every sentence cites its source: dogs up to 50 lb are allowed, with a one-time refundable $300 deposit plus $35 a month in pet rent, and service and support animals are not treated as pets. Questions the documents don’t cover go to the leasing team.

Illustrative example Fictional business, people and data. Not a real client, conversation or result.

HOW A RAG PIPELINE WORKS

How retrieval-augmented generation works, step by step

Most of the answer quality gets decided before the model writes a word. If retrieval brings back the wrong passage, the best model in the world will still get it wrong. That’s why most of our RAG development time goes into the first four steps.

  1. 01

    Collect and clean

    We connect the sources (shared drives, wikis, help centers, databases), remove duplicates and outdated versions, and record who’s allowed to see each document.

  2. 02

    Chunk and enrich

    We split documents into passages that keep headings, tables and section numbers intact, and tag them with metadata like date, product or location.

  3. 03

    Embed and index

    Each passage becomes a vector stored in a vector database, alongside a keyword index, so the system can match both meaning and exact terms.

  4. 04

    Retrieve and rerank

    At question time, hybrid search finds candidate passages, a reranker puts them in order and permission filters drop anything the user isn’t allowed to see.

  5. 05

    Generate with citations

    The model writes the answer only from the retrieved passages, cites each one and says “I don’t know” when they don’t contain the answer.

  6. 06

    Log and evaluate

    Questions, passages and answers are logged and scored against a test set, so any change to any step gets measured before it ships.

RAG VS FINE-TUNING VS PROMPTING

RAG vs fine-tuning vs prompt-only: picking the right fit

Use RAG when the model needs to know your facts. Use fine-tuning when it needs to learn a format, tone or narrow task it keeps getting wrong. Use a well-written prompt alone when the knowledge fits in the instructions. Most production systems are RAG plus good prompting. Fine-tuning is the exception, and we add it only when the evaluation shows it helps.

  • Teaches the model

    RAG (Recommended)
    Your facts, at question time
    LLM fine-tuning
    A style, format or narrow skill
    Prompt-only
    Rules that fit in the instructions
  • Content changes often

    RAG (Recommended)
    Yes
    LLM fine-tuning
    No
    Prompt-only
    Partly
  • Cites sources

    RAG (Recommended)
    Yes
    LLM fine-tuning
    No
    Prompt-only
    No
  • Respects document permissions

    RAG (Recommended)
    Yes
    LLM fine-tuning
    No
    Prompt-only
    Partly
  • Needs training data

    RAG (Recommended)
    No
    LLM fine-tuning
    Hundreds to thousands of curated examples
    Prompt-only
    No
  • Upfront effort

    RAG (Recommended)
    Medium: data pipeline and evaluation
    LLM fine-tuning
    High: datasets, training runs, re-testing
    Prompt-only
    Low
  • Main risk

    RAG (Recommended)
    Poor retrieval gives wrong answers
    LLM fine-tuning
    Outdated knowledge baked in
    Prompt-only
    Breaks as rules and content grow

General comparison. These techniques are often combined. For the longer version, see our blog article on RAG vs fine-tuning.

LLM APPLICATION DEVELOPMENT

What our LLM development company builds

Our LLM application development covers six kinds of systems. Most clients start with one assistant over one set of documents, then add sources and channels once the answers hold up.

RAG chatbots for customers

Website or in-app chat that answers from your help center, catalog and policies, cites the article and hands off to support when it should.

Knowledge base AI assistants

An internal assistant for staff in Slack, Teams or a web app that answers from SOPs, HR policies and past tickets.

Document Q&A and search

Ask questions across contracts, reports or technical manuals and get answers with the clause, page or section linked.

Structured extraction

Pull fields, dates and obligations out of long documents into your database, with the source passage stored next to each value.

Fine-tuned models

When evaluation shows a base model can’t hold a format or a narrow task, we fine-tune on curated examples and test it against the baseline.

MCP servers

Your knowledge base or product exposed as a Model Context Protocol server, so AI assistants can search it and call its tools with scoped access.

MCP server development: MCP servers
EXPLAINED

Vector databases, fine-tuning and MCP servers, explained

These terms come up in every RAG conversation. Here’s what each one means for your project, in plain English.

01 / 04

What a vector database does in a RAG system

A vector database stores each passage of your content as a list of numbers (an embedding) that captures its meaning, so the system can find passages about “canceling a lease early” even when the document says “early termination.” It works alongside a keyword index, because exact terms like SKUs, policy numbers and names still matter.

  • Options range from a Postgres extension to managed services
  • We pick based on volume, hosting rules and cost
  • Metadata filters enforce dates, locations and permissions
02 / 04

Why chunking decides answer quality

Before anything gets embedded, documents are split into passages. Split a table in half or separate a rule from its exception, and retrieval will hand the model half the truth. We chunk by structure (headings, sections, rows) and test several strategies against your questions before choosing one.

  • Headings and section numbers kept with each passage
  • Tables and lists kept whole
  • Strategy chosen by evaluation, not by default
03 / 04

When LLM fine-tuning is worth it

Fine-tuning retrains a model on your examples so it reliably produces a specific format, tone or classification. It doesn’t keep the model up to date on your facts, and it costs more to maintain. We recommend it only when a RAG system with good prompts still misses the bar on your evaluation set. Our article on RAG vs fine-tuning goes deeper.

  • Good for: fixed formats, classification, domain style
  • Poor for: facts that change
  • Always compared against the untuned baseline
04 / 04

What an MCP server adds

The Model Context Protocol is an open standard for connecting AI assistants to data and tools. An MCP server wraps your knowledge base or system so any compatible assistant can search it or call its functions, with authentication and scopes you control, instead of building a separate integration for each AI tool. More in our guide to the Model Context Protocol.

  • One integration, many AI clients
  • Read-only tools by default
  • Every tool call logged
ENTERPRISE RAG

Enterprise RAG: permissions, freshness and an audit trail

Enterprise RAG is a RAG system built for many users, many sources and real access rules. The chat window is the easy part. The hard part is making sure a sales rep never gets an answer pulled from an HR file, and that last week’s price change replaces last quarter’s.

We enforce permissions at search time using the access rules of the source system, re-index on a schedule or on change, keep versions so you can see which document an answer came from, and log every question for review. If the assistant also needs to act, like opening a ticket or updating a record, that’s where AI integration with your systems comes in.

  • Permission filters inherited from the source
  • Scheduled or event-driven re-indexing
  • Answer, sources and user logged together
  • Hosting that fits your data rules
SOURCES AND STACK

Where your knowledge lives, and what we build on

RAG development starts with the sources you already have. These are the categories and common tools we work with. We confirm connectors for your specific accounts during scoping.

Document sources

  • SharePoint
  • Google Drive
  • Confluence
  • Notion
  • PDF and Office files

Help centers and tickets

  • Help desk articles
  • Past tickets
  • Product docs

Databases

  • PostgreSQL
  • MySQL
  • Data warehouse exports

Vector search

  • pgvector
  • Managed vector databases
  • Hybrid keyword + vector search

Models

  • OpenAI
  • Anthropic
  • Open-weight models on your infrastructure

Channels

  • Website chat
  • Slack
  • Microsoft Teams
  • Your app via API
HONEST LIMITS

What RAG can’t fix

RAG is only as good as the documents behind it. In these situations, we’ll tell you to fix something else first.

Contradictory documents

If three versions of the refund policy disagree, the assistant will find all three. Someone has to decide which one is true.

Answers that need math across records

“What was our margin by region last quarter?” is a database query, not a document search. We route those to SQL tools or reports.

Knowledge nobody wrote down

If the answer lives in a senior employee’s head, retrieval has nothing to find. Capturing it is step one.

Scanned images with poor quality

Faded faxes and photos of handwritten notes need OCR and cleanup before anyone can search them, and some never will be searchable.

Zero-error decisions

Clinical, legal or financial determinations stay with qualified people. The assistant can find the passage. It doesn’t make the call.

Tiny document sets

If everything fits in a few pages, a well-written prompt may do the job without a retrieval pipeline.

CUSTOMIZATION

How you shape the assistant’s answers

Every rule below gets written into the system and covered by the evaluation set, so you can see it hold up before launch.

  • Sources: which folders, spaces and databases are in, and which are explicitly out
  • Citation style: link, page number or section reference on every claim
  • Tone: concise for staff, warmer for customers, in your vocabulary
  • Languages: questions and answers in English or Spanish, from documents in either
  • Refusals: topics it declines and the exact wording it uses
  • “I don’t know” threshold: how confident retrieval must be before it answers
  • Handoff: where unanswered questions go, with the conversation attached
  • Feedback: thumbs up or down, flagged answers and a weekly review queue
RAG DEVELOPMENT SERVICES

How our RAG development services run, from documents to launch

We prove answer quality on your real questions before anything touches a live channel.

Step 1 of 5

Source audit

We inventory your documents, find duplicates and gaps, and agree on what the assistant may use.

Deliverable: a source map with owners, access rules and refresh frequency.

Step 2 of 5

Question set

Your team gives us real questions and the correct answers. This becomes the evaluation set every version is scored on.

Step 3 of 5

Retrieval prototype

We test chunking, embeddings and search settings until the right passages come back, then add generation and citations.

You see a scored report: which questions pass, which fail and why.

Step 4 of 5

Harden and connect

Permissions, re-indexing, guardrails, logging, the user interface or channel, and handoff to your team.

Step 5 of 5

Pilot and tune

A small group uses it for real work. We review flagged answers every week and add sources or users once it holds up.

USE CASES

RAG chatbots and knowledge assistants by industry

The pattern repeats: a pile of documents people search every day, and a real cost when they grab the wrong version. These are examples of what we build, not reported results.

  • Healthcare

    Front-desk policy assistant

    Answers staff questions on insurance, cancellations and visit preparation from the practice’s own policies, with links.

  • Dental

    Benefits and procedure lookup

    Explains what the office’s written procedures say about a treatment, so staff give patients the same answer every time.

  • Physical therapy

    Clinic protocol search

    Lets therapists search home-exercise guidance and clinic protocols in plain language, citing the source document.

  • Restaurants

    Allergen and menu answers

    Answers guest questions about ingredients from the current menu specs, and hands uncertain cases to the manager.

  • Software

    Support RAG chatbot

    A customer chat that answers from the help center and passes unresolved cases to a support rep, context included.

  • Professional services

    Contract and policy Q&A

    Search across signed agreements for renewal terms, notice periods and liability caps, with the clause linked.

WHY APPYWEB

Why teams choose us for LLM and RAG development

We build and run AI agents and MCP servers in production, so retrieval, permissions and evaluation are everyday work for us, not experiments.

Evaluation from day one

Every version is scored on your questions. You see pass rates and failures, not a polished demo.

Model-agnostic

We compare providers on your data and structure the code so the model can change without a rebuild.

Documented handover

Pipeline design, prompts, tests and runbooks, all in writing, from a team headquartered in Austin, Texas, with engineers in Spain.

RELATED SERVICES

Build on top of your knowledge base

FAQ

LLM and RAG development questions

What does an LLM development company do?

An LLM development company builds applications on top of large language models: assistants that answer from your documents, extraction pipelines, chat and search experiences, and the integrations around them. The model usually comes from OpenAI, Anthropic or an open-weight project. The company designs retrieval, prompts, guardrails, evaluation and operations.

What is a RAG chatbot?

A RAG chatbot is a chat assistant that searches your own content before answering and writes the reply from what it found, with the source cited. Unlike a general chatbot, it can answer about your products, policies and procedures, and it can say “I don’t know” when the documents don’t cover the question.

What is enterprise RAG?

Enterprise RAG is retrieval-augmented generation built for an organization: many sources, many users and real access rules. It adds permission filtering at search time, scheduled re-indexing, version tracking, audit logs and hosting that meets the company’s data requirements.

Should we use RAG or fine-tune a model?

For knowledge that changes, such as policies, prices or product details, use RAG. It stays current and cites sources. Fine-tuning helps when a model must reliably follow a format, tone or narrow task. Plenty of projects need only RAG and good prompts. We add fine-tuning when the evaluation set shows a clear gain.

Do we need a vector database?

Most RAG systems use one, but it doesn’t have to be a separate product. For moderate volumes, a vector extension on a database you already run can be enough. We choose based on the size of your content, hosting rules and cost, and we usually combine vector search with keyword search.

Is an MCP server the same as RAG?

No. RAG is a technique for answering from your documents. An MCP server is a standard way to expose data and tools to AI assistants. They work well together: an MCP server can offer a “search the knowledge base” tool backed by a RAG pipeline, plus other tools such as looking up an order.

Can a RAG system respect who is allowed to see what?

Yes, if it’s designed that way from the start. We store each passage with the access rules of its source and filter results by the user’s permissions before the model sees them, so the assistant can’t quote a document the person couldn’t open.

How do you measure whether the answers are good?

With an evaluation set of real questions and expected answers from your team. We score whether the right passages were retrieved, whether the answer is faithful to them and whether citations are correct, and we rerun the set before every release.

Tell us what your assistant should know

List the documents or systems it should answer from, who’ll use it and where. We’ll get back to you with a recommended approach, what we need from your team and what a first version would include.

  1. 01 We read what you sent before we talk.
  2. 02 A free call to map the workflows worth automating.
  3. 03 A written proposal with scope and a fixed price.

We need an email or a phone number to get back to you.