How to Build an AI Chatbot That Answers From Your Own Documents

Every team has the same problem. The answers already exist, in an SOP, a client brief, a pricing sheet or a long WhatsApp forward someone saved as a PDF. But nobody can find them, so they ask the one person who knows. That person spends half the day answering the same five questions.

A chatbot that answers only from your own documents fixes this. The technical name is RAG (retrieval augmented generation): the AI first finds the relevant parts of your files, then writes the answer from those parts instead of from the internet.

I recently built one of these live with a learner in my AI Business Automation Program who leads a small design team. His use case was simple and very real. Client offers and updated instructions often sit in documents that never make it to the website, so his designers keep interrupting him to ask. A document chatbot lets them ask the bot instead.

You do not have to start where we ended. Here are three levels, from zero effort to a real app you can share.

Level 1: Gemini Notebook (free, no code, 10 minutes)

Gemini Notebook is the new name for NotebookLM (Google renamed it in July 2026). It is the fastest way to see what a document chatbot feels like.

1. Go to Gemini Notebook with your Google account and create a new notebook.
2. Add your sources: PDFs, Google Docs, Google Slides, website links or YouTube links.
3. Ask questions in the chat.

Why I recommend it first:

• It answers only from your sources. Google says chat responses only use data from your sources, so it will not wander off to the internet.
• It shows citations. Hover over a citation to see the exact quoted text, or click it to jump to that spot in the document. This is how you check it is not making things up.
• The free plan is generous. Up to 100 notebooks and 50 sources per notebook, with each source up to 500,000 words or 200 MB.

Good first notebooks: your company leave policy, a product manual, all your client briefs for one account, or the syllabus and notes for a course you are studying.

Level 2: ChatGPT or Claude Projects (for your daily work)

If you already use ChatGPT or Claude every day, a Project gives you a document chatbot inside the tool you know.

• ChatGPT Projects are available on every plan, including Free. You can upload reference files and add instructions. Free users can add 5 files per project, Go and Plus users 25.
• Claude Projects are also on the free plan, with up to five projects. You upload files to the project knowledge and set project instructions. On paid plans, Claude switches to RAG automatically when your files get too big for its memory, which lets a project hold up to 10 times more.

The secret is the instruction. Add something like this to the project instructions:

Answer only using the files in this project. If the answer is not in the files, say "I could not find this in the documents" instead of guessing. Mention which file the answer came from.

One note on custom GPTs: OpenAI says new GPT creation is not available on personal plans (Free, Go, Plus and Pro) and it is retiring custom GPTs. So if an old tutorial tells you to "build a GPT", use a Project instead.

Level 3: Build your own app that your team can open

Levels 1 and 2 are personal. If you want a link your whole team can use, you need a small app. This is what my learner and I built, and you do not have to write the code yourself. Claude writes it.

Here is the path we followed:

1. Start a Claude project for the build. Upload a test document (he used his own LinkedIn profile exported as a PDF, which is a smart, safe first test) and describe what you want: "a chatbot that answers questions only from this document".
2. Get a model to power it. We used a Gemini API key from Google AI Studio, because the Gemini API has a free tier for experimenting.
3. Run it on your computer first. Claude gives you a few Python files (one for the chat screen, one to read the documents, one to find the right parts and answer). Test it with questions that are in the document, and questions that are not. A good chatbot should say "this is not in the document" for the second kind.
4. Put the code in a private GitHub repository. A link that starts with "localhost" only works on your own laptop. Your team cannot open it.
5. Deploy it on Vercel. Connect Vercel to your GitHub repository and it builds a public link for you.

Be ready for small errors. Our first deployment showed a "page not found" error, which we pasted back to Claude to fix and redeploy. Then the Gemini free tier returned a "high demand" error. This is normal. Building with AI is mostly about reading the error, giving it back to the AI, and trying again.

Your API key is like your ATM PIN

This is the part beginners skip, and it is the part that costs money.

An API key is a password that lets an app use an AI model on your account. If someone else gets it, they can use the service in your name, and once billing is turned on, you pay for their usage.

Three rules:

• Never paste the key into your code or a document you share. Keep it in a file called .env, and make sure that file is never uploaded to GitHub. The .env.example file is only a template with blank values.
• Add the key to Vercel separately, in your project's environment variables. Vercel encrypts these values. Remember that a changed variable only applies to new deployments, so redeploy after you add it.
• If you think a key leaked, delete it and create a new one immediately.

The privacy question you must ask first

Before you upload company or client files anywhere, check how that service uses your data.

Google is clear about this: on the free tier of the Gemini API and in Google AI Studio, it uses what you submit to improve its products, and human reviewers may read it. Google's own advice is not to send sensitive, confidential or personal information to these free services. Paid Gemini API usage is treated differently.

So my rule of thumb:

• Public or harmless documents: free tiers are fine for learning.
• Client data, contracts, salaries, health or ID details: use a paid plan your company has approved, or run a model on your own computer. My guide on how to install Ollama shows how to run a local model, so your files never leave your laptop.
• For the settings worth checking in ChatGPT, Claude and Gemini, read how to keep your data private.

Separate chatbots per client are only needed when confidentiality demands it. If your clients' information can be seen by your whole team, one chatbot with all the documents is simpler.

Where to start this week

Pick one document your team asks about all the time. Put it in Gemini Notebook today and ask it ten real questions your colleagues asked you last month. If the answers are right and the citations check out, you have your proof. Then decide whether you need Level 2 or Level 3.

If you want to understand the building blocks behind Level 3 (APIs, MCP and connectors), start with my explainer on what MCP is.

And if you want to build something like this for your own business, with someone guiding you through every error, that is exactly what my 1:1 programs are for.