Skip to content

Latest commit

 

History

History

01-intro

Module 1: Introduction

In this module, we will learn what LLM and RAG are and implement a simple RAG pipeline to answer questions about the FAQ Documents from our Zoomcamp courses

What we will do:

1.1 Introduction to LLM and RAG

  • LLM
  • RAG
  • RAG architecture
  • Course outcome

1.2 Preparing the Environment

  • Installing libraries
  • Alternative: installing anaconda or miniconda
pip install tqdm notebook==7.1.2 openai elasticsearch pandas scikit-learn ipywidgets

1.3 Retrieval

1.4 Generation with OpenAI

  • Invoking OpenAI API
  • Building the prompt
  • Getting the answer

If you don't want to use a service, you can run an LLM locally refer to module 2 for more details.

In particular, check "2.7 Ollama - Running LLMs on a CPU" - it can work with OpenAI API, so to make the example from 1.4 work locally, you only need to change a few lines of code.

1.4.2 OpenAI API Alternatives

Open AI Alternatives

1.5 Cleaned RAG flow

  • Cleaning the code we wrote so far
  • Making it modular

1.6 Searching with ElasticSearch

  • Run ElasticSearch with Docker
  • Index the documents
  • Replace MinSearch with ElasticSearch

Running ElasticSearch:

docker run -it \
    --rm \
    --name elasticsearch \
    -m 4GB \
    -p 9200:9200 \
    -p 9300:9300 \
    -e "discovery.type=single-node" \
    -e "xpack.security.enabled=false" \
    docker.elastic.co/elasticsearch/elasticsearch:8.4.3

If the previous command doesn't work (i.e. you see "error pulling image configuration"), try to run ElasticSearch directly from Docker Hub:

docker run -it \
    --rm \
    --name elasticsearch \
    -p 9200:9200 \
    -p 9300:9300 \
    -e "discovery.type=single-node" \
    -e "xpack.security.enabled=false" \
    elasticsearch:8.4.3

Index settings:

{
    "settings": {
        "number_of_shards": 1,
        "number_of_replicas": 0
    },
    "mappings": {
        "properties": {
            "text": {"type": "text"},
            "section": {"type": "text"},
            "question": {"type": "text"},
            "course": {"type": "keyword"} 
        }
    }
}

Query:

{
    "size": 5,
    "query": {
        "bool": {
            "must": {
                "multi_match": {
                    "query": query,
                    "fields": ["question^3", "text", "section"],
                    "type": "best_fields"
                }
            },
            "filter": {
                "term": {
                    "course": "data-engineering-zoomcamp"
                }
            }
        }
    }
}

We use "type": "best_fields". You can read more about different types of multi_match search in elastic-search.md.

1.7 Homework

More information here.

Extra materials

  • If you're curious to know how the code for parsing the FAQ works, check this video

Notes