2.1 API Connection

First, of course, we need a connection to OpenAI and setup an API account:
https://openai.com/product

Now, as we have set up an account and received the api key, we save it in an external .txt file on the hard disk. From there, we can use the rgpt package to load the key into the R environment, if required.

rgpt3::gpt3_authenticate("~/api_access_key.txt")

As a further precondition to be able to run the subsequent python scripts in the R environment, we need to ensure to have the reticulate package installed.

install.packages("reticulate")

Obiously, if you don’t already have Python installed, download a version from the website at python.org.
Here, we also used the reticulate package to install the latest Version of 3.10

reticulate::install_python("3.10:latest")

To test if the installation has been successful, we can simply run a test like:

reticulate::py_run_string('print("Python is setup and running!")')

2.2 Initial Python setup

After Python has been properly installed, we create a virtual environment in which we load all the necessary packages for our project. Like in R projects, we need only to load the packages once !
As we are referring to a langchain project, we name the environment “langchain_env” and load the necessary packages:

reticulate::virtualenv_create(envname = "langchain_env", # name environment 
                  packages = c("langchain","unstructured","openai","python-dotenv","pypdf","chromadb","bs4","tiktoken") # load initial packages
                  ) 

In case you need to install further packages after creating the environment, you can simply use py_install()

reticulate::py_install(packages = c("numpy"), envname = "langchain_env")

2.3 Project’s common entry point

After the initial setup, this is the entry point for reopening the project. Here, we reactivate our virtual environment and load the API key into R.

reticulate::use_virtualenv("langchain_env")
rgpt3::gpt3_authenticate("~/api_access_key.txt")

Since we will need the API key in the Python code, we need to convert it once from the r object to a Python object.

py_api_key <- reticulate::r_to_py(api_access_key)

3.1 Text splitting

Before we can ask Chatgpt questions about our market documents, we first have some preparation and preliminary work to do:
As the next step, we create a folder to store our PDFs, like “gpt_doc” and save the path in the following Python script. For a better traceability of the individual steps and breakdowns, we read in the 1 document “goldmansachs_credit_outlook_24.pdf” for further processing.

Via langchain we import the document and check the number of pages loaded in:

reticulate::py_run_string('from langchain.document_loaders import PyPDFDirectoryLoader
pdf_loader = PyPDFDirectoryLoader("~/gpt_doc/")
all_pages = pdf_loader.load()')

Another great convenience of the reticulate package is that we can use the Python objects such as ‘all_pages’ directly in R. This means that we can either throw a ‘print’ message directly in the Python code or conveniently analyze temporary results in R.

We now have to break the text down into smaller blocks to enable Chatgpt later on to find matching content and return it to us. Using langchains' split_document we see the 25 pages are split up into 29 blocks:

reticulate::py_run_string('from langchain.text_splitter import RecursiveCharacterTextSplitter
splitted_doc = RecursiveCharacterTextSplitter().split_documents(all_pages)')

The more blocks the text is split into, the more granular the search. But if the text section is too small, the more confusing the answer will be. So we ought to define an appropriate block size here, which we can specify as the border size in the Python code. We have therefore increased the number of blocks from 29 to 97 with a maximum size of 900.

reticulate::py_run_string('chunk_size = 900
chunk_overlap = 150
from langchain.text_splitter import CharacterTextSplitter
new_splitted_doc = CharacterTextSplitter(chunk_size=chunk_size, chunk_overlap=chunk_overlap, separator=" ").split_documents(all_pages)')

Let’s take a quick look at the 97 blocks as an interim result and see if the output meets our expectations.

py$new_splitted_doc

3.2 Check Chatgpt costs

If we now want to upload the blocks to Chatgpt in order to receive the answers to our questions, we should first check what costs we will have to expect. In order to do this, we need to know the total number of tokens from our research document. In this case, we can use the OpenAIR package, which uses the following purrr::map function to sum up the tokens of all 97 blocks, which is in total 15027.

purrr::map_int(py$new_splitted_doc, ~ TheOpenAIR::count_tokens(.x$page_content)) |> 
  sum()

The cost of our research with $0.0001 per 1K tokens is at roughly: 15027/1000 * 0.0001 = $0.0015

So - good to go …

4.1 Setup Vector Database

To be able to save the embeddings, we can use Chroma’s local storage service and save everything in a newly created folder.

reticulate::py_run_string('import os
from langchain.embeddings.openai import OpenAIEmbeddings
os.environ["OPENAI_API_KEY"] = r.py_api_key
embedding_obj = OpenAIEmbeddings()
from langchain.vectorstores import Chroma

# Place to store Eemdeddings
chroma_store_directory = "~/gpt_chroma_db"
# Setup Vector Database
vectordb = Chroma.from_documents(
    documents=new_splitted_doc,
    embedding=embedding_obj,
    persist_directory=chroma_store_directory)')

4.2 Setup OpenAI Connection

in our use case, we use Chatgpt’s LLM gpt-3.5-turbo model and link this to our vector database. The result is a Q&A concatenation that we can use for our questions.

reticulate::py_run_string('import openai
from langchain.chat_models import ChatOpenAI

# Assign Chatgpt model
llm_model = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)

# Using langchains RetrievalQA to create a Q&A chain
from langchain.chains import RetrievalQA
qa_chain = RetrievalQA.from_chain_type(llm_model,retriever=vectordb.as_retriever())')