Chat with Local Documents
LocalGPT is a project that allows you to chat with your documents on your local device using GPT models. No data leaves your device and 100% private. You can use LocalGPT to ask questions to your documents without an internet connection, using the power of LLMs. LocalGPT is built with LangChain and Vicuna-7B and InstructorEmbeddings.
LocalGPT requires Python 3.10 or higher to run. It will not work with older versions of Python.
A C++ compiler may be required to build a wheel during
pip install process, This can cause an error message during the process.
For Windows 10/11
To set up a C++ compiler on Windows 10/11, you need to do the following::
- Install Visual Studio 2022.
- Make sure to select the following components:
- Universal Windows Platform development
- C++ CMake tools for Windows
- Download the MinGW installer from the MinGW website.
- Run the installer and select the “gcc” component.
To run the code here, you need to install all requirements first:
pip install -r requirements.txt
This repo uses a Constitution of USA as an example.
Instructions for ingesting your own dataset
To load your documents, you need to do two things. First, move all your files with .txt, .pdf, or .csv extensions to the SOURCE_DOCUMENTS directory. Second, change the docs_path variable to the full path of your SOURCE_DOCUMENTS directory in the load_documents() function.
To ingest all the data, run the following command:
This will build an index with the local vectorstore. The duration depends on your document size. You can add any number of documents and they will all be stored in the local embeddings database. To start with an empty database, remove the
Note: When you run this for the first time, it will download take time as it has to download the embedding model. In the subseqeunt runs, no data will leave your local enviroment and can be run without internet connection.
Ask questions to your documents, locally!
To ask a question, run a command like this:
And wait for the script to require your input.
> Enter a query:
Press enter. The LLM model will process the prompt and generate the answer. It will also show the 4 sources from your documents that it used as context. You can ask more questions without restarting the script. Just wait for the prompt again.
exit to finish the script.
Run it on CPU
run_localGPT.py scripts in localGPT can use your GPU by default. This makes them run faster. If you only have a CPU, you can still run them, but they will be slower. To do this, add
--device_type cpu to both scripts.
Run the following for Ingestion:
python ingest.py --device_type cpu
In order to ask a question, run a command like:
python run_localGPT.py --device_type cpu
LangChain local models and power, you can process everything locally, keeping your data secure and fast.
ingest.pyuses tools from
LangChainto analyze the document and create local embeddings with
InstructorEmbeddings. It then saves the result in a local vector database with
run_localGPT.pyuses a local LLM (Vicuna-7B in this case) to comprehend questions and generate answers. The context for the answers is retrieved from the local vector store using a similarity search to find the right piece of context from the docs.
- You can swap this local LLM with any other LLM from the HuggingFace. Make sure whatever LLM you choose is in the HF format.
Join Guidady AI Mail List
Subscribe to our mailing list and get interesting stuff and updates to your email inbox.