Christian Heilmann

PrivateGPT – Running “ChatGPT” offline on local documents

Saturday, May 27th, 2023 at 4:04 pm

You can download the GPT model and interrogate local files using Python without having to send any data to the cloud.

PrivateGPT is a python script to interrogate local files using GPT4ALL, an open source large language model. It is pretty straight forward to set up:

  • Clone the repo
  • Download the LLM – about 10GB – and place it in a new folder called `models`.
  • Place the documents you want to interrogate into the `source_documents` folder – by default, there’s a text of the last US state of the union in there.
  • Run the `ingest.py` script – this can a long time. On this MacBook M1 it was 2 minutes though
  • Run the `privateGPT.py` script and you get a prompt in your Terminal:

I asked it what the US president has to say about the situation in the Ukraine and it gave me a synopsis and where in the document the information is from.

Screenshot of the terminal with a question providing an analysis of the situation in the Ukraine and the answer

Neat!

Share on Mastodon (needs instance)

Share on Twitter

My other work: