Here is a fully functioning code that will allow you to run LLM (large language model) on your device locally. Think of it as GPT that works without an internet connection and protects your privacy.

Feel free to use it for your own AI projects!

Check out other tutorials:LLMs - Resources & Tutorials

Code base: Python

Framework: GPT4All

Model: GPT4All Falcon

from gpt4all import GPT4All

model = GPT4All(model_name="ggml-model-gpt4all-falcon-q4_0.bin", model_path="./")
output = model.generate("write me a short story about lonely koala", max_tokens=500)
print(output)

If you wish to use it with Langchain framework, use the following code:

from langchain.llms import GPT4All
from langchain import ConversationChain, LLMChain, PromptTemplate
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

callbacks = [StreamingStdOutCallbackHandler()]
model = GPT4All(model="./ggml-model-gpt4all-falcon-q4_0.bin", max_tokens=500, temp=1.5)

template = "Aswer the following question: {question}"

prompt = PromptTemplate(input_variables=["question"], template=template)

localgpt_chain = LLMChain(
    llm=model,
    prompt=prompt,
    verbose=True
)

question = "write me a rhyming poem about a quiet night"

localgpt_chain.run(question = question, callbacks=callbacks)

How to make it work**:**

1. Run Visual Studio Code as an admin
2. Create a virtual environment in VS Code: In the search bar type ‘>Python: Create Environment venv’
3. install gpt4all: In the Terminal type: pip install gpt4all
4. Download the model you want to use from the gpt4all webpage and save it to the project folder (you can pick any model there)

1. Update model_name and model_path variables in the code if needed
2. Run you app💫

From the author :