Technology is constantly evolving, and in the world of AI, things are getting more and more interesting. Today, I will talk to you about a project that is not only interesting but also very useful: RAGstack.
The idea is simple and allows you to deploy a private alternative to ChatGPT on your VPC (Virtual Private Cloud – your virtual private cloud network) and offers to explore open-source LLM models such as Llama 2, Falcon, and GPT4All.
RAGstack is a reliable solution for companies that want to use the so-called Retrieval Augmented Generation (RAG) technique to improve the capabilities of LLMs by retrieving information from other systems. Deployed in business, this will allow your employees to save time in their daily lives. In addition, it is less expensive and faster than fine-tuning.
The RAGstack project supports different models, including GPT4all, Llama2, and Falcon-7B. It can be deployed on GCP, AWS, and Azure. But you can also run it on your own server like this:
Copy ragstack-ui/local.env to ragstack-ui/.env and copy server/example.env to server/.env.
In server/.env, replace YOUR_SUPABASE_URL with the URL of your Supabase project and YOUR_SUPABASE_KEY with the secret API key of your Supabase.
In ragstack-ui/.env, replace YOUR_SUPABASE_URL with the URL of your Supabase project and YOUR_SUPABASE_PUBLIC_KEY with the secret API key of your Supabase.
You can find these values in your Supabase dashboard under Settings > API.
Then, in Supabase, create a ragstack_users table with the following columns:
id, uuidapp_id,
uuidsecret_key,
uuidemail,
textavatar_url,
textfull_name, textIf you added row-level security in the base, make sure that inserts and selects have a WITH CHECK expression of (auth.uid() = id).
Run scripts/local/run-dev. This will download ggml-gpt4all-j-v1.3-groovy.bin to server/llm/local/ and run the server, LLM, and Qdrant vector database locally.
All services will then be ready when you see the following message:
INFO: Application startup complete.
Well done, it’s functional! You can also try an online version.
The source code for the RAGstack project can be found on GitHub.
