2024-05-22

Own AI mentor

I build a tool which lets you search through a database of transcripts which are about books which are about famous people in any space, mostly entrepreneur ship and business. The tool is kind of ment to be like a "mentor" sort of whom you can ask questions or rather keywords, until now.

In the future i not only want to add keywords but whole sentences so that the questions can be more broad and better, this I want to do with some vector embeddings.

This is how everything currently works.
1

The AI process at the time is really slow (>5 min) due to me running the LLMs locally with Ollama and not using some kind of API. For my LLM, I'm using mistral. I tried to run the LLM on Google collab because I did not want to rent any GPUs. I tried to run it on Google collab to get one of their T4s and then use ngrok to connect it to my localhost, which resulted in a 404 error (text me if you know why).

Thanks,

Finn