Hi 
I’d like to use AI with my L5 so I installed Alpaca app but then I’ve to download a model and there are a lot and I don’t know which one to use.
Please some suggestion considering privacy and security?
Thank you
1 Like
Up to you and the available resources on your Librem 5. Since you already use ChatGPT, you can also consider asking them what AI they recommend.
1 Like
If you find and answer to this, please let me know what you find. I assumed I wouldn’t be able to run AI on my Librem 5 because the AI was probably infested with nonfree ideology even for the free AI.
2 Likes
I had something about this in How-to: Installing AI to L5 and running it locally offline with ollama.ai The gist of it is that L5 has limited resources, so you want to select a model that is small in size - and it still will be slow. I recommend trying tinyllama first (purely from size standpoint), or anything around 1Gb (you can probably take it to 2 to 3Gb but I’m not sure when L5 simply runs out of steam - usually the memory is the limit with sizes).
Then, there are different models for different uses and you can find some model card databases that give some description about how they were made and for what (and what license they use and how open they are).
Ollama list is (see also the tab for variations of models): library
More in depth info of models from Huggingface’s database (where Ollama gets them mostly): Models - Hugging Face
Btw. using Alpaca GUI from flathub may add some overhead to your L5, so you may have to use a tad smaller models than what Ollama is able from command prompt uses (haven’t compared them yet). That being said, Alpaca is probably more user friendly - by far.
2 Likes
What I could about “TinyLlama” online makes it sound like it was created using the structure of Facebook’s model and a bunch of the outputs of ChatGPT that were snapshotted off and then it was trained to guess those, or something like that.
Their source code repo is Apache License, but I heard that those often lie and say that and then the actual model weights have additional licensing. HuggingFace didn’t seem to bring me to a TinyLlama model weights licensing like I expected. Do you know if it’s easy to find?
1 Like
It’s not straight forward and I’ve not been digging around it enough to say. There are thousands of variations at this point. But a license covers what’s inside - that’s the idea of a license. App license and model license are different things but license for weights is… odd (I haven’t come across). That would be another version of a model and even then the original license set boundaries what you can do (depending on the license).
For a more appropriately licensed models, Huggingface does have search based on those too.
Anyway, here’s the tinyllama model card you were looking for (the one that’s for the Ollama tinyllama model): TinyLlama/TinyLlama-1.1B-Chat-v0.6 · Hugging Face (there’s a link in the Ollama library list.)
For further reading, HF blog has some interesting posts: OpenRAIL: Towards open and responsible AI licensing frameworks and Creating Privacy Preserving AI with Substra etc. Also remember other open AI resources Open Source AI Deep Dive – Open Source Initiative (etc.)
2 Likes