" v. You will not use the Llama Materials or any output or results of the
Llama Materials to improve any other large language model (excluding Llama 2 or
derivative works thereof)."
That this AI technology is nonfree is not surprising, considering that it is created by facebook, an enemy of software freedom and digital rights (including privacy).
I think that varies - completely open and free for any use is (as with a lot of software and services) a challenge and rarity. Some models have different licenses. Hopefully this will get better and some of the worst offenders get left behind (OpenAI and ChatGPT are very problemmatic). Regarding AI openness, one of the FOSDEM talks showed these slides, which add to your list in the AI context:
[edit to add: ollama itself is MIT license]
[edit to add2: Microsoft’s research has also developed the phi model, which is small(ish) and has several derivatives like llama, and it uses the MIT license (see bottom) - free but from M$, pick your poison]
[edit to add3: noticed that also available as small(ish) version is StarCoder which has it’s own AI-relevant license]
you could train and run your finished (trained) A.I. on your Phone to. But you should train it on a Desktop or with Big-data or internet access on a Server-Farm or something.
Its about Training an A.I. is expensive, like feed and schooling a child for knowledge, if you copy its brain state later you have access on its thoughts and knowledge later. Like your mind is more useful later after some work for learning or discovering things, its like the A.I. too. And the daily growing Process is expansive. So run it not on your phone.
Like you would use a script or Program - with the knowledge of the Wikipedia, after Decades of spending time an polishing.
I’m not saying it’s a bad thing but putting it to perspective regarding all of software where our preferred type is comparatively small collection (for some that may be enough, admittedly). And if our existence wouldn’t be a challenge, we wouldn’t even having this conversation because open and free (as FOSS interpretation, not all the other shades of gray) would be the default. And, like I pointed out, “open” is getting new areas that need to be considered with regards to AI - the second slide kinda illustrates that as many systems are open in different areas but maybe not all. Having PureOS or GNU system open is good but that should only be a starting point for humanity. As it has been in the past with open and free, there are development steps to take and more to come, and that’s what I see with AI as well - some of the big examples are not pretty (not open or free or even ethical in other aspects) but there is a lot of effort (not just grassroots like us but apparently in the legislative level too, albeit with mixed results) to get things better. So, when I say challenge and rarity, I guess I mean that there is still work to be done and this is just the starting point now regarding for what’s possible in the future with truly open AI systems and software (perhaps even open AI-hardware someday).
Yes, you seem to understand what the AI models are and what Ollama does. The metaphore seems apt. They are not trained on the phone, although I think with some you can input some data to them. Some of the “brain states” are a bit confused sometimes and the phone has the capacity to maybe have only half a brain, I think
[edit to add: You can find links from Ollama model library to Hugingface library, which has much larger collection and much more info on various facets of AI models. There is more info on the models and some of them list info about what the models include, how they were trained, how much computing was used etc.]
The more time I spend with PureOS and with seeing how much code is free software in stuff that I ignorantly used prior to that, the more the rest of the world doesn’t make sense to me.
And maybe this is a personal problem in my interpretation. But, what development steps that go beyond the truly free technology were actually warranted and not simply some evil to lead me astray? I’m saying this as someone who was led astray, very often and for most of my life! But what, would we call the Android kernel a development of the kernel of linux because they convinced users to stop customizing or replacing their kernels?
Anyway, with regards to AI, you know how the saying goes:
“God reached AI singularity a billion years ago. God laughs at people doing AI.”
An now back to our regularly scheduled programming original topic…
Does anyone have a handy setup with ssh and a machine more capable of running an AI model? Can you run remote AI (possibly a larger and more potent) and get better quality answers faster?
If you can install Ollama on that and run it, I suppose it does. I was thinking more of a home server type but you may have more computing power on that server (no idea what the specs of that may be).
sudo docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
sudo docker exec -it ollama ollama run llama2-uncensored
Here is an excerpt of the “conversation” so far:
>>> Hello
Hello, how can I help you?
>>> What is llama2-uncensored?
llama2-uncensored is an open source tool designed to provide uncensored
access to content on the internet. It utilizes a network of servers around
the world to bypass restrictions and filters imposed by governments,
corporations, and ISPs.
>>> How does llama2-uncensored bypass restrictions and filters imposed by
... governments, corporations, and ISPs?
llama2-uncensored employs a combination of techniques to achieve this.
Firstly, it utilizes a distributed network of servers around the world
that are connected via a peer-to-peer protocol. This helps to disperse
traffic and make it more difficult for any single entity to block or
restrict access.
Secondly, llama2-uncensored uses encryption and anonymization technologies
to ensure that user data remains private and secure. Additionally, the
tool can automatically detect and circumvent IP blacklists and other forms
of network filtering.
>>> How does llama2-uncensored automatically detect and circumvent IP
... blacklists and other forms of network filtering?
llama2-uncensored employs a variety of techniques to circumvent IP
blacklists and other forms of network filtering. For example, it can
utilize proxies or VPNs to route traffic through different IP addresses in
order to avoid detection. Additionally, the tool can dynamically change
its IP address based on changing geolocation data in order to stay one
step ahead of any potential blocks or restrictions.
>>> What is the threat model of llama2-uncensored?
The threat model for llama2-uncensored takes into account both technical
and non-technical threats. On the technical side, there are always risks
associated with using any type of software that connects to the internet,
including the potential for viruses or malware infections. Additionally,
users should be aware of the potential security vulnerabilities of
peer-to-peer protocols and networks.
On the non-technical side, llama2-uncensored has a risk associated with
its use in countries where access to certain types of content is
restricted or prohibited by law. Users should be aware that they may be
subject to legal repercussions if caught using this tool in such
jurisdictions.
Seems simple enough. You can do a bit of performance measuring using /set verbose at llama2 prompt - it will give you statistics on response time - we can compare those to what the model gives on L5 (problem being though, that it probably won’t give the same answer to the same question).
But anyway, you can now try installing some larger model (see for instance the llama2-uncensored library page’s tag tab - how about a ten times bigger 70billion model instead of normal 7b… if your server can handle it) and use it remotely privately.
Here are multiple sequential questions using ‘verbose’ mode with llama2-uncensored:
>>> /set verbose
Set 'verbose' mode.
>>> What are the controversies surrounding Purism SPC?
The main controversy around Purism SPC is that it is a closed-source
operating system, which means that only certain people can access the
source code. This has led to concerns over privacy and security, as well
as questions about transparency in the development process. Additionally,
some critics argue that the company's focus on data protection and
security comes at the expense of user experience and innovation. However,
supporters of Purism SPC believe that its closed-source approach allows
for more secure and private software development.
total duration: 1m27.204596687s
load duration: 1.502986ms
prompt eval count: 33 token(s)
prompt eval duration: 13.325142s
prompt eval rate: 2.48 tokens/s
eval count: 111 token(s)
eval duration: 1m13.874077s
eval rate: 1.50 tokens/s
>>> What are the controversies surrounding Purism?
Purism has been involved in several controversies over the years,
including their stance on surveillance, privacy, and encryption. One of
the most notable controversies was when the company refused to cooperate
with government agencies during the investigation into the WikiLeaks
website. This led to accusations that Purism was engaging in political
activism rather than being a technology company focused solely on creating
innovative products. Additionally, some have criticized Purism for its
lack of diversity and inclusion efforts within their company.
total duration: 1m23.139962614s
load duration: 950.141µs
prompt eval count: 29 token(s)
prompt eval duration: 11.063985s
prompt eval rate: 2.62 tokens/s
eval count: 112 token(s)
eval duration: 1m12.071648s
eval rate: 1.55 tokens/s
>>> What are the controversies surrounding myGovID?
MyGovID has been involved in several controversies, including its use as a
centralized identity verification platform by governments around the
world. Some critics argue that this approach violates privacy rights and
could lead to surveillance of individuals without their knowledge or
consent. Additionally, there have been concerns raised about how myGovID
handles user data and whether it is secure enough to protect personal
information from hacking and other cyber threats. Finally, some have
criticized the company for its lack of transparency in terms of how the
platform works and what exactly happens when users submit their identity
information through the site.
total duration: 1m47.506579377s
load duration: 1.606476ms
prompt eval count: 31 token(s)
prompt eval duration: 12.647663s
prompt eval rate: 2.45 tokens/s
eval count: 129 token(s)
eval duration: 1m34.847361s
eval rate: 1.36 tokens/s
>>> What are the controversies surrounding the Competition Bureau?
The Competition Bureau has been involved in several controversies over the
years, including its handling of antitrust cases involving large
corporations and the impact that such actions have on consumers. Some
critics argue that the bureau is too lenient with big businesses and does
not do enough to protect consumer interests. Additionally, there have been
concerns raised about how the Competition Bureau handles confidential
information and whether it has the resources necessary to effectively
investigate and prosecute cases of antitrust violations.
total duration: 1m42.034195413s
load duration: 1.234664ms
prompt eval count: 31 token(s)
prompt eval duration: 17.455753s
prompt eval rate: 1.78 tokens/s
eval count: 103 token(s)
eval duration: 1m24.56909s
eval rate: 1.22 tokens/s
The VPS Elite only has 8 GB of RAM; see this thread below for a reminder of the details:
It gives bad answers and takes time to do so
It looks like it assumes that there are controversy to beginwith due to how the question is structured and then just considers everything controversial, regardless what side or stance was actually taken by … some similar company. That’s statistical machine learning - a bit of mix’n’match.
Thanks for testing!
Other things for testing, if someone needs a challenge: the llava model is supposedly developed to identify what’s on images. It would be interesting if some day that could
a) automatically create a spreadsheet/db of all the pics taken with camera, and
b) be an optional feature/plugin with L5 camera app, helping to identify strange objects , get additional info, translate foreign languages it sees etc.
(I remember from 6-7 years ago, when L5 was just a distant dream, there was a blind person visiting this forum who would probably have befitted of that kind of feature too)
What is the benefit of putting AI on a Librem 5? It seems to me that the Librem 5 has difficulty enough, just running a trimmed-down version of PureOS. Without several high speed cores, lots of RAM, and a big hard drive or SSD drive, any AI program wouldn’t stand a chance at doing anything productive. What we need to do is use the google model. Build a data center in your house and log in to your data center from your Librem 5. Without external data centers, your average Samsung phone wouldn’t be much better than your Librem 5 at crunching data.
In a typical application (example: navigation), your Android phone sends the address you entered in to the navigation program, to a data center. The data center does all of the navigation and communications calculations, and sends the results back to your Android phone. So your phone appears to have super powers. But it’s not much more than a glorified dumb terminal. If we take away all of the data centers, your Android phone gets very stupid, very quickly. The Librem 5 can’t do much better unless you add a lot more resources, like some kind of datacenter to greatly expand the hardware resourcing. If you try to add AI to the phone’s extremely small hardware resources, I doubt that it will work at all. Put the AI on a server at home and log in to it from your Librem 5.
Plus sends a copy of the addresses etc. to the Google-borg for future advertising / law enforcement / …
FTFY.
That would be my approach but isn’t the essence of it that you own your own Purism hardware and you can do what you want with it, including impractical things that cause problems?
I have a Mini with several AI’s installed - Tiny Llama, Samantha Mistral, and Dolphin Mistral. The Mini works more as a server than a workstation, as I do most of my work on a docked Librem 5. When I want to utilize an AI, I ssh into the Mini from the Librem 5, and run the AI that way.
Tiny Llama would probably work (slowly) directly on the L-5. But for me it isn’t worth taking up the precious disk space!
