Ecosia and Qwant building a European privacy-first search index

[amarok]

Article: https://www.euronews.com/next/2024/11/12/europes-answer-to-google-ecosia-and-qwant-partner-to-build-new-search-index

• will be operational in 2025
• infrastructure, called European Search Perspectives (EUSP), based in Paris
• infrastructure will be available for other independent search engines and technology companies
  
  

[FranklyFlawless]

Here is a quote from the article that caught my eye:

The aim is to build European digital sovereignty and use the platform to provide a foundation for artificial intelligence (AI) infrastructure.

[amarok]

Hopefully, the AI aspect is in the context of improving indexing and search results.

Here’s a more in-depth article: Ecosia and Qwant, two European search engines, join forces on an index to shrink reliance on Big Tech | TechCrunch

[FranklyFlawless]

Here is a quote from the latter article:

Both search engines have already dabbled in integrating GenAI features. Expect more on this front, although they aren’t planning to develop AI model development themselves. They say they will continue to rely on API access to major platforms’ large language models (LLMs) to power these additions.

I assume that their usage of GenAI within their search engines will be answering queries as text summaries.

[irvinewade]

Won’t Ecosia and Qwant just become Big Tech then? Even so, anything to reduce reliance on Google is to be applauded.

Likely some of the problems with LLMs and GenAI will just be replicated within the new infrastructure.

[FranklyFlawless]

No, both entities are only focusing on creating an independent privacy-first search index, not creating proprietary operating systems and/or social media platforms. The rest of the infrastructure, notably LLMs, remains heavily reliant on various third-parties’ API access, which are still as susceptible to EULAs, pricehikes, ratelimiting, etc.

[irvinewade]

In respect of LLMs etc., I was (very unclearly) commenting on a hypothetical future where even that was brought into the new “sovereign” digital infrastructure. In other words, yes, you are surely right about the situation today, as described in the article, but the logical extension would be to sever all dependencies … but there will still be e.g. “hallucinations”. So it’s a bit like putting lipstick on a pig.

@FranklyFlawless

focusing on creating an independent privacy-first search index

… which would definitely be a step forward but I was considering some of the wider issues with “Big Tech”. In fact, when I see the word “sovereign”, that rings alarm bells.

[FranklyFlawless]

Right, “European digital sovereignty” is inherently a political term, but the current situation is that most search engines are based in the US:

List of search engines - Wikipedia (on my Wikiless instance)

As quoted earlier, Ecosia and Qwant are not planning to develop AI models themselves, so the reliance on third-party LLMs and their APIs will continue to exist unless stated otherwise. Their plan is to provide the infrastructure for others to use it, so I assume that doing so will bring more attention to them and potentially attract European investment to further their mission.

[Ick]

And another chance wasted to create an European open search index. Instead we get something “fancy with AI”. I didn’t expect anything else from companies.

[TiX0]

FYI there already is an anonymous privacy-respecting European search engine:

Startpage - Private Search Engine. No Tracking. No Search History.

And it’s a pretty good one at that too! Integrates nicely with FF

[FranklyFlawless]

Startpage uses Google as its search engine under the hood.

[TiX0]

Are you sure? I thought they used Bing (because in fact Google didn’t want to make a deal with them…)

[FranklyFlawless]

Absolutely.

[TiX0]

Ok, That would be good news - Google is still better than any other (unfortunately)

[FranklyFlawless]

You can use the list of search engines I linked from my Wikiless instance above if you want to compare the current available search engines, or if you want more backstory surrounding my usage of Startpage, you can read about it on the Purism community forums:

Backdoor in FireFox in Kali and Parrot OSes (and others)

After DuckDuckGo, I used Gigablast for quite a while, but it looks like it was recently shutdown without any announcement or notice. After using several search engines, including Startpage, SearX(NG) and LibreX, I started using Mojeek for the last few months, but very recently decided to go back to Startpage for now. It is one of the few ties I have with Google, even though the relationship is indirect.

Librem 14 updated and working after two years of trying

When I get stumped, I usually use Startpage; it uses the Google search engine as the backend. Occasionally I use Wikipedia or the ArchWiki.

@FranklyFlawless

I use Startpage, which uses the Google search engine backend, but I have been looking to reduce my dependency on it for a while now. Currently, these are the listed search engine options available on LibRedirect:

• SearXNG
• SearX (unmaintained)
• Whoogle
• LibreX (unmaintained)/LibreY
• 4get

I already used most of these proxy front-ends before, but due to rate limiting issues, I ended up swapping back to Startpage.

Currently I use my own Whoogle instance:

https://whoogle.franklyflawless.org/

[TiX0]

Nice!
Your Whoogle instance would give the same results as if it were a Google search?

[FranklyFlawless]

I would not know, as I have stopped directly using Google services a long time ago. What I do know is that my Whoogle instance fetches Google results and has not been ratelimited since I have deployed it.

[irvinewade]

@FranklyFlawless

Startpage uses Google as its search engine under the hood.

… and I would hope that when the project as per this topic is complete, Startpage changes over to using the fruits of this project.

[FranklyFlawless]

Here is another quote from the latter article:

In other words, in order for a search index to be useful, it also needs to be able to freely crawl the internet. But with Big Tech more jealously guarding info inside its own platforms these days, as giants compete to monetize user data afresh for training LLMs, this is also complicating the business of trying to get out from under their shadow by indexing the internet for search… From a rock to a hard place, then.

Even if an independent European privacy-first search engine is realized and Startpage migrates over to this “solution”, Big Tech can and will use technical and legal measures to prevent gratis indexing of their proprietary platforms, effectively locking projects out of usable search results unless they pay for the “privilege”.

[irvinewade]

@FranklyFlawless

Big Tech can and will use technical and legal measures to prevent indexing of their proprietary platforms.

Up to a point that is reasonable. If the information is only accessible by logging in and even then only accessible for personal use then it may be reasonable to prevent crawling or scraping. In any case, a polite web crawler should respect the “robot directives” - and a company in this position should be issuing same.

Where this would get very dodgy is if, say, a Big Tech company offers search and offers web site hosting - and they allow their crawler to crawl those web sites but they prevent any other crawlers from crawling those web sites, or by default they prevent that.

I have no problem with a social media site outright preventing all web crawling.

Then there are the free, public web sites that want to prevent scraping in order to monetise their content - but are not averse to being indexed. (That can be handled by the site owner indexing its own content and then handing the index to the crawler.)

You are right though that there are many legal stoushes and many regulatory stoushes between where we are now and any kind of comprehensive European “sovereign” search / indexing capability.