Yaci - A decentralized search engine based on P2P

[dredger]

Hi all,

Just read an interresting article which describe a new search engine (visibly from germany) which could lead to a real google competitor.
This topic may mainly interrest IT guys, but this would concern anyone.

First, here's the french article which describes it :

Yacy – Le moteur de recherche sans censure

On l’a encore vu récemment, Google n’est pas vraiment neutre et pratique la censure. Il en va de même pour tous les moteurs de recherche qui fonctionnent de manière centralisée. Alors b…

korben.info

and here's the automatic google translate link to english

So, it's not search engine page as we know, but an application you install so you host your own search engine, and this application communicates with any other owner of this application to share a decentralized database !
The idea is pretty good and i can't help to think about STO and STS here. Google and any company which is owned by one are a few people is a representation of the STS hierarchy, pyramidal, with a keyword which is "ownership", when for STO, it's all at the same level, and the keyword (or one of the) is share, or sharing, or networking too. And here this is the first time i hear about a search engine which is clearly STO-oriented, if i could say.

Here's the YaCy website :

Home - YaCy

YaCy P2P - Decentralized Search Engine

yacy.de

Maybe interresting to test, to start to install on our computers, the more people will use it, the stronger and more accurate the global search engine (which is finally the sum of all people using it, trully and STO concept) will become. This may even be a good idea to promote this to any SOTT/forum member, and dedicate a thread for people having issue to install/run/use it. This may interrest @Scottie who seems to like also IT on top of electronics.

That's all for today :)

 

[Vajra2]

That sounds promising D - we very badly need a less commercialised search capability on the web.
The big question is perhaps how reliable and user friendly YaCy might be at this point.
Does anybody have any experience?.

What is clear is that we have a huge problem with the currently available (Google monopoly based?) search engines. Even supposedly independent examples as the years have gone by bring up an ever shorter list of vaguely related but highest paying results with little regard for the information actually sought in a search.
Output is by now down to about a page before it starts to repeat.
It's next to impossible to access the sort of technical and special interest content that searches were routinely bringing up by the 1990s. What little there is is typically (overtly or covertly) infommercial in form and requires membership and/or a payment/and/or being subjected to unwanted advertising to access.
It seems likely that the many that back in those years worked to develop and publish informative content have more or less given up too - presumably because short of having the website address in advance their content can't be found.
It's ultimately as you suggest yet another example of how value free narrow thinking focused solely on maximisation of short term profit is entropic in nature - of how it destroys what it touches.
What was a fabulous tool by which to share information has been reduced to minimal usefulness.
Hollywood and the cinema industry have done much the same - they ate their seed corn by focusing on the sale of over priced tickets and 'refreshments', the building of high overhead and grandiose cinemas and by refusing to invest in films that they didn't regard as a safe bet - endless repeats of franchises, simplistic B movie quality stuff etc.
On a much larger scale the tying of our capital up in non-value adding financial market speculations instead of wealth generating investments by the financial 'industry' has had similar effects on whole economies.
It's the story of our collapsing Western cultures really - we badly need to start seeing people with higher values and more holistic views finding their way into positions of influence and authority.
That's not going to happen however until we move away from the hyper authoritarian and hierarchical forms of business organisation that have become the norm (call centres and corporates are especially problematical in this regard) - where employees are selected for their readiness to accept authority, frightened they will lose their jobs, concerned mostly with staying clean within the hierarchy and intent upon ignoring all else.
The resulting business cultures have proven to very effectively promote the value-free into positions of authority - with disastrous consequences...

 

[Vajra2]

Just a follow up on a related topic. (the forcing of content on users by search engines, browsers etc)

I noticed a few here mention using Brave (which isn't a search engine per se and only starts to nibble at the edges of the search engine problem described above) and installed it yesterday.

They call it a browser Secure, Fast, & Private Web Browser with Adblocker | Brave but it in reality seems to sit on top of existing browsers (?) but shuts down adverts and their manipulations of the searches saved in the browser's list of saved frequently used searches. It seemingly enables a long list of other security and privacy functions too.

The items brought up in searches seem more or less as normal for the selected search engine - Duck Duck Go in this case.

One big positive is that so far it seems to completely prevent the forced playing of ads and distortion of searches that now comes with YouTube videos.

Does anybody more expert in these matters know more about the positives and negatives of it? Is it safe? Who developed it, and how do they generate revenue I wonder? Are there non-distorting search engine options about other than the usual?

 

[Goemon_]

Does anybody more expert in these matters know more about the positives and negatives of it? Is it safe? Who developed it, and how do they generate revenue I wonder? Are there non-distorting search engine options about other than the usual?

I'm certainly no expert. But they surely generate some revenue by opening a page, once in a while, to ask you for donation.

 

[KJS]

Does anybody more expert in these matters know more about the positives and negatives of it? Is it safe? Who developed it, and how do they generate revenue I wonder? Are there non-distorting search engine options about other than the usual?

Brave Software is led by Brendan Eich. He has some integrity and values, especially regarding the COVID hysteria. On the other hand, Brave's business model doesn't seem to be radically different from others... They still need to generate revenue, and they do that in a kind of parasitical way, by removing ads from services and displaying their own. They are also known to push new services that are disliked by users.

I can't really tell much about their privacy statements, but I'm not sure if their browser is all that secure. As it's a modified Chrome, security patches can take some time to be included in their release cycle. Another thing is that they are pushing updates that often include and install additional services on your computer, like ipfs or VPN. That means, that apart from the browser, you now have additional software that can have security holes and updates tied to the Brave release cycle.

My last remark is that currently, with clever browser fingerprinting techniques, it's a better strategy to have more privacy by blending-in with the crowd, than using a distinct browser with small market penetration. So personally, I'd go with Google Chrome or Safari or even Ungoogled Chromium via a popular and trusted (uncompromised yet) VPN like Mullvad with their DNS filtering enabled.

As for search engines, Brave Search works very well for me, especially that you can modify their ranking algorithm via Googles feature.

 

[Vajra2]

Thank you very much KJS and G.

It'll take a while to work through the implications of what you have said - although Brave has already attempted to push their VPN.

 

[washable_orbit]

Thank you very much KJS and G.

It'll take a while to work through the implications of what you have said - although Brave has already attempted to push their VPN.

I know nothing about the technically of search engines or VPNs, but I wouldn’t trust a built-in VPN, or any free VPN for that matter.

I am very happy using Proton VPN (take a shot every time I say that word). It gets rid of all ads and limits tracking and other stuff. Again, I don’t know what I’m talking about but it has served me very well.

As for the search engines. I use three: Google, Brave and DuckDuckGo. Google is unfortunately just very convenient for most things. I use the Brave search engine through their browser which is also Chromium based. But it falls short in a lot of ways for me. Lately, they just seem to push the most popular or the highest paid websites, but the results are never as good or direct as Google. DuckDuckGo is the only one that doesn’t seem to do any of that, but it is also the one that I use the least. Go figure.

 

[Ina]

YaCy looks like fun! I cannot wait to make my very own first web crawler.

 

[Nienna]

This is just a fwiw:

Negating all VPNs may have been possible since 2002

Researchers at the Leviathan Security Group have publicized an exploit that can force a VPN user to transmit unencrypted internet traffic outside of the VPN tunnel, exposing...

www.techspot.com

IN BRIEF: Many users consider VPNs essential for maintaining digital privacy. However, researchers have discovered an exploit that can completely neutralize the technology without the target knowing, and every VPN on every operating system except Android is vulnerable. Furthermore, the only foolproof workaround is currently exclusive to Linux.

Researchers at the Leviathan Security Group have publicized an exploit that can force a VPN user to transmit unencrypted internet traffic outside of the VPN tunnel, exposing them to snooping and defeating the entire purpose of the technology. Currently, no method to fully address the problem exists on popular operating systems like Windows, macOS, or iOS. Although the researchers have found no evidence of active exploitation, it may have been possible for over two decades.

By running a DHCP server on the same network as their target, a malicious actor can route traffic meant for a VPN through a gateway and read it without encryption. The method is particularly sneaky because an affected user won't notice anything unusual.

You can read the rest of the article at the above link.

No matter how hard we try to keep our privacy, it seems that those in control can thwart our efforts anyway.

 

[dredger]

Hello,

Sadly I haven't yet tested Yaci, but as soon as I read about I wanted to share this good idea here. I hope to have time to test it in the future, or i'll ask one of my colleagues to do so.

Here's a "trick" I use, I don't know if this really "works" but it should, logically. Aim is to prevent big tech to spy me, or minimize it.
This is an idea i had when configuring my actual PC (win10) last year. As usual, i installed a couple of browsers and manage to install all of them in a specific directory (not the usual C:\Programs or ..\Program Files) - I searched for portable versions for each.
So far i have 2x Brave, 2x Maxthon, 1x Chromium and and old Firefox version

Then i disabled the automatic updates for all and customised their settings. For instance, the defaults download folders are configured on a specific disk, in a specific download directory, one sub-directory per browser. It helps to gather all my downloads made on a specific disk (not only the ones made by browsers) and +/- sorted.

And the trick i mention is the following : on my 2 brave browsers, the ones i use from day to day, i did not login to any google service (or any social media platform but i do not have any account, or not anymore) : no login on one of my gmail account, nor youtube, nothing. I used the 2 Maxthon (I have 2 different and separated versions) when i have to access one google service. My aim here is to have my browsing history kept private, or kept distant from being accessed by any bigtech service. The lone exception is X/Twitter, which I access from one of my Brave installation.
Here's a small visual/screenshoot of the directories from the file explorer of windows, the 6th browsers i setup, Chromium, is installed somewhere else on the same disk - i avoided the default system disk (C:\ drive) and installed all on the D:\ )

The number after the name is the version, ie : Brave143 is Brave version 1.43.89


For the forum, i use one of my 2 Brave installation, the one i configured by default with an high security (that i can lower individually site per site). Note that i did not spent days on it and I'm not expert in IT security, I do not know too if, as discussed here, Brave is 100% reliable or did not spent time to inform on it more than that.
Note also that i sometimes have an error message from Brave telling me that it has a problem and need to be re-installed but i ignore it. Having 2 separated versions of Brave cause some little problems has they visibly share some common settings set in Windows, but these are minors ones and this does not prevent me to use them. I'm using this config since 9 months now and it's doing fine.

For the updates, when one browser starts to have too much problems displaying the websites, i update it manually, but this kind of "maintenance" is done less than once a year.

The "problem" is that in order to achieve this you need to have already a good knowledge in IT. It should be possible to prepare such package (a bundle of portable browsers in one single zipped file to download, unzip and it's ready), but this would require a good amount of work & testing.

That's all, FWIW