Testing anubis to prevent LLM bots from taking down source.puri.sm

Oh, could be taken either way. But it is a mighty sentence, that’s for sure.

2 Likes

Store to tape and lock it up.

Changes require signature sign-off on a piece of paper by two officials †.

Also lock up the sign-off list.

(“Tape” is a metaphor, store to any offline medium.)

Compare the online file(s) with the offline file(s) frequently.

† We used to call this “two-person integrity”. We’d use it just to open the door of a SCIF. That kind of security has gone by the wayside it seems.

1 Like

… which was kind of my point.

You can go to great lengths to create a robots.txt file with a hundred lines (literally, here, q.v.), steering robots away from a long list of unsuitable URLs but bad robots are just going to ignore it anyway - or even use it to reach disconnected parts of the URL graph. So why not “all robots, disallow all URLs”? In other words, my question was actually about good robots, not bad robots.

Or to put that (rhetorical) question a different way … what benefit does Purism accrue from good robots? and does the benefit outweigh the cost (from good and bad robots)?

1 Like

Coming back more directly to topic … did anyone test from a mobile phone? (might need lapdock - no idea how well that web site works on a phone-sized screen)

At least here, mobile phones are most likely to be behind CGNAT and I am not sure how sticky the public IP address is, which then influences the behaviour of any state (e.g. cookies) that is confined to a specific IP address. However the documentation was not crystal clear on whether a successful result can be shared between IP addresses.

1 Like

I think cookie is per browser, not per IP address. So changing your IP address should not invalidate the cookie. Firefox on byzantium and crimson/bookworm seems to work fine. Gitlab is very responsive to mobile screens.

1 Like

I think if the robots don’t take the whole server down and if other users are not affected, it would be fine.

1 Like

I think so too - but that leaves open the obvious attack vector that one member of a botnet does the proof of work but then shares the cookie with all the other members that will actually be traversing your site at high speed thereby DDoSing it.

(I have the same issue on some of my web sites with the anti-spam mechanism, which uses crypto to incorporate a bunch of information in a cookie, including the IP address to which the cookie will be sent. Should it then validate that the cookie is handed back by the same IP address? It’s a trade-off. By including the IP address it leaves open the possibility that a future sysadmin option would allow the IP address to be validated as the same.)

1 Like

I’d let the anubis author figure out the best defenses for now, that works against most of the bots, even if not stopping 100% of the bots.

1 Like

This is test post to check link previews Instance Configuration · Help · GitLab

1 Like

I got a 503 error (Service Unavailable)

1 Like

Because I made a mistake in the configuration. Ip address should be in CIDR format. It should work now.

2 Likes

It does. Animegirl was happy.

1 Like

Link titles are coming correctly after an exception was added for forums.puri.sm in bot policy.

1 Like

Works fine.