Skip to content

Guides: keeping an installation reachable under rate limiting (static homepage + custom 429 page via the reverse proxy) - #12655

Open
ErykKul wants to merge 4 commits into
developfrom
rate-limiting-reachability-docs
Open

Guides: keeping an installation reachable under rate limiting (static homepage + custom 429 page via the reverse proxy)#12655
ErykKul wants to merge 4 commits into
developfrom
rate-limiting-reachability-docs

Conversation

@ErykKul

@ErykKul ErykKul commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

What this PR does / why we need it:

When anonymous traffic (search-engine and AI crawlers) exhausts the shared tier 0 rate-limit bucket, every guest gets 429s — including on the homepage and the login page they'd need to reach a higher tier, so the installation looks down. This adds a field-tested subsection to the Rate Limiting docs with two reverse-proxy recipes that keep the front door usable: serving the homepage as a static file from Apache (zero rate-limit budget, survives app-server saturation) and a styled 429 error page that explains the situation and points guests to login. Includes downloadable sample pages, an nginx pointer, and curl verification steps. This is operational guidance from a production installation, generalized. Docs-only; no linked issue.

… homepage and custom 429 page via the reverse proxy)
@ErykKul
ErykKul requested a review from pdurbin August 28, 2026 14:59
@ErykKul

ErykKul commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

@pdurbin can you review? A community-contributed subsection for the Rate Limiting docs, based on what we run in production at KU Leuven after bot traffic made the repository look down to guests. @DieuwertjeBloemen pinging you as this documents our setup, feel free to add anything I missed from the operational side.

@DieuwertjeBloemen

Copy link
Copy Markdown
Contributor

The one thing to definitely include in the documentation is the advice to include a "login" link or button to the login page from both the error page and homepage, as this is the key to properly accessing a Dataverse in these cases. (Without it, people are informed, but can't actually get to the authentication step necessary to by pass the anonymous rate limit).

@ErykKul

ErykKul commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

The one thing to definitely include in the documentation is the advice to include a "login" link or button to the login page from both the error page and homepage, as this is the key to properly accessing a Dataverse in these cases. (Without it, people are informed, but can't actually get to the authentication step necessary to by pass the anonymous rate limit).

Good point, made the advice explicit in 1ec2e82.

@Kris-LIBIS

Copy link
Copy Markdown
Contributor

Is there a need to add that the static landing page is also a good place to add site metadata, like FAIRiCat and have that always available?

@pdurbin pdurbin moved this to Ready for Triage in IQSS Dataverse Project Aug 28, 2026
@ErykKul

ErykKul commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

Is there a need to add that the static landing page is also a good place to add site metadata, like FAIRiCat and have that always available?

Good suggestion, added in 7606244 and refined in 788b956 with the linkset content type and profile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Ready for Triage

Development

Successfully merging this pull request may close these issues.

4 participants