so the developer of bazqux -- a $30 dollar per year RSS reader -- had his app temporarily blocked by the poast nitter because of rate limiting
the developer uses several VPS to fetch the content and send it to his users so it took a while but he finally got blocked automatically and somebody complained so he decided to bypass this restriction by misrepresenting the app and pretending to be a Windows 10 desktop running Chrome.
I've now caught him bypassing the permanent block on his IP range using a VPN. so he's set up VPN on all his servers to bypass this second block.
@graf@jonwzee I'm not quite sure who you're blocking (none of the IPs listed are mine). BazQux is not currently being blocked. Use dig bots.bazqux.com to get an actual list of BazQux IPs to block. Or grep for User-Agent BazQux.
I'm guilty as charged in that I changed my User-Agent from BazQux to Chrome in November. I sometimes do this as a shortcut to avoid asking admins (who often don't know how blocking/unblocking works). But I actually changed it back to BazQux and moved to the original servers (discourse.bazqux.com/t/nitter-poast-org-not-updating/341/11?u=vshabanov) a few days ago, so it's as legit as it can be from now on.
BazQux was doing about 2k requests a day. I added a rate limit, so now it makes about 1.3k requests (one every 67s). I can limit it even more if you want.
If you care about the number of requests, a hosted RSS reader will make fewer requests than several self-hosted "free" ones, because the feeds are shared and there are no custom rules like update every minute. $30/year is less than what you would pay for hosting and work to set everything up yourself. And that money actually goes to an indie software developer instead of a BigCo.
@graf@jonwzee Unfortunately, I cannot guarantee that I will never use a browser's user agent. Sometimes site admins don't respond, can't or won't change their rules discourse.bazqux.com/t/deviantart-issue/336/6?u=vshabanov and changing the user-agent at least temporarily (along with increased rate limiting) is the only way to get around the blocking.
It's not done often (and I actually removed the browser user-agents for the 3 remaining sites yesterday and they work fine), but it's a useful option to have.
BazQux doesn't make a lot of requests (precisely because it's not a "free" service with millions of users), so user-agent spoofing is used for badly configured sites with unresponsive admins, not to circumvent rate limits.
In the case of poast.org, it was laziness, which I admit is wrong.
