so the developer of bazqux -- a $30 dollar per year RSS reader -- had his app temporarily blocked by the poast nitter because of rate limiting
the developer uses several VPS to fetch the content and send it to his users so it took a while but he finally got blocked automatically and somebody complained so he decided to bypass this restriction by misrepresenting the app and pretending to be a Windows 10 desktop running Chrome.
I've now caught him bypassing the permanent block on his IP range using a VPN. so he's set up VPN on all his servers to bypass this second block.
@graf As someone that uses bazqux for news aggregation, I pay the yearly fee to cover his server costs, his time and effort to provide support. I don't think he's a corporation, just someone that created a news aggregator that also captures comments. IMHO, much better than Inoreader. When I go to nitter.poast.org I don't see a way to donate. I'd do that if you allowed modest news aggregators like bazqux to continue to funnel your content and thus lower your server costs.
@jonwzee no. the developer took malicious steps to bypass rate limiting. instead of reaching out to find out why, he doubled, tripled and quadruped down. that’s malicious and parasitic behavior and the #1 thing i keep off the nitter. theres a reason ours is basically the last fully functional one.
bazqux is the only one blocked and it will continue to be permanently. all others are fine because they have devs that don’t try to bypass limits to hold onto the paltry sum of users they have left
BazQux has never used VPN or 3rd party proxies as it’s impossible to guarantee that they are accessing a real website.
there is one very specific user that I've never seen fetched before, it's fetched so infrequently that I know it's his client, it's the same one listed in the OP.
@Skringot@jonwzee `BazQux uses 7 proxy servers (my own Linode nanonodes) in the USA to download most of the feeds. This was done due to
Hetzner servers were being blocked a lot by sites (I guess because there are a lot of bots running there thanks to Hetzner’s almost free internet traffic)`
@graf@jonwzee I'm not quite sure who you're blocking (none of the IPs listed are mine). BazQux is not currently being blocked. Use dig bots.bazqux.com to get an actual list of BazQux IPs to block. Or grep for User-Agent BazQux.
I'm guilty as charged in that I changed my User-Agent from BazQux to Chrome in November. I sometimes do this as a shortcut to avoid asking admins (who often don't know how blocking/unblocking works). But I actually changed it back to BazQux and moved to the original servers (discourse.bazqux.com/t/nitter-poast-org-not-updating/341/11?u=vshabanov) a few days ago, so it's as legit as it can be from now on.
BazQux was doing about 2k requests a day. I added a rate limit, so now it makes about 1.3k requests (one every 67s). I can limit it even more if you want.
If you care about the number of requests, a hosted RSS reader will make fewer requests than several self-hosted "free" ones, because the feeds are shared and there are no custom rules like update every minute. $30/year is less than what you would pay for hosting and work to set everything up yourself. And that money actually goes to an indie software developer instead of a BigCo.
>I do this as it’s faster than dealing with site admins
the issue i take with you is a user approached you to find out why it wasn't working -- it wasn't working because we have security limits in place to prevent scraping from multiple IPs sharing the same user agent. this is due in large part to us being scraped by over 2 million brazillian IPs. it was automated. however, the automated bans are escalating in nature, so he may have not seen that immediately and it may have taken several hour, days long bans before he was able to make a complaint about it, and by then its usually permament.
instead of you asking about it, like most people would do (i get emails daily about it, in fact) you chose to deliberately misrepresent your app to our nitter servers. to me, that is malicious in nature because that is exactly what it is. you changed your user agent to mask who you were to the server. thats how you earned the permanent ban. not for any other reason other than that.
@graf@jonwzee Unfortunately, I cannot guarantee that I will never use a browser's user agent. Sometimes site admins don't respond, can't or won't change their rules discourse.bazqux.com/t/deviantart-issue/336/6?u=vshabanov and changing the user-agent at least temporarily (along with increased rate limiting) is the only way to get around the blocking.
It's not done often (and I actually removed the browser user-agents for the 3 remaining sites yesterday and they work fine), but it's a useful option to have.
BazQux doesn't make a lot of requests (precisely because it's not a "free" service with millions of users), so user-agent spoofing is used for badly configured sites with unresponsive admins, not to circumvent rate limits.
In the case of poast.org, it was laziness, which I admit is wrong.
@graf Ya know, i'm a old person, RSS works for me, it's sad that you are hating on a good aggregation website. So will you block all the RSS aggregation websites?
@jonwzee >Ya know, i'm a old person, RSS works for me
that's cool because we aren't blocking RSS readers
>it's sad that you are hating on a good aggregation website
the developer admits to deliberately bypassing our security to continue to use our service. that's not a "good" anything, thats malicious in intent and the developer came here to argue it, see here poa.st/objects/fefa05a1-6ce8-48f0-b203-b95b5cf10bdb
>So will you block all the RSS aggregation websites?
no, retard. i literally said it was JUST bazqux and the justification for it
@jonwzee@Witch_Hunter_Siegfired nigger your RSS reader's developer purposely circumvented security, admitted to doing it even. what the fuck do you mean "cunty"
Add comment