Forcing RST sending hack

Hi,

Some developers are running app instances in Azure cloud, and Microsoft has decided 128 connections ought to be enough for everybody. i.e. SNAT allows 128 outgoing connections to same host/port combo.

To really piss people off, they have decided that it takes 240 seconds aka 4 minutes to clear an entry in the SNAT table, and allow a new connection. So basically we can open one new connection every 2 seconds !!!!! They suggest to use connection pooling / reuse. But we talk to multiple names on the same IP so would have to have a pretty low limit per name to ensure sum of all connections stays below the limit. Say 4 names = 128/4 = 32 connections per name. But we need to go lower due to the 2 minute timeouts. nginx on receiving end recycles after 100 requests. So even setting it to max 1 connection per name might exhaust the 128 on a busy day.

Now, if the Microsoft firewall receives a RST, then the connection is released 15 seconds later. So we can have the same app instance handle 16 times more connections.

Now to the Linux part:

Since I am an old school hacker, my idea was to replace any outgoing FIN packets towards the Microsoft IPs with RST. That should be doable with packet mangling (?)

It is too dirty, and not compliant, but are there other better solutions ? Say wait for the connection to go in TIME_WAIT / FIN_WAIT then wait 5 seconds (to allow for oacket loss/ retransmit) and then send reset ? I would need to have SEQ numbers of last packet I guess.

Another solution would be running pcap/tcpdump, finding the FIN packets towards the Microsoft IPs, wait 5 secs, and then send reset. The pcap will have the SEQ numbers right.

The right solution would be to move a pure Linux server rather than the app server stuff, then I would have 63k connections to use, and I could set the tcp_fin_timeout to to 10 seconds rather than the 240 seconds.

On top of all this, we now have 4 IP addresses to the backend server, but are still discussing best DNS setup for DNS round robin.

Really hate arbitrary limits, especially if they are ultra-low like here. Fanning out horizontal (more app instances) scales linear, which is why we would rather put a multiplier on somewhere.