Not enough public IPs in farm despite having a free public ips

[A-Harby]

Describe the bug

Mainnet, 2.4.2.

Dashboard is giving error Not enough public IPs in farm despite having a free public ips on the farm.

To Reproduce

Steps to reproduce the behavior:
Just deploy any instance on the dashboard using IPv4.

Screenshots

[AhmedHanafy725]

Further investigation, these 2 IPs on farm ID 1 look free on graphql.

However, they are reserved on the chain.

    {
      ip: 185.69.166.147/24
      gateway: 185.69.166.1
      contractId: 15946
    },
    {
      ip: 185.69.166.157/24
      gateway: 185.69.166.1
      contractId: 16178
    }

but @sameh-farouk found these contracts are deleted, but they are still on the farm object and reserving these IPs

[sameh-farouk]

Update:
I attempted to debug this issue today.
I found the calls that created and canceled these contracts was executed with runtime 116
I inspected the codebase tagged with v2.1.4 to see if it contained any bugs, and I couldn't find anything obvious with the logic of reserving and unreserving IPs.
Digging more into what happened vs how it happened, I found that:

• These contracts were created with 0 public IPs, so this shouldn't trigger the reserve IP logic.
• There have been no events to release these IPs due to these contracts' cancelation, which makes sense, since these contracts weren't reserved any public IPs.

https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Ftfchain.grid.tf%2Fws#/explorer/query/5844419

What could be causing these incorrect states? It's still not clear to me, but it's probably either:

• Likely, an old migration that interfered with some of the public IPs' states
• Less likely, a hidden bug in the runtime 116

Now, how to fix this:

• The council can call force_reset_farm_ip to force unreserve these IPs

It's important to conduct checks to ensure that there are no similar cases with other IPs on different farms as well. I'll follow up.

[sameh-farouk]

@LeeSmet @renauter Can we approve the suggested solution here so I can open tickets for ops to prepare and execute related proposals on mainnet and free these IPs ?
Otherwise, they will remain reserved forever

[LeeSmet]

I'm personally a bit uneasy with the force reset if the root cause is not identified. Since the working theory seems to be that something messed with the state in an unexpected way, not removing the IP's without knowing the full extend could lead to some more dangling state