Sheffield Council not working after council page changed

[JonScaife]

Name of Council

Sheffield Council

Issue Information

Setup fails with error Failed setup, will retry: Unexpected error: 'NoneType' object has no attribute 'find_all'
I think this is because the URL has changed?
The data URL certainly used to be https://wasteservices.sheffield.gov.uk/property/{property-id}
This now appears to have changed to https://wasteservices.sheffield.gov.uk/recycling-rubbish/property-search/{new-property-id}/your-collection-days
It appears that the code of the council site has also changed so just updating the URL alone probably won't resolve the issue. I'm not sure how recently hit was changed but I think it must be in the last week or 2.
My python is not good enough to work out how to change the parsing rules for the new html page so I haven't been able to work out a fix myself.
Thanks for the great work on this. Hope there will be a fix soon

Verification

• I searched for similar issues at https://github.com/robbrad/UKBinCollectionData/issues?q=is:issue and found no duplicates
• I have checked my address/postcode/UPRN works on the council's website
• I have provided a detailed explanation of the issue as well as steps to replicate the issue
• I understand that this project is run by volunteer contributors therefore completion of this issue cannot be guaranteed

[Tom-Whi]

I’ve done some additional testing and it looks like the old UPRN-based lookup still works, but the site frontend and HTML structure have changed.

The current parser appears to fail because it still expects the old table-based HTML:

rows = soup.find("table", {"class": re.compile("table")}).find_all("tr")

The new Sheffield page no longer contains that structure, so soup.find(...) returns None, which causes:

'NoneType' object has no attribute 'find_all'

I tested the Sheffield API endpoint directly and can confirm it still works when using the original UPRN value:

POST https://wasteservices.sheffield.gov.uk/api/getCalendarData

Payload:

{"councilId":"1","uprn":"<redacted-uprn>"}

This returns valid JSON collection data including:

service
actual_scheduled_date
service_group
service_color
frequency
actual_scheduled_day

Example response structure:

{
  "message": "OK",
  "searchedBy": "uprn",
  "statusCode": 200,
  "data": [
    {
      "records": [
        {
          "service": "Black Bin",
          "actual_scheduled_date": "2026-05-11T00:00:00Z"
        }
      ]
    }
  ]
}

The newer public URL format:

/recycling-rubbish/property-search/<id>/your-collection-days

appears to use a different internal identifier rather than the UPRN, and using that ID against the API returns 404.

So it looks like the existing UPRN configuration value is still valid, but the integration probably needs to switch from HTML scraping to parsing the API response instead.

[st3nic]

@Tom-Whi is the UPRN the long number at the end of the old URL we were using?

Its ok i have tried it and i works, i might be able to create a custom integration just for the Sheffield council site while a full fix is found.

[Tom-Whi]

@Tom-Whi is the UPRN the long number at the end of the old URL we were using?

Sorry - slow to reply. But yes it's the older long number. The new shorter number that the council website uses now doesn't work with the API I found.

[st3nic]

@Tom-Whi @JonScaife

https://github.com/st3nic/bin_collections_sheffield

Its the first release but it seems to be working, if you can try it please let me know.