[Bug]: Kingston Gov Date format change

[the-jeffski]

I Have A Problem With:

A specific source

What's Your Problem

It seems the date format in the API has changed to provide / instead of - as a separator. I amended my local copy for the date comparisons and this fixed the errors. Assume this is permanent or needs a if loop in case they revert.

Source (if relevant)

kingston_gov_uk

Logs

Traceback (most recent call last):
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 158, in fetch
    entries: Iterable[Collection] = self._source.fetch()
                                    ^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/kingston_gov_uk.py", line 76, in fetch
    date=datetime.strptime(
         ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/_strptime.py", line 554, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/_strptime.py", line 333, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data '2024/11/29 12:00:00' does not match format '%Y-%m-%d %H:%M:%S'

Relevant Configuration

No response

Checklist Source Error

• Use the example parameters for your source (often available in the documentation) (don't forget to restart Home Assistant after changing the configuration)
• Checked that the website of your service provider is still working
• Tested my attributes on the service provider website (if possible)
• I have tested with the latest version of the integration (master) (for HACS in the 3 dot menu of the integration click on "Redownload" and choose master as version)

Checklist Sensor Error

• Checked in the Home Assistant Calendar tab if the event names match the types names (if types argument is used)

Required

• I have searched past (closed AND opened) issues to see if this bug has already been reported, and it hasn't been.
• I understand that people give their precious time for free, and thus I've done my very best to make this problem as easy as possible to investigate.

[ilbarone87]

My Kingston API seems broken as well since a few days due to change in time data format.
I've tried to edit kingston_gov_uk.py file and set the correct format to "%Y/%m/%d %H:%M:%S" but after restarting HA I'm getting the below error

fetch failed for source The Royal Borough of Kingston Council: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 716, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 404, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1061, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 472, in connect _match_hostname(cert, self.assert_hostname or server_hostname) File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 545, in _match_hostname match_hostname(cert, asserted_hostname) File "/usr/local/lib/python3.12/site-packages/urllib3/util/ssl_match_hostname.py", line 150, in match_hostname raise CertificateError( urllib3.util.ssl_match_hostname.CertificateError: hostname 'kingston-self.achieveservice.com' doesn't match either of 'eu1.servisbot.com', '*.eu-1.servisbotcdn.com', '*.production.helium.servismatrixcdn.com', '*.production.helium.servismatrix.com', '*.eu1.servisbot.com', '*.servisbot.com', '*.eu1.servisbotcdn.com', '*.eu-1.servisbot.com' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 802, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 594, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kingston-self.achieveservice.com', port=443): Max retries exceeded with url: /service/In_my_Area_Results?uprn=100021772910&displaymode=collections&altVal= (Caused by SSLError(CertificateError("hostname 'kingston-self.achieveservice.com' doesn't match either of 'eu1.servisbot.com', '*.eu-1.servisbotcdn.com', '*.production.helium.servismatrixcdn.com', '*.production.helium.servismatrix.com', '*.eu1.servisbot.com', '*.servisbot.com', '*.eu1.servisbotcdn.com', '*.eu-1.servisbot.com'"))) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 158, in fetch entries: Iterable[Collection] = self._source.fetch() ^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/kingston_gov_uk.py", line 43, in fetch r0 = s.get(API_URLS["session"], headers=HEADERS) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 698, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='kingston-self.achieveservice.com', port=443): Max retries exceeded with url: /service/In_my_Area_Results?uprn=100021772910&displaymode=collections&altVal= (Caused by SSLError(CertificateError("hostname 'kingston-self.achieveservice.com' doesn't match either of 'eu1.servisbot.com', '*.eu-1.servisbotcdn.com', '*.production.helium.servismatrixcdn.com', '*.production.helium.servismatrix.com', '*.eu1.servisbot.com', '*.servisbot.com', '*.eu1.servisbotcdn.com', '*.eu-1.servisbot.com'")))

Certificate did not match expected hostname: kingston-self.achieveservice.com. Certificate: {'subject': ((('commonName', 'eu1.servisbot.com'),),), 'issuer': ((('countryName', 'US'),), (('organizationName', 'Amazon'),), (('commonName', 'Amazon RSA 2048 M02'),)), 'version': 3, 'serialNumber': '07DA90D939BA1A334318582B926DB1A8', 'notBefore': 'Sep 14 00:00:00 2024 GMT', 'notAfter': 'Oct 13 23:59:59 2025 GMT', 'subjectAltName': (('DNS', 'eu1.servisbot.com'), ('DNS', '*.eu-1.servisbotcdn.com'), ('DNS', '*.production.helium.servismatrixcdn.com'), ('DNS', '*.production.helium.servismatrix.com'), ('DNS', '*.eu1.servisbot.com'), ('DNS', '*.servisbot.com'), ('DNS', '*.eu1.servisbotcdn.com'), ('DNS', '*.eu-1.servisbot.com')), 'OCSP': ('http://ocsp.r2m02.amazontrust.com',), 'caIssuers': ('http://crt.r2m02.amazontrust.com/r2m02.cer',), 'crlDistributionPoints': ('http://crl.r2m02.amazontrust.com/r2m02.crl',)}

[ilbarone87]

I hope I'm wrong but by a first look at it, my python proficiency is not great, seems they have changed the API to only be allowed to be called from certain domains and this is really not a good news. Hope someone can confirm I'm wrong... 😢

[ilbarone87]

Tried to re-configure from GUI and getting this

[ilbarone87]

#3096 this PR should fix it.
Changes tested on local request is working again.

[5ila5]

fixed in #3096

This will be part of the next release or is available now as master (git) version

install master/git: Developer tools -> actions -> select update: Install update -> click Coose entity and select WCS -> select version and type master in the input field -> run the action -> restart HA

Older versions of HACS:

install master with HACS version < 2.0: (3-dot menu -> redownload -> change version to master -> fully restart HA (this will increase update cycle by a lot)

[cmsj]

@ilbarone87 I think they might have changed something again, I'm now seeing:

2024-12-16 09:47:12.097 ERROR (SyncWorker_8) [custom_components.waste_collection_schedule.waste_collection_schedule.source_shell] fetch failed for source The Royal Borough of Kingston Council:
Traceback (most recent call last):
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 158, in fetch
    entries: Iterable[Collection] = self._source.fetch()
                                    ~~~~~~~~~~~~~~~~~~^^
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/kingston_gov_uk.py", line 79, in fetch
    date=datetime.strptime(
         ~~~~~~~~~~~~~~~~~^
        data["echo_refuse_next_date"], "%Y/%m/%d %H:%M:%S"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ).date(),
    ^
  File "/usr/local/lib/python3.13/_strptime.py", line 573, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
                                    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/_strptime.py", line 352, in _strptime
    raise ValueError("time data %r does not match format %r" %
                     (data_string, format))
ValueError: time data 'Friday 20 December' does not match format '%Y/%m/%d %H:%M:%S'

[ilbarone87]

Not sure if will be able to fix, as my python knowledge is very basic.
Please open a new issue and will try to have a look.

[the-jeffski]

So they changed again and now its a string - not much use is that! Maybe we need to revert to using the ics link instead.

[ilbarone87]

@the-jeffski and @cmsj I've set the new date format to "%A %d %B" but doesn't seem to pull any date.
Not sure if I need to edit my sensor too:

- platform: waste_collection_schedule
  name: "TBC waste collections"
  details_format: "appointment_types"
  value_template: '{{value.types|join(", ")}}'
  date_template: 'in {{value.daysTo}} days'
  add_days_to: true
  types:
      - food waste bin
      - mixed recycling bin
      - paper and card recycling bin
      - refuse bin

[cmsj]

Filed #3224 so we can discuss the new fun there