Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cookiebot detecting over 10,000 subpages #140

Closed
ShannonTrust opened this issue Sep 27, 2024 · 19 comments
Closed

Cookiebot detecting over 10,000 subpages #140

ShannonTrust opened this issue Sep 27, 2024 · 19 comments
Assignees
Labels
bug Something isn't working sla

Comments

@ShannonTrust
Copy link
Collaborator

Please select the priority level by adding one of the following labels to this issue?

Priority-3 (Normal e.g. Functionality is not working for a subset of users)

Describe the bug and the expected behaviour.

Further to an email I sent Kristian about an increase in what we're paying for Cookiebot, I contacted their support team to understand why we're being charged more when we definitely don't have more than 3,500 subpages as their subscription page outline. They got back to me and said they detected over 10,000 pages - all of which come up as the news and views pages, but not the actual stories. Would you be able to investigate this further for us please as I'm really unsure why these pages are showing up in the scan when they're not even real pages.

Steps To Reproduce

No response

Screenshots or a link to a Loom Recording

Response from Cookiebot and the file they attached to show the pages detected

urlindex-2071909.csv
[Cookiebot Support] Re_ Increase in subscription cost.pdf

What browsers are you seeing the problem on?

No response

Anything else?

No response

@cyberteenie
Copy link
Member

Thanks for looking into that Megan! We will look into it.

@cyberteenie
Copy link
Member

@ShannonTrust - can you please forward the login verifcation email that was sent to media@shannontrust to [email protected] so that he can look into the Cookiebot issue a bit deeper please?

@ShannonTrust
Copy link
Collaborator Author

Hey @cyberteenie, we only had an email to activate the account. I clicked on the link and it says our account has now been verified, but we already have an account, so not sure why this was needed.

@AbdelhalimOJ
Copy link
Collaborator

AbdelhalimOJ commented Oct 1, 2024

Hi @ShannonTrust The pages identified by the Cookiebot scanner are not distinct; they are all variations of the same page, differentiated only by a query parameter that indicates the page number due to the paginated news collection. The solution is to redesign the news page to load all news items at once instead of four at a time.

If you agree with this approach, please let us know, and we can begin implementing the changes on the news page.

Update:
I updated the robots.txt file to prevent the scanner from indexing these pages. I also initiated a scan on Cookiebot to check if this change fixed the issue. If it hasn't, we’ll need to eliminate pagination and display all news items at once.

@ShannonTrust
Copy link
Collaborator Author

Hi @AbdelhalimOJ, thanks for looking into this. Here is the report:
Cookie scan report October 2024.pdf

I can't see where the pages are shown, but the compliance issue is coming up again.

I'm happy for you to go ahead with the pagination edits if you think that's best.

@AbdelhalimOJ
Copy link
Collaborator

Hi @ShannonTrust I'll look into the compliance issue. I've disabled pagination and applied another solution. You won't notice any changes, it's just a different implementation. This should stop the subpages from being generated.

@AbdelhalimOJ
Copy link
Collaborator

They say "Check the attached report to find out which scripts are loaded before consent" in the PDF file, is there any other report attached to this?

@ShannonTrust
Copy link
Collaborator Author

Thanks, @AbdelhalimOJ. Sorry, I thought you would have been able to click the link in the file in the PDF. I can't attach it separately as it's a HTML document. But if you go into the Cookiebot account and click into the reports tab, it's the first one that comes up.

@AbdelhalimOJ
Copy link
Collaborator

Thanks @ShannonTrust I checked all the cookies that weren't blocked until the user accepted them and made the necessary fixes on the website. However, there's still one script loading before consent, which is related to fonts. We're using a font from Adobe called "Aller," and I need to know if we're using Adobe Fonts from your account or ours.

If we are using your Adobe account, could you please check if you can get me the Adobe Fonts API Key? This will help me fix the issue with the fonts script.

@ShannonTrust
Copy link
Collaborator Author

Hi @AbdelhalimOJ, thanks for making the changes. Looking in our Adobe account, we don't have any API tokens. When we rebranded, we worked with a designer who sent us the fonts, so I'm assuming they're likely from his account. Will it still work if I set up a new API token, or will it need to be through the designer?

@AbdelhalimOJ
Copy link
Collaborator

if possible to get it through the designer it is preferable. @ShannonTrust

@ShannonTrust
Copy link
Collaborator Author

@AbdelhalimOJ OK, leave it with me and I'll get back to you when I know more.

@ShannonTrust
Copy link
Collaborator Author

Hi @AbdelhalimOJ, this is the response I had from the designer:

_When I finished the rebrand with you guys in 2021 I handed over the open artwork and brand guidelines to your developers. 'Aller' was being used as an Adobe Typekit font my end, but wouldn't have been included in the open artwork as you can't transfer fonts that are being streamed in via the creative cloud. At that point my involvement came to an end and the original developers would have had to source a version of 'Aller' in order to populate the new website. So I don't think you'd have had access to anything from my Adobe cloud account.

There might be an issue with the font API they've originally used, has your website hosting platform been updated recently? Sometimes newer versions of the CMS you're using can cause some issues with older fonts/APIs if they're not kept up to date.

The only other way around I can think of would be to purchase a copy of the desktop license from the original source: https://www.daltonmaag.com/font-library/aller.html#retail_

Is it possible that we're using the font from your account?

@AbdelhalimOJ
Copy link
Collaborator

Hi @ShannonTrust thanks for getting back to me. I am checking with the team if we used the Adobe API key from our account.

@AbdelhalimOJ
Copy link
Collaborator

AbdelhalimOJ commented Oct 9, 2024

Latest Update
re subpages I requested generation of a new scan report to see if disabling pagination solved the issue of 10,000 subpages or not.

@ShannonTrust if Cookiebot scanner can still see those pages then we will have to go on a meeting with their team to solve this issue.

@ShannonTrust
Copy link
Collaborator Author

Thanks for the update @AbdelhalimOJ. Is there anything else I need to do?

We've had another report come through and it looks like the compliance issue is sorted.

@AbdelhalimOJ
Copy link
Collaborator

Hi @ShannonTrust ! Yes I just saw it is now 121 subpages only ^^

image

Nothing else to do. Thanks and feel free to close this issue.

@AbdelhalimOJ
Copy link
Collaborator

AbdelhalimOJ commented Oct 10, 2024

@ShannonTrust I think you can downgrade to Premium Small subscription now since you have less than 350 subpages (pricing page here)
image

I have sent a request to the support team to downgrade, you are expected to get a response on [email protected] within 1 business day.

Subject: Request to downgrade to Premium Small
Body: We had an issue on our website, which caused your scanner to detect over 10,000 subpages. The issue is now fixed, and we only have 121 subpages. We are eligible to downgrade to the Premium Small plan.
Please downgrade our plan. Thanks!

@ShannonTrust
Copy link
Collaborator Author

Super, thanks so much @AbdelhalimOJ :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working sla
Projects
None yet
Development

No branches or pull requests

4 participants