Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update puppeteer to version 23.4.0 #18773

Open
timvandermeij opened this issue Sep 22, 2024 · 5 comments
Open

Update puppeteer to version 23.4.0 #18773

timvandermeij opened this issue Sep 22, 2024 · 5 comments
Labels
dependencies test

Comments

@timvandermeij
Copy link
Contributor

In #18771 we found that updating Puppeteer from version 23.3.0 to 23.4.0 caused new test failures. This is also reproducible locally. The logs from the bots can be found at http://54.193.163.58:8877/f9b8d1c303f789c/output.txt (Windows) and http://54.241.84.105:8877/902fcf449da1c69/output.txt (Linux). Version 23.3.1 doesn't have this issue locally, so we need to investigate what changed between 23.3.1 and 23.4.0 that causes some tests to fail and/or harden the tests against (intermittent) failure.
@timvandermeij timvandermeij mentioned this issue Sep 22, 2024
Merged
@timvandermeij
Copy link
Contributor Author

timvandermeij commented Sep 22, 2024 •
edited
Loading

I have compared the Puppeteer 23.3.1 release with the Puppeteer 23.4.0 release and found the following commits: puppeteer/puppeteer@puppeteer-v23.3.1...puppeteer-v23.4.0. There isn't really much interesting in there other than the Chrome update, and in the logs above I noticed that the failures are all in Chrome. However, the problems in Chrome might point to pre-existing issues in the tests themselves that only surface now.

@jamesgpearce
Copy link

Unrelated to pdf.js but I am also getting unusual errors from 23.4 relative to 23.3. Basic things like not being able to connect to the server for the first few tests.

@timvandermeij
Copy link
Contributor Author

timvandermeij commented Oct 5, 2024 •
edited
Loading

The issue exists in Puppeteer 23.5.0 too. The following logs contain the failures for me locally when running the integration tests against that version:

Puppeteer 23.5.0 integration test logs 
Failures:
1) FreeText Editor Freetext must stay focused after having been moved must keep the focus
  Message:
    In chrome: Expected 0 to equal 1.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/freetext_editor_spec.mjs:2326:14
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/freetext_editor_spec.mjs:2250:7)

2) Highlight Editor Highlight with the keyboard must check that some text has been highlighted
  Message:
    ProtocolError: Waiting for selector `#pdfjs_internal_editor_0` failed: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:322:36)
        at IsolatedWorld.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:86:30)
        at CdpJSHandle.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:147:37)
        at WaitTask.rerun (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:100:47)

3) Highlight Editor Highlight and caret browsing must check that selection is correctly highlighted on arrow down key pressed
  Message:
    ProtocolError: Waiting for selector `#pdfjs_internal_editor_0` failed: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:322:36)
        at IsolatedWorld.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:86:30)
        at CdpJSHandle.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:147:37)
        at WaitTask.rerun (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:100:47)

4) Highlight Editor Use a toolbar overlapping an other highlight must check that the toolbar is usable
  Message:
    ProtocolError: Waiting for selector `#pdfjs_internal_editor_0` failed: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:322:36)
        at IsolatedWorld.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:86:30)
        at CdpJSHandle.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:147:37)
        at WaitTask.rerun (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:100:47)

5) Ink Editor Basic operations must draw, undo/redo and check that the editor don't move
  Message:
    In chrome: Expected 420 to equal 121.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/ink_editor_spec.mjs:133:14
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/ink_editor_spec.mjs:103:7)
  Message:
    In chrome: Expected 331 to equal 32.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/ink_editor_spec.mjs:137:14
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/ink_editor_spec.mjs:103:7)

6) Interaction in issue15092.pdf must check that a value is correctly updated on a field and its siblings
  Message:
    ProtocolError: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:322:36)
        at IsolatedWorld.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:86:30)
        at CdpJSHandle.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:147:37)
        at WaitTask.rerun (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:100:47)

7) Interaction in bug1802888.pdf must check field value is treated by default as a number
  Message:
    In chrome: Expected '479' to equal '579'.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/scripting_spec.mjs:1596:58
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/scripting_spec.mjs:1580:7)

8) Interaction Textfield with a zip code starting with 0 must check the zip code is correctly formatted
  Message:
    ProtocolError: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:270:36)
        at IsolatedWorld.evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:97:30)
        at CdpJSHandle.evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:139:37)
        at awaitPromise (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/test_utils.mjs:129:18)

9) Stamp Editor Basic operations must load a SVG, delete it and undo
  Message:
    ProtocolError: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:322:36)
        at IsolatedWorld.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:86:30)
        at CdpJSHandle.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:147:37)
        at WaitTask.rerun (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:100:47)

10) Text layer Text selection using mouse doesn't jump when hovering on an empty area in a single page
  Message:
    In chrome:
        Expected '' to roughly match 'code sequences, records
        them, and compiles them to fast native code. We call suc'.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/text_layer_spec.mjs:138:18
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/text_layer_spec.mjs:116:11

In contrast, running the integration test locally with the currently pinned Puppeteer 23.3.1 version gives the following much cleaners logs (the only failure here is tracked in #18774):

Puppeteer 23.3.1 integration test logs 
Failures:
1) Text layer Text selection using mouse doesn't jump when hovering on an empty area in a single page
  Message:
    In chrome:
        Expected '' to roughly match 'code sequences, records
        them, and compiles them to fast native code. We call suc'.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/text_layer_spec.mjs:138:18
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/text_layer_spec.mjs:116:11)

@timvandermeij timvandermeij mentioned this issue Oct 6, 2024
Merged
This was referenced Nov 3, 2024
Merged
Open
@timvandermeij timvandermeij mentioned this issue Nov 17, 2024
Merged
@timvandermeij
Copy link
Contributor Author

timvandermeij commented Nov 17, 2024 •
edited
Loading

If PR #19064 is merged we should be down to two categories of issues with the newer versions of Puppeteer:

  • The Stamp Editor Basic operations must load a SVG, delete it and undo integration test, which fails pretty consistently locally and also on the bots sometimes already. This might actually be an issue in the editor code, which I have explained further in [Meta] Fix intermittent issues in the integration tests #18396 (comment) but don't actually know a good fix for yet.
  • The highlight editor integration tests from Update puppeteer to version 23.4.0 #18773 (comment), which fail consistently locally but not (or barely) on the bots. This might also be an issue in the editor code or the tests that the new Puppeteer version merely surfaced: the editor never actually shows up after making the selection. I have had some luck with reproducing this by following the exact steps from the test in the viewer too, but I'm not sure on the root cause or fix here too.

/cc @calixteman If you have any ideas here that'd be great, since you're much more familiar with this code/these tests than I am. Those should be the final blockers before we can update Puppeteer again. The test failures can easily be reproduced by setting Puppeteer to version ^23.8.0 locally in package.json, running npm install and then running npx gulp integrationtest (to speed it up, only enable the highlight and stamp specs in integration-boot.js).

@timvandermeij
Copy link
Contributor Author

The stamp editor issue is fixed by #19069, so only the highlight editor ones seem to remain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
dependencies test
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jamesgpearce @timvandermeij