Skip to content

Commit

Permalink
Merge upstream (#4)
Browse files Browse the repository at this point in the history
* fix: polyfill 'fetch' for Node target

* fix(tests): remove 'fetch' workaround

This workaround was making the tests pass even when `fetch` was
undefined. That made sense if you assume that the only Node-based
clients of `scratch-storage` are its own tests, but it doesn't help (for
example) Node-based tests in `scratch-vm`. Since we now ensure that
`fetch` is polyfilled for Node, this workaround is no longer necessary.

* fix(fetchtool): catch fetch exception on get resource

* fix(fetchtool): update another fetch

* fix(fetchtool, webhelper): handle asset requests with a 404 response

FetchTool should resolve null for the data request if the request gets a 404 response. WebHelper
should properly handle this case and resolve null for the asset (instead of an asset skeleton with
null 'data' field) if the asset is nowhere to be found. WebHelper should also report errors
properly.

BREAKING CHANGE: The WebHelper was not previously resolving with null assets even though this was
the documented intended behavior of ScratchStorage.load

builds on and closes scratchfoundation#363 (thanks @AndreyNikitin!), closes scratchfoundation#156, might be related to scratchfoundation#367

* test(fetchtool): test handling 404 response to get request

Add a test that FetchTool.get resolves null when it encounters a 404 for the get request

* fix(fetchtool): create Uint8Array on promise result instead of promise

We were previously calling new Uint8Array on a promise instead of the result of the promise

* fix(fetchworkertool.worker): update `get` to handle 404 response

* test(fetchtool): update tests to handle promise reject and resolve

* fix(fetchworkertool): don't create Uint8Array for null response

* fix(worker): report worker errors as strings

Error objects can't be sent with `postMessage`, so report errors using strings instead.

* fix(worker): protect against null/undefined errors

Co-authored-by: Karishma Chadha <[email protected]>

* feat(circleci): add circleci

* ci: use node orb and node 16

* test: fix and modernize tests

* test: stop using deprecated methods

* chore: upgrade package-lock.json to lockfileVersion 2

```sh
npm ci && npm i --package-lock-only && npm i --package-lock-only
```

* refactor: use cross-fetch instead of node-fetch

Previously, we had Webpack's ProvidePlugin injecting `node-fetch` as a
polyfill into Node builds, and web builds just relied on the `fetch`
built into the browser (if it's there).

Now, we use `cross-fetch` everywhere, and as a ponyfill instead of a
polyfill. This broke the unit tests in `fetch-tool.js` because they were
overriding `global.fetch` to mock `node-fetch`. Now, those tests use
`tap.mock` to mock `cross-fetch`. Of course, `tap.mock` didn't exist in
the old version of `node-tap`, so I also upgraded that.

* chore: insert scratchFetch into FetchTool's pipeline

* feat: allow adding metadata to scratchFetch

* fix: apply metadata to FetchWorkerTool too

* fix: don't use ?. in src/ because VM isn't ready for that

* feat: allow removing request metadata

* docs: clarify (and fix) jsdoc for is*Supported in FetchTool

* fix: send Headers to fetch worker safely

Without this, the Headers object gets corrupted by postMessage and
results in CORS errors complaining about a header field called "map"

* test: expose scratchFetch API better to support testing

* refactor: store & merge headers using Headers object

This code will more directly follow Headers rules. In particular:
- Attempting to add metadata with a name that isn't a valid header name
  will cause an error immediately instead of in `applyMetadata` (likely
  at fetch time).
- Header names are not case sensitive. Using a Headers object internally
  means that the metadata API will respect that. For example,
  `setMetadata('a', ...)` will override an earlier value from
  `setMetadata('A', ...)`. Previously, this happened "accidentally" in
  `applyMetadata` instead of in the metadata container itself.

* test: mock cross-fetch more completely

Also emphasize that the `Headers` class used in `scratchFetch.js` comes
from `cross-fetch`

* refactor: turn mockFetch into a full module mock

This should make it harder to make a mistake mocking `cross-fetch` again

* test: add test for case sensitivity in metadata keys

* fix: rename headers to fit HTTP convention

* chore(deps): pin node orb to 5.1.0

* refactor: don't use !arraybuffer-loader! notation

* test: wait for checkAsset in load-default-assets tests

* test: convert all tests from node-tap to jest

* test: mock cross-fetch for download-known-assets, too

* test: fix transformer for backslashes

Injecting the path into the "code" was causing the backslashes to be
interpreted as escape characters instead of path separators. I could
have escaped them, but this approach is more similar to what Webpack
does with `arraybuffer-loader` and (IMO) easier to read.

* feat: control scratchFetch with query param

* test: add '?scratchMetadata=1' in metadata test

* fix: tell browsers to use browser build

* test: verify default asset sizes

This checks for things like accidentally using the filename as the file
content, etc.

* chore: repo rename for npm publish

* chore: package-lock update

* build: github actions build

* build: ci pin node version

---------

Co-authored-by: Christopher Willis-Ford <[email protected]>
Co-authored-by: Karishma Chadha <[email protected]>
Co-authored-by: meyerhot95 <[email protected]>
Co-authored-by: meyerhot95 <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
  • Loading branch information
6 people authored Oct 30, 2023
1 parent d12fc4a commit 22bf32b
Show file tree
Hide file tree
Showing 35 changed files with 35,630 additions and 12,198 deletions.
70 changes: 70 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@

version: 2.1
orbs:
commitlint: conventional-changelog/[email protected]
node: circleci/[email protected]

jobs:
build-and-test:
executor:
name: node/default
resource_class: medium
tag: '16.19'
steps:
- checkout
- node/install-packages
- run: npm run build
- run: npm run test
- persist_to_workspace:
root: ~/project
paths:
- .
deploy:
executor:
name: node/default
resource_class: medium
tag: '16.19'
steps:
- attach_workspace:
at: ~/project
- run: npm run semantic-release

workflows:
commitlint:
when:
not:
or:
- equal: [ master, <<pipeline.git.branch>> ]
- equal: [ develop, <<pipeline.git.branch>> ]
- matches: { pattern: "^hotfix.*", value: <<pipeline.git.branch>> }
- matches: { pattern: "^release.*", value: <<pipeline.git.branch>> }
jobs:
- commitlint/lint:
target-branch: develop

build-and-test-workflow:
when:
not:
or:
- equal: [ master, <<pipeline.git.branch>> ]
- equal: [ develop, <<pipeline.git.branch>> ]
- matches: { pattern: "^hotfix.*", value: <<pipeline.git.branch>> }
- matches: { pattern: "^release.*", value: <<pipeline.git.branch>> }
jobs:
- build-and-test

deploy-workflow:
when:
or:
- equal: [ master, <<pipeline.git.branch>> ]
- equal: [ develop, <<pipeline.git.branch>> ]
- matches: { pattern: "^hotfix.*", value: <<pipeline.git.branch>> }
- matches: { pattern: "^release.*", value: <<pipeline.git.branch>> }

jobs:
- build-and-test
- deploy:
context:
- scratch-npm-creds
requires:
- build-and-test
45 changes: 45 additions & 0 deletions .github/workflows/ci-cd.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: CI/CD

on:
pull_request: # Runs whenever a pull request is created or updated
push: # Runs whenever a commit is pushed to the repository...
branches: [main, develop, hotfix/*] # ...on any of these branches
workflow_dispatch: # Allows you to run this workflow manually from the Actions tab

concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
cancel-in-progress: true

permissions:
contents: write # publish a GitHub release
pages: write # deploy to GitHub Pages
issues: write # comment on released issues
pull-requests: write # comment on released pull requests

jobs:
ci-cd:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: wagoid/commitlint-github-action@v5
if: github.event_name == 'pull_request'
- uses: actions/setup-node@v3
with:
node-version: '16'
cache: 'npm'
- name: Info
run: |
cat <<EOF
Node version: $(node --version)
NPM version: $(npm --version)
GitHub ref: ${{ github.ref }}
GitHub head ref: ${{ github.head_ref }}
EOF
- name: Setup
run: npm ci
- run: npm run build
- run: npm test
- name: semantic-release
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
run: npx --no -- semantic-release
24 changes: 0 additions & 24 deletions .travis.yml

This file was deleted.

3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
## scratch-storage
#### Scratch Storage is a library for loading and storing project and asset files for Scratch 3.0

[![Build Status](https://travis-ci.org/LLK/scratch-storage.svg?branch=develop)](https://travis-ci.org/LLK/scratch-storage)
[![CircleCI](https://circleci.com/gh/LLK/scratch-storage/tree/develop.svg?style=shield&circle-token=171bbb6a82280e98afbfcedd8aa90b95b13b5de3)](https://circleci.com/gh/LLK/scratch-storage?branch=develop)

[![Coverage Status](https://coveralls.io/repos/github/LLK/scratch-storage/badge.svg?branch=develop)](https://coveralls.io/github/LLK/scratch-storage?branch=develop)
[![Greenkeeper badge](https://badges.greenkeeper.io/LLK/scratch-storage.svg)](https://greenkeeper.io/)

Expand Down
5 changes: 5 additions & 0 deletions jest.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module.exports = {
transform: {
'\\.(png|svg|wav)$': '<rootDir>/test/transformers/arraybuffer-loader.js'
}
};
Loading

0 comments on commit 22bf32b

Please sign in to comment.