Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/master' into update-from-upstream
Browse files Browse the repository at this point in the history
# Conflicts:
#	.gitignore
#	package-lock.json
#	package.json
  • Loading branch information
StasDoskalenkoHover committed Sep 27, 2023
2 parents af03249 + 10e23dd commit 3586186
Show file tree
Hide file tree
Showing 16 changed files with 5,690 additions and 179 deletions.
68 changes: 68 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
# ******** NOTE ********

name: "CodeQL"

on:
push:
branches: [ master ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ master ]
schedule:
- cron: '15 20 * * 3'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
language: [ 'javascript' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ]
# Learn more...
# https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#overriding-automatic-language-detection

steps:
- name: Checkout repository
uses: actions/checkout@v2

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main

# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl

# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language

#- run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
33 changes: 33 additions & 0 deletions .github/workflows/node.js.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions

name: Node.js CI

on:
push:
branches: [ master ]
pull_request:
branches: [ master ]

jobs:
build:

runs-on: ubuntu-latest
environment: test
strategy:
matrix:
node-version: [lts/*]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/

steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v2
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- run: npm install
- run: echo "${{ secrets.SERVICE_ACCOUNT }}" | gpg --quiet --batch --yes --decrypt --passphrase="${{ secrets.KEY }}" --output test-storage.json
- run: npm test
- run: rm -f ./test-storage.json

2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ node_modules
*~
storage.json
test/storage.json
.idea
.idea
5 changes: 5 additions & 0 deletions .idea/.gitignore

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions .idea/modules.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions .idea/vcs.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions .idea/zip-bucket.iml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions .npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.github
gpg
node_modules
*~
13 changes: 0 additions & 13 deletions .travis.yml

This file was deleted.

89 changes: 8 additions & 81 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,97 +1,24 @@
# zip-bucket



[![Build Status](https://travis-ci.org/DrPaulBrewer/zip-bucket.svg?branch=master)](https://travis-ci.org/DrPaulBrewer/zip-bucket)
[![Coverage Status](https://coveralls.io/repos/github/DrPaulBrewer/zip-bucket/badge.svg?branch=master)](https://coveralls.io/github/DrPaulBrewer/zip-bucket?branch=master)
![Build Status](https://github.com/DrPaulBrewer/zip-bucket/actions/workflows/node.js.yml/badge.svg)
[![Known Vulnerabilities](https://snyk.io/test/github/DrPaulBrewer/zip-bucket/badge.svg)](https://snyk.io/test/github/DrPaulBrewer/zip-bucket)

Creates a .zip archive from a collection of files in a Google Cloud Storage[tm] bucket. Include nodejs library and simple command-line program.

Keep the resulting .zip file locally, upload back to Google Cloud Storage, or both.

## Command Line Program

### Installation

For global installation of the command line program, so that all users can use "zip-bucket":

sudo npm install -g zip-bucket

### Usage

Enter `zip-bucket --help` to print this reminder message:

<pre>

Usage: zip-bucket [options] fromBucketPath [toBucketPath]

Options:

-h, --help output usage information
-V, --version output the version number
--key <keyfile> keyfile to access Google Cloud Storage API
--keep <keep> path in local filesystem to keep a copy of the .zip file
--progress show progress messages
--json output parameters and manifest in json at program completion

</pre>

`<fromBucketPath>` has the format `gs://bucketName/path`

`<toBucketPath>` needs to include the zip file name, e.g. `gs://bucketName/some/other/path/myzipfile.zip`

### Examples

#### Example #1

Goal: zip up the files found at gs://my-bucket/2016 to another bucket gs://backup-bucket/my2016.zip, show progress messages (not shown)

$ zip-bucket --progress --key /path/to/secret/key.json gs://my-bucket/2016 gs://backup-bucket/my2016.zip

#### Example #2

Goal: zip up the files found at gs://my-bucket/20170402T0616/U to a local file /tmp/sims2.zip and output the manifest (and parameters) in JSON

$ zip-bucket --json --key /path/to/secret/key.json --keep /tmp/sims2.zip gs://my-bucket/20170402T0616/U/
{
"keep": "/tmp/sims2.zip",
"fromBucket": "my-bucket",
"fromPath": "20170402T0616/U/",
"manifest": [
[
"20170402T0616/U/effalloc.csv",
"U/effalloc.csv"
],
[
"20170402T0616/U/md5.json",
"U/md5.json"
],
[
"20170402T0616/U/ohlc.csv",
"U/ohlc.csv"
],
[
"20170402T0616/U/profit.csv",
"U/profit.csv"
],
[
"20170402T0616/U/sim.json",
"U/sim.json"
],
[
"20170402T0616/U/trade.csv",
"U/trade.csv"
],
[
"20170402T0616/U/volume.csv",
"U/volume.csv"
]
]
}
The `zip-bucket` command line program was split off
into its own module in v2.0.0.

install [zip-bucket-bin](https://npmjs.com/zip-bucket-bin) for the `zip-bucket` command line program

## Library

The v2.0.0 library is the same as v1.9.0. The only breaking change
is the removal of the `zip-bucket` command line program and associated dependencies.

### Installation

npm i zip-bucket -S
Expand Down
69 changes: 0 additions & 69 deletions bin.js

This file was deleted.

8 changes: 6 additions & 2 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -120,9 +120,13 @@ module.exports = (storage) => (options) => {
});
}

function zipEachFile(filelist) {
async function zipEachFile(filelist) {
const {concurrentLimit = 1} = options;
return asyncPool(concurrentLimit, filelist, zipFile);
const results = [];
for await (const result of asyncPool(concurrentLimit, filelist, zipFile)) {
results.push(result);
}
return results;
}

function finalize() {
Expand Down
Loading

0 comments on commit 3586186

Please sign in to comment.