GitHub 1C Enterprise Daily Trending

Anttwo/SuGaR

2023-12-22T01:25:25Z

Official implementation of SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering

SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering

Antoine Guédon Vincent Lepetit
LIGM, Ecole des Ponts, Univ Gustave Eiffel, CNRS

| Webpage | arXiv | Presentation video |

Our method extracts meshes from 3D Gaussian Splatting reconstructions and build hybrid representations
that enable easy composition and animation in Gaussian Splatting scenes by manipulating the mesh.

Abstract

We propose a method to allow precise and extremely fast mesh extraction from 3D Gaussian Splatting (SIGGRAPH 2023). Gaussian Splatting has recently become very popular as it yields realistic rendering while being significantly faster to train than NeRFs. It is however challenging to extract a mesh from the millions of tiny 3D Gaussians as these Gaussians tend to be unorganized after optimization and no method has been proposed so far. Our first key contribution is a regularization term that encourages the 3D Gaussians to align well with the surface of the scene. We then introduce a method that exploits this alignment to sample points on the real surface of the scene and extract a mesh from the Gaussians using Poisson reconstruction, which is fast, scalable, and preserves details, in contrast to the Marching Cubes algorithm usually applied to extract meshes from Neural SDFs. Finally, we introduce an optional refinement strategy that binds Gaussians to the surface of the mesh, and jointly optimizes these Gaussians and the mesh through Gaussian splatting rendering. This enables easy editing, sculpting, rigging, animating, or relighting of the Gaussians using traditional softwares (Blender, Unity, Unreal Engine, etc.) by manipulating the mesh instead of the Gaussians themselves. Retrieving such an editable mesh for realistic rendering is done within minutes with our method, compared to hours with the state-of-the-art method on neural SDFs, while providing a better rendering quality in terms of PSNR, SSIM and LPIPS.

Hybrid representation (Mesh + Gaussians on the surface)

Underlying mesh without texture

BibTeX

@article{guedon2023sugar,
  title={SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering},
  author={Gu{\'e}don, Antoine and Lepetit, Vincent},
  journal={arXiv:2311.12775},
  year={2023},
}

Updates and To-do list

Updates

[12/20/2023] Added a short notebook showing how to render images with the hybrid representation using the Gaussian Splatting rasterizer.
[12/18/2023] Code release.

To-do list

Using SuGaR with Windows: Adapt the code to make it compatible with Windows. Due to path-writing conventions, the current code is not compatible with Windows.
Synthetic datasets: Add the possibility to use the NeRF synthetic dataset (which has a different format than COLMAP scenes)
Composition and animation: Finish to clean the code for composition and animation, and add it to the sugar_scene/sugar_compositor.py script.
Composition and animation: Make a tutorial on how to use the scripts in the blender directory and the sugar_scene/sugar_compositor.py class to import composition and animation data into PyTorch and apply it to the SuGaR hybrid representation.

Overview

As we explain in the paper, SuGaR optimization starts with first optimizing a 3D Gaussian Splatting model for 7k iterations with no additional regularization term. In this sense, SuGaR is a method that can be applied on top of any 3D Gaussian Splatting model, and a Gaussian Splatting model optimized for 7k iterations must be provided to SuGaR.

Consequently, the current implementation contains a version of the original 3D Gaussian Splatting code, and we built our model as a wrapper of a vanilla 3D Gaussian Splatting model. Please note that, even though this wrapper implementation is convenient for many reasons, it may not be the most optimal one for memory usage, so we might change it in the future.

After optimizing a vanilla Gaussian Splatting model, the SuGaR pipeline consists of 3 main steps, and an optional one:

SuGaR optimization: optimizing Gaussians alignment with the surface of the scene
Mesh extraction: extracting a mesh from the optimized Gaussians
SuGaR refinement: refining the Gaussians and the mesh together to build a hybrid representation
Textured mesh extraction (Optional): extracting a traditional textured mesh from the refined SuGaR model

We provide a dedicated script for each of these steps, as well as a script train.py that runs the entire pipeline. We explain how to use this script in the next sections.

Please note that the final step, Textured mesh extraction, is optional but is enabled by default in the train.py script. Indeed, it is very convenient to have a traditional textured mesh for visualization, composition and animation using traditional softwares such as Blender. However, this step is not needed to produce, modify or animate hybrid representations.

Hybrid representation (Mesh + Gaussians on the surface)

Underlying mesh with a traditional colored UV texture

Below is another example of a scene showing a robot with a black and specular material. The following images display the hybrid representation (Mesh + Gaussians on the surface), the mesh with a traditional colored UV texture, and a depth map of the mesh:

Hybrid representation - Textured mesh - Depth map of the mesh

Installation

0. Requirements

The software requirements are the following:

Conda (recommended for easy setup)
C++ Compiler for PyTorch extensions
CUDA toolkit 11.8 for PyTorch extensions
C++ Compiler and CUDA SDK must be compatible

Please refer to the original 3D Gaussian Splatting repository for more details about requirements.

1. Clone the repository

Start by cloning this repository:

# HTTPS
git clone https://github.com/Anttwo/SuGaR.git --recursive

# SSH
git clone git@github.com:Anttwo/SuGaR.git --recursive

2. Install the required Python packages

To install the required Python packages and activate the environment, go inside the SuGaR/ directory and run the following commands:

conda env create -f environment.yml
conda activate sugar

If this command fails to create a working environment

Then you can try to install the required packages manually by running the following commands:

conda create --name sugar -y python=3.9
conda activate sugar
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install pytorch3d==0.7.4 -c pytorch3d
conda install -c plotly plotly
conda install -c conda-forge rich
conda install -c conda-forge plyfile==0.8.1
conda install -c conda-forge jupyterlab
conda install -c conda-forge nodejs
conda install -c conda-forge ipywidgets
pip install open3d
pip install --upgrade PyMCubes

3. Install the Gaussian Splatting rasterizer

Run the following commands inside the sugar directory to install the additional Python submodules required for Gaussian Splatting:

cd gaussian_splatting/submodules/diff-gaussian-rasterization/
pip install -e .
cd ../simple-knn/
pip install -e .
cd ../../../

Please refer to the 3D Gaussian Splatting repository for more details.

Quick Start

Start by optimizing a vanilla Gaussian Splatting model for 7k iterations by running the script gaussian_splatting/train.py, as shown below. Please refer to the original 3D Gaussian Splatting repository for more details. This optimization should be very fast, and last only a few minutes.

python gaussian_splatting/train.py -s <path to COLMAP dataset> --iterations 7000 -m <path to the desired output directory>

Then, run the script train.py in the root directory to optimize a SuGaR model.

python train.py -s <path to COLMAP dataset> -c <path to the Gaussian Splatting checkpoint> -r <"density" or "sdf">

The most important arguments for the train.py script are the following:

Parameter	Type	Description
`--scene_path` / `-s`	`str`	Path to the source directory containing a COLMAP dataset.
`--checkpoint_path` / `-c`	`str`	Path to the checkpoint directory of the vanilla 3D Gaussian Splatting model.
`--regularization_type` / `-r`	`str`	Type of regularization to use for optimizing SuGaR. Can be `"density"` or `"sdf"`. For reconstructing detailed objects centered in the scene with 360° coverage, `"density"` provides a better foreground mesh. For a stronger regularization and a better balance between foreground and background, choose `"sdf"`.
`--eval`	`bool`	If True, performs an evaluation split of the training images. Default is `True`.
`--low_poly`	`bool`	If True, uses the standard config for a low poly mesh, with `200_000` vertices and `6` Gaussians per triangle.
`--high_poly`	`bool`	If True, uses the standard config for a high poly mesh, with `1_000_000` vertices and `1` Gaussian per triangle.
`--refinement_time`	`str`	Default configs for time to spend on refinement. Can be `"short"` (2k iterations), `"medium"` (7k iterations) or `"long"` (15k iterations).
`--export_uv_textured_mesh` / `-t`	`bool`	If True, will optimize and export a traditional textured mesh as an `.obj` file from the refined SuGaR model, after refinement. Computing a traditional color UV texture should take less than 10 minutes. Default is `True`.

We provide more details about the two regularization methods "density" and "sdf" in the next section. For reconstructing detailed objects centered in the scene with 360° coverage, "density" provides a better foreground mesh. For a stronger regularization and a better balance between foreground and background, choose "sdf".
The default configuration is high_poly with refinement_time set to "long". Results are saved in the output/ directory.

As we explain in the paper, this script extracts a mesh in 30~35 minutes on average on a single GPU. After mesh extraction, the refinement time only takes a few minutes when using --refinement_time "short", but can take up to an hour when using --refinement_time "long". A short refinement time is enough to produce a good-looking hybrid representation in most cases.

Please note that the optimization time may vary (from 20 to 45 minutes) depending on the complexity of the scene and the GPU used. Moreover, the current implementation splits the optimization into 3 scripts that can be run separately (SuGaR optimization, mesh extraction, model refinement) so it reloads the data at each part, which is not optimal and takes several minutes. We will update the code in a near future to optimize this.

Below is a detailed list of all the command line arguments for the train.py script.

All command line arguments for train.py

Data and initial 3D Gaussian Splatting optimization

Parameter	Type	Description
`--scene_path` / `-s`	`str`	Path to the source directory containing a COLMAP data set.
`--checkpoint_path` / `-c`	`str`	Path to the checkpoint directory of the vanilla 3D Gaussian Splatting model.
`--iteration_to_load` / `-i`	`int`	Iteration to load from the 3DGS checkpoint directory. If not specified, loads the iteration `7000`.
`--eval`	`bool`	If True, performs an evaluation split of the training images. Default is `True`.

SuGaR optimization

Parameter	Type	Description
`--regularization_type` / `-r`	`str`	Type of regularization to use for optimizing SuGaR. Can be `"density"` or `"sdf"`.
`--gpu`	`int`	Index of GPU device to use. Default is `0`.

Mesh extraction

Parameter	Type	Description
`--surface_level` / `-l`	`int`	Surface level to extract the mesh at. Default is `0.3`.
`--n_vertices_in_mesh` / `-v`	`int`	Number of vertices in the extracted mesh. Default is `1_000_000`.
`--bboxmin` / `-b`	`str`	Min coordinates to use for foreground bounding box, formatted as a string `"(x,y,z)"`.
`--bboxmax` / `-B`	`str`	Max coordinates to use for foreground bounding box, formatted as a string `"(x,y,z)"`.
`--center_bbox`	`bool`	If True, centers the bbox. Default is True.

SuGaR and mesh refinement (Hybrid representation)

Parameter	Type	Description
`--gaussians_per_triangle` / `-g`	`int`	Number of gaussians per triangle. Default is `1`.
`--refinement_iterations` / `-f`	`int`	Number of refinement iterations. Default is `15_000`.

(Optional) Parameters for traditional textured mesh extraction

Parameter	Type	Description
`--export_uv_textured_mesh` / `-t`	`bool`	If True, will optimize and export a textured mesh as an .obj file from the refined SuGaR model. Computing a traditional colored UV texture should take less than 10 minutes. Default is `True`.
`--square_size`	`int`	Size of the square to use for the UV texture. Default is `10`.
`--postprocess_mesh`	`bool`	If True, postprocess the mesh by removing border triangles with low-density. This step takes a few minutes and is not needed in general, as it can also be risky. However, it increases the quality of the mesh in some cases, especially when very thin objects are visible only from one side in the images. Default is `False`.
`--postprocess_density_threshold`	`float`	Threshold to use for postprocessing the mesh. Default is `0.1`.
`--postprocess_iterations`	`int`	Number of iterations to use for postprocessing the mesh. Default is `5`.

(Optional) Default configurations

Parameter	Type	Description
`--low_poly`	`bool`	If True, uses standard config for a low poly mesh, with `200_000` vertices and `6` Gaussians per triangle.
`--high_poly`	`bool`	If True, uses standard config for a high poly mesh, with `1_000_000` vertices and `1` Gaussians per triangle.
`--refinement_time`	`str`	Default configs for time to spend on refinement. Can be `"short"` (2k iterations), `"medium"` (7k iterations) or `"long"` (15k iterations).

Tips for using SuGaR on your own data and obtain better reconstructions

1. Capture images or videos that cover the entire surface of the scene

Using a smartphone or a camera, capture images or a video that cover the entire surface of the 3D scene you want to reconstruct. The easiest way to do this is to move around the scene while recording a video. Try to move slowly and smoothly in order to avoid motion blur. For consistent reconstruction and easier camera pose estimation with COLMAP, maintaining a uniform focal length and a constant exposure time is also important. We recommend to disable auto-focus on your smartphone to ensure that the focal length remains constant.

For better reconstructions, try to cover objects from several and different angles, especially for thin and detailed parts of the scene. Indeed, SuGaR is able to reconstruct very thin and detailed objects, but some artifacts may appear if these thin objects are not covered enough and are visible only from one side in the training images.

Detailed explanation

SuGaR applies Poisson reconstruction with 3D points sampled on the parts of the surface that are visible in the training images. This visibility constraint is important to prevent sampling points on the backside of the Gaussian level sets, located behind the surface of the scene, which would produce a lot of self-collisions and many unnecessary vertices in the mesh after applying Poisson reconstruction. However, this visibility constraint also means that SuGaR cannot reconstruct parts of the surface that are not visible in the training images. If thin objects are visible only from one side in the training images, the Poisson reconstruction will try to reconstruct a closed surface, and will extend the surface of the thin objects, which produces an inaccurate mesh.

TODO: Add images illustrating such artifacts.

However, such artifacts are not visible in the hybrid representation, because the gaussian texturing gives low-opacity to these artifacts during refinement.

We already have simple ideas that could help to avoid such artifacts, such as (a) identifying new camera poses that cover parts of the surface non-visible in the training images that are likely to be on the same level set as the visible parts, and (b) adding these camera poses to the set of cameras used for sampling the points when applying Poisson reconstruction. We will update the code in a near future to include this.

To convert a video to images, you can install ffmpeg and run the following command:

ffmpeg -i <Path to the video file> -qscale:v 1 -qmin 1 -vf fps=<FPS> %04d.jpg

where <FPS> is the desired sampling rate of the video images. An FPS value of 1 corresponds to sampling one image per second. We recommend to adjust the sampling rate to the length of the video, so that the number of sampled images is between 100 and 300.

2. Estimate camera poses with COLMAP

Please first install a recent version of COLMAP (ideally CUDA-powered) and make sure to put the images you want to use in a directory <location>/input. Then, run the script gaussian_splatting/convert.py from the original Gaussian splatting implementation to compute the camera poses from the images using COLMAP. Please refer to the original 3D Gaussian Splatting repository for more details.

python gaussian_splatting/convert.py -s <location>

Sometimes COLMAP fails to reconstruct all images into the same model and hence produces multiple sub-models. The smaller sub-models generally contain only a few images. However, by default, the script convert.py will apply Image Undistortion only on the first sub-model, which may contain only a few images.

If this is the case, a simple solution is to keep only the largest sub-model and discard the others. To do this, open the source directory containing your input images, then open the sub-directory <Source_directory>/distorted/sparse/. You should see several sub-directories named 0/, 1/, etc., each containing a sub-model. Remove all sub-directories except the one containing the largest files, and rename it to 0/. Then, run the script convert.py one more time but skip the matching process:

python gaussian_splatting/convert.py -s <location> --skip_matching

Note: If the sub-models have common registered images, they could be merged into a single model as post-processing step using COLMAP; However, merging sub-models requires to run another global bundle adjustment after the merge, which can be time consuming.

3. Density or SDF? Choose a regularization method that fits your scene

As we explain in the paper, we provide two separate regularization methods for SuGaR: a density regularization and an SDF regularization. The density regularization is the simplest one and works well with objects centered in the scene. The SDF provides a stronger regularization, especially in background regions. As a consequence, the SDF regularization produces higher metrics on standard datasets. However, for reconstructing an object centered in the scene with images taken from all around the object, the simpler density regularization generally produces a better mesh.

Therefore, we recommend the following when using the train.py script:

For reconstructing detailed objects centered in the scene with 360° coverage (such as the toys we reconstructed in our presentation video), start with the density regularization -r 'density'. However, this may result in more chaotic Gaussians in the background.
For reconstructing more challenging scenes or enforcing a stronger regularization in the background, use the SDF regularization -r 'sdf'.

4. (Optional) Adapt the scale and the bounding box of the scene

As it is explained in the original 3D Gaussian Splatting repository, the method is expected to reconstruct a scene with reasonable scale. For reconstructing much larger datasets, like a city district, the original authors recommend to lower the learning rates of the positions and scaling factors of the Gaussians. The more extensive the scene, the lower these values should be.

Concerning SuGaR, such learning rates should also be lowered when reconstructing a very large scene. Moreover, as we explain in the supplementary material of the paper, for extracting a mesh from the Gaussians with an optimal repartition of vertices, we apply two Poisson reconstructions in practice: one on foreground Gaussians, and one on background Gaussians. The foreground Gaussians are defined as the Gaussians located inside a predefined bounding box, and the background Gaussians are defined as the Gaussians located outside this bounding box.

By default, this bounding box is computed as the bounding box of all camera centers. This general approach is coherent with how the original 3D Gaussian Splatting scales the learning rates. We used this default bounding box for all the reconstructions shown in the paper and the presentation video.

However, this bounding box might not be optimal in very specific cases, especially when the user wants to reconstruct with high details a very specific object located somewhere in the scene, or if the scene is very large, or if the camera centers are very far from the scene. The user is free to provide a custom bounding box to the train.py script, using the parameters --bboxmin and --bboxmax. Please note that the bounding box must be provided as strings, formatted as "(x,y,z)", where x, y and z are the coordinates of the min and max points of the bounding box.

Rendering, composition and animation

The metrics.py script provides an example of how to load a refined SuGaR model for rendering a scene with the hybrid representation and the Gaussian Splatting rasterizer. We will add more details about this in a near future, as well as a notebook with a detailed tutorial.

We also provide in the blender directory several python scripts to export from Blender composition and animation data of SuGaR meshes modified or animated within Blender. Additionally, we provide in the sugar_scene/sugar_compositor.py script a Python class that can be used to import such animation or composition data into PyTorch and apply it to the SuGaR hybrid representation.

The hybrid representation allows for high-quality rendering of the scene with the Gaussian Splatting rasterizer, as shown below.

The usage of these scripts and class may be a bit tricky, so we will add a detailed tutorial on how to use them in a near future.

Evaluation

To evaluate the quality of the reconstructions, we provide a script metrics.py that computes the PSNR, SSIM and LPIPS metrics on test images. Start by optimizing SuGaR models for the desired scenes and a regularization method ("density" or "sdf"), then create a .json config file containing the paths to the scenes in the following format: {source_images_dir_path: vanilla_gaussian_splatting_checkpoint_path}.

Finally, run the script as follows:

python metrics.py --scene_config <Path to the .json file> -r <"sdf" or "density">

Results are saved in a .json file in the output/metrics/ directory. Please refer to the script for more details on the command line arguments.

babaohuang/GeminiProChat

2023-12-22T01:25:25Z

A collection of Solana programs maintained by Solana Labs

Solana Program Library

The Solana Program Library (SPL) is a collection of on-chain programs targeting the Sealevel parallel runtime. These programs are tested against Solana's implementation of Sealevel, solana-runtime, and some are deployed to Mainnet Beta. As others implement Sealevel, we will graciously accept patches to ensure the programs here are portable across all implementations.

For more information see the SPL documentation and the Token TypeDocs.

Deployments

Only a subset of programs within the Solana Program Library repo are deployed to the Solana Mainnet Beta. Currently, this includes:

Program	Version
token	3.4.0
associated-token-account	1.1.0
token-2022	1.0.0
governance	3.1.0
stake-pool	1.0.0
account-compression	0.1.3
shared-memory	1.0.0
feature-proposal	1.0.0
name-service	0.3.0
memo	3.0.0

In addition, one program is planned for deployment to Solana Mainnet Beta:

Program	Version
single-pool	1.0.0

Audits

Only a subset of programs within the Solana Program Library repo are audited. Currently, this includes:

Program	Last Audit Date	Version
token	2022-08-04 (Peer review)	4fadd55
associated-token-account	2022-08-04 (Peer review)	c00194d
token-2022	2023-11-03	e924132
stake-pool	2023-11-14	6ed7254
account-compression	2022-12-05	6e81794
shared-memory	2021-02-25	b40e0dd
single-pool	2023-08-08	735d729

All other programs may be updated from time to time. These programs are not audited, so fork and deploy them at your own risk. Here is the full list of unaudited programs:

More information about the repository's security policy at SECURITY.md.

The security-audits repo contains all past and present program audits.

Program Packages

Package	Description	Version	Docs
`spl-token`	ERC20-like token program on Solana
`spl-token-2022`	Token program compatible with `spl-token`, with extensions
`spl-associated-token-account`	Stateless protocol defining a canonical "associated" token account for a wallet
`spl-governance`	DAO program using tokens for voting
`spl-account-compression`	Program for managing compressed accounts stored in an off-chain merkle tree
`spl-feature-proposal`	Program using tokens to vote on enabling Solana network features
`spl-noop`	Program that does nothing, used for logging instruction data
`spl-memo`	Program for logging signed memos on-chain
`spl-name-service`	Program for managing ownership of data on-chain
`spl-shared-memory`	Program for sharing data between programs
`spl-stake-pool`	Program for pooling stake accounts, managed by another entity
`spl-instruction-padding`	Program to padding to other instructions
`spl-concurrent-merkle-tree`	Library for on-chain representation of merkle tree
`spl-math`	Library for on-chain math
`spl-token-lending`	Over-collateralized lending program for tokens
`spl-token-swap`	AMM for trading tokens
`spl-token-upgrade`	Protocol for burning one token type in exchange for another

CLI Packages

Package	Description	Version
`spl-token-cli`	CLI for the token, token-2022, and associated-token-account programs
`spl-stake-pool-cli`	CLI for the stake-pool program
`spl-feature-proposal-cli`	CLI for the feature-proposal program
`spl-token-lending-cli`	CLI for the token-lending program
`spl-token-upgrade-cli`	CLI for the token-upgrade program

JavaScript Packages

Package	Description	Docs
`@solana/spl-token`	Bindings for the token, token-2022, and associated-token-account programs
`@solana/spl-governance`	Bindings for the governance program	N/A
`@solana/spl-account-compression`	Bindings for the account-compression program
`@solana/spl-memo`	Bindings for the memo program	N/A
`@solana/spl-name-service`	Bindings for the name-service program	N/A
`@solana/spl-stake-pool`	Bindings for the stake-pool program	N/A
`@solana/spl-token-lending`	Bindings for the token-lending program	N/A
`@solana/spl-token-swap`	Bindings for the token-swap program	N/A

Development

Environment Setup

Install the latest Solana tools.
Install the latest Rust stable. If you already have Rust, run rustup update to get the latest version.
Install the libudev development package for your distribution (libudev-dev on Debian-derived distros, libudev-devel on Redhat-derived).

Build

Build on-chain programs

# To build all on-chain programs
$ cargo build-sbf

# To build a specific on-chain program
$ cd <program_name>/program
$ cargo build-sbf

Build clients

# To build all clients
$ cargo build

# To build a specific client
$ cd <program_name>/cli
$ cargo build

Test

Unit tests contained within all projects can be run with:

$ cargo test      # <-- runs host-based tests
$ cargo test-sbf  # <-- runs BPF program tests

To run a specific program's tests, such as SPL Token:

$ cd token/program
$ cargo test      # <-- runs host-based tests
$ cargo test-sbf  # <-- runs BPF program tests

Integration testing may be performed via the per-project .js bindings. See the token program's js project for an example.

Common Issues

Solutions to a few issues you might run into are mentioned here.

Failed to open: ../../deploy/spl_<program-name>.so

Update your Rust and Cargo to the latest versions and re-run cargo build-sbf in the relevant <program-name> directory, or run it at the repository root to rebuild all on-chain programs.

Error while loading shared libraries. (libssl.so.1.1)

A working solution was mentioned here. Install libssl.

wget http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1l-1ubuntu1.2_amd64.deb
sudo dpkg -i libssl1.1_1.1.1l-1ubuntu1.2_amd64.deb

CPU or Memory usage at 100%

This is to be expected while building some of the programs in this library. The simplest solution is to add the --jobs 1 flag to the build commands to limit the number of parallel jobs to 1 and check if that fixes the issue. Although this will mean much longer build times.

Clippy

$ cargo clippy

Coverage

$ ./coverage.sh  # Help wanted! Coverage build currently fails on MacOS due to an XCode `grcov` mismatch...

MacOS

You may need to pin your grcov version, and then rustup with the apple-darwin nightly toolchain:

$ cargo install grcov --version 0.6.1
$ rustup toolchain install nightly-x86_64-apple-darwin

Release Process

SPL programs are currently tagged and released manually. Each program is versioned independently of the others, with all new development occurring on master. Once a program is tested and deemed ready for release:

Bump Version

Increment the version number in the program's Cargo.toml
Run cargo build-sbf <program> to build binary. Note the location of the generated spl_<program>.so for attaching to the Github release.
Open a PR with these version changes and merge after passing CI.

Create Github tag

Program tags are of the form <program>-vX.Y.Z. Create the new tag at the version-bump commit and push to the solana-program-library repository, eg:

$ git tag token-v1.0.0 b24bfe7
$ git push upstream --tags

Publish Github release

Go to GitHub Releases UI
Click "Draft new release", and enter the new tag in the "Tag version" box.
Title the release "SPL vX.Y.Z", complete the description, and attach the spl_<program>.so binary
Click "Publish release"

Publish to Crates.io

Navigate to the program directory and run cargo package to test the build. Then run cargo publish.

Disclaimer

All claims, content, designs, algorithms, estimates, roadmaps, specifications, and performance measurements described in this project are done with the Solana Labs, Inc. (“SL”) best efforts. It is up to the reader to check and validate their accuracy and truthfulness. Furthermore nothing in this project constitutes a solicitation for investment.

Any content produced by SL or developer resources that SL provides, are for educational and inspiration purposes only. SL does not encourage, induce or sanction the deployment, integration or use of any such applications (including the code comprising the Solana blockchain protocol) in violation of applicable laws or regulations and hereby prohibits any such deployment, integration or use. This includes use of any such applications by the reader (a) in violation of export control or sanctions laws of the United States or any other applicable jurisdiction, (b) if the reader is located in or ordinarily resident in a country or territory subject to comprehensive sanctions administered by the U.S. Office of Foreign Assets Control (OFAC), or (c) if the reader is or is working on behalf of a Specially Designated National (SDN) or a person subject to similar blocking or denied party prohibitions.

The reader should be aware that U.S. export control and sanctions laws prohibit U.S. persons (and other persons that are subject to such laws) from transacting with persons in certain countries and territories or that are on the SDN list. Accordingly, there is a risk to individuals that other persons using any of the code contained in this repo, or a derivation thereof, may be sanctioned persons and that transactions with such persons would be a violation of U.S. export controls and sanctions law.

solidjs/solid-start

2023-12-22T01:25:25Z

SolidStart, the Solid app framework

SolidStart

Start has just entered a new Beta Phase

v0.4.x marks a significant change in the project. Please check the updated docs and example projects to see how things have changed. A summary of the chances can be found in the RFC.

This is the home of the Solid app framework. This is still a work in progress. Many features are missing or incomplete. Experimental status does not even mean beta status. Patch releases will break everything.

Features

File-system based routing
Supports all rendering modes:
- Server-side rendering (SSR)
- Streaming SSR
- Client-side rendering (CSR)
- Static Site Generation (SSG)
Streaming
Build optimizations with Code splitting, tree shaking and dead code elimination
API Routes
Built on Web standards like Fetch, Streams, and WebCrypto
Adapters for deployment to all popular platforms
CSS Modules, SASS/SCSS Support
TypeScript-first

Getting started

Create a SolidStart application and run a development server using your preferred package manager:

mkdir my-app
cd my-app

# with npm
npm init solid@latest
npm install
npm run dev

# or with pnpm
pnpm create solid@latest
pnpm install
pnpm dev

# or with Bun
bunx create-solid@latest
bun install
bun run dev

Development

You should use a node version manager compatible with .node-version (asdf-vm is a great option mac/linux users)

The monorepo uses pnpm as the package manager. To install pnpm, run the following command in your terminal.

npm install -g pnpm

Run pnpm install to install all the dependencies for the packages and examples in your monorepo.

Monorepo & `project.json` `"workspace"` support

If you are using Solid Start within a monorepo that takes advantage of the package.json "workspaces" property (e.g. yarn workspaces) with hoisted dependencies (the default for yarn), you must include solid-start within the optional "nohoist" (for yarn v2 or higher, see further down for instructions) workspaces property.

In the following, "workspace root" refers to the root of your repository while "project root" refers to the root of a child package within your repository

For example, if specifying "nohoist" options from the workspace root (i.e. for all packages):

// in workspace root
{
  "workspaces": {
    "packages": [
      /* ... */
    ],
    "nohoist": ["**/solid-start"]
  }
}

If specifying "nohoist" options for a specific package using solid-start:

// in project root of a workspace child
{
  "workspaces": {
    "nohoist": ["solid-start"]
  }
}

Regardless of where you specify the nohoist option, you also need to include solid-start as a devDependency in the child package.json.

The reason why this is necessary is because solid-start creates an index.html file within your project which expects to load a script located in /node_modules/solid-start/runtime/entry.jsx (where / is the path of your project root). By default, if you hoist the solid-start dependency into the workspace root then that script will not be available within the package's node_modules folder.

Yarn v2 or higher

The nohoist option is no longer available in Yarn v2+. In this case, we can use the installConfig property in the package.json (either workspace package or a specific project package) to make sure our deps are not hoisted.

// in project root of a workspace child
{
  "installConfig": {
    "hoistingLimits": "dependencies"
  }
}

Credits

Dyalog/library-core

2023-12-22T01:26:28Z

Core application development library for Dyalog APL

Dyalog/dyalog-tube-ws

2023-12-22T01:26:28Z

Source for building the dfns tube planner workspace (tube.dws)

EbolaMan-YT/IPTOOLKIT

2023-12-23T01:25:07Z

just sum simple network stuff

rsm28/lethal_company_batch_files

2023-12-23T01:25:07Z

SebLague/Tiny-Chess-Bot-Challenge-Results

2023-12-23T01:25:42Z

Games, results, and source code of bots entered into the Tiny Chess Bot Coding Challenge

You can find the rankings of the main tournament in Swiss/Results.txt (note that #57 was removed as a duplicate of #62 prior to creating the knockout pairings). The format of the file is: ID | Bot Name | Author Name/s | Score (win = 2, draw = 1, loss = 0) | Tiebreak

Krypteria/AtlasLdr

2023-12-23T01:25:59Z

Reflective x64 PE/DLL Loader implemented using Dynamic Indirect Syscalls

Atlas is a reflective x64 loader that has the following features:

Features

Retrieve of DLL and PE from a remote server
Manual Mapping on a remote process
Position independent code
Use of indirect Syscalls
- ZwAllocateVirtualMemory
- ZwProtectVirtualMemory
- ZwQuerySystemInformation
- ZwFreeVirtualMemory
- ZwCreateThreadEx
Single stub for all Syscalls
- Dynamic SSN retrieve
- Dynamic Syscall address resolution
Atlas also uses
- LdrLoadDll
- NtWriteVirtualMemory
Custom implementations of
- GetProcAddress
- GetModuleHandle
API hashing
Cleanup on error
Variable EntryPoint

Usage

Compilation

Atlas needs to be compiled using x86_64-w64-mingw32-g++, once you have it on your system, just execute make (or mingw32-make.exe) on the project folder

Future work

Improve the way the syscalls are made
x86 support
Give more flexibility to the HTTP client
Implement some form of encryption on the remote connection

Disclaimer

There are improvements that could be made such as modifying the way syscalls are executed with a consistent and error-proof assembly code. In the future this could be implemented.

SJTU-IPADS/PowerInfer

2023-12-22T01:27:27Z

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU

TL;DR

PowerInfer is a CPU/GPU LLM inference engine leveraging activation locality for your device.

Demo 🔥

https://github.com/SJTU-IPADS/PowerInfer/assets/34213478/fe441a42-5fce-448b-a3e5-ea4abb43ba23

PowerInfer v.s. llama.cpp on a single RTX 4090(24G) running Falcon(ReLU)-40B-FP16 with a 11x speedup!

_{Both PowerInfer and llama.cpp were running on the same hardware and fully utilized VRAM on RTX 4090.}

Abstract

We introduce PowerInfer, a high-speed Large Language Model (LLM) inference engine on a personal computer (PC) equipped with a single consumer-grade GPU. The key underlying the design of PowerInfer is exploiting the high locality inherent in LLM inference, characterized by a power-law distribution in neuron activation.

This distribution indicates that a small subset of neurons, termed hot neurons, are consistently activated across inputs, while the majority, cold neurons, vary based on specific inputs. PowerInfer exploits such an insight to design a GPU-CPU hybrid inference engine: hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons are computed on the CPU, thus significantly reducing GPU memory demands and CPU-GPU data transfers. PowerInfer further integrates adaptive predictors and neuron-aware sparse operators, optimizing the efficiency of neuron activation and computational sparsity.

Evaluation shows that PowerInfer attains an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs (including OPT-175B) on a single NVIDIA RTX 4090 GPU, only 18% lower than that achieved by a top-tier server-grade A100 GPU. This significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy.

Features

PowerInfer is a high-speed and easy-to-use inference engine for deploying LLMs locally.

PowerInfer is fast with:

Locality-centric design: Utilizes sparse activation and 'hot'/'cold' neuron concept for efficient LLM inference, ensuring high speed with lower resource demands.
Hybrid CPU/GPU Utilization: Seamlessly integrates memory/computation capabilities of CPU and GPU for a balanced workload and faster processing.

PowerInfer is flexible and easy to use with:

Easy Integration: Compatible with popular ReLU-sparse models.
Local Deployment Ease: Designed and deeply optimized for local deployment on consumer-grade hardware, enabling low-latency LLM inference and serving on a single GPU.
Backward Compatibility: While distinct from llama.cpp, you can make use of most of examples/ the same way as llama.cpp such as server and batched generation. PowerInfer also supports inference with llama.cpp's model weights for compatibility purposes, but there will be no performance gain.

You can use these models with PowerInfer today:

Falcon-40B
Llama2 family

We have tested PowerInfer on the following platforms:

x86-64 CPU (with AVX2 instructions) on Linux
x86-64 CPU and NVIDIA GPU on Linux
Apple M Chips on macOS (As we do not optimize for Mac, the performance improvement is not significant now.)

And new features coming soon:

Mistral-7B model
Metal backend for sparse inference on macOS

Getting Started

Setup and Installation

Get the Code

git clone https://github.com/SJTU-IPADS/PowerInfer
cd PowerInfer
pip install -r requirements.txt # install Python helpers' dependencies

Build

In order to build PowerInfer you have two different options. These commands are supposed to be run from the root directory of the project.

Using CMake(3.13+) on Linux or macOS:

If you have an NVIDIA GPU:

cmake -S . -B build -DLLAMA_CUBLAS=ON
cmake --build build --config Release

If you just CPU:

cmake -S . -B build
cmake --build build --config Release

Model Weights

PowerInfer models are stored in a special format called PowerInfer GGUF based on GGUF format, consisting of both LLM weights and predictor weights.

Download PowerInfer GGUF via Hugging Face

You can obtain PowerInfer GGUF weights at *.powerinfer.gguf as well as profiled model activation statistics for 'hot'-neuron offloading from each Hugging Face repo below.

Base Model	PowerInfer GGUF
LLaMA(ReLU)-2-7B	PowerInfer/ReluLLaMA-7B-PowerInfer-GGUF
LLaMA(ReLU)-2-13B	PowerInfer/ReluLLaMA-13B-PowerInfer-GGUF
Falcon(ReLU)-40B	PowerInfer/ReluFalcon-40B-PowerInfer-GGUF
LLaMA(ReLU)-2-70B	PowerInfer/ReluLLaMA-70B-PowerInfer-GGUF

We suggest downloading/cloning the whole repo so PowerInfer can automatically make use of such directory structure for feature-complete model offloading:

.
├── *.powerinfer.gguf (Unquantized PowerInfer model)
├── *.q4.powerinfer.gguf (INT4 quantized PowerInfer model, if available)
├── activation (Profiled activation statistics for fine-grained FFN offloading)
│   ├── activation_x.pt (Profiled activation statistics for layer x)
│   └── ...
├── *.[q4].powerinfer.gguf.generated.gpuidx (Generated GPU index at runtime for corresponding model)

Convert from Original Model Weights + Predictor Weights

Hugging Face limits single model weight to 50GiB. For unquantized models >= 40B, you can convert PowerInfer GGUF from the original model weights and predictor weights obtained from Hugging Face.

Base Model	Original Model	Predictor
LLaMA(ReLU)-2-7B	SparseLLM/ReluLLaMA-7B	PowerInfer/ReluLLaMA-7B-Predictor
LLaMA(ReLU)-2-13B	SparseLLM/ReluLLaMA-13B	PowerInfer/ReluLLaMA-13B-Predictor
Falcon(ReLU)-40B	SparseLLM/ReluFalcon-40B	PowerInfer/ReluFalcon-40B-Predictor
LLaMA(ReLU)-2-70B	SparseLLM/ReluLLaMA-70B	PowerInfer/ReluLLaMA-70B-Predictor

You can use the following command to convert the original model weights and predictor weights to PowerInfer GGUF:

# make sure that you have done `pip install -r requirements.txt`
python convert.py --outfile /PATH/TO/POWERINFER/GGUF/REPO/MODELNAME.powerinfer.gguf /PATH/TO/ORIGINAL/MODEL /PATH/TO/PREDICTOR
# python convert.py --outfile ./ReluLLaMA-70B-PowerInfer-GGUF/llama-70b-relu.powerinfer.gguf ./SparseLLM/ReluLLaMA-70B ./PowerInfer/ReluLLaMA-70B-Predictor

For the same reason, we suggest keeping the same directory structure as PowerInfer GGUF repos after conversion.

Inference

For CPU-only and CPU-GPU hybrid inference with all available VRAM, you can use the following instructions to run PowerInfer:

./build/bin/main -m /PATH/TO/MODEL -n $output_token_count -t $thread_num -p $prompt
# ./build/bin/main -m ./ReluFalcon-40B-PowerInfer-GGUF/falcon-40b-relu.q4.powerinfer.gguf -n 128 -t 8 -p "Once upon a time"

If you want to limit the VRAM usage of GPU:

./build/bin/main -m /PATH/TO/MODEL -n $output_token_count -t $thread_num -p $prompt --vram-budget $vram_gb
# ./build/bin/main -m ./ReluLLaMA-7B-PowerInfer-GGUF/llama-7b-relu.powerinfer.gguf -n 128 -t 8 -p "Once upon a time" --vram-budget 8

Under CPU-GPU hybrid inference, PowerInfer will automatically offload all dense activation blocks to GPU, then split FFN and offload to GPU if possible.

Quantization

PowerInfer has optimized quantization support for INT4(Q4_0) models. You can use the following instructions to quantize PowerInfer GGUF model:

./build/bin/quantize /PATH/TO/MODEL /PATH/TO/OUTPUT/QUANTIZED/MODEL Q4_0
# ./build/bin/quantize ./ReluFalcon-40B-PowerInfer-GGUF/falcon-40b-relu.powerinfer.gguf ./ReluFalcon-40B-PowerInfer-GGUF/falcon-40b-relu.q4.powerinfer.gguf Q4_0

Then you can use the quantized model for inference with PowerInfer with the same instructions as above.

Evaluation

We evaluated PowerInfer vs. llama.cpp on a single RTX 4090(24G) with a series of FP16 ReLU models under inputs of length 64, and the results are shown below. PowerInfer achieves up to 11x speedup on Falcon 40B and up to 3x speedup on Llama 2 70B.

_{The X axis indicates the output length, and the Y axis represents the speedup compared with llama.cpp. The number above each bar indicates the end-to-end generation speed (total prompting + generation time / total tokens generated, in tokens/s).}

We also evaluated PowerInfer on a single RTX 2080Ti(11G) with INT4 ReLU models under inputs of length 8, and the results are illustrated in the same way as above. PowerInfer achieves up to 8x speedup on Falcon 40B and up to 3x speedup on Llama 2 70B.

Please refer to our paper for more evaluation details.

FAQs

What if I encountered CUDA_ERROR_OUT_OF_MEMORY?
- You can try to run with --reset-gpu-index argument to rebuild the GPU index for this model to avoid any stale cache.
- Due to our current implementation, model offloading might not be as accurate as expected. You can try with --vram-budget with a slightly lower value or --disable-gpu-index to disable FFN offloading.
Does PowerInfer support mistral, original llama, Qwen, ...?
- Now we only support models with ReLU/ReGLU/Squared ReLU activation function. So we do not support these models now. It's worth mentioning that a paper has demonstrated that using the ReLU/ReGLU activation function has a negligible impact on convergence and performance.
Why is there a noticeable downgrade in the performance metrics of our current ReLU model, particularly the 70B model?
- In contrast to the typical requirement of around 2T tokens for LLM training, our model's fine-tuning was conducted with only 5B tokens. This insufficient retraining has resulted in the model's inability to regain its original performance. We are actively working on updating to a more capable model, so please stay tuned.
What if...
- Issues are welcomed! Please feel free to open an issue and attach your running environment and running parameters. We will try our best to help you.

TODOs

We will release the code and data in the following order, please stay tuned!

Release core code of PowerInfer, supporting Llama-2, Falcon-40B.
Support Mistral-7B
Support Windows
Support text-generation-webui
Release perplexity evaluation code
Support Metal for Mac
Release code for OPT models
Release predictor training code
Support online split for FFN network
Support Multi-GPU

Paper and Citation

More technical details can be found in our paper.

If you find PowerInfer useful or relevant to your project and research, please kindly cite our paper:

@misc{song2023powerinfer,
      title={PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU}, 
      author={Yixin Song and Zeyu Mi and Haotong Xie and Haibo Chen},
      year={2023},
      eprint={2312.12456},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Acknowledgement

We are thankful for the easily modifiable operator library ggml and execution runtime provided by llama.cpp. We also extend our gratitude to THUNLP for their support of ReLU-based sparse models. We also appreciate the research of Deja Vu, which inspires PowerInfer.

webui-dev/webui

2023-12-22T01:27:27Z

Use any web browser as GUI, with your preferred language in the backend and HTML5 in the frontend, all in a lightweight portable lib.

WebUI v2.4.2

WebUI is not a web-server solution or a framework, but it allows you to use any web browser as a GUI, with your preferred language in the backend and HTML5 in the frontend. All in a lightweight portable lib.

Download

Features

Portable (Needs only a web browser at runtime)
One header file
Lightweight (Few Kb library) & Small memory footprint
Fast binary communication protocol
Multi-platform & Multi-Browser
Using private profile for safety

Showcase

This text editor is a lightweight and portable example written in C using WebUI as the GUI library.

UI & The Web Technologies

Borislav Stanimirov discusses using HTML5 in the web browser as GUI at the C++ Conference 2019 (YouTube).

Web application UI design is not just about how a product looks but how it works. Using web technologies in your UI makes your product modern and professional, And a well-designed web application will help you make a solid first impression on potential customers. Great web application design also assists you in nurturing leads and increasing conversions. In addition, it makes navigating and using your web app easier for your users.

Why Use Web Browsers?

Today's web browsers have everything a modern UI needs. Web browsers are very sophisticated and optimized. Therefore, using it as a GUI will be an excellent choice. While old legacy GUI lib is complex and outdated, a WebView-based app is still an option. However, a WebView needs a huge SDK to build and many dependencies to run, and it can only provide some features like a real web browser. That is why WebUI uses real web browsers to give you full features of comprehensive web technologies while keeping your software lightweight and portable.

How Does it Work?

Think of WebUI like a WebView controller, but instead of embedding the WebView controller in your program, which makes the final program big in size, and non-portable as it needs the WebView runtimes. Instead, by using WebUI, you use a tiny static/dynamic library to run any installed web browser and use it as GUI, which makes your program small, fast, and portable. All it needs is a web browser.

Runtime Dependencies Comparison

	WebView	Qt	WebUI
Runtime Dependencies on Windows	WebView2	QtCore, QtGui, QtWidgets	*A Web Browser*
Runtime Dependencies on Linux	GTK3, WebKitGTK	QtCore, QtGui, QtWidgets	*A Web Browser*
Runtime Dependencies on macOS	Cocoa, WebKit	QtCore, QtGui, QtWidgets	*A Web Browser*

Documentation

Note We are currently writing documentation.

Build

Windows

# GCC
mingw32-make

# MSVC
nmake

Windows SSL/TLS (Optional)

Download and install the OpenSSL pre-compiled binaries for Windows:

MSVC: x64 OpenSSL v3.1.3 or 32Bit. Please check this Wiki list for more info.
MinGW: Curl for Windows win OpenSSL

# GCC
mingw32-make WEBUI_USE_TLS=1 WEBUI_TLS_INCLUDE="C:\curl-xxx-xxx-mingw\curl-xxx-xxx-mingw\include" WEBUI_TLS_LIB="C:\curl-xxx-xxx-mingw\curl-xxx-xxx-mingw\lib"

# MSVC
nmake WEBUI_USE_TLS=1 WEBUI_TLS_INCLUDE="C:\Program Files\OpenSSL-xxx\include" WEBUI_TLS_LIB="C:\Program Files\OpenSSL-xxx\lib"

Linux

# GCC
make

# Clang
make CC=clang

Linux SSL/TLS (Optional)

sudo apt update
sudo apt install libssl-dev

# GCC
make WEBUI_USE_TLS=1

# Clang
make WEBUI_USE_TLS=1 CC=clang

macOS

make

macOS SSL/TLS (Optional)

brew install openssl
make WEBUI_USE_TLS=1

Examples

Wrappers

Language	Status	Link
Go	✔️	Go-WebUI
Nim	✔️	Nim-WebUI
Pascal	✔️	Pascal-WebUI
Python	✔️	Python-WebUI
Rust	not complete	Rust-WebUI
TypeScript / JavaScript	✔️	Deno-WebUI
V	✔️	V-WebUI
Zig	not complete	Zig-WebUI
Odin	not complete	Odin-WebUI

Supported Web Browsers

Browser	Windows	macOS	Linux
Mozilla Firefox	✔️	✔️	✔️
Google Chrome	✔️	✔️	✔️
Microsoft Edge	✔️	✔️	✔️
Chromium	✔️	✔️	✔️
Yandex	✔️	✔️	✔️
Brave	✔️	✔️	✔️
Vivaldi	✔️	✔️	✔️
Epic	✔️	✔️	not available
Apple Safari	not available	coming soon	not available
Opera	coming soon	coming soon	coming soon

License

Licensed under MIT License.

Stargazers

nmcfarl/donetoday

2023-12-22T01:28:21Z

Simple to-done list UI, with support for adding things done today and viewing past days.

sabraham/songbird

2023-12-22T01:28:21Z

jafingerhut/thalia

2023-12-22T01:28:21Z

A collection of documentation for Clojure symbols

learning-process/ppc-2023-mpi-reports

2023-12-23T01:26:33Z

Ritika-Agrawal811/css-and-js-code-snippets

2023-12-22T01:29:01Z

A collection of real-life code snippets written in CSS and JavaScript for every web development project.

css-and-js-code-snippets 💛

A collection of real-life code snippets written in CSS and JavaScript for every web development project. Each folder contains the stylesheet and script file (if required).

Categories 🍁

This repository is divided into mentioned categories. You may navigate accordingly to find cool web components to add to your project!

aalmiray/java-champions

2023-12-23T01:27:00Z

A list of Java Champions

Java Champions

What is a Java Champion?

One of the best descriptions was given in the 2009 JavaOne Conference PDF:

"The Java Champions community was started by Sun at the 2005 JavaOne conference to recognize key influencers in the Java community. Java Champions are influential Rock Star presenters and Java technology educators, authors, and consultants; Java platform event organizers; and others within the Java technology ecosystem. For the third consecutive year, Java Champions have contributed to the JavaOne conference process as technical reviewers for paper submissions, have been recognized for their achievements, and have shared their thoughts about the state of the Java platform at their BOF sessions." From javaone-2009.pdf

Note: This list is maintained by the Java Champions themselves.

huggingface/candle-paged-attention

2023-12-23T01:26:58Z

SemlerByte/Create-Host

2023-12-23T01:27:48Z

Here This Is Used To Make Your Free Host

navikt/dp-service-template

2023-12-23T01:27:48Z

A template repository (primarily) for our Behov solver services

Dagpenger-SERVICENAME

Komme i gang

Gradle brukes som byggverktøy og er bundlet inn.

./gradlew build

Sjekkliste for ny app

Bytt navn på app i nais.yaml, settings.gradle og dockerfile.
Fjern kommentering i deploy.yaml (står beskrevet hva man skal kommentere ut)
Legge til deploysecreten i repoet.

Henvendelser

Spørsmål knyttet til koden eller prosjektet kan rettes mot:

André Roaldseth, andre.roaldseth@nav.no
Eller en annen måte for omverden å kontakte teamet på

For NAV-ansatte

Interne henvendelser kan sendes via Slack i kanalen #dagpenger.

eooce/nodejs-argo

2023-12-23T01:27:48Z

node环境通用项目，输出sub.txt文件和域名/sub订阅或域名:端口/sub订阅，也可访问查看节点信息

说明（部署前请仔细阅读完）

本项目是针对node环境的paas平台和游戏玩具而生，采用Argo隧道部署节点，集成哪吒探针服务。
node玩具平台只需上传index.js和package.json即可，paas平台需要docker部署的才上传Dockerfile。
如需是链接github部署，请先删除README.md说明文件，安全起见，已混淆主代码部分。
不填写ARGO_DOMAIN和ARGO_AUTH两个变量即启用临时隧道，反之则使用固定隧道。
若遇到已获取到临时隧道但节点不通，说明域名被墙，重启即可
无需设置NEZHA_TLS,当哪吒端口为443时，自动开启--tls。
右边的Releases中已适配FreeBSD，自行下载，类似的平台Serv00，CT8

PaaS 平台设置的环境变量，index.js中的1至12行中设置

变量名	是否必须	默认值	备注
URL	否	https://www.google.com	项目分配的域名
PORT	否	3000	http服务监听端口，也是订阅端口
ARGO_PORT	否	8080	argo隧道端口，固定隧道token需和cloudflare后台设置的一致
UUID	否	89c13786-25aa-4520-b2e7-12cd60fb5202	UUID
TIME	否	2 * 60 * 1000	自动访问间隔时间（默认2分钟）
NEZHA_SERVER	否		哪吒服务端域名，例如nz.aaa.com
NEZHA_PORT	否	5555	当哪吒端口为443时，自动开启tls
NEZHA_KEY	否		哪吒客务端专用KEY
ARGO_DOMAIN	否		argo固定隧道域名
ARGO_AUTH	否		argo固定隧道json或token
CFIP	否	skk.moe	节点优选域名或ip
CFPORT	否	443	节点端口
NAME	否	ABCD	节点名称前缀，例如：Glitch，Replit
FILE_PATH	否	temp	运行目录

节点信息

本项目采用Argo隧道，输出sub文件和订阅，默认在temp文件夹内，域名/sub或域名:端口/sub查看节点信息,也是订阅连接

其他

本项目已添加自动访问保活功能，仅支持不重启停机的平台，需在第2行中添加项目分配的域名。建议配合外部自动访问保活，保活项目地址：https://github.com/eoovve/Auto-keep-online
Replit，Codesanbox，Glitch，Render，koyeb，Fly，Northfrank，back4app，Alwaysdate，Zeabur，Doprax及数十个游戏玩具平台均已测试ok。
Render及其他比较严格的容器平台，请使用docker image部署，Dockerfile地址：https://github.com/eoovve/nodejs-argo-image

免责声明

本程序仅供学习了解, 非盈利目的，请于下载后 24 小时内删除, 不得用作任何商业用途, 文字、数据及图片均有所属版权, 如转载须注明来源。
使用本程序必循遵守部署免责声明，使用本程序必循遵守部署服务器所在地、所在国家和用户所在国家的法律法规, 程序作者不对使用者任何不当行为负责。

sabiwara/aja

2023-12-23T01:28:24Z

Extension of the Elixir standard library focused on data stuctures, data manipulation and performance

anmolitor/protoc-gen-elm

2023-12-22T01:30:55Z

An Elm plugin for protoc

smies/bitPrice

2023-12-23T01:28:46Z

ramoncardias/sd_mc_workshop

2023-12-23T01:29:00Z

Workshop Atomistic Spin Dynamics: Fundamentals and Applications

Yatchanek/Platformer

2023-12-22T01:32:01Z

Game project for Fauna's 1M sub : a dungeon crawler

GhostTroops/scan4all

2023-12-22T01:32:23Z

Official repository vuls Scan: 15000+PoCs; 23 kinds of application password crack; 7000+Web fingerprints; 146 protocols and 90000+ rules Port scanning; Fuzz, HW, awesome BugBounty( ͡° ͜ʖ ͡°)...

💬

README_中文 • Compile/Install/Run • Parameter Description • How to use • Scenario • POC List • Custom Scan • Best Practices

Features

Free one id Multi-target web netcat for reverse shell
What is scan4all: integrated vscan, nuclei, ksubdomain, subfinder, etc., fully automated and intelligent。red team tools Code-level optimization, parameter optimization, and individual modules, such as vscan filefuzz, have been rewritten for these integrated projects. In principle, do not repeat the wheel, unless there are bugs, problems
Cross-platform: based on golang implementation, lightweight, highly customizable, open source, supports Linux, windows, mac os, etc.
Support [23] password blasting, support custom dictionary, open by "priorityNmap": true
- RDP
- VNC
- SSH
- Socks5
- rsh-spx
- Mysql
- MsSql
- Oracle
- Postgresql
- Redis
- FTP
- Mongodb
- SMB, also detect MS17-010 (CVE-2017-0143, CVE-2017-0144, CVE-2017-0145, CVE-2017-0146, CVE-2017-0147, CVE-2017-0148), SmbGhost (CVE- 2020-0796)
- Telnet
- Snmp
- Wap-wsp (Elasticsearch)
- RouterOs
- HTTP BasicAuth(Authorization), contains Webdav、SVN（Apache Subversion） crack
- Weblogic, enable nuclei through enableNuclei=true at the same time, support T3, IIOP and other detection
- Tomcat
- Jboss
- Winrm(wsman)
- POP3/POP3S
By default, http password intelligent blasting is enabled, and it will be automatically activated when an HTTP password is required, without manual intervention
Detect whether there is nmap in the system, and enable nmap for fast scanning through priorityNmap=true, which is enabled by default, and the optimized nmap parameters are faster than masscan Disadvantages of using nmap: Is the network bad, because the traffic network packet is too large, which may lead to incomplete results Using nmap additionally requires setting the root password to an environment variable

  export PPSSWWDD=yourRootPswd

More references: config/doNmapScan.sh By default, naabu is used to complete port scanning -stats=true to view the scanning progress Can I not scan Ports?

noScan=true ./scan4all -l list.txt -v
# nmap result default noScan=true 
./scan4all -l nmapRssuilt.xml -v

Fast 15000+ POC detection capabilities, PoCs include:
- nuclei POC
Nuclei Templates Top 10 statistics

TAG	COUNT	AUTHOR	COUNT	DIRECTORY	COUNT	SEVERITY	COUNT	TYPE	COUNT
cve	1430	daffainfo	631	cves	1407	info	1474	http	3858
panel	655	dhiyaneshdk	584	exposed-panels	662	high	1009	file	76
edb	563	pikpikcu	329	vulnerabilities	509	medium	818	network	51
lfi	509	pdteam	269	technologies	282	critical	478	dns	17
xss	491	geeknik	187	exposures	275	low	225
wordpress	419	dwisiswant0	169	misconfiguration	237	unknown	11
exposure	407	0x_akoko	165	token-spray	230
cve2021	352	princechaddha	151	workflows	189
rce	337	ritikchaddha	137	default-logins	103
wp-plugin	316	pussycat0x	133	file	76

281 directories, 3922 files.

vscan POC
- vscan POC includes: xray 2.0 300+ POC, go POC, etc.
scan4all POC

Support 7000+ web fingerprint scanning, identification:
- httpx fingerprint
  - vscan fingerprint
  - vscan fingerprint: including eHoleFinger, localFinger, etc.
- scan4all fingerprint
Support 146 protocols and 90000+ rule port scanning
- Depends on protocols and fingerprints supported by nmap
Fast HTTP sensitive file detection, can customize dictionary
Landing page detection
Supports multiple types of input - STDIN/HOST/IP/CIDR/URL/TXT
Supports multiple output types - JSON/TXT/CSV/STDOUT
Highly integratable: Configurable unified storage of results to Elasticsearch [strongly recommended]
Smart SSL Analysis:
- In-depth analysis, automatically correlate the scanning of domain names in SSL information, such as *.xxx.com, and complete subdomain traversal according to the configuration, and the result will automatically add the target to the scanning list
- Support to enable *.xx.com subdomain traversal function in smart SSL information, export EnableSubfinder=true, or adjust in the configuration file
Automatically identify the case of multiple IPs associated with a domain (DNS), and automatically scan the associated multiple IPs
Smart processing:
- 1. When the IPs of multiple domain names in the list are the same, merge port scans to improve efficiency
- 1. Intelligently handle http abnormal pages, and fingerprint calculation and learning
Automated supply chain identification, analysis and scanning
Link python3 log4j-scan
- This version blocks the bug that your target information is passed to the DNS Log Server to avoid exposing vulnerabilities
- Added the ability to send results to Elasticsearch for batch, touch typing
- There will be time in the future to implement the golang version how to use?

mkdir ~/MyWork/;cd ~/MyWork/;git clone https://github.com/hktalent/log4j-scan

Intelligently identify honeypots and skip Targets. This function is disabled by default. You can set EnableHoneyportDetection=true to enable
Highly customizable: allow to define your own dictionary through config/config.json configuration, or control more details, including but not limited to: nuclei, httpx, naabu, etc.
support HTTP Request Smuggling: CL-TE、TE-CL、TE-TE、CL_CL、BaseErr
Support via parameter Cookie='PHPSession=xxxx' ./scan4all -host xxxx.com, compatible with nuclei, httpx, go-poc, x-ray POC, filefuzz, http Smuggling

work process

how to install

download from Releases

go install github.com/GhostTroops/scan4all@2.8.9
scan4all -h

how to use

1. Start Elasticsearch, of course you can use the traditional way to output, results

mkdir -p logs data
docker run --restart=always --ulimit nofile=65536:65536 -p 9200:9200 -p 9300:9300 -d --name es -v $PWD/logs:/usr/share/elasticsearch/logs -v $PWD /config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v $PWD/config/jvm.options:/usr/share/elasticsearch/config/jvm.options -v $PWD/data:/ usr/share/elasticsearch/data hktalent/elasticsearch:7.16.2
# Initialize the es index, the result structure of each tool is different, and it is stored separately
./config/initEs.sh

# Search syntax, more query methods, learn Elasticsearch by yourself
http://127.0.0.1:9200/nmap_index/_doc/_search?q=_id:192.168.0.111
where 92.168.0.111 is the target to query

Please install nmap by yourself before use Using Help

go build
# Precise scan szUrl list UrlPrecise=true
UrlPrecise=true ./scan4all -l xx.txt
# Disable adaptation to nmap and use naabu port to scan its internally defined http-related Ports
priorityNmap=false ./scan4all -tp http -list allOut.txt -v

Work Plan

Integrate web-cache-vulnerability-scanner to realize HTTP smuggling smuggling and cache poisoning detection
Linkage with metasploit-framework, on the premise that the system has been installed, cooperate with tmux, and complete the linkage with the macos environment as the best practice
Integrate more fuzzers , such as linking sqlmap
Integrate chromedp to achieve screenshots of landing pages, detection of front-end landing pages with pure js and js architecture, and corresponding crawlers (sensitive information detection, page crawling)
Integrate nmap-go to improve execution efficiency, dynamically parse the result stream, and integrate it into the current task waterfall
Integrate ksubdomain to achieve faster subdomain blasting
Integrate spider to find more bugs
Semi-automatic fingerprint learning to improve accuracy; specify fingerprint name, configure

Q & A

how use Cookie?
libpcap related question

more see: discussions

References

Thanks Donors

Contributors

https://github.com/GhostTroops/scan4all/graphs/contributors

Changelog

2023-10-01 Optimize support for nuclei@latest
2022-07-28 Added substr and aes_cbc dsl helper by me nuclei v2.7.7
2022-07-20 fix and PR nuclei #2301 Concurrent multi-instance bug
2022-07-20 add web cache vulnerability scanner
2022-07-19 PR nuclei #2308 add dsl function: substr aes_cbc
2022-07-19 Add dcom Protocol enumeration network interfaces
2022-06-30 Embedded integrated private version nuclei-templates A total of 3744 YAML POC; 1. Integrate Elasticsearch to store intermediate results 2. Embed the entire config directory into the program
2022-06-27 Optimize fuzzy matching to improve accuracy and robustness; integrate ksubdomain progress
2022-06-24 Optimize fingerprint algorithm; add workflow chart
2022-06-23 Added parameter ParseSSl to control the default of not deeply analyzing DNS information in SSL and not scanning DNS in SSL by default; Optimization: nmap does not automatically add .exe bug; Optimize the bug of cache files under Windows not optimizing the size
2022-06-22 Integrated weak password detection and password blasting for 11 protocols: ftp, mongodb, mssql, mysql, oracle, postgresql, rdp, redis, smb, ssh, telnet, and optimized support for plug-in password dictionary
2022-06-20 Integrate Subfinder, domain name blasting, startup parameter export EnableSubfinder=true, note that it is very slow after startup; automatic deep drilling of domain name information in the ssl certificate allows you to define your own dictionary through config/config.json configuration, or set related switch
2022-06-17 Optimize the situation where one domain name has multiple IPs. All IPs will be port scanned, and then follow the subsequent scanning process.
2022-06-15 This version adds several weblogic password dictionaries and webshell dictionaries obtained in past actual combat
2022-06-10 Complete the integration of the core, including of course the integration of the core template
2022-06-07 Add similarity algorithm to detect 404
2022-06-07 Added http url list precision scanning parameters, turned on according to the environment variable UrlPrecise=true

Communication group (WeChat, QQ，Tg)

Wechat	Or	QQchat	Or	Tg

💖Star

Donation

Wechat Pay	AliPay	Paypal	BTC Pay	BCH Pay
		paypal miracletalent@gmail.com

RUB-NDS/Terrapin-Scanner

2023-12-22T01:32:23Z

This repository contains a simple vulnerability scanner for the Terrapin attack present in the paper "Terrapin Attack: Breaking SSH Channel Integrity By Sequence Number Manipulation".

Terrapin Vulnerability Scanner

The Terrapin Vulnerability Scanner is a small utility program written in Go, which can be used to determine the vulnerability of an SSH client or server against the Terrapin Attack. The vulnerability scanner requires a single connection with the peer to gather all supported algorithms. However, it does not perform a fully fledged SSH key exchange, will never attempt authentication on a server, and does not perform the attack in practice. Instead, vulnerability is determined by checking the supported algorithms and support for known countermeasures (strict key exchange). This may falsely claim vulnerability in case the peer supports countermeasures unknown to this tool.

Building

For convenience, we are providing pre-compiled binaries for all major desktop platforms. These can be found on the Release page.

However, we understand that you might prefer building tools, that connect to your SSH server, yourself. To do this, ensure that you have at least Go v1.21 installed. To compile and install the Terrapin Vulnerability Scanner Go package, run the command below.

go install github.com/RUB-NDS/Terrapin-Scanner@latest

This will download, compile, and install the Go package for your local system. The compiled binary will become available at $GOBIN/Terrapin-Scanner. If the GOBIN environment variable is not set, Go will default to using $GOPATH/bin or $HOME/go/bin, depending on whether the $GOPATH environment variable is set.

Usage

# Scan the SSH server available at localhost port 2222
./Terrapin-Scanner --connect localhost:2222

# If no port is specified, the tool will default to port 22 instead
./Terrapin-Scanner --connect localhost

# To scan an SSH client, specify the listen command instead
# After running the command, you will need to connect with your SSH client to port 2222
./Terrapin-Scanner --listen 0.0.0.0:2222

# When binding to localhost, you can omit the interface address
# The following command will listen for incoming connections on 127.0.0.1:2222
./Terrapin-Scanner --listen 2222

The scanner supports outputting the scan result as json. To do so, provide the --json flag when calling the scanner. The output is structured as follows:

{
    "Banner": "SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.5",
    "SupportsChaCha20": true,
    "SupportsCbcEtm": false,
    "SupportsStrictKex": true,
    "Vulnerable": false
}

youngsoul/DynamicPropertiesPlugin

2023-12-22T01:32:49Z

Plugin to support dynamic properties on a Domain class

IntersectMBO/plutus

2023-12-22T01:33:03Z

The Plutus language implementation and tools

Plutus Core

Table of Contents

Introduction

Plutus Core is the scripting language embedded in the Cardano ledger and forms the basis of the Plutus Platform, an application development platform for developing distributed applications using the Cardano blockchain.

For more information about the projects, see the [user-documentation].

This repository contains:

The implementation, specification, and mechanized metatheory of Plutus Core
Plutus Tx, the compiler from Haskell to Plutus Core.

For people who want to use the project, please consult the [user-documentation].

Development

How to develop and contribute to the project

Run nix develop to enter the development shell and you will be presented with a list of available commands.

*Please see CONTRIBUTING for comprehensive documentation on how to contribute to the project, including development and submitting changes

How to submit an issue

Issues can be filed in the GitHub Issue tracker.

How to depend on the project from another Haskell project

The plutus libraries are published via CHaP. See the information there for how to use CHaP. After setting it up you should just be able to depend on the plutus packages as normal and cabal will find them.

Documentation

User documentation

The main documentation is located here.

The latest documentation for the metatheory can be found here.

Talks

Functional Smart Contracts on Cardano (2020): an overview of the ideas behind the Plutus Platform.
The Plutus Platform (2020): an overview of the Platform as a whole (including the Application Framework) at the time.

Specifications and design

Plutus Technical Report (draft): a technical report and design document for the project.
Plutus Core Specification: the formal specification of the core language.
Extended UTXO Model: a design document for the core changes to the Cardano ledger.

Academic papers

Unraveling Recursion: a description of some of the compilation strategies used in Plutus IR (published version).
System F in Agda: a formal model of System F in Agda (published version).
The Extended UTXO Model: a full presentation of the EUTXO ledger extension (published version).
UTXOma: UTXO with Multi-Asset Support: a full presentation of the multi-asset ledger extension (published version).
Native Custom Tokens in the Extended UTXO Model: a discussion of the interaction of the multi-asset support with EUTXO (published version).
Translation Certification for Smart Contracts: a certifier of Plutus IR compiler passes written in Coq.

Licensing

You are free to copy, modify, and distribute this software under the terms of the Apache 2.0 license.

See the LICENSE and NOTICE files for details.

chrisdone/hell

2023-12-23T01:30:20Z

Haskell-based shell scripting language

hell

Welcome to Hell 😈

Table of Contents

hell

Description

Hell is an interpreted, statically-typed, shell scripting language based on Haskell.

It's a WIP.

Informal description

See examples/ for a list of example scripts.

Example program:

main = do
  Text.putStrLn "Please enter your name and hit ENTER:"
  name :: Text <- Text.getLine
  Text.putStrLn "Thanks, your name is: "
  Text.putStrLn name

Supports:

UTF-8 and binary file I/O
UTF-8 text operations (via text)
Stdout/stderr/stdin I/O
Directory, arguments, environment variables
Concurrency (via async)
Recursion (via fix)
Running processes (via typed-process)

Design philosophy

Turtle, Shelly, shell-conduit and Shh are "do shell scripting in Haskell", but lack something. GHC is a large dependency to require for running scripts, and the Haskell ecosystem is not capable of the stability required. Scripts written in them are brittle and clunky.

My definition of a shell scripting language:

A small interpreted language capable of launching processes
No abstraction or re-use capabilities from other files/modules/packages
Small, portable binary
Stable, does not change in backwards-incompatible ways

Hell can satisfy these criteria.

The other point of view that I've arrived at is that quotation like that of Bash and so on was never a good idea. I explored a Haskell that is quoted by default, such as ls $(echo x), and realised that this way leads to madness. What you write in a REPL and what you write in a script file can be different.

The other design decisions are:

Use existing Haskell naming convention, don't rename things for the sake of it. Even if the names Haskell chose aren't great, or are long.
Lean on and re-use concepts in the host system, even if they're flawed. Haskell's standard libraries get a lot of things right, and some things wrong. But stick to the intuitions that already are there where possible.
Don't break common shell understanding. Current directory and environment variables are process-wide, even if one would prefer otherwise. If you want "local" directories, carry a path around.
Use established API patterns that already work. (In particular this applies to the process launching API, which "script in Haskell" always tend to re-invent. I'm just re-using the API of typed-process.)

Names mirror their equivalent Haskell names (typically the package). Data.Text is Text.*, Control.Concurrent.Async is async, etc.

One exception to this rule is avoiding type String. Sorry, it's hard to justify when Text is established.

Also, avoiding operators, because operators are a bit harder to deal with combined with type applications.

There is only one monad, IO. So all monadic actions are specialised upon it.

mapM/forM are specialised on lists (like Haskell 98), and live under the IO. namespace. In future, there could be Maybe.mapM, etc. It's also possible to have traverse @IO @[] type of thing, but that seems unnecessarily verbose. List is mostly fine, especially for scripting purposes.

More formal description

The language is a simply-typed lambda calculus, with Haskell syntax.
Some primitives that can be polymorphic (but require immediately applied type applications).
Polymorphic primitives such as id require passing the type of the argument as id @Int 123. You cannot define polymorphic lambdas of your own. It's not full System-F.
Recursion is not supported. Use Function.fix.
Supports type-classes (Eq, Ord and Show only), but the type must still be explicitly supplied. You can't define classes, or data types, of your own.
The types and functions available lean directly on the host language (Haskell) and are either directly lifted, or a simplified layer over the original things.
There is presently no type inference (but I will add it). All parameters of lambdas, or do-notation let bindings, must have their type declared via a pattern signature: \(x :: Int) -> x
Globals of any kind must be fully qualified (Main.foo and Text.putstrLn), including the current module.

Instructions

Running

Presently the hell binary type-checks and interprets immediately a program in IO.

$ hell examples/01-hello-world.hell
Hello, World!

See https://github.com/chrisdone/hell/releases for a statically-linked amd64 Linux binary.

Building

Build statically for Linux in a musl distribution:

stack build --ghc-options="-static -optl-static"

Performance

I did a quick fib test and it does fine compared with runhaskell. There might be some undue strictness or something; I haven't looked deeply into it. What's important is that it's not dog slow, and it isn't.

import Data.Function
import Data.Bool
main = print $ fib (30::Int)

fib :: Int -> Int
fib = fix (\fib i ->
    bool
        (bool
           ( (fib (subtract 1 i))
            + (fib (subtract 2 i)))
          1
          (i == 1))
        0
        (i == 0)
    )

main = do
  Text.putStrLn (Int.show (Main.fib 30))

fib =
  Function.fix @(Int -> Int)
    (\(fib :: Int -> Int) -> \(i :: Int) ->
      Bool.bool @Int
        (Bool.bool @Int
           (Int.plus (fib (Int.subtract 1 i))
                     (fib (Int.subtract 2 i)))
           1
           (Int.eq i 1))
        0
        (Int.eq i 0)
    )

$ GHCRTS='-s' runhaskell test.hs
832040
     962,232,208 bytes allocated in the heap
      30,899,272 bytes copied during GC
       9,916,872 bytes maximum residency (4 sample(s))
         187,960 bytes maximum slop
              24 MiB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0       109 colls,     0 par    0.022s   0.022s     0.0002s    0.0037s
  Gen  1         4 colls,     0 par    0.067s   0.067s     0.0168s    0.0216s

  TASKS: 5 (1 bound, 4 peak workers (4 total), using -N1)

  SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled)

  INIT    time    0.001s  (  0.001s elapsed)
  MUT     time    3.758s  (  3.804s elapsed)
  GC      time    0.089s  (  0.089s elapsed)
  EXIT    time    0.001s  (  0.007s elapsed)
  Total   time    3.849s  (  3.900s elapsed)

  Alloc rate    256,067,353 bytes per MUT second

  Productivity  97.6% of total user, 97.5% of total elapsed

$ stack run -- examples/12-fib.hell +RTS -s
832040
   1,892,556,080 bytes allocated in the heap
       1,588,368 bytes copied during GC
         150,208 bytes maximum residency (2 sample(s))
          87,360 bytes maximum slop
              41 MiB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0       451 colls,   451 par    0.299s   0.158s     0.0004s    0.0006s
  Gen  1         2 colls,     1 par    0.001s   0.001s     0.0004s    0.0006s

  Parallel GC work balance: 4.14% (serial 0%, perfect 100%)

  TASKS: 18 (1 bound, 17 peak workers (17 total), using -N8)

  SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled)

  INIT    time    0.004s  (  0.002s elapsed)
  MUT     time    1.971s  (  1.588s elapsed)
  GC      time    0.300s  (  0.159s elapsed)
  EXIT    time    0.002s  (  0.001s elapsed)
  Total   time    2.278s  (  1.750s elapsed)

  Alloc rate    960,062,184 bytes per MUT second

  Productivity  86.5% of total user, 90.8% of total elapsed

CMS-Enterprise/batcave-tf-efs

2023-12-22T01:33:02Z

To house repositories related to the deployment of the batCAVE platform

CMS-Enterprise/batcave-tf-vpc

2023-12-22T01:33:02Z

Terraform modules supporting EKS deployments

CMS-Enterprise/batcave-tf-buckets

2023-12-22T01:33:02Z

Terraform modules supporting EKS deployments

linkedin/dph-framework

2023-12-22T01:33:17Z

CoreUI Angular is free Angular 17 admin template based on Bootstrap 5

CoreUI Free Admin Dashboard Template for Angular 17

CoreUI is meant to be the UX game changer. Pure & transparent code is devoid of redundant components, so the app is light enough to offer ultimate user experience. This means mobile devices also, where the navigation is just as easy and intuitive as on a desktop or laptop. The CoreUI Layout API lets you customize your project for almost any device – be it Mobile, Web or WebApp – CoreUI covers them all!

Versions

CoreUI Pro

Only customers with Enterpise Membership Plan have access to private GitHub CoreUI Pro repository.

Quick Start

Download the latest release
Clone the repo: git clone https://github.com/coreui/coreui-free-angular-admin-template.git

Prerequisites

Before you begin, make sure your development environment includes Node.js® and an npm package manager.

Node.js

Angular 17 requires Node.js LTS version ^18.13 or ^20.09.

To check your version, run node -v in a terminal/console window.
To get Node.js, go to nodejs.org.

Angular CLI

Install the Angular CLI globally using a terminal/console window.

npm install -g @angular/cli

Installation

$ npm install
$ npm update

Basic usage

# dev server with hot reload at http://localhost:4200
$ npm start

Navigate to http://localhost:4200. The app will automatically reload if you change any of the source files.

Build

Run build to build the project. The build artifacts will be stored in the dist/ directory.

# build for production with minification
$ npm run build

What's included

Within the download you'll find the following directories and files, logically grouping common assets and providing both compiled and minified variations. You'll see something like this:

coreui-free-angular-admin-template
├── src/                         # project root
│   ├── app/                     # main app directory
|   │   ├── containers/          # layout containers
|   |   │   └── default-layout/  # layout containers
|   |   |       └── _nav.js      # sidebar navigation config
|   │   ├── icons/               # icons set for the app
|   │   └── views/               # application views
│   ├── assets/                  # images, icons, etc.
│   ├── components/              # components for demo only
│   ├── scss/                    # scss styles
│   └── index.html               # html template
│
├── angular.json
├── README.md
└── package.json

Documentation

The documentation for the CoreUI Admin Template is hosted at our website CoreUI for Angular

This project was generated with Angular CLI version 17.0.0.

Versioning

For transparency into our release cycle and in striving to maintain backward compatibility, CoreUI Free Admin Template is maintained under the Semantic Versioning guidelines.

See the Releases section of our project for changelogs for each release version.

Development server

Run ng serve for a dev server. Navigate to http://localhost:4200/. The app will automatically reload if you change any of the source files.

Code scaffolding

Build

Run ng build to build the project. The build artifacts will be stored in the dist/ directory.

Running unit tests

Run ng test to execute the unit tests via Karma.

Running end-to-end tests

Run ng e2e to execute the end-to-end tests via a platform of your choice. To use this command, you need to first add a package that implements end-to-end testing capabilities.

Further help

To get more help on the Angular CLI use ng help or go check out the Angular CLI Overview and Command Reference page.

Creators

Łukasz Holeczek

CoreUI team

https://github.com/orgs/coreui/people

Community

Get updates on CoreUI's development and chat with the project maintainers and community members.

Follow @core_ui on Twitter.
Read and subscribe to CoreUI Blog.

Support CoreUI Development

CoreUI is an MIT-licensed open source project and is completely free to use. However, the amount of effort needed to maintain and develop new features for the project is not sustainable without proper financial backing. You can support development by buying the CoreUI PRO or by becoming a sponsor via Open Collective.

Platinum Sponsors

Support this project by becoming a Platinum Sponsor. A large company logo will be added here with a link to your website.

Gold Sponsors

Support this project by becoming a Gold Sponsor. A big company logo will be added here with a link to your website.

Silver Sponsors

Support this project by becoming a Silver Sponsor. A medium company logo will be added here with a link to your website.

Bronze Sponsors

Support this project by becoming a Bronze Sponsor. The company avatar will show up here with a link to your OpenCollective Profile.

Backers

Thanks to all the backers and sponsors! Support this project by becoming a backer.

Copyright and License

Code released under the MIT license. There is only one limitation you can't re-distribute the CoreUI as stock. You can’t do this if you modify the CoreUI. In the past, we faced some problems with persons who tried to sell CoreUI based templates.

Balackburn/Apollo

2023-12-23T01:30:36Z

Altstore source and website for Apollo for Reddit (with ImprovedCustomApi)

Original Tweak

Website

From

https://github.com/JeffreyCA/Apollo-ImprovedCustomApi

https://apolloapp.io/

jepsen-io/elle

2023-12-22T01:33:48Z

Black-box transactional safety checker based on cycle detection

vanila-io/wireflow

2023-12-23T01:31:26Z

Wireflow - user flow chart real-time collaborative tool

Wireflow - flow chart collaboration app

Alpha version of Wireflow app made by The Vanila Team and UX Store.

Official Website: Wireflow.co

Around the web:

Develop Locally

yarn
yarn start
open http://localhost:3000

Using docker-compose

docker-compose up -d

Credits

Contributors

This project exists thanks to all the people who contribute.

Backers

Thank you to all our backers! 🙏 [Become a backer]

vitozyf/lucky-draw

2023-12-23T01:31:26Z

年会抽奖程序

年会抽奖

演示地址

使用说明

配置建议

本程序理论支持配置百万级别人数，超过 10 万以上时初始化会比较耗时，初始化后可以正常抽奖。
如果人数较多，不建议导入名单或照片，按号码抽奖即可。

抽奖配置

设置抽奖总人数、奖项及每个奖项的人数，默认包含两个奖项，如果不想抽取默认的奖项，可以设置该奖项数量为 0
可以新增自定义的奖项，新增后必须将奖项人数设置大于 0 才会在抽奖奖项列表中显示

抽奖结果

显示抽取的结果，点击号码允许从结果中删除该号码。删除后该号码可以参与剩余的抽奖，否则已经中的号码无法参与剩余奖项的抽奖（除非开启全员参与功能）

开始

开始抽奖，需要选取抽取的奖项、本次抽取的人数和是否开启全员抽奖功能。
本次抽取的人数可以选择 1 人、5 人、一次性抽取完或者自定义抽取数量，不能大于奖项剩余的数量
点击停止抽取完成

重置

重置数据恢复到初始状态
可选的重置选项： 1.重置全部数据 2.重置抽奖配置 3.重置名单 4.重置照片 5.重置抽奖结果

导入名单

按照格式导入名单，可以多次输入。若号码有对应的姓名，则在抽取过程及结果中会显示号码及姓名，若没有对应的姓名，则只显示号码。

导入相册

按照抽奖号-照片的一对一导入，（可在现场将号码发出签到后，每发一个号码，导入一个照片）。抽奖结果将以照片形式展示。
照片格式支持.jpg和.png，照片大小不能超过 150kb,建议 20-50kb，建议尺寸为 160*160px

温馨提示

本抽奖程序无暗箱操作，无后台，无后门。
名单和照片显示只需导入一种即可,无导入数据则使用抽奖号码。
建议使用最新的 Chrome 浏览器打开体验最佳。
由于背景音乐加载较慢，可以在抽奖前提前打开缓存好。或者安装 windows 版本软件到本地，无需联网即可使用。

windows 软件版本

本抽奖程序的 windows 软件版本下载安装包

Donation

如果这个开源项目对您有帮助，请作者喝杯咖啡吧

Email: vito.z@qq.com

License

MIT

DuendeSoftware/IdentityServer

2023-12-23T01:31:26Z

The most flexible and standards-compliant OpenID Connect and OAuth 2.x framework for ASP.NET Core

Duende IdentityServer

The most flexible and standards-compliant OpenID Connect and OAuth 2.x framework for ASP.NET Core

Web site and documentation

How to build

Install Git
Clone this repo
Install the required .NET Core SDK
Run build.ps1 or build.sh in the root of the cloned repo

eclipse-eclemma/.eclipsefdn

2023-12-23T01:31:50Z

Repository to host configurations related to the Eclipse Foundation.

SocialComplexityLab/life2vec

2023-12-22T01:34:49Z

Using Sequences of Life-events to Predict Human Lives

This repository contains code for the Using Sequences of Life-events to Predict Human Lives (life2vec) paper.

Source Code

This repository contains scripts and several notebooks for data processing, life2vec training, statistical analysis, and visualization. The model weights, experiment logs, and associated model outputs can be obtained in accordance with the rules of Statistics Denmark's Research Scheme.

Paths (e.g., to data, or model weights) were redacted before submitting scripts to GitHub.

Overall Structure

We use Hydra to run the experiments. The /conf folder contains configs for the experiments:

/experiment contains configuration yaml for pretraining and finetuning,
/tasks contain the specification for data augmentation in MLM, SOP, etc.,
/trainer contains configuration for logging (not used) and multithread training (not used),
/data_new contains configs for data loading and processing,
/datamodule contains configs that specify how data should be loaded to PyTorch and PyTorch Lightning
callbacks.yaml specifies the configuration for the PyTorch Lightning Callbacks ,
prepare_data.yaml can be used to run data preprocessing.

The /analysis folder contains ipynb notebooks for post-hoc evaluation:

/embedding contains the analysis of the embedding spaces,
/metric contains notebooks for the model evaluation,
/visualisation contains notebooks for the visualisation of spaces,
/tcav includes TCAV implementation,
/optimization hyperparameter tuning.

The source folder, /src, contains the data loading and model training codes. Due to the specifics of the hydra package. Here is the overview of the /src folder:

The /src/data_new contains scripts to preprocess data as well as prepare data to load into the PyTorch or PyTorch Lightning,
The /src/models contains the implementation of baseline models,
The /src/tasks include code specific to the particular task, aka MLM, SOP, Mortality Prediction, Emigration Prediction, etc.
/src/tranformer contains the implementation of the life2vec model:
1. In performer.py, we overwrite the functionality of the performer-pytorch package,
2. In cls_model.py, we have an implementation of the finetuning stage for the binary classification tasks (i.e. early mortality and emigration),
3. In hexaco_model.py, we have an implementation of the finetuning stage for the personality nuance prediction task,
4. models.py contains the code for the life2vec pretraining (aka the base life2vec model),
5. The transformer_utils.py contains the implementation of custom modules, like losses, activation functions, etc.
6. The metrics.py contains code for the custom metric,
7. The modules.py, attention.py, att_utils.py, and embeddings.py contain the implementation of modules used in the transformer network (aka life2vec encoders).

Scripts such as train.py, test.py, tune.py, and val.py used to run a particular stage of the training, while prepare_data.py was used to run the data processing (see below the example).

Run the script

To run the code, you would use the following commands:

# run the pretraining:
HYDRA_FULL_ERROR=1 python -m src.train experiment=pretrain trainer.devices=[7]

# finetuning of the hyperparamaters (for the pretraining)
HYDRA_FULL_ERROR=1 python -m src.train experiment=pretrain_optim

# assemble general dataset (GLOBAL_SET)
HYDRA_FULL_ERROR=1 python -m src.prepare_data +data_new/corpus=global_set target=\${data_new.corpus}

# assemble dataset for the mortality prediction task (SURVIVAL_SET)
HYDRA_FULL_ERROR=1 python -m src.prepare_data +data_new/population=survival_set target=\${data_new.population}


# assemble labour source
python -m src.prepare_data +data_new/sources=labour target=\${data_new.sources}

# run emigration finetuning
HYDRA_FULL_ERROR=1 python -m src.train experiment=emm trainer.devices=[0] version=0.01

Another Code Contributors

Søren Mørk Hartmann.

How to cite

Research Square Preprint

@article{savcisens2023using,
  title={Using Sequences of Life-events to Predict Human Lives},
  author={Savcisens, Germans and Eliassi-Rad, Tina and Hansen, Lars Kai and Mortensen, Laust and Lilleholt, Lau and Rogers, Anna and Zettler, Ingo and Lehmann, Sune},
  year={2023}
}

Code

@misc{life2vec_code,
  author = {Germans Savcisens},
  note = {Zenodo},
  title = {SocialComplexityLab/life2vec},
  year = {2023},
  howpublished = {\url{https://doi.org/10.5281/zenodo.10118621}},
}

neulab/gemini-benchmark

2023-12-22T01:34:49Z

Foundational Models for State-of-the-Art Speech and Text Translation

Seamless Intro

Seamless is a family of AI models that enable more natural and authentic communication across languages. SeamlessM4T is a massive multilingual multimodal machine translation model supporting around 100 languages. SeamlessM4T serves as foundation for SeamlessExpressive, a model that preserves elements of prosody and voice style across languages and SeamlessStreaming, a model supporting simultaneous translation and streaming ASR for around 100 languages. SeamlessExpressive and SeamlessStreaming are combined into Seamless, a unified model featuring multilinguality, real-time and expressive translations.

Links

Demos

	SeamlessM4T v2	SeamlessExpressive	SeamlessStreaming
Demo	SeamlessM4T v2 Demo	SeamlessExpressive Demo
HuggingFace Space Demo	🤗 SeamlessM4T v2 Space	🤗 SeamlessExpressive Space	🤗 SeamlessStreaming Space

Papers

Seamless

EMMA

SONAR

Blog

AI at Meta Blog

Tutorial

An exhaustive tutorial given at the NeurIPS 2023 - Seamless EXPO, which is a one-stop shop to learn how to use the entire suite of Seamless models. Please feel free to play with the notebook.

SeamlessM4T

SeamlessM4T is our foundational all-in-one Massively Multilingual and Multimodal Machine Translation model delivering high-quality translation for speech and text in nearly 100 languages.

SeamlessM4T models support the tasks of:

Speech-to-speech translation (S2ST)
Speech-to-text translation (S2TT)
Text-to-speech translation (T2ST)
Text-to-text translation (T2TT)
Automatic speech recognition (ASR)

🌟 We are releasing SeamlessM4T v2, an updated version with our novel UnitY2 architecture. This new model improves over SeamlessM4T v1 in quality as well as inference latency in speech generation tasks.

To learn more about the collection of SeamlessM4T models, the approach used in each, their language coverage and their performance, visit the SeamlessM4T README or 🤗 Model Card.

[!NOTE] Seamless M4T is also available in the 🤗 Transformers library. Visit this section for more details.

SeamlessExpressive

SeamlessExpressive is a speech-to-speech translation model that captures certain underexplored aspects of prosody such as speech rate and pauses, while preserving the style of one's voice and high content translation quality.

To learn more about SeamlessExpressive models, visit the SeamlessExpressive README or 🤗 Model Card

SeamlessStreaming

SeamlessStreaming is a streaming translation model. The model supports speech as input modality and speech/text as output modalities.

The SeamlessStreaming model supports the following tasks:

Speech-to-speech translation (S2ST)
Speech-to-text translation (S2TT)
Automatic speech recognition (ASR)

To learn more about SeamlessStreaming models, visit the SeamlessStreaming README or 🤗 Model Card

Seamless

The Seamless model is the unified model for expressive streaming speech-to-speech translations.

What's new

[12/18/2023] We are open-sourcing our Conformer-based W2v-BERT 2.0 speech encoder as described in Section 3.2.1 of the paper, which is at the core of our Seamless models.
[12/14/2023] We are releasing the Seamless tutorial given at NeurIPS 2023.

Quick Start

Installation

[!NOTE] One of the prerequisites is fairseq2 which has pre-built packages available only for Linux x86-64 and Apple-silicon Mac computers. In addition it has a dependency on libsndfile which might not be installed on your machine. If you experience any installation issues, please refer to its README for further instructions.

pip install .

[!NOTE] Transcribing inference audio for computing metric uses Whisper, which is automatically installed. Whisper in turn requires the command-line tool ffmpeg to be installed on your system, which is available from most package managers.

Running inference

SeamlessM4T Inference

Here’s an example of using the CLI from the root directory to run inference.

S2ST task:

m4t_predict <path_to_input_audio> --task s2st --tgt_lang <tgt_lang> --output_path <path_to_save_audio>

T2TT task:

m4t_predict <input_text> --task t2tt --tgt_lang <tgt_lang> --src_lang <src_lang>

Please refer to the inference README for detailed instruction on how to run inference and the list of supported languages on the source, target sides for speech, text modalities.

For running S2TT/ASR natively (without Python) using GGML, please refer to the unity.cpp section.

SeamlessExpressive Inference

[!NOTE] Please check the section on how to download the model.

Here’s an example of using the CLI from the root directory to run inference.

expressivity_predict <path_to_input_audio> --tgt_lang <tgt_lang> --model_name seamless_expressivity --vocoder_name vocoder_pretssel --output_path <path_to_save_audio>

SeamlessStreaming and Seamless Inference

Streaming Evaluation README has detailed instructions for running evaluations for the SeamlessStreaming and Seamless models. The CLI has an --no-scoring option that can be used to skip the scoring part and just run inference.

Running SeamlessStreaming Demo

You can duplicate the SeamlessStreaming HF space to run the streaming demo.

You can also run the demo locally, by cloning the space from here. See the README of the SeamlessStreaming HF repo for more details on installation.

Running SeamlessM4T & SeamlessExpressive Gradio demos locally

To launch the same demo Space we host on Hugging Face locally:

cd demo
pip install -r requirements.txt
python app.py

Resources and usage

Model

SeamlessM4T models

Model Name	#params	checkpoint	metrics
SeamlessM4T-Large v2	2.3B	🤗 Model card - checkpoint	metrics
SeamlessM4T-Large (v1)	2.3B	🤗 Model card - checkpoint	metrics
SeamlessM4T-Medium (v1)	1.2B	🤗 Model card - checkpoint	metrics

SeamlessExpressive models

🤗 Model card

To access and download SeamlessExpressive, please request the model artifacts through this request form. Upon approval, you will then receive an email with download links to each model artifact.

Please note that SeamlessExpressive is made available under its own License and Acceptable Use Policy.

SeamlessStreaming models

Model Name	#params	checkpoint	metrics
SeamlessStreaming	2.5B	🤗 Model card - monotonic decoder checkpoint - streaming UnitY2 checkpoint	metrics

Seamless models

Seamless model is simply the SeamlessStreaming model with the non-expressive vocoder_v2 swapped out with the expressive vocoder_pretssel. Please check out above section on how to acquire vocoder_pretssel checkpoint.

W2v-BERT 2.0 speech encoder

Model Name	#params	checkpoint
W2v-BERT 2.0	600M	🤗 Model card - checkpoint

Here's how you should do a foward pass through the speech encoder:

import torch

from fairseq2.data.audio import AudioDecoder, WaveformToFbankConverter
from fairseq2.memory import MemoryBlock
from fairseq2.nn.padding import get_seqs_and_padding_mask
from pathlib import Path
from seamless_communication.models.conformer_shaw import load_conformer_shaw_model


audio_wav_path, device, dtype = ...
audio_decoder = AudioDecoder(dtype=torch.float32, device=device)
fbank_converter = WaveformToFbankConverter(
    num_mel_bins=80,
    waveform_scale=2**15,
    channel_last=True,
    standardize=True,
    device=device,
    dtype=dtype,
)
collater = Collater(pad_value=1)

model = load_conformer_shaw_model("conformer_shaw", device=device, dtype=dtype)
model.eval()

with Path(audio_wav_path).open("rb") as fb:
    block = MemoryBlock(fb.read())

decoded_audio = audio_decoder(block)
src = collater(fbank_converter(decoded_audio))["fbank"]
seqs, padding_mask = get_seqs_and_padding_mask(src)

with torch.inference_mode():
  seqs, padding_mask = model.encoder_frontend(seqs, padding_mask)
  seqs, padding_mask = model.encoder(seqs, padding_mask)

Evaluation

SeamlessM4T Evaluation

To reproduce our results, or to evaluate using the same metrics over your own test sets, please check out the README here.

SeamlessExpressive Evaluation

Below is the script for efficient batched evaluation.

export MODEL_DIR="/path/to/SeamlessExpressive/model"
export TEST_SET_TSV="input.tsv" # Your dataset in a TSV file, with headers "id", "audio"
export TGT_LANG="spa" # Target language to translate into, options including "fra", "deu", "eng" ("cmn" and "ita" are experimental)
export OUTPUT_DIR="tmp/" # Output directory for generated text/unit/waveform
export TGT_TEXT_COL="tgt_text" # The column in your ${TEST_SET_TSV} for reference target text to calcuate BLEU score. You can skip this argument.
export DFACTOR="1.0" # Duration factor for model inference to tune predicted duration (preddur=DFACTOR*preddur) per each position which affects output speech rate. Greater value means slower speech rate (default to 1.0). See expressive evaluation README for details on duration factor we used.
expressivity_evaluate ${TEST_SET_TSV} \
  --gated-model-dir ${MODEL_DIR} --task s2st --tgt_lang ${TGT_LANG} \
  --audio_root_dir "" --output_path ${OUTPUT_DIR} --ref_field ${TGT_TEXT_COL} \
  --model_name seamless_expressivity --vocoder_name vocoder_pretssel \
  --text_unk_blocking True --duration_factor ${DFACTOR}

Please check out this README section

SeamlessStreaming and Seamless Evaluation

Streaming Evaluation README has detailed instructions for running evaluations on the SeamlessStreaming and Seamless models.

Unity.cpp

To enable Seamless Communication Everywhere, we implemented unity.cpp so users could run SeamlessM4T models in GGML - a C tensor library allowing easier integration on verbose platforms.

To transcribe/translte a given audio,

./ggml/bin/unity --model seamlessM4T_medium.ggml input.wav

For details of build and more usage please check out unity.cpp

Expressive Datasets

We created two expressive speech-to-speech translation datasets, mExpresso and mDRAL, between English and five other languages -- French, German, Italian, Mandarin and Spanish. We currently open source the speech-to-text of mExpresso for out-of-English directions, and we will open source the remaining part of the datasets soon. For details, please check out README

SeamlessAlignExpressive

We’re introducing the first expressive speech alignment procedure. Starting with raw data, the expressive alignment procedure automatically discovers pairs of audio segments sharing not only the same meaning, but the same overall expressivity. To showcase this procedure, we are making metadata available to create a benchmarking dataset called SeamlessAlignExpressive, that can be used to validate the quality of our alignment method. SeamlessAlignExpressive is the first large-scale (11k+ hours) collection of multilingual audio alignments for expressive translation. More details can be found on the SeamlessAlignExpressive README.

Converting raw audio to units

Please check out the README here. Note that SeamlessM4T v1 model uses reduced units and other models use non-reduced units.

Libraries

Seamless Communication depends on 4 libraries developed by Meta.

fairseq2

fairseq2 is our next-generation open-source library of sequence modeling components that provides researchers and developers with building blocks for machine translation, language modeling, and other sequence generation tasks. All SeamlessM4T models in this repository are powered by fairseq2.

SONAR and BLASER 2.0

SONAR, Sentence-level multimOdal and laNguage-Agnostic Representations is a new multilingual and -modal sentence embedding space which outperforms existing sentence embeddings such as LASER3 and LabSE on the xsim and xsim++ multilingual similarity search tasks. SONAR provides text and speech encoders for many languages. SeamlessAlign was mined based on SONAR embeddings.

BLASER 2.0 is our latest model-based evaluation metric for multimodal translation. It is an extension of BLASER, supporting both speech and text. It operates directly on the source signal, and as such, does not require any intermediate ASR system like ASR-BLEU. As in the first version, BLASER 2.0 leverages the similarity between input and output sentence embeddings. SONAR is the underlying embedding space for BLASER 2.0. Scripts to run evaluation with BLASER 2.0 can be found in the SONAR repo.

stopes

As part of the seamless communication project, we've extended the stopes library. Version 1 provided a text-to-text mining tool to build training dataset for translation models. Version 2 has been extended thanks to SONAR, to support tasks around training large speech translation models. In particular, we provide tools to read/write the fairseq audiozip datasets and a new mining pipeline that can do speech-to-speech, text-to-speech, speech-to-text and text-to-text mining, all based on the new SONAR embedding space.

SimulEval

SimulEval is a library used for evaluating simulaneous translation models. SimulEval also provides a backend for generation using partial/incremental inputs with flexible/extensible states, which is used to implement streaming inference. Users define agents which implement SimulEval's interface, which can be connected together in a pipeline. You can find agents implemented for SeamlessStreaming here.

[Legacy] SeamlessM4T v1 instructions

Finetuning SeamlessM4T v1 models

Please check out the README here.

On-device models

Apart from Seamless-M4T large (2.3B) and medium (1.2B) models, we are also releasing a small model (281M) targeted for on-device inference. To learn more about the usage and model details check out the README here.

SeamlessAlign mined dataset

We open-source the metadata to SeamlessAlign, the largest open dataset for multimodal translation, totaling 270k+ hours of aligned Speech and Text data. The dataset can be rebuilt by the community based on the SeamlessAlign readme.

Citation

If you use Seamless in your work or any models/datasets/artifacts published in Seamless, please cite :

@inproceedings{seamless2023,
   title="Seamless: Multilingual Expressive and Streaming Speech Translation",
   author="{Seamless Communication}, Lo{\"i}c Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, John Hoffman, Min-Jae Hwang, Hirofumi Inaguma, Christopher Klaiber, Ilia Kulikov, Pengwei Li, Daniel Licht, Jean Maillard, Ruslan Mavlyutov, Alice Rakotoarison, Kaushik Ram Sadagopan, Abinesh Ramakrishnan, Tuan Tran, Guillaume Wenzek, Yilin Yang, Ethan Ye, Ivan Evtimov, Pierre Fernandez, Cynthia Gao, Prangthip Hansanti, Elahe Kalbassi, Amanda Kallet, Artyom Kozhevnikov, Gabriel Mejia, Robin San Roman, Christophe Touret, Corinne Wong, Carleigh Wood, Bokai Yu, Pierre Andrews, Can Balioglu, Peng-Jen Chen, Marta R. Costa-juss{\`a}, Maha Elbayad, Hongyu Gong, Francisco Guzm{\'a}n, Kevin Heffernan, Somya Jain, Justine Kao, Ann Lee, Xutai Ma, Alex Mourachko, Benjamin Peloquin, Juan Pino, Sravya Popuri, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, Anna Sun, Paden Tomasello, Changhan Wang, Jeff Wang, Skyler Wang, Mary Williamson",
  journal={ArXiv},
  year={2023}
}

License

We have three license categories.

The following non-generative components are MIT licensed as found in MIT_LICENSE:

W2v-BERT 2.0 speech encoder
Code
Text only part of the mExpresso dataset found in the SeamlessExpressive README.
UnitY2 forced alignment extractor found in the UnitY2 Aligner README.
Speech toxicity tool with the etox dataset found in the Toxicity README.

The following models are CC-BY-NC 4.0 licensed as found in the LICENSE:

SeamlessM4T models (v1 and v2).
SeamlessStreaming models.

The following models are Seamless licensed as found in SEAMLESS_LICENSE:

Seamless models.
SeamlessExpressive models.

YuKongA/Updater

2023-12-22T01:35:08Z

[APP] 一个简单的 HyperOS/MIUI 更新链接获取软件

Notes:

无论是否登录, 都可以查看服务器上已存在版本的基础信息

通过 v1 接口 (未登录小米账号状态) 可获取正式版下载链接

使用 v2 接口 (登陆拥有权限的小米账号) 可获取开发版下载链接

Instructions for use:

正式版后缀:

S/T/U (安卓版本)
NC/NB/MA/.. (机型内部代号)
CN/TW/MI/IN/KR/JP/TR/RU/.. (地区版本)
XM/DM/CC/.. (运营商版本)

示例:

以 小米 14, 版本号 OS1.0.26.0.UNCCNXM 为例, 其中:
U = Android14(U), NC = 内部代号(C=3, N 世代 3 等机), CN = 中国, XM = 全网通

以 小米 14 Pro, 版本号 OS1.0.27.0.UNBCNXM 为例, 其中:
U = Android14(U), NB = 内部代号(B=2, N 世代 2 等机), CN = 中国, XM = 全网通

以 小米 13 Ultra, 版本号 V14.0.5.0.TMAMIXM 为例, 其中:
T = Android13(T), MA = 内部代号(A=1, M 世代 1 等机), MI = 全球, XM = 全网通

设备代号后缀:

设备代号: 中国正式版, 中国开发版公测
设备代号_pre: 中国开发版内测
设备代号_global: 全球正式版
设备代号_pre_global: 全球开发版内测
设备代号_国家地区代码_global: 国家/地区专属正式版
设备代号_pre_国家地区代码_global: 国家/地区专属开发版内测

已知国家/地区相关后缀：

"_global", "_eea_global", "_pre_eea_global", "_pre_eea_miui15_global", "_h3g_global", "_eea_hg_global", "_eea_ee_global", "_pre_eea_ee_global", "_eea_or_global", "_eea_tf_global", "_eea_by_global", "_eea_vf_global", "_eea_sf_global", "_eea_ti_global", "_ru_global", "_pre_ru_global", "_in_global", "_pre_in_global", "in_global", "_in_fk_global", "_in_jo_global", "in_in_global", "_id_global", "_pre_id_global", "_tr_global", "_pre_tr_global", "_kr_global", "_kr_gu_global", "_kr_kt_global", "_kr_sk_global", "_jp_global", "_jp_kd_global", "_jp_sb_global", "_jp_rk_global", "_tw_global", "_pre_tw_global", "_global", "_tw_global", "_eea_global", "_ru_global", "_id_global", "_in_global", "in_global", "_in_fk_global", "_kr_global", "in_in_global", "_tr_global", "_jp_global", "_mx_global", "_lm_global", "_th_global", "_pe_global", "_za_global", "_jp_kd_global", "_kr_gu_global", "_kr_kt_global", "_kr_sk_global", "_h3g_global", "_eea_hg_global", "_eea_or_global", "_eea_tf_global", "_eea_by_global", "_eea_vf_global", "_mx_tc_global", "_mx_at_global", "_lm_cr_global", "_cl_en_global", "_cl_global", "_eea_sf_global", "_eea_ti_global", "_th_as_global", "_lm_ms_global", "_pe_ms_global", "_za_vc_global", "_za_mt_global", , "_pre_dpp_global", "_dev_soter_global", "_dc_global", "_test_pre_global", "_pre_miui14_global", "_pre_miui15_global", "_mx_global", "_lm_global", "_th_global", "_pe_global", "_za_global", "_mx_tc_global", "_mx_at_global", "_pre_mx_tc_global", "_pre_mx_at_global", "_lm_cr_global", "_cl_en_global", "_pre_cl_en_global", "_cl_global", "_th_as_global", "_lm_ms_global", "_lm_cr_global", "_pre_lm_cr_global", "_pe_ms_global", "_za_vc_global", "_za_mt_global", "_it_tim_global", "_it_vodafone_global", "_mx_telcel_global", "_es_vodafone_global", "_dck_global", "_gpp_pre_global", "_gt_tg_global", "_gt_global", "_gpp_global", "_qc_global", "_mcaas_global", "_cl_wom_global", "_cl_movistar_global", "_ita_vodafone_global", "_tr_turkcell_global", "_p70_global", "_fr_orange_global", "_wlnd_global"

其他已知后缀 (2023/12/16):

"_pre", "_factory", "_demo", "_ep_yunke", "_soter", "_mfw", "_pre_miui14", "_pre_miui15", "_dev_soter", "_shxc", "_stable_soter", "_hi25", "_by", "_qiy", "_tianyi", "_tq", "_xman", "_yh", "_yfan", "_new", "_3sat", "_beta", "_dev", "_gajw", "_zc360", "zq_360", "_pre_wechat", "_mtk", "_nio", "_bs", "_clb", "_gq", "_hhxa", "_hwl", "_justsafed", "_mdsw", "_ntb", "_xiuyixiu", "_hi2", "_test", "_gpp_pre", "_gpp", "_pre_7475vbl", "_pre_dpp", "_pre_gpp", "_chenfeng", "_dameng", "_ahjw", "_cqjw", "_didi", "_hbjw", "_hnjwxd", "_hnxdjw", "_jili", "_lnjw", "_scjw", "_sjt", "_tianshanjw", "_tangshanjw", "_wanguo", "_whjw", "_ydjw", "_yunnanjw", "_yunke", "_camera", "_port", "_aikesai", "_aochuang", "_hujing", "_kplus", "_ldrh", "_liuniu", "_lsjw", "_qsh", "_tl", "_tly", "_tongzhou", "_tongzhuo", "_wd", "_stable_cmcc", "_stable_cmcc01", "_stable_ct", "_ep_stdee", "_ep_xy", "_ep_kywl", "_ep_cqrcb", "_ep_ec", "_ep_sxht", "_ep_yfan", "_ep_yx", "_ep_stdce", "_ep_xdja", "_ep_litee", "_ep_yy", "_ep_by", "_ep_tq", "_ep_ui", "_ep_wosq", "_ep_xzm", "_ep_dhao", "_ep_qiy", "_ep_tly", "_ep_tlkj", "_ep_zc", "_ep_zdjt", "_ep_zzyglkg", "_ep_zyyglkg", "_ep_zzybp", "_ep_sdlybjcg", "_ep_justsafe", "_ep_justsafed", "_ep_nio", "_ep_txzx", "_ep_dameng", "_ep_yxyun", "_ep_hujing", "_ep_jwm", "_ep_daote", "_ep_jd", "_ep_tpkj", "_ep_tjzf", "_ep_tpybx", "_ep_bds", "_ep_hfw", "_ep_hn", "_ep_jyrj", "_ep_xysw", "_guazi", "_cf", "_gaotu", "_gz", "_hkdw", "_huaxun", "_jds", "_jlxf", "_wanglong", "_wlnd", "_yhai", "_yuxun", "_zkcd", "_cm", "_ct", "_beike", "_yf", "_yskj", "_zyb", "_ep_rb", "_dxo", "_yaohui", "_bcwl", "_czht", "_txzx", "_ep_daotetest", "_jkpd_factory", "_miui13_pre", "_test_pre", "_hmh", "_rrc", "_zy", "_ep_sdlyjcb", "_pre_ep_stdee", "_haozu", "_szkx", "_xmzy", "_yhhl", "_huatian", "_mcaas", "_qkzq", "_qzxx", "_shzl", "_sjyc", "_gzdt", "_lls", "_miui_factory", "_shrq", "_shrx", "_taier", "_tmg", "wuweilab", "_cmcc", "_ajy", "_dp", "_zjzy", "_fs", "_langtuo", "_ep_gy58tc", "_ep_jds", "_ep_yhai", "_zhutai", "_bb2021", "ep_czht", "_ep_mjwxns", "_ep_qdyh", "_research", "_sdlybjcg", "_8475_pre", "_ep_sbgl", "_miui_demo_factory", "_bindsim", "_ent_ct", "_fxtc", "_gzxc", "_dqgx"

Credits:

关于设备代号("Code name"), 请参阅: 小米手机型号汇总

关于国家地区代码("Country codes"), 请参阅: ISO 3166-2

HuolalaTech/batteryfinder-android

2023-12-22T01:35:08Z

A detection tool of Android App power consumption

English Readme

BatteryFinder

如果你的App还在为高耗电而被用户投诉，却无法知道是哪里引发了高耗电，那么这款耗电检测工具BatteryFinder(电源探测者)应该可以帮到你。它利用了编译时插桩的机制，将必要的功耗api进行hook，同时通过耗电基准文件进行加权得到耗电量，同时可通过堆栈打印，调用者跟踪等等分析手段帮助开发者发现异常功耗问题。

目前支持：定位，蓝牙，Alarm，PowerWakeLock，WifiWakeLock，Sensor等多层次功耗检测

详细介绍

https://juejin.cn/post/7311343224546574346

使用介绍

编译引入插件

《声明》由于目前业务项目还在AGP版本还在7以下，所以当前仅支持7以下的插件，后续补充7以上的插件

根目录build.gradle中配置插件

classpath 'cn.huolala:batteryplugin:1.0.2'

gradle 配置

使用module中build.gradle里面添加插件依赖即可

apply plugin: 'com.battery.plugins'

同时选择开启插桩的类型，见app 中build.gradle

BatteryHookConfig{
// 因为以下涉及到hook操作，可根据配置进行是否开启，true则开启，默认都是false不开启
// Alarm闹钟 
  alarm = true   
//传感器
  sensor = true  
//蓝牙
  blueTooth = true 
//定位
  location = true 
//wakelock
  powerWakeLock = true 
  wifiWakeLock = true  、
// 设置不插桩的classname
  whiteList = ['whiteList1','whiteList2']
}

使用module中添加aar依赖

implementation 'cn.huolala:batteryapi:1.0.2'

必要的框架初始化设置

step1

在Application 中进行必要的设置（几乎无损耗）

 BatteryFinderDataCenter.setConfig(
            // 是否是debug
            DataConfig.Builder()
                // 是否是debug模式，用于框架内部排查
                .setIsDebug(false)
                // 可选 是否记录多进程模式下的数据【仅支持定位】
                .setIsOpenIPC(true)
                // 是否运行打开耗电计算，可用于线上开关配置
                .setIsOpenBatteryFinder(true)
                // 是否打开堆栈跟踪，打开后会记录堆栈，如果用在频繁函数，存在一定的堆栈记录损耗
                .setOpenStackTrace(true)
                // 设置达到次数后才回调，0就是立即回调，配置设置于定位 【以下推荐默认值即可】
                .setLocationInvokeTime(0)
                // 等同于上，设置蓝牙达到次数
                .setBlueToothInvokeTime(0)
                // 等同于上，设置sensor达到次数
                .setSensorInvokeTime(0)
                // 等同于上，设置wakelock 达到次数
                .setPowerWakeLockInvokeTime(0)
                .setWifiWakeLockInvokeTime(0)
                // alarm 调度次数
                .setAlarmInvokeTime(0)
                .build()
        )

step2

设置自己需要的基准电源文件（power_profile.xml）

在assets中放入power_profile.xml文件，如果不关心具体机型差异带来的功耗差异表现，可以直接用demo项目中的power_profile.xml文件，位于（app/src/main/assets）中

如果需要机型特定的power_profile.xml文件，可以通过反编译获取手机的

adb pull /system/framework/framework-res.apk

通过反编译framework-res.apk获取power_profile文件，放入assets即可

step3

定义一个实现DataChangeInvoke的类，里面可以选择复写方法，当有对应的数据来时，就会通过spi机制回调，参考【DemoInvoker】

比如调用requestLocationUpdates时或者removeUpdates 时，都会回调一次onLocationInvoke，在里面可以获取功耗数据

override fun onLocationInvoke(
recordState: InvokeState,
locationReportData: LocationReportData,
batteryData: BatteryData,
cpuStatData: CpuStatData
) {
   super.onLocationInvoke(recordState, locationReportData, batteryData, cpuStatData)
   Log.e("demo_battery_finder","耗电量 已完成 + 记录中 ${locationReportData.getCurrentBatteryConsume()}")
   Log.e("demo_battery_finder","耗电量已完成 ${locationReportData.getRecordedBatteryConsume()}")
   Log.e("demo_battery_finder","耗电量记录中 ${locationReportData.getRecordingBatteryConsume()}")
   Log.e("demo_battery_finder","$recordState $locationReportData")
}

然后通过Android 标准SPI流程，在resources/META-INF/services/ 下建立一个com.battery.api.DataChangeInvoker文件，里面内容填写为自定义的实现DataChangeInvoke类名即可，可参考app目录

回调参数

数据类	含义
AlarmData	记录闹钟的数据类，以下几类闹钟均被记录 setAlarmClock setExact setExactAndAllowWhileIdle
BatteryData	通过广播记录的整机耗电，作为排查辅助
ScreenData	前台屏幕使用时间数据
BlueToothData	蓝牙使用时间与扫描模式ScanSettings 等
CpuStatData	通过读取/proc/self/stat 获取当前的cpu数据，作为排查辅助
LocationReportData	持续定位耗电数据包含多个子数据LocationBaseData
LocationBaseData	持续定位耗电子数据以一个listener/pending intent为维度记录
SensorData	传感器数据，使用时长/精度
WakeLockData	PowerWakeLock与WifiWakeLock等通用数据，记录使用时常，次数等

项目层级介绍

app下是使用例子
lib_battery_api 是api层调用，也是hook的具体实现
lib_battery_plugin 是asm 插桩的具体实现

问题交流

如果你发现了bug或者有其他功能诉求，欢迎提issue。如果想贡献代码，可以直接发起PR。

环境准备

建议直接用最新的稳定版本Android Studio打开工程.

许可证

采用Apache 2.0协议，详情参考LICENSE

leanprover-community/lean-graph

2023-12-23T01:32:28Z

beeper/phone-registration-provider

2023-12-23T01:32:43Z

A small service that generates iMessage registration data on a jailbroken iPhone

tzachar/cmp-ai

2023-12-22T01:35:50Z

cmp-ai

AI source for hrsh7th/nvim-cmp

This is a general purpose AI source for cmp, easily adapted to any restapi supporting remote code completion.

For now, HuggingFace SantaCoder, OpenAI Chat and Google Bard are implemeted.

Install

Dependencies

You will need plenary.nvim to use this plugin.
For using OpenAI or HuggingFace, you will also need curl.
For using Google Bard, you will need dsdanielpark/Bard-API.

Using a plugin manager

Using Lazy:

return require("lazy").setup({
    {'tzachar/cmp-ai', dependencies = 'nvim-lua/plenary.nvim'},
    {'hrsh7th/nvim-cmp', dependencies = {'tzachar/cmp-ai'}},
})

And later, tell cmp to use this plugin:

require'cmp'.setup {
    sources = {
        { name = 'cmp_ai' },
    },
}

Setup

Please note the use of : instead of a .

To use HuggingFace:

local cmp_ai = require('cmp_ai.config')

cmp_ai:setup({
  max_lines = 1000,
  provider = 'HF',
  notify = true,
  notify_callback = function(msg)
    vim.notify(msg)
  end,
  run_on_every_keystroke = true,
  ignored_file_types = {
    -- default is not to ignore
    -- uncomment to ignore in lua:
    -- lua = true
  },
})

You will also need to make sure you have the Hugging Face api key in you environment, HF_API_KEY.

To use OpenAI:

local cmp_ai = require('cmp_ai.config')

cmp_ai:setup({
  max_lines = 1000,
  provider = 'OpenAI',
  provider_options = {
    model = 'gpt-4',
  },
  notify = true,
  notify_callback = function(msg)
    vim.notify(msg)
  end,
  run_on_every_keystroke = true,
  ignored_file_types = {
    -- default is not to ignore
    -- uncomment to ignore in lua:
    -- lua = true
  },
})

You will also need to make sure you have the OpenAI api key in you environment, OPENAI_API_KEY.

Available models for OpenAI are gpt-4 and gpt-3.5-turbo.

To use Google Bard:

local cmp_ai = require('cmp_ai.config')

cmp_ai:setup({
  max_lines = 1000,
  provider = 'Bard',
  notify = true,
  notify_callback = function(msg)
    vim.notify(msg)
  end,
  run_on_every_keystroke = true,
  ignored_file_types = {
    -- default is not to ignore
    -- uncomment to ignore in lua:
    -- lua = true
  },
})

You will also need to follow the instructions on dsdanielpark/Bard-API to get the __Secure-1PSID key, and set the environment variable BARD_API_KEY accordingly (note that this plugin expects BARD_API_KEY without a leading underscore).

To use Ollama:

local cmp_ai = require('cmp_ai.config')

cmp_ai:setup({
  max_lines = 100,
  provider = 'Ollama',
  provider_options = {
    model = 'codellama:7b-code',
  },
  notify = true,
  notify_callback = function(msg)
    vim.notify(msg)
  end,
  run_on_every_keystroke = true,
  ignored_file_types = {
    -- default is not to ignore
    -- uncomment to ignore in lua:
    -- lua = true
  },
})

`notify`

As some completion sources can be quit slow, setting this to true will trigger a notification when a completion starts and ends using vim.notify.

`notify_callback`

The default notify function uses vim.notify, but an override can be configured. For example:

notify_callback = function(msg)
  require('notify').notify(msg, vim.log.levels.INFO, {
    title = 'OpenAI',
    render = 'compact',
  })
end

`max_lines`

How many lines of buffer context to use

`run_on_every_keystroke`

Generate new completion items on every keystroke.

`ignored_file_types` `(table: <string:bool>)`

Which file types to ignore. For example:

local ignored_file_types = {
  html = true,
}

cmp-ai will not offer completions when vim.bo.filetype is html.

Dedicated `cmp` keybindings

As completions can take time, and you might not want to trigger expensive apis on every keystroke, you can configure cmp-ai to trigger only with a specific key press. For example, to bind cmp-ai to <c-x>, you can do the following:

cmp.setup({
  ...
  mapping = {
    ...
    ['<C-x>'] = cmp.mapping(
      cmp.mapping.complete({
        config = {
          sources = cmp.config.sources({
            { name = 'cmp_ai' },
          }),
        },
      }),
      { 'i' }
    ),
  },
})

Also, make sure you do not pass cmp-ai to the default list of cmp sources.

Pretty Printing Menu Items

You can use the following to pretty print the completion menu (requires lspkind and patched fonts (https://www.nerdfonts.com)):

local lspkind = require('lspkind')

local source_mapping = {
  buffer = '[Buffer]',
  nvim_lsp = '[LSP]',
  nvim_lua = '[Lua]',
  cmp_ai = '[AI]',
  path = '[Path]',
}

require('cmp').setup({
  sources = {
    { name = 'cmp_ai' },
  },
  formatting = {
    format = function(entry, vim_item)
      -- if you have lspkind installed, you can use it like
      -- in the following line:
      vim_item.kind = lspkind.symbolic(vim_item.kind, { mode = 'symbol' })
      vim_item.menu = source_mapping[entry.source.name]
      if entry.source.name == 'cmp_ai' then
        local detail = (entry.completion_item.labelDetails or {}).detail
        vim_item.kind = ''
        if detail and detail:find('.*%%.*') then
          vim_item.kind = vim_item.kind .. ' ' .. detail
        end

        if (entry.completion_item.data or {}).multiline then
          vim_item.kind = vim_item.kind .. ' ' .. '[ML]'
        end
      end
      local maxwidth = 80
      vim_item.abbr = string.sub(vim_item.abbr, 1, maxwidth)
      return vim_item
    end,
  },
})

Sorting

You can bump cmp-ai completions to the top of your completion menu like so:

local compare = require('cmp.config.compare')
cmp.setup({
  sorting = {
    priority_weight = 2,
    comparators = {
      require('cmp_ai.compare'),
      compare.offset,
      compare.exact,
      compare.score,
      compare.recently_used,
      compare.kind,
      compare.sort_text,
      compare.length,
      compare.order,
    },
  },
})

chaseSpace/k8s-tutorial-cn

2023-12-22T01:36:05Z

nix-community/napalm

2023-12-22T01:37:14Z

Support for building npm packages in Nix and lightweight npm registry [maintainer @jtojnar]

FrankBijnen/ExifToolGui

2023-12-23T01:35:16Z

A GUI for ExifTool

jevinskie/llvm-etc

2023-12-23T01:35:33Z

Various files that I use while working with LLVM

mshinall/resize-images

2023-12-23T01:35:33Z

resize-images

Use ImageMagick identify and convert to bulk resize one or more images to a different size. You need to have ImageMagick installed.

nafiesl/uptime-monitor

2023-12-23T01:35:28Z

A self-hosted web monitoring tool, built with laravel

Uptime Monitor

Uptime Monitor is a self-hosted web monitoring tool, built with laravel.

Features

Monitor your web uptime per minutes (or any time interval)
Record response time on each web
Show uptime badges in 3 colors: green for up, yellow for warning, red for down, based on response time
Send telegram notification when you site down for 5 minutes (based on check periode)

Why I need this?

Open-source, modify as you need
Self-hosted, deploy on your own server
Store and control your monitoring logs yourself
Let you know when your websites are down
For freelancer/agency, increase your client's trust because you monitor their website

How to Install

Server Requirements

This application can be installed on local server and online server with these specifications:

PHP 8.1 (and meet Laravel 10.x requirements).
MySQL or MariaDB Database.
SQLite (for automated testing).

Installation Steps

Clone repository: git clone https://github.com/nafiesl/uptime-monitor.git
$ cd uptime-monitor
Install PHP dependencies: $ composer install
Install javscript dependencies: $ npm install
Copy .env.example to .env: $ cp .env.example .env
Generate application key: $ php artisan key:generate
Create a MySQL or MariaDB database.

Configure database and environment variables .env.

APP_URL=http://localhost:8000
APP_TIMEZOME="Asia/Jakarta"

DB_DATABASE=homestead
DB_USERNAME=homestead
DB_PASSWORD=secret

TELEGRAM_NOTIFER_TOKEN=

Run database migration: $ php artisan migrate --seed
Build assets: $ npm run build
Run task scheduler: $ php artisan schedule:work
Start server in a separeted terminal tab: $ php artisan serve
Open the web app: http://localhost:8000.
Login using default user credential:
- Email: admin@example.net
- Password: password
Go to Customer Site menu.
Add some new customer sites (name and URL).
After adding customer sites, go to Dashboard
Click Start Monitoring to update the uptime badge per minute.

Telegram Notifier Setup

In order to get notified in Telegram when the customer sites are down, we need to use a Telegram Bot and a Chat ID

Create a Telegram Bot (how to)
Get a Chat ID of the Telegram Bot (how to)
Update .env file, set TELEGRAM_NOTIFER_TOKEN=your_telegram_bot_token
Set our Chat ID in the Profile Page.
- Go to User Profile -> Click Edit Profile
- Fill the Telegram Chat ID field with your_chat_id
- Click Update Profile
By default, we will have 5 minutes inteval when the customer sites are down. But we can change the interval per customer sites.
- Go to Customer Site menu
- Select one of the customer site and click Edit link
- Set the Notify User Interval field, between 0 to 60.
- Set the Notify User Interval field to 0 if you don't want to get notified.

Screenshot

Dashboard

Dashboard in mobile device

Monitoring graph on customer site detail

Monitoring log tab on customer site detail

User profile menu

Telegram notification sample

Lisensi

Uptime Monitor project is an open-sourced software licensed under the Lisensi MIT.

Log1x/filament-starter

2023-12-23T01:35:28Z

My preferred starting point for TALL stack projects using Filament. ⚡

Filament Starter

This is my preferred starting point to quickly bootstrap TALL stack projects utilizing Filament for the admin panel.

Features

🚀 Quick, clean starting point with an example Post model and resource.
🧑‍💻 Fully pre-configured and customized Filament panel with best practices in place.
🎨 Clean, minimally styled frontend powered by Livewire.
💄 TailwindCSS and Vite ready for immediate use.
👷 Pre-bundled Livewire and Alpine for easy extendability.
⚡️ SPA-ready in both Filament and the frontend.
🔨 GitHub Actions workflows for Pint with pre-configured Dependabot for dependencies.
🔍️ Easy programmatic SEO using romanzipp/laravel-seo.
📝 Automatic sitemap generation using spatie/laravel-sitemap.

Requirements

Make sure all dependencies have been installed before moving on:

PHP >= 8.1
Composer
Node.js >= 18
Yarn

Getting Started

Start by cloning the Filament Starter repository and initializing the .env file:

git clone --depth=1 git@github.com:log1x/filament-starter.git
cd filament-starter
cp .env.example .env
composer install
php artisan key:generate

After .env is configured, you can proceed to migrate & seed the database:

php artisan migrate:fresh --seed

Once the database is seeded, you can login at /admin using the default admin user:

Username: admin
Password: admin

Build Assets

The project assets are compiled using Vite. This can be done by installing the dependencies and running the build command with Yarn.

yarn install
yarn build

Plugins Used

The following Filament plugins come fully implemented and configured out of the box:

Plugin	Description	Author
Curator	A beautiful media library.	awcodes
Gravatar	Easy avatar integration powered by Gravatar.	awcodes
Exceptions	A simple but powerful Exception viewer.	bezhansalleh
Jobs Monitor	Easily monitor background jobs and their progress.	croustibat
Breezy	Customizable user profile pages and 2FA support.	jeffgreco13
Peek	Quick & efficient front-end previews of resources.	pboivin
Logger	Zero-config resource activity logging.	z3d0x

Bug Reports

If you discover a bug in Filament Starter, please open an issue.

Contributing

Contributing whether it be through PRs, reporting an issue, or suggesting an idea is encouraged and appreciated.

License

Filament Starter is provided under the MIT License.

yebekhe/TVC

2023-12-23T01:35:28Z

V2ray Collector

This project is intended for educational purposes only. Any other use of it, including commercial, personal, or non-educational use, is not accepted!

This is a PHP script that collects V2Ray Configs from various Channels and saves them to different Subscription files based on their protocol type (SHADOWSOCKS, VLESS, VMESS, TROJAN, TUIC and HYSTERIA2).

Instructions & Usage

To use this project, simply import the subscription links into your preferred client. Ensure that your client supports at least SHADOWSOCKS, VLESS, VMESS, TROJAN, TUIC and HYSTERIA2.

You can access the project's subscription links through our Telegram bot: TVC BOT or use Tables bellow.

Core	Mix
Xray	Group Mix Link
Xray (Base64)	Group Mix Link
Singbox	Group Mix Link
Clash	Group Mix Link
Clash.Meta	Group Mix Link
Surfboard	Group Mix Link

Core	VMess	VLess	Reality	Trojan	Shadowsocks	Tuic	Hysteria2
Xray	VMess Link	VLess Link	Reality Link	Trojan Link	Shadowsocks Link	Tuic Link	Hysteria2 Link
Xray (Base64)	VMess Link	VLess Link	Reality Link	Trojan Link	Shadowsocks Link	Tuic Link	Hysteria2 Link
Singbox	VMess Link	VLess Link	Reality Link	Trojan Link	Shadowsocks Link	Tuic Link	Hysteria2 Link
Clash	VMess Link	VLess Not Supported	Reality Not Supported	Trojan Link	Shadowsocks Link	Tuic Not Supported	Hysteria2 Not Supported
Clash.Meta	VMess Link	VLess Link	Reality Link	Trojan Link	Shadowsocks Link	Tuic Not Supported	Hysteria2 Not Supported
Surfboard	VMess Link	VLess Not Supported	Reality Not Supported	Trojan Link	Shadowsocks Link	Tuic Not Supported	Hysteria2 Not Supported

Node Sources

This project currently utilizes Channels as the source of v2ray nodes.

You can access the Channels list through our Telegram bot: TVC BOT.

Contribution

If you have a Channel and you want to add it to this project or you know Channels that can be used in this project, you can ask for addition in the Issues section or in our Telegram group.

License

This project is licensed under the AGPL-3.0 license - see the LICENSE file for details.

PostgREST/plmustache

2023-12-22T01:38:48Z

Logic-less templates for Postgres

Mr-Proxy-source/BadUSB-Payloads

2023-12-23T01:36:10Z

Repository for Flipper Zero/USB Rubber Ducky payloads 😈

😈 BadUSB 😈

Description

Contact

Acknowledgments

Unleash the power of your Flipper/BadUSB 💻

Description

[!IMPORTANT]
This is for educational and learning purposes only. Do not intend for illegal use!

Payloads	Description	Plug'n'Play	Author
Script-Paster	Copy payload from the link then paste it in few powershell in seconds.	⛔	MrProxy
Lazagne	Grabbing passwords and credentials from vulnerable systems.	✅	MrProxy
Downloader	Download file from your url and then execute it on targets system.	✅	MrProxy
PHPexfil	Gather essential information such as computer name, hardware ID,...	⛔	MrProxy
IOS-Website	Open your desired website on iOS 17 and older versions!	✅	MrProxy
Google-Exfil	Upload Google Data to gofile.io and sends link to desired platform.	✅	MrProxy

Contact

📱 My Socials 📱

YouTube

Twitter

Instagram

Discord

TikTok

Acknowledgments

MrSec
All Credits To Willy & I am Jakoby

MicrosoftDocs/appcenter-docs

2023-12-23T01:36:10Z

content repo for Visual Studio App Center on docs.microsoft.com

App Center documentation

Welcome! This repo contains source files for App Center technical documentation. The topics are published to App Center documentation.

Code of conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information, see the Code of conduct FAQ, or contact opencode@microsoft.com with any additional questions or comments.

Contributing to the documentation

We welcome your contributions to help us improve the App Center docs. All the articles in this repository use GitHub flavored markdown.

To contribute to this documentation, see the Contributing guide.

If you're an internal docs contributor (Microsoft or GitHub), refer to the Contributing guide – Internal

Several feature areas of App Center have their own folders in this repo, such as build for topics on Build, crashes for topics on the Crash Reporting service, and so forth. The /images subfolder in each folder contains art files for the topics. The Contributing guide has more information.

SWI-Prolog/packages-clpqr

2023-12-22T01:39:12Z

CLP(Q,R): constraints over rational numbers and floats by Leslie de Koninck

SWI-Prolog/packages-chr

2023-12-22T01:39:12Z

CHR: Leuven Constraint Handling Rules

SWI-Prolog/contrib-protobufs

2023-12-22T01:39:12Z

An interface to Google Protocol Buffers (protobuf)

spyboy-productions/CloakQuest3r

2023-12-22T01:39:29Z

完全免费开源，基于 Requests 模块实现：TikTok 主页/视频/图集/原声；抖音主页/视频/图集/收藏/直播/原声/合集/评论/账号/搜索/热榜数据采集工具

TikTokDownloader

🔥 TikTok 主页/视频/图集/原声；抖音主页/视频/图集/收藏/直播/原声/合集/评论/账号/搜索/热榜数据采集工具：完全开源，基于 Requests 模块实现的免费工具；批量下载抖音账号发布、喜欢、收藏作品；批量下载 TikTok 账号主页作品；下载抖音链接或 TikTok 链接作品；获取抖音直播推流地址；下载抖音直播视频；采集抖音作品评论数据；批量下载抖音合集作品；采集抖音账号详细数据；采集抖音用户 / 作品 / 直播搜索结果；采集抖音热榜数据。

⭐ Windows 10 及以上用户可前往 Releases 下载已编译的 exe 程序，开箱即用！

❤️ 作者仅在 GitHub 发布 TikTokDownloader，未与任何个人或网站合作，且没有任何收费计划！

📝 功能清单(Function)

✅ 下载抖音无水印视频/图集
✅ 下载 TikTok 无水印视频/图集
✅ 批量下载抖音账号发布/喜欢/收藏作品
✅ 批量下载 TikTok 账号发布/喜欢作品
✅ 采集抖音 / TikTok 详细数据
✅ 批量下载链接作品
✅ 多账号批量下载作品
✅ 自动跳过已下载的文件
✅ 持久化保存采集数据
✅ 下载动态/静态封面图
✅ 获取抖音直播推流地址
✅ 调用 ffmpeg 下载直播
✅ Web UI 交互界面
✅ 采集抖音作品评论数据
✅ 批量下载抖音合集作品
✅ 记录点赞收藏等统计数据
✅ 筛选作品发布时间
✅ 支持账号作品增量下载
✅ 支持使用代理采集数据
✅ 支持局域网远程访问
✅ 采集抖音账号详细数据
✅ 作品统计数据更新
✅ 自动更新账号昵称
✅ 部署至私有服务器
✅ 部署至公开服务器
✅ 采集抖音搜索数据
✅ 采集抖音热榜数据
✅ 记录已下载作品 ID
✅ 扫码登陆获取 Cookie
✅ 支持 Web API 调用
✅ 支持多线程下载作品
✅ 文件完整性处理机制
✅ 自定义规则筛选作品

💻 程序界面(Screenshot)

终端命令行模式：

Web UI 交互模式：

Web API 接口模式：

📽 运行演示(Example)

批量下载账号发布作品

🎥 点击图片观看演示视频，建议通过配置文件管理账号，更多介绍请查阅文档

📈 项目状态(Status)

🟢 Releases 发布的源码已完成测试，所有功能均可正常使用
🟡 未来可能使用协程技术优化项目代码
🟡 未来可能新增可视化编辑配置文件功能
🟡 未来可能支持更多抖音热榜类型
🟡 未来可能新增终端文本用户界面(TUI)模式
🟡 未来可能支持全功能版的 Web UI 交互模式
🟡 未来可能新增监听剪贴板下载作品功能
🟡 未来可能新增账号新作品监测功能
🟡 未来可能新增合集新作品监测功能
🟡 未来可能新增直播开播监测功能
🟡 未来可能新增获取账号关注列表功能
🟡 未来可能新增获取账号收藏合集列表功能
🟡 未来可能优化 TikTok 平台批量下载功能
🔴 最新版本的源码可能存在不稳定的 Bug
🔴 如果在使用过程中发现程序 Bug，请及时告知作者修复

📁 项目结构(Structure)

TikTokDownloader
├─ main.py                                 // 项目程序启动入口
├─ requirements.txt                        // 程序所需第三方模块信息
├─ settings.json                           // 运行参数配置文件
├─ src                                     // 项目模块源码文件夹
│    ├─ CookieTool.py                      // Cookie 处理模块
│    ├─ Customizer.py                      // 项目代码调整模块
│    ├─ Configuration.py                   // 配置文件处理模块
│    ├─ DataAcquirer.py                    // 接口数据获取模块
│    ├─ DataExtractor.py                   // 数据提取储存模块
│    ├─ DataDownloader.py                  // 作品文件下载模块
│    ├─ FileManager.py                     // 作品文件管理模块
│    ├─ Parameter.py                       // 加密参数生成模块
│    ├─ Recorder.py                        // 日志及数据记录模块
│    ├─ StringCleaner.py                   // 非法字符处理模块
│    ├─ main_complete.py                   // 终端命令行模式启动入口
│    ├─ main_server.py                     // 服务器部署模式启动入口
│    ├─ main_api_server.py                 // Web API 接口模式启动入口
│    └─ main_web_UI.py                     // Web UI 交互模式启动入口
├─ cache                                   // 缓存数据文件夹
│    ├─ temp                               // 下载文件临时文件夹
│    ├─ AccountCache.json                  // 账号管理缓存数据
│    └─ IDRecorder.txt                     // 作品下载记录数据
├─ static                                  // 静态资源文件夹
├─ templates                               // HTML 模板文件夹
└─ docs                                    // 项目文档资源文件夹

📋 项目说明(Instructions)

快速入门

下载 EXE 程序 或者 配置运行环境
1. 下载 Releases 发布的 EXE 程序压缩包
2. 解压后打开程序文件夹，双击运行 main.exe
1. 安装不低于 3.12 版本的 Python 解释器
2. 下载最新的源码或 Releases 发布的源码至本地
3. 运行 pip install -r requirements.txt 命令安装程序所需模块
4. 运行 main.py
查看屏幕输出的 TikTokDownloader 免责声明，根据提示输入内容
将 Cookie 信息写入配置文件
1. 参考 Cookie 提取教程，复制所需 Cookie 至剪贴板
2. 选择 复制粘贴写入 Cookie 选项，按照提示将 Cookie 写入配置文件
1. 选择 扫码登陆写入 Cookie 选项，程序会显示登录二维码图片，并使用默认应用打开图片
2. 使用抖音 APP 扫描二维码并登录账号
3. 按照提示操作，将 Cookie 写入配置文件
返回程序界面，依次选择 终端命令行模式 --> 批量下载链接作品
输入抖音或 TikTok 作品链接即可下载作品文件
更多详细说明请查看 项目文档

⭐ 推荐使用 Windows 终端（Windows 11 自带默认终端）

关于 Cookie

点击查看 Cookie 获取教程

程序功能	是否需要登录
下载账号发布作品	⭕建议登录
下载账号喜欢作品	⭕建议登录
下载链接作品	⭕建议登录
获取直播推流地址	❌无需登录
下载直播视频	❌无需登录
获取作品评论数据	⭕建议登录
下载合集作品	⭕建议登录
获取账号数据	⭕建议登录
采集搜索结果	⭕建议登录
采集热榜数据	❌无需登录
下载账号收藏作品	✔️需要登录

Cookie 仅需在失效后重新写入配置文件，并非每次运行程序都要写入配置文件！

程序获取数据失败时，可以尝试更新 Cookie 或者使用已登录的 Cookie！

其他说明

程序提示用户输入时，直接回车代表返回上级菜单，输入 Q 或 q 代表结束运行
由于获取账号喜欢作品和收藏作品数据仅返回喜欢 / 收藏作品的发布日期，不返回操作日期，因此程序需要获取全部喜欢 / 收藏作品数据再进行日期筛选；如果作品数量较多，可能会花费较长的时间；可通过 max_pages 参数控制请求次数
获取私密账号的发布作品数据需要登录后的 Cookie，且登录的账号需要关注该私密账号
批量下载账号作品或合集作品时，如果对应的昵称或标识发生变化，程序会自动更新已下载作品文件名称中的昵称和标识
程序下载文件时会先将文件下载至临时文件夹，下载完成后再移动至储存文件夹；程序运行结束时会清空临时文件夹
如果想要修改程序功能，可以直接修改 src/Customizer.py 文件内容，支持自定义大部分程序功能
批量下载收藏作品模式 目前仅支持下载当前已登录 Cookie 对应账号的收藏作品，暂不支持多账号
如果想要程序使用代理，必须在 settings.json 设置 proxies 参数，否则程序不会使用代理
部分使用者反馈，新发布的作品过早下载会下载到低分辨率的文件，一段时间后才能下载到高分辨率文件，但时间规律尚不明确
退出程序时，请以正常方式结束运行或者按下 Ctrl + C 结束运行，不要直接点击终端窗口的关闭按钮结束运行，否则会导致数据丢失
程序默认不启用请求延时，但是建议使用者编辑 src/Customizer.py 文件启用随机延时或固定延时，避免频繁请求导致被抖音风控
如果您的计算机没有合适的程序编辑 JSON 文件，建议使用 JSON 在线工具编辑配置文件内容
当程序请求用户输入内容或链接时，请注意避免输入的内容或链接包含换行符，这可能会导致预期之外的问题

EXE 更新

方案一： 下载并解压文件，将旧版本的 cache 文件夹和 settings.json 文件复制到 _internal 文件夹。
方案二： 下载并解压文件，复制全部文件，直接覆盖旧版本文件。

⚠️ 免责声明(Disclaimers)

使用者对本项目的使用由使用者自行决定，并自行承担风险。作者对使用者使用本项目所产生的任何损失、责任、或风险概不负责。
本项目的作者提供的代码和功能是基于现有知识和技术的开发成果。作者尽力确保代码的正确性和安全性，但不保证代码完全没有错误或缺陷。
使用者在使用本项目时必须严格遵守 GNU General Public License v3.0 的要求，并在适当的地方注明使用了 GNU General Public License v3.0 的代码。
使用者在任何情况下均不得将本项目的作者、贡献者或其他相关方与使用者的使用行为联系起来，或要求其对使用者使用本项目所产生的任何损失或损害负责。
使用者在使用本项目的代码和功能时，必须自行研究相关法律法规，并确保其使用行为合法合规。任何因违反法律法规而导致的法律责任和风险，均由使用者自行承担。
本项目的作者不会提供 TikTokDownloader 项目的付费版本，也不会提供与 TikTokDownloader 项目相关的任何商业服务。
基于本项目进行的任何二次开发、修改或编译的程序与原创作者无关，原创作者不承担与二次开发行为或其结果相关的任何责任，使用者应自行对因二次开发可能带来的各种情况负全部责任。

在使用本项目的代码和功能之前，请您认真考虑并接受以上免责声明。如果您对上述声明有任何疑问或不同意，请不要使用本项目的代码和功能。如果您使用了本项目的代码和功能，则视为您已完全理解并接受上述免责声明，并自愿承担使用本项目的一切风险和后果。

✉️ 联系作者(Contact)

QQ: 2437596031（联系请说明来意）
QQ Group: 点击扫码加入群聊
Email: yonglelolu@gmail.com

TikTokDownloader 是我个人独立维护的一个开源项目，鉴于个人精力有限，请理解项目进展可能较为缓慢，我会尽力保持更新和维护，以确保项目的稳定性和功能的不断改进。

如果您通过 Email 联系我，我可能无法及时查看并回复信息，我会尽力在七天内回复您的邮件；如果有紧急事项或需要更快的回复，请通过其他方式与我联系，谢谢理解！

如果您对小红书感兴趣，可以了解一下我的另一个开源项目 XHS-Downloader

♥️ 支持项目(Support)

如果 TikTokDownloader 对您有帮助，请考虑为它点个 Star ⭐，感谢您的支持！

微信(WeChat)	支付宝(Alipay)

如果您愿意，可以考虑提供资助为 TikTokDownloader 提供额外的支持！

💡 代码参考(Refer)

andrew-codechimp/HA-Battery-Notes

2023-12-22T01:39:29Z

A Home Assistant integration to provide battery notes of devices

Home Assistant Battery Notes

Integration to add battery notes to a device, with automatic discovery via a growing battery library for devices

Please ⭐ this repo if you find it useful

This integration will set up the following platforms.

Platform	Description
`sensor`	Show battery type.

Installation

HACS

Or Search for Battery Notes in HACS and install it under the "Integrations" category.

Restart Home Assistant

Important

Add the following entry to your configuration.yaml

battery_notes:

Restart Home Assistant a final time In the HA UI go to "Configuration" -> "Integrations" click "+" and search for "Battery Notes"

Manual Installation

Using the tool of choice open the directory (folder) for your HA configuration (where you find configuration.yaml). If you do not have a custom_components directory (folder) there, you need to create it. In the custom_components directory (folder) create a new folder called battery_notes. Download all the files from the custom_components/battery_notes/ directory (folder) in this repository. Place the files you downloaded in the new directory (folder) you created. Restart Home Assistant Add the following entry to your configuration.yaml

battery_notes:

Restart Home Assistant a final time In the HA UI go to "Configuration" -> "Integrations" click "+" and search for "Battery Notes"

Configuration is done in the UI

On the "Configuration" -> "Integrations" -> "Battery Notes" screen add a new device, pick your device with a battery and add the battery type. The battery type will then be displayed as a diagnostic sensor on the device.

FAQ's

When is the library updated?
It updates when Home Assistant is restarted and approximately every 24 hours after that.
It will pull the latest devices that have been merged into the main branch, if you have recently submitted a pull request for a new device it will not appear until it has been manually reviewed and merged.
How do I remove a battery note on a device?
Go into the Settings -> Integrations -> Battery Notes, use the menu on the right of a device and select Delete, this will only delete the battery note, not the whole device.
Why does the device icon change?
Unfortunately where there are multiple integrations associated with a device Home Assistant seems to choose an icon at random, I have no control over this.
Can I edit a battery note?
Go into Settings -> Integrations -> Battery Notes and click Configure on the device you want to edit.

Automatic discovery

Battery Notes will automatically discover devices (as long as you have followed the installation instructions above) that it has in its library and create a notification to add a battery note.

If you wish to disable this functionality then change your configuration.yaml to set enable_autodiscovery to false

battery_notes:
  enable_autodiscovery: false

Contributing to the Battery Library

Fork the repository, add your device details to the JSON document custom_components/battery_notes/data/library.json, and then submit a pull request.

The manufacturer and model should be exactly what is displayed on the Device screen within Home Assistant.
The make & model names may be different between integrations such as Zigbee2MQTT and ZHA, if you see a similar device please duplicate the entry rather than changing it.
Please keep devices in alphabetical order by manufacturer/model.
The battery_quantity data is numeric (no quotes) and optional. If a device only requires a single battery, it should be omitted.
The battery_type data should follow the most common naming for general batteries (ex. AAA, D) and the IEC naming for battery cells according to Wikipedia (ex. CR2032, 18650)
If a device has a bespoke rechargeable battery you can use "battery_type": "Rechargeable"
For devices like smoke alarms where the battery is not replaceable you can use "battery_type": "Irreplaceable"

For the example image below, your JSON entry will look like this:

{
    "manufacturer": "Philips",
    "model": "Hue motion sensor (9290012607)",
    "battery_type": "AAA",
    "battery_quantity": 2
},

Contributions are welcome!

If you want to contribute to this please read the Contribution guidelines.

Acknowledgements

A lot of the inspiration for this integration came from the excellent PowerCalc by bramstroker, without adapting code from PowerCalc I'd never have worked out how to add additional sensors to a device.

Thanks to everyone who has submitted devices to the library.

huggingface/distil-whisper

2023-12-22T01:39:29Z

Distilled variant of Whisper for speech recognition. 6x faster, 50% smaller, within 1% word error rate.

Distil-Whisper

[Paper] [Models] [Colab] [Training Code]

Distil-Whisper is a distilled version of Whisper that is 6 times faster, 49% smaller, and performs within 1% word error rate (WER) on out-of-distribution evaluation sets:

Model	Params / M	Rel. Latency ↑	Short-Form WER ↓	Long-Form WER ↓
large-v2	1550	1.0	9.1	11.7

distil-large-v2	756	5.8	10.1	11.6
distil-medium.en	394	6.8	11.1	12.4
distil-small.en	166	5.6	12.1	12.8

For applications where latency and accuracy are important, we recommend the distil-medium.en or distil-large-v2 checkpoints. For resource-constrained applications, such as on-device or mobile applications, the distil-small.en is a great choice, since it is only 166M parameters, while performing within 3% WER of Whisper large-v2.

Note: Distil-Whisper is currently only available for English speech recognition. We are working with the community to distill Whisper on other languages. If you are interested in distilling Whisper in your language, check out the provided training code. We will soon update the repository with multilingual checkpoints when ready!

1. Usage

Distil-Whisper is supported in Hugging Face 🤗 Transformers from version 4.35 onwards. To run the model, first install the latest version of the Transformers library. For this example, we'll also install 🤗 Datasets to load a toy audio dataset from the Hugging Face Hub:

pip install --upgrade pip
pip install --upgrade transformers accelerate datasets[audio]

Short-Form Transcription

Short-form transcription is the process of transcribing audio samples that are less than 30-seconds long, which is the maximum receptive field of the Whisper models. This means the entire audio clip can be processed in one go without the need for chunking.

First, we load Distil-Whisper via the convenient AutoModelForSpeechSeq2Seq and AutoProcessor classes.

We load the model in float16 precision and make sure that loading time takes as little time as possible by passing low_cpu_mem_usage=True. In addition, we want to make sure that the model is loaded in safetensors format by passing use_safetensors=True:

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model_id = "distil-whisper/distil-large-v2"

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
)
model.to(device)

processor = AutoProcessor.from_pretrained(model_id)

The model and processor can then be passed to the pipeline. Note that if you would like to have more control over the generation process, you can directly make use of model.generate(...) as shown here.

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    max_new_tokens=128,
    torch_dtype=torch_dtype,
    device=device,
)

Next, we load an example short-form audio from the LibriSpeech corpus:

from datasets import load_dataset

dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
sample = dataset[0]["audio"]

Finally, we can call the pipeline to transcribe the audio:

result = pipe(sample)
print(result["text"])

To transcribe a local audio file, simply pass the path to your audio file when you call the pipeline:

result = pipe("audio.mp3")
print(result["text"])

For more information on how to customize the automatic speech recognition pipeline, please refer to the ASR pipeline docs. We also provide an end-to-end Google Colab that benchmarks Whisper against Distil-Whisper.

Long-Form Transcription

Distil-Whisper uses a chunked algorithm to transcribe long-form audio files longer than 30-seconds. In practice, this chunked long-form algorithm is 9x faster than the sequential algorithm proposed by OpenAI in the Whisper paper (see Table 7 of the Distil-Whisper paper).

We can load the model and processor as before:

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model_id = "distil-whisper/distil-large-v2"

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
)
model.to(device)

processor = AutoProcessor.from_pretrained(model_id)

To enable chunking, pass the chunk_length_s parameter to the pipeline. For Distil-Whisper, a chunk length of 15-seconds is optimal. To activate batching, pass the argument batch_size:

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    max_new_tokens=128,
    chunk_length_s=15,
    batch_size=16,
    torch_dtype=torch_dtype,
    device=device,
)

The argument max_new_tokens controls the maximum number of generated tokens per-chunk. In the typical speech setting, we have no more than 3 words spoken per-second. Therefore, for a 15-second input, we have at most 45 words (approx 60 tokens). We set the maximum number of generated tokens per-chunk to 128 to truncate any possible hallucinations that occur at the end of the segment. These tokens get removed at the chunk borders using the long-form chunking transcription algorithm, so it is more efficient to truncate them directly during generation to avoid redundant generation steps in the decoder.

Now, let's load a long-form audio sample. Here, we use an example of concatenated samples from the LibriSpeech corpus:

from datasets import load_dataset

dataset = load_dataset("distil-whisper/librispeech_long", "clean", split="validation")
sample = dataset[0]["audio"]

Finally, we can call the pipeline to transcribe the audio:

result = pipe(sample)
print(result["text"])

For more information on how to customize the automatic speech recognition pipeline, please refer to the ASR pipeline docs.

Speculative Decoding

Distil-Whisper can be used as an assistant model to Whisper for speculative decoding. Speculative decoding mathematically ensures the exact same outputs as Whisper are obtained while being 2 times faster. This makes it the perfect drop-in replacement for existing Whisper pipelines, since the same outputs are guaranteed.

For speculative decoding, we need to load both the teacher: openai/whisper-large-v2. As well as the assistant (a.k.a student) distil-whisper/distil-large-v2.

Let's start by loading the teacher model and processor. We do this in much the same way we loaded the Distil-Whisper model in the previous examples:

from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
import torch

device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model_id = "openai/whisper-large-v2"

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
)
model.to(device)

processor = AutoProcessor.from_pretrained(model_id)

Now let's load the assistant. Since Distil-Whisper shares exactly same encoder as the teacher model, we only need to load the 2-layer decoder as a "Decoder-only" model:

from transformers import AutoModelForCausalLM
assistant_model_id = "distil-whisper/distil-large-v2"

assistant_model = AutoModelForCausalLM.from_pretrained(
    assistant_model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
)
assistant_model.to(device)

The assistant model shares the same processor as the teacher, so there's no need to load a student processor.

We can now pass the assistant model to the pipeline to be used for speculative decoding. We pass it as a generate_kwarg with the key "assistant_model" so that speculative decoding is enabled:

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    max_new_tokens=128,
    generate_kwargs={"assistant_model": assistant_model},
    torch_dtype=torch_dtype,
    device=device,
)

As before, we can pass any sample to the pipeline to be transcribed:

from datasets import load_dataset

dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
sample = dataset[0]["audio"]

result = pipe(sample)
print(result["text"])

Note: speculative decoding should be on average 2x faster than using "only" Whisper large-v2 at a mere 8% increase in VRAM memory usage while mathematically ensuring the same results. This makes it the perfect replacement for Whisper large-v2 in existing speech recognition pipelines.

For more details on speculative decoding, refer to the following resources:

Speculative decoding for 2x faster Whisper inference blog post by Sanchit Gandhi
Assisted Generation: a new direction toward low-latency text generation blog post by Joao Gante
Fast Inference from Transformers via Speculative Decoding paper by Leviathan et. al.

Additional Speed & Memory Improvements

You can apply additional speed and memory improvements to Distil-Whisper which we cover in the following.

Flash Attention

We recommend using Flash Attention 2 if your GPU allows for it. To do so, you first need to install Flash Attention:

pip install flash-attn --no-build-isolation

You can then pass use_flash_attention_2=True to from_pretrained to enable Flash Attention 2:

- model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True)
+ model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True, use_flash_attention_2=True)

Torch Scale-Product-Attention (SDPA)

If your GPU does not support Flash Attention, we recommend making use of BetterTransformers. To do so, you first need to install optimum:

pip install --upgrade optimum

And then convert your model to a "BetterTransformer" model before using it:

model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True)
+ model = model.to_bettertransformer()

Exporting to Other Libraries

Distil-Whisper has support in the following libraries with the original "sequential" long-form transcription algorithm. Click the links in the table to see the relevant code-snippets for each:

Library	distil-small.en	distil-medium.en	distil-large-v2
OpenAI Whisper	link	link	link
Whisper cpp	link	link	link
Transformers js	link	link	link
Candle (Rust)	link	link	link

Updates will be posted here with the integration of the "chunked" long-form transcription algorithm into the respective libraries.

For the 🤗 Transformers code-examples, refer to the sections Short-Form and Long-Form Transcription.

2. Why use Distil-Whisper? ⁉️

Distil-Whisper is designed to be a drop-in replacement for Whisper on English speech recognition. Here are 5 reasons for making the switch to Distil-Whisper:

Faster inference: 6 times faster inference speed, while performing to within 1% WER of Whisper on out-of-distribution audio:

Robustness to noise: demonstrated by strong WER performance at low signal-to-noise ratios:

Robustness to hallucinations: quantified by 1.3 times fewer repeated 5-gram word duplicates (5-Dup.) and 2.1% lower insertion error rate (IER) than Whisper:

Designed for speculative decoding: Distil-Whisper can be used as an assistant model to Whisper, giving 2 times faster inference speed while mathematically ensuring the same outputs as the Whisper model.
Permissive license: Distil-Whisper is MIT licensed, meaning it can be used for commercial applications.

3. Approach ✍️

To distill Whisper, we copy the entire encoder module and freeze it during training. We copy only two decoder layers, which are initialised from the first and last decoder layers from Whisper. All other decoder layers from Whisper are discarded:

Distil-Whisper is trained on a knowledge distillation objective. Specifically, it is trained to minimise the KL divergence between the distilled model and the Whisper model, as well as the cross-entropy loss on pseudo-labelled audio data.

We train Distil-Whisper on a total of 22k hours of pseudo-labelled audio data, spanning 10 domains with over 18k speakers:

This diverse audio dataset is paramount to ensuring robustness of Distil-Whisper to different datasets and domains.

In addition, we use a WER filter to discard pseudo-labels where Whisper mis-transcribes or hallucinates. This greatly improves WER performance of the downstream distilled model.

For full details on the distillation set-up and evaluation results, refer to the Distil-Whisper paper.

4. Training Code

Training code to reproduce Distil-Whisper can be found in the directory training. This code has been adapted be general enough to distill Whisper for multilingual speech recognition, facilitating anyone in the community to distill Whisper on their choice of language.

5. Acknowledgements

OpenAI for the Whisper model and original codebase
Hugging Face 🤗 Transformers for the model integration
Google's TPU Research Cloud (TRC) program for Cloud TPU v4s

6. Citation

If you use this model, please consider citing the Distil-Whisper paper:

@misc{gandhi2023distilwhisper,
      title={Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling}, 
      author={Sanchit Gandhi and Patrick von Platen and Alexander M. Rush},
      year={2023},
      eprint={2311.00430},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

And also the Whisper paper:

@misc{radford2022robust,
      title={Robust Speech Recognition via Large-Scale Weak Supervision}, 
      author={Alec Radford and Jong Wook Kim and Tao Xu and Greg Brockman and Christine McLeavey and Ilya Sutskever},
      year={2022},
      eprint={2212.04356},
      archivePrefix={arXiv},
      primaryClass={eess.AS}
}

aml4td/website

2023-12-23T01:37:12Z

Website sources for Applied Machine Learning for Tabular Data

Welcome! This is a work in progress. We want to create a practical guide to developing quality predictive models from tabular data. We'll publish materials here as we create them and welcome community contributions in the form of discussions, suggestions, and edits.

We also want these materials to be reusable and open. The sources are in the source GitHub repository with a Creative Commons license attached (see below).

Our intention is to write these materials and, when we feel we're done, pick a publishing partner to produce a print version.

The book takes a holistic view of the predictive modeling process and focuses on a few areas that are usually left out of similar works. For example, the effectiveness of the model can be driven by how the predictors are represented. Because of this, we tightly couple feature engineering methods with machine learning models. Also, quite a lot of work happens after we have determined our best model and created the final fit. These post-modeling activities are an important part of the model development process and will be described in detail.

To cite this work, we suggest:

@online{aml4td,
  Author = {Kuhn, M and Johnson, K},
  title = {{Applied Machine Learning for Tabular Data}},
  year = {2023},
  url = { https://aml4td.org},
  urldate = {2023-11-20}
}

License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Our goal is to have an open book where people can reuse and reference the materials but can't just put their names on them and resell them (without our permission).

Intended Audience

Our intended audience includes data analysts of many types: statisticians, data scientists, professors and instructors of machine learning courses, laboratory scientists, and anyone else who desires to understand how to create a model for prediction. We don't expect readers to be experts in these methods or the math behind them. Instead, our approach throughout this work is applied. That is, we want readers to use this material to build intuition about the predictive modeling process. What are good and bad ideas for the modeling process? What pitfalls should we look out for? How can we be confident that the model will be predictive for new samples? What are advantages and disadvantages of different types of models? These are just some of the questions that this work will address.

Some background in modeling and statistics will be extremely useful. Having seen or used basic regression models is good, and an understanding of basic statistical concepts such as variance, correlation, populations, samples, etc., is needed. There will also be some mathematical notation, so you'll need to be able to grasp these abstractions. But we will keep this to those parts where it is absolutely necessary. There are a few more statistically sophisticated sections for some of the more advanced topics.

If you would like a more theoretical treatment of machine learning models, then we recommend Hastie et al. (2017). Other books for gaining a more in-depth understanding of machine learning are Bishop and Nasrabadi (2006), Arnold et al. (2019) and, for more of a deep learning focus, Goodfellow et al. (2016).

Is there code?

We definitely want to decouple the content of this work from specific software. One of our other books on modeling had computing sections. Many people found these sections to be a useful resource at the time of the book's publication. However, code can quickly become outdated in today's computational environment. In addition, this information takes up a lot of page space that would be better used for other topics.

We will create computing supplements to go along with the materials. Since we use R's tidymodels framework for calculations, the supplement currently in-progress is:

tidymodels.aml4td.org

If you are interested in working on a python/scikit-learn supplement, please file an issue

Are there exercises?

Many readers found the Exercise sections of Applied Predictive Modeling to be helpful for solidifying the concepts presented in each chapter. The current set can be found at exercises.aml4td.org

How can I ask questions?

If you have questions about the content, it is probably best to ask on a public forum, like cross-validated. You'll most likely get a faster answer there if you take the time to ask the questions in the best way possible.

If you want a direct answer from us, you should follow what I call Yihui's Rule: add an issue to GitHub (labeled as "Discussion") first. It may take some time for us to get back to you.

If you think there is a bug, please file an issue.

Can I contribute?

There is a contributing page with details on how to get up and running to compile the materials (there are a lot of software dependencies) and suggestions on how to help.

If you just want to fix a typo, you can make a pull request to alter the appropriate .qmd file.

Please feel free to improve the quality of this content by submitting pull requests. A merged PR will make you appear in the contributor list. It will, however, be considered a donation of your work to this project. You are still bound by the conditions of the license, meaning that you are not considered an author, copyright holder, or owner of the content once it has been merged in.

Raku/L10N

2023-12-22T01:40:06Z

The home of supported localizations of the Raku Programming Language

tsoding/bada

2023-12-22T01:40:40Z

The ⭐️ Open Source Assistants API allows you to build AI assistants within your own applications with your own models. 75% Cheaper & 23x Faster Assistants. Same API/SDK.

⭐️ Open Source Assistants API

Build Powerful AI Assistants In-House, On Your Terms

75% Cheaper & 23x Faster Assistants. Same API/SDK.

📞 Need Support? We're here for you.
🖼️ How it Works – Visual Guide
✨ Suggest a Feature
❤️‍🩹 Found a Bug? Let us know.

Quickstart

Get started in less than a minute through GitHub Codespaces:

Why Open Source Assistants API?

Full Control: Own your data, your models, and your destiny.
No Hidden Costs: Absolutely free. Seriously, no strings attached.
Customizable: Tailor the AI to your specific needs and use cases.
Offline Capabilities: Perfect for edge cases or internet-free zones.
OpenAI Compatibility: Love OpenAI's API? We play nice with that too.
Simplicity: Easy setup, no steep learning curve.

What's Cooking? – Latest News

[2023/08/19] 🔥 New example: Open source LLM with code interpreter. Learn more.
[2023/08/12] 🔥 New example: Open source LLM with function calling. Learn more.
[2023/29/11] 🔥 New example: Using mistral-7b, an open source LLM. Check it out.

Key Features

Code Interpreter: Runs Python code in a sandboxed environment. (Beta)
Knowledge Retrieval: Retrieves external knowledge or documents.
Function Calling: Defines and executes custom functions.
File Handling: Supports a range of file formats.
Multimodal: Supports audio, images, and text.
- image audio text
- audio text
- image text (soon)
- text

What can you build with Assistants?

Perplexity for Astronauts: Stellar Amenities uses Assistants to let the LLM automatically find the right information on the spacecraft to help Astronauts

Join the Movement

For Developers: We've got the docs, tools, and a community ready to help you build what's next.
For Innovators: Looking for an edge in AI? Here's where you leapfrog the competition.
For the Visionaries: Dreamt of a custom AI assistant? Let's make it a reality.

Deployment

Please follow this documentation.

FAQ

What's the difference with LangChain?

LangChain offers detailed control over AI conversations, while OpenAI's Assistants API simplifies the process, managing conversation history, data/vector store, and tool switching for you.

Are you related to OpenAI?

No.

I don't use Assistants API. Can I use this?

We recommend switching to the Assistants API for a more streamlined experience, allowing you to focus more on your product than on infrastructure.

Does the Assistants API support audio and images?

Images soon, working on it. Audio in a few weeks.

vincent-herlemont/native_db

2023-12-22T01:40:40Z

Drop-in embedded database

Native DB

Here's a drop-in, fast, embedded database for multi-platform apps (server, desktop, mobile). Sync Rust types effortlessly. Enjoy! 😌🍃.

Features

Simple API 🦀.
Support for multiple indexes (primary, secondary, unique, non-unique, optional).
Minimal boilerplate.
Transparent serialization/deserialization using native_model.
Automatic model migration 🌟.
Thread-safe and fully ACID-compliant transactions provided by redb.
Real-time subscription with filters for insert, update and delete operations.
Compatible with all Rust types (enum, struct, tuple etc.).
Hot snapshots.

Installation

Add this to your Cargo.toml:

[dependencies]
native_db = "0.5.1"
native_model = "0.4.2"

NOTE: native_db requires native_model to work.

Status

Active development. The API is not stable yet and may change in the future.

Usual API

DatabaseBuilder
- define a model.
- create / open a database.
- create_in_memory an in-memory database.
Database
- snapshot the database.
- rw_transaction open a read-write transaction.
  - insert a new item.
  - update an existing item.
  - remove an existing item.
  - commit the transaction.
  - migrate a model.
  - plus all read-only transaction APIs.
- r_transaction open a read-only transaction.
  - get
    - primary an item by its primary key.
    - secondary an item by its secondary key.
  - scan
    - primary
      - all items.
      - start_with items with a primary key starting with a given value.
      - range items with a primary key in a given range.
    - secondary
      - all items with a given secondary key.
      - start_with items with a secondary key starting with a given value.
      - range items with a secondary key in a given range.
  - len
    - primary the number of items.
    - secondary the number of items with a given secondary key.
- watch real-time subscriptions via std channel based or tokio channel based depending on the feature tokio.
  - get
    - primary an item by its primary key.
    - secondary an item by its secondary key.
  - scan
    - primary
      - all items.
      - start_with items with a primary key starting with a given value.
      - range items with a primary key in a given range.
    - secondary
      - all items with a given secondary key.
      - start_with items with a secondary key starting with a given value.
      - range items with a secondary key in a given range.

Example

use serde::{Deserialize, Serialize};
use native_db::*;
use native_model::{native_model, Model};

#[derive(Serialize, Deserialize, PartialEq, Debug)]
#[native_model(id = 1, version = 1)]
#[native_db]
struct Item {
    #[primary_key]
    id: u32,
    #[secondary_key]
    name: String,
}

fn main() -> Result<(), db_type::Error> {
    let mut builder = DatabaseBuilder::new();
    // Initialize the model
    builder.define::<Item>()?;
    
    // Create a database in memory
    let mut db = builder.create_in_memory()?;
    
    // Insert data (open a read-write transaction)
    let rw = db.rw_transaction().unwrap();
    rw.insert(Item { id: 1, name: "red".to_string() })?;
    rw.insert(Item { id: 2, name: "green".to_string() })?;
    rw.insert(Item { id: 3, name: "blue".to_string() })?;
    rw.commit()?;
    
    // Open a read-only transaction
    let r = db.r_transaction()?;
    // Retrieve data with id=3 
    let retrieve_data: Item = r.get().primary(3_u32)?.unwrap();
    println!("data id='3': {:?}", retrieve_data);
    // Iterate items with name starting with "red"
    for item in r.scan().secondary::<Item>(ItemKey::name)?.start_with("red") {
        println!("data name=\"red\": {:?}", item);
    }
    
    // Remove data (open a read-write transaction)
    let rw = db.rw_transaction()?;
    rw.remove(retrieve_data)?;
    rw.commit()?;
    Ok(())
}

Contributors

_{Akshith Madhur}
💻

astriaorg/astria

2023-12-23T01:38:02Z

A monorepo containing all the custom components of the Astria network, a decentralized system that replaces traditional sequencers, offering a shared, permissionless sequencer network.

Astria

Astria replaces centralized sequencers, allowing many rollups to share a single decentralized network of sequencers that’s simple and permissionless to join. This shared sequencer network provides out-of-the-box censorship resistance, fast block confirmations, and atomic cross-rollup composability – all while retaining each rollup’s sovereignty.

This repository contains the custom Astria components that make up the Astria network. Other components of the Astria network can be found in the astriaorg organization.

To run locally, we utilize a dev-cluster which can be found at astriaorg/dev-cluster.

To learn more about Astria, please visit astria.org.

Components

conductor: conducts blocks from the data availability layer to the execution layer.
proto: relevant protobufs for Astria types.
sequencer: ABCI application that defines the sequencer state transition logic.
sequencer-relayer: relays blocks from the sequencer chain to the data availability layer.

Build

To build the relevant Astria binaries, you only need Rust installed.

Then:

git clone https://github.com/astriaorg/astria.git
cd astria
cargo build --release

Docker build

To build a docker image locally you will first need docker installed. With docker installed you can use the following just command:

# Full command:
just docker-build <CRATE> <TAG=local>
#
# Replace CRATE with what the target binary is ie `astria-sequencer`
# TAG defaults to `local` but can be changed.

# this command will build a local image tagged as 'astria-sequencer:local' 
just docker-build astria-sequencer

# this command will build a local image tagged as 'astria-sequencer:debug' 
just docker-build astria-sequencer debug

Running locally

The entire stack consists of many different binaries. It's recommended to use the setup located in astriaorg/dev-cluster.

Testing

To run unit tests:

cargo test

Note that the astria-proto generates its code by running tests (and verifying that nothing changed). In order for its tests to run you also need Buf installed.

Formatting

This project uses rustfmt to format rust sources, taplo to format toml files, and markdownlint-cli2 for markdown.

Rust

# Install rustfmt
rustup +nightly-2023-08-18 component add rustfmt
# Run rustfmt
just fmt rust

Toml

# Install for macOS
brew install taplo
# Install for Arch Linux
sudo pacman -S taplo

# Run
just fmt toml

Markdown

# Install for macOS w/ homebrew
brew install markdownlint-cli2
# Install for Arch Linux
sudo pacman -S markdownlint-cli2
# Install with NPM
npm install markdownlint-cli2 --global

# Run
just lint md

# Run with docker
docker run -v $PWD:/workdir davidanson/markdownlint-cli2:v0.8.1 "**/*.md" "#.github"

Contributing

Pull requests should be created against the main branch. In general, we follow the "fork-and-pull" Git workflow.

Fork the repo on GitHub
Clone the project to your own machine
Commit changes to your own branch
Push your work back up to your fork
Submit a Pull request so that we can review your changes

NOTE: Be sure to merge the latest from upstream before making a pull request!

Issues

If you encounter any issues while using this project or have any questions, please open an issue in this repository here.

geminicaprograms/fp_learning

2023-12-22T01:41:03Z

Template engine for Xitrum based on Scalate

Xitrum <http://xitrum-framework.github.io/>_ allows you to choose template engines. This library is a template engine for Xitrum. It wraps Scalate <http://scalate.github.io/scalate/>_.

See CHANGELOG to know which version of xitrum-scalate to use for which version of Xitrum.

Config your Xitrum project to use this template engine


Edit your Xitrum project's project/plugins.sbt:

::

  // For precompiling Scalate templates in the compile phase of SBT
  addSbtPlugin("org.scalatra.scalate" % "sbt-scalate-precompiler" % "1.9.6.0")

Edit build.sbt:

::

  // Template engine for Xitrum
  libraryDependencies += "tv.cntt" %% "xitrum-scalate" % "2.9.2"

  // Precompile Scalate templates
  import org.fusesource.scalate.ScalatePlugin._
  ScalateKeys.scalateTemplateConfig in Compile := Seq(TemplateConfig(
    baseDirectory.value / "src" / "main" / "scalate",
    Seq(),
    Seq(Binding("helper", "xitrum.Action", importMembers = true))
  ))

Edit xitrum.conf:

::

  xitrum {
    ...
    template {
      "xitrum.view.Scalate" {
        defaultType = jade  # jade, mustache, scaml, or ssp
      }
    }
    ...
  }

"type" option
~~~~~~~~~~~~~

In xitrum.conf, you config "defaultType" (see above).

When calling Xitrum's renderView method, if you want to use template type other
than the defaultType, set the last argument (options) like this:

::

   renderView(Map("type" -> "mustache")

"date" option
~~~~~~~~~~~~~

If you don't specify `DateFormat <http://docs.oracle.com/javase/7/docs/api/java/text/DateFormat.html>`_,
``java.text.DateFormat.getDateInstance(DateFormat.DEFAULT, lo)`` will be used,
where ``lo`` is ``java.util.Locale.forLanguageTag(currentAction.language)``.

::

  renderView(Map("date" -> myDateFormat)

"number" option
~~~~~~~~~~~~~~~

If you don't specify `NumberFormat <http://docs.oracle.com/javase/7/docs/api/java/text/NumberFormat.html>`_,
``java.text.NumberFormat.getInstance(lo)`` will be used,
where ``lo`` is ``java.util.Locale.forLanguageTag(currentAction.language)``.

::

  renderView(Map("date" -> myNumberFormat)

If you want to display an integer number as is, without any number format,
instead of (Jade example):

::

  = myObject.myInt

Use:

::

  = myObject.myInt.toString

Other utility methods
~~~~~~~~~~~~~~~~~~~~~

You can use methods [view and collection](http://scalate.github.io/scalate/documentation/user-guide.html#Views) of Scalate.

`xitrum.view.Scalate` provides some utility methods so that you can easily
use Scalate features. See the `API doc <http://xitrum-framework.github.io/xitrum-scalate/>`_.

Ex:

::

  import xitrum.Config
  import xitrum.view.Scalate

  // In your action:
  val scalate  = Config.xitrum.template.get.asInstanceOf[Scalate]
  val template = "p This Jade template is from a string, not from a file."
  val string   = scalate.renderJadeString(template)
  respondInlineView(string)

Log
~~~

To avoid unnecessary log, you should add these to config/logback.xml:

::

  <logger name="org.fusesource.scalate" level="INFO"/>
  <logger name="org.fusesource.scalate.util.ClassPathBuilder" level="ERROR"/>

hmrc/pbd-4334-live-test

2023-12-23T01:38:32Z

pbd-4334-live-test

This is a placeholder README.md for a new repository

License

This code is open source software licensed under the Apache 2.0 License.

cmu-csd-phd/csd-blog

2023-12-22T01:41:12Z

Apache Seata(incubating) Website

aniketrajnish/SpiderVersePostProcess-Unreal-Unity

2023-12-23T01:38:48Z

Spiderverse-insprired stylized post processing-effect for Unreal Engine and Unity

wukongdaily/diy-nas-onescript

2023-12-23T01:38:51Z

这是一个适用于在debian系统上部署nas常用软件的脚本。它可以帮助用户快速安装想要的软件环境和docker应用。

diy-nas-onescript

这是一个适用于在deepin v23beta2 系统上部署nas常用软件的脚本。它可以帮助用户快速安装想要的软件环境和docker应用。

❤使用方法 Wiki

‼️提BUG

适配系统

✅ Deepin v23beta2
✅ Deepin v20.9

截图

king-hacking/King-Hacking

2023-12-23T01:38:51Z

Now you can download the most powerful tools using this simple script

Screenshot

Installations

git clone https://github.com/king-hacking/King-Hacking.git
cd King-Hacking
bash King-Tools.sh

My Accounts

bgd-labs/aave-helpers

2023-12-23T01:39:24Z

chainguard-dev/rules_apko

2023-12-22T01:42:02Z

Bazel rules for apko

D-Sketon/hexo-theme-reimu

2023-12-23T01:39:44Z

一款博丽灵梦风格的Hexo主题 | A Hakurei Reimu style Hexo theme.

RADar-AZDelta/Keun

2023-12-22T01:42:10Z

Keun (West Flemish for rabbit) is a web based modern variant of the Usagi OMOP CDM mapping tool.

TheResonanceTeam/TrollApps

2023-12-22T01:42:16Z

App Store alternative for TrollStore

TrollApps

An App Store alternative for TrollStore.

What is TrollApps?

TrollApps is an alternative to the iOS App Store for TrollStore users. TrollApps allows users to download apps from repositories and install them directly in TrollStore. TrollApps works on iOS 14.0 and up; make sure to be on the latest version of TrollStore.

Requirements

You require a TollStore 1.0 / 2.0 compatible device, and TrollStore must be installed to use this application. It is also required that you enable schema urls within TrollStore.

Check out our documentation!

Information on TrollApps sources and our URL Scheme can be found here

Found a bug / want to send feedback?

Join our Discord!

open-telemetry/opentelemetry-swift

2023-12-22T01:42:16Z

OpenTelemetry API for Swift

opentelemetry-swift

A swift OpenTelemetry client

Installation

This package includes several libraries. The OpenTelemetryApi library includes protocols and no-op implementations that comprise the OpenTelemetry API following the specification. The OpenTelemetrySdk library is the reference implementation of the API.

Libraries that produce telemetry data should only depend on OpenTelemetryApi, and defer the choice of the SDK to the application developer. Applications may depend on OpenTelemetrySdk or another package that implements the API.

Adding the dependency

opentelemetry-swift is designed for Swift 5. To depend on the opentelemetry-swift package, you need to declare your dependency in your Package.swift:

.package(url: "https://github.com/open-telemetry/opentelemetry-swift", from: "1.0.0"),

and to your application/library target, add OpenTelemetryApi or OpenTelemetrySdkto your dependencies, e.g. like this:

.target(name: "ExampleTelemetryProducerApp", dependencies: ["OpenTelemetryApi"]),

.target(name: "ExampleApp", dependencies: ["OpenTelemetrySdk"]),

Documentation

Official documentation for the library can be found in the official opentelemetry documentation page, including:

Documentation about installation and manual instrumentation
Libraries that provide automatic instrumentation

Current status

API and SDK

Tracing and Baggage are considered stable

Logs are considered beta quality

Metrics is implemented using an outdated spec, is fully functional but will change in the future

Supported exporters and importers

Traces

Exporters: Stdout, Jaeger, Zipkin, Datadog and OpenTelemetry (OTLP) collector
Importers: OpenTracingShim

Metrics

Exporters: Prometheus, Datadog, and OpenTelemetry (OTLP) collector
Importers: SwiftMetricsShim

Logs

Exporters: OpenTelemetry (OTLP) collector

NOTE: OTLP exporters are supported both in GRPC and HTTP, only GRPC is production ready, HTTP is still experimental

Instrumentation libraries

URLSession
NetworkStatus
SDKResourceExtension
SignPostIntegration

Examples

The package includes some example projects with basic functionality:

Datadog Sample - Shows the Datadog exporter used with a Simple Exporter, showing how to configure for sending.
Logging Tracer - Simple api implementation of a Tracer that logs every api call
Network Tracer - Shows how to use the URLSessionInstrumentation instrumentation in your application
Simple Exporter - Shows the Jaeger an Stdout exporters in action using a MultiSpanExporter. Can be easily modified for other exporters
Prometheus Sample - Shows the Prometheus exporter reporting metrics to a Prometheus instance
OTLP Exporter - Shows the OTLP exporter reporting traces to Zipkin and metrics to a Prometheus via the otel-collector

unaguil/metodos_numericos

2023-12-23T01:40:25Z

Ejercicios de la asignatura de métodos numéricos I de la UNED

subhylahiri/MikTeX-local

2023-12-23T01:40:25Z

Bib files, macro lists etc. for including in all other LaTeX projects

MikTeX-local

Bib files, macro lists etc. for including in all other LaTeX projects

CTAN packages

You'll need

a4wide
ntgclass
natbib
url
xcolor
mptopdf
beamer
pgf
ms
sansmathaccent
filehook
appendixnumberbeamer
etoolbox
adjustbox
collectbox
tcolorbox
environ
trimspaces
type1cm
fp
caption
paralist
cancel
appendix

crowdsecurity/cs-whm-plugin

2023-12-23T01:40:46Z

WHM plugin to better control the Security Engine

pastelsky/tsdocs

2023-12-22T01:42:55Z

Collection of TypeScript packages that consume and interact with the Injective Chain

🌟 Injective's TypeScript Monorepo

Access Limitless DeFi Markets with Zero Barriers.

injective-ts is a TypeScript monorepo that contains packages which can be used to interact with Injective from a Node.js or browser environments and which provide simple abstractions over core data structures, serialization, key management, and API request generation, etc. The packages can be found in the packages folder and each package is a npm module that is published on the npm registry.

Learn more on our Docs

📜 Contribution

General contribution guidelines can be found on the Contributing page. There can be a package-specific contribution guides, please make sure you read them before you open a PR.

⛑ Support

Reach out to us at one of the following places!

Website at injective.com
Twitter at @Injective
Discord at Discord
Telegram at Telegram

🔓 License

Originally released by Injective Labs Inc. under:
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/

_{Powering the future of decentralized finance.}

unjs/h3

2023-12-22T01:42:55Z

⚡️ Minimal H(TTP) framework built for high performance and portability

H3

H3 (pronounced as /eɪtʃθriː/, like h-3) is a minimal h(ttp) framework built for high performance and portability.

👉 Online Playground

👉 Online Examples Playground

Features

✔️ Portable: Works perfectly in Serverless, Workers, and Node.js

✔️ Minimal: Small and tree-shakable

✔️ Modern: Native promise support

✔️ Extendable: Ships with a set of composable utilities but can be extended

✔️ Router: Super fast route matching using unjs/radix3

✔️ Compatible: Compatibility layer with node/connect/express middleware

Install

# Using npm
npm install h3

# Using yarn
yarn add h3

# Using pnpm
pnpm add h3

Using Nightly Releases

If you are directly using h3 as a dependency:

{
  "dependencies": {
    "h3": "npm:h3-nightly@latest"
  }
}

If you are using a framework (Nuxt or Nitro) that is using h3:

pnpm and yarn:

{
  "resolutions": {
    "h3": "npm:h3-nightly@latest"
  }
}

npm:

{
  "overrides": {
    "h3": "npm:h3-nightly@latest"
  }
}

Note: Make sure to recreate lockfile and node_modules after reinstall to avoid hoisting issues.

Usage

import { createServer } from "node:http";
import { createApp, eventHandler, toNodeListener } from "h3";

const app = createApp();
app.use(
  "/",
  eventHandler(() => "Hello world!"),
);

createServer(toNodeListener(app)).listen(process.env.PORT || 3000);

Example using listhen for an elegant listener:

import { createApp, eventHandler, toNodeListener } from "h3";
import { listen } from "listhen";

const app = createApp();
app.use(
  "/",
  eventHandler(() => "Hello world!"),
);

listen(toNodeListener(app));

Router

The app instance created by h3 uses a middleware stack (see how it works) with the ability to match route prefix and apply matched middleware.

To opt-in using a more advanced and convenient routing system, we can create a router instance and register it to app instance.

import { createApp, eventHandler, createRouter } from "h3";

const app = createApp();

const router = createRouter()
  .get(
    "/",
    eventHandler(() => "Hello World!"),
  )
  .get(
    "/hello/:name",
    eventHandler((event) => `Hello ${event.context.params.name}!`),
  );

app.use(router);

Tip: We can register the same route more than once with different methods.

Routes are internally stored in a Radix Tree and matched using unjs/radix3.

For using nested routers, see this example

More app usage examples

// Handle can directly return object or Promise<object> for JSON response
app.use(
  "/api",
  eventHandler((event) => ({ url: event.node.req.url })),
);

// We can have better matching other than quick prefix match
app.use(
  "/odd",
  eventHandler(() => "Is odd!"),
  { match: (url) => url.substr(1) % 2 },
);

// Handle can directly return string for HTML response
app.use(eventHandler(() => "<h1>Hello world!</h1>"));

// We can chain calls to .use()
app
  .use(
    "/1",
    eventHandler(() => "<h1>Hello world!</h1>"),
  )
  .use(
    "/2",
    eventHandler(() => "<h1>Goodbye!</h1>"),
  );

// We can proxy requests and rewrite cookie's domain and path
app.use(
  "/api",
  eventHandler((event) =>
    proxyRequest(event, "https://example.com", {
      // f.e. keep one domain unchanged, rewrite one domain and remove other domains
      cookieDomainRewrite: {
        "example.com": "example.com",
        "example.com": "somecompany.co.uk",
        "*": "",
      },
      cookiePathRewrite: {
        "/": "/api",
      },
    }),
  ),
);

// Legacy middleware with 3rd argument are automatically promisified
app.use(
  fromNodeMiddleware((req, res, next) => {
    req.setHeader("x-foo", "bar");
    next();
  }),
);

// Lazy loaded routes using { lazy: true }
app.use("/big", () => import("./big-handler"), { lazy: true });

Utilities

H3 has a concept of composable utilities that accept event (from eventHandler((event) => {})) as their first argument. This has several performance benefits over injecting them to event or app instances in global middleware commonly used in Node.js frameworks, such as Express. This concept means only required code is evaluated and bundled, and the rest of the utilities can be tree-shaken when not used.

👉 You can check list of exported built-in utils from JSDocs Documentation.

Body

readRawBody(event, encoding?)
readBody(event)
readValidatedBody(event, validate)
readMultipartFormData(event)

Request

getQuery(event)
getValidatedQuery(event, validate)
getRouterParams(event, { decode? })
getRouterParam(event, name, { decode? })
getValidatedRouterParams(event, validate, { decode? })
getMethod(event, default?)
isMethod(event, expected, allowHead?)
assertMethod(event, expected, allowHead?)
getRequestHeaders(event, headers) (alias: getHeaders)
getRequestHeader(event, name) (alias: getHeader)
getRequestURL(event)
getRequestHost(event)
getRequestProtocol(event)
getRequestPath(event)
getRequestIP(event, { xForwardedFor: boolean })

Response

send(event, data, type?)
sendNoContent(event, code = 204)
setResponseStatus(event, status)
getResponseStatus(event)
getResponseStatusText(event)
getResponseHeaders(event)
getResponseHeader(event, name)
setResponseHeaders(event, headers) (alias: setHeaders)
setResponseHeader(event, name, value) (alias: setHeader)
appendResponseHeaders(event, headers) (alias: appendHeaders)
appendResponseHeader(event, name, value) (alias: appendHeader)
defaultContentType(event, type)
sendRedirect(event, location, code=302)
isStream(data)
sendStream(event, data)
writeEarlyHints(event, links, callback)

Sanitize

sanitizeStatusMessage(statusMessage)
sanitizeStatusCode(statusCode, default = 200)

Error

sendError(event, error, debug?)
createError({ statusCode, statusMessage, data? })

Route

useBase(base, handler)

Proxy

sendProxy(event, { target, ...options })
proxyRequest(event, { target, ...options })
fetchWithEvent(event, req, init, { fetch? }?)
getProxyRequestHeaders(event)

Cookie

parseCookies(event)
getCookie(event, name)
setCookie(event, name, value, opts?)
deleteCookie(event, name, opts?)
splitCookiesString(cookiesString)

Session

useSession(event, config = { password, maxAge?, name?, cookie?, seal?, crypto? })
getSession(event, config)
updateSession(event, config, update)
sealSession(event, config)
unsealSession(event, config, sealed)
clearSession(event, config)

Cache

handleCacheHeaders(event, opts)

Cors

handleCors(options) (see h3-cors for more detail about options)
isPreflightRequest(event)
isCorsOriginAllowed(event)
appendCorsHeaders(event, options) (see h3-cors for more detail about options)
appendCorsPreflightHeaders(event, options) (see h3-cors for more detail about options)

Community Packages

You can use more H3 event utilities made by the community.

Please check their READMEs for more details.

PRs are welcome to add your packages.

h3-typebox
- validateBody(event, schema)
- validateQuery(event, schema)
h3-zod
- useValidatedBody(event, schema)
- useValidatedQuery(event, schema)
h3-valibot
- useValidateBody(event, schema)
- useValidateParams(event, schema)
h3-compression
- useGZipCompression(event, response)
- useDeflateCompression(event, response)
- useBrotliCompression(event, response)
- useCompression(event, response)
- useGZipCompressionStream(event, response)
- useDeflateCompressionStream(event, response)
- useCompressionStream(event, response)
@intlify/h3
- defineI18nMiddleware(options)
- useTranslation(event)
- getHeaderLocale(event, options)
- getHeaderLocales(event, options)
- getCookieLocale(event, options)
- setCookieLocale(event, options)
- getPathLocale(event, options)
- getQueryLocale(event, options)

License

MIT

radishes-music/radishes

2023-12-23T01:40:53Z

Cross-platform copyright-free music platform（跨平台的无版权音乐平台）. 支持 windows / macos / linux / web

Radishes 2.0.0

Radishes是这个项目的名称，它是由萝卜翻译而来。可以在这里试听和下载“灰色”歌曲哟

界面表现和功能参考 windows 网易云音乐界面和 ios 的网易云音乐，会对一些功能进行移动端支持。希望能找到合适的基于 vue3 的移动端打包工具（非套壳）。

项目运行

克隆项目

git clone https://github.com/Linkontoask/radishes.git

安装依赖

cd radishes/
yarn bootstrap

运行项目
web: yarn dev
electron: yarn dev:electron
构建项目
web: yarn build
electron:win: yarn build:electron
electron:linux: yarn build)linux
electron:macos: yarn build)macos

支持的功能

browser: 浏览器支持
electron: 桌面级支持
mobile: 移动端支持(在线链接: ~~http://112.74.169.178/mobile/music/~~，分支：next-mobile)
√: 已经支持
×: 不打算支持

功能	browser	electron	mobile
登录和注册（邮箱暂不支持注册）	√	√
每日推荐歌单和歌曲	√	√	√
每日轮播图以及新歌播放、查看歌单详情	√	√	√
歌单搜索、查看、收藏、下载	√	√	√
歌曲排行榜以及排行榜的分类支持	√	√	√
支持查看歌手专辑、详情、相似歌手以及歌手按地区等方式过滤	√	√	√
歌曲播放、暂停、歌词(横向、纵向)、音量 MediaSession(Chrome)	√	√	√
浮动歌词支持（在 PC 上为一个子窗口），歌词进度显示	√	√	×
支持设置播放源（QQ、酷狗等）、下载品质和下载位置修改	√	√
支持搜索歌手、专辑、歌单和单曲	√	√	√
本地音乐支持（支持拖放播放歌曲、歌曲标签 ID3v2 显示）	√	√	×
缓存本地所有状态（比如当前播放歌曲、进度、主题颜色、大小等）	√	√	√
朋友动态查看、点赞等
音乐云盘	√	√
视频播放、收藏
通知(已登录)	×		×
音效(3D 环绕、淡入淡出、温柔、混合)	√	√

Electron

目前已发布第一版，包含听歌等功能，前往 https://github.com/Linkontoask/radishes/releases/tag/2.0.0-alpha.1 下载 windows 版本。
已发布mac版本。v2.0.0-alpha.4 在开发的同时会快速兼容 Electron 。所谓的快速就是调用很少的 API 实现 PC 端的功能，比如放大、缩小、通知、歌词、音乐拖放上传。

目前Electron已经支持对歌曲的标签写入与读取，专辑封面还未压缩处理。所有在radishes平台下载的歌曲都会自动生成歌曲的歌手、专辑、标题等内容。
到现在为止，Electron平台相对于Web平台已经完成了下载进度展示、本地音乐的同步和播放、浮动歌词的展示。

`vue3.x`已发布，音乐项目已完成基本架构

目前会利用闲暇时间进行重构，如果你也想参与进来，可以发邮件或在这里告诉我

Star History

borusyak/are213

2023-12-22T01:25:25Z

Rapid prototyping of web apps using plain english 👶🤖

mesquidar/ForensicsTools

2023-12-22T01:25:25Z

This repository contains low level design resources to improve coding skills and prepare for interviews.

GT-RIPL/Awesome-LLM-Robotics

2023-12-22T01:25:25Z

A comprehensive list of papers using large language/multi-modal models for Robotics/RL, including papers, codes, and related websites

Casper64/css

2023-12-23T01:41:05Z

A CSS parser written V

FelipeFFerreira/ITA-CORES

2023-12-23T01:41:13Z

RISC-V SoC Physical Implementation in 180 nm CMOS with a Quark Core Based on FemtoRV32

dauris/Pond_Window

2023-12-23T01:41:19Z

xiaoxiaolulu/EasyPost

2023-12-22T01:43:23Z

MetaLowCode——美乐低代码，企业级全栈低代码开发平台。私有部署，源码交付。

xiaoqing-tan/vue3dashboard

2023-12-23T01:41:27Z

一款基于Vue3+tailwindcss+Element Plus的后台管理系统

donald-ngo/donald

2023-12-23T01:42:01Z

Donald's Training Repo

This repo will store my test code.

subugoe/Salvador

2023-12-23T01:42:01Z

This Repository isn't used anymore!

New URL

The new Location of this repository is at GitLab.

Salvador - SUB Ant Library for Web Development

#Introduction This is a Library of Apache Ant Tasks that might be useful for Web developers. The main purpose is the management of external tool dependencies, so you don't need to manage them by yourself. It is also able to use a embedded web server, precompile PHP files to static HTML and provides some Ruby tools like SASS and Compass.

#Modules Lockated in the 'modules' sub directory.

scm.xml - Tasks for Git and Subversion.
jetty.xml - Tasks for Jetty
javascript.xml - Several dependencies for JavaScript related development
jruby.xml - JRuby mainly used by sass.xml and compass.xml
compass.xml - Compass CSS tools
sass.xml - SASS CSS tools
php.xml - Integrated PHP compiler
exist.xml - integration of eXist related tasks
tinypng.xml - integration of the TinyPNG web service (you need a API key to use this)
jtidy.xml - integration of JTidy
groovy.xml - integration of Groovy
basex.xml - integration of BaseX

##Unfinished modules

envjs.xml - EnvJS integration (not finished).
jekyll.xml - Jekyll integration (doesn't work since stupid Jekyll relies on native crap)
webjar.xml - Webjar integration (not finished).

#Usage ##Basics To use the libraries import the desired module in your Ant project:

<import file="./build/modules/jetty.xml"/>

Add the requested task as a dependency of your task:

<target name="war.run" depends="salvador.jetty.install"/>

Then either use the dependency declarations of your Ant tasks to get the functionality of the required tools and libraries (for dependency related tasks) or use the supplied macros for specific functionality. ##Naming of tasks

Tasks containing "download" in their name, get the libraries and set up the classpath accordingly, you can use them to create your own macros.
Tasks containing "install" in their name rely on the download and set up macro definitions.

##build.xml This is the base package of Salvador, it provides Maven tasks and Ant enhancements. ###Additional Tasks

The target salvador.base.ant.httptask.install provides the Missing Link Ant HTTP Tasks
The target salvador.base.ant.contrib.install provides the Ant Contrib Tasks
The target salvador.base.maven.install provides the Maven Ant Tasks

##scm.xml This file provides Git and Subversion integration. ###Additional Tasks

User the target salvador.scm.jgit.install to get the JGIT Ant Tasks

###Macros

salvador.scm.svn.co, part of the target salvador.scm.svn.install, takes the attributes 'url' and 'todir' to check out the latest revision of a subersion retository.

##jetty.xml This file provides Jetty integration. ###Additional Tasks

The target salvador.jetty.install provides the Jetty 9 Ant Tasks

###Macros

The target salvador.jetty.install also provides the macro salvador.jetty.run: It takes the attributes 'path', 'webxml' (might be empty), 'port' (default: 8080) and 'contextpath' (default: /ant). This is a wrapper around the <jetty> task, it will scan for changes ever 5 seconds.

##javascript.xml This file provides several JavaScript tools. ###Additional Tasks

The target salvador.js.jsdoctoolkit.install provides the JSDoc Ant Tasks

###Macros The target salvador.js.yui.install provides the following YUI tasks:

salvador.js.yui.compress.js: Takes a 'src' and 'destfile' attribute and compress a JS file.
salvador.js.yui.compress.css: Takes a 'src' and 'destfile' attribute and compress a CSS file.

##jruby.xml This file provides JRuby. I's mainly used by SASS and Compass, but can also be used to get access to other Ruby Gems. It operate on an internal JRuby distribution, not the one provided by your system. ###Macros The target salvador.jruby.gems provides two macros:

salvador.jruby.gem.check checks if the ruby Gem given by the attribute 'gem' is installed. Check the property salvador.jruby.gem.check.return.code for the value '0' (zero).
salvador.jruby.gem.install installs the ruby Gem given by the attribute 'gem'

##compass.xml This file provides Compass related tasks. ###Macros

The target salvador.sass.install provides the macro salvador.sass.compile.watch. It takes two attributes 'src' and 'dest'. It watches the 'src' directory and writes changes to the 'dest' directory using the poll method.

##sass.xml This file provides SASS related tasks. ###Macros The target salvador.sass.macros provides the following macros:

salvador.sass.convert.watch: It takes two attributes 'src' and 'dest'. It watches the 'src' directory and writes changes to the 'dest' directory using the poll method.
salvador.sass.convert: It takes two attributes 'src' and 'destfile'. It converts the 'src' file and writes the result to 'destfile'.
salvador.sass.convert.dir: It takes two attributes 'refid' and 'todir'. It converts the fileSet given by the reference 'refid' file and writes the result to 'todir' using salvador.sass.convert.

##php.xml This file provides an PHP compiler. ###Macros

The target salvador.tinypng.macros provides the macros:
salvador.php.compile: Takes a 'src' and 'destfile' attribute, compiles PHP files to static HTML (or whatever the PHP file emits)
salvador.php.compile.dir: It takes three attributes 'refid', 'todir' and suffix (default ".html"). It converts the fileSet given by the reference 'refid' file and writes the result to 'todir' using salvador.php.compile, use another 'suffix' if the results shouldn't end with html.

##exist.xml This file provides integration of eXist.

Warning: this takes some time on the first usage, since it needs to fetch a complete eXist distribution. ###Additional Tasks

The target salvador.exist.install provides the eXist Ant Tasks

##tinypng.xml This file provides integration of the TinyPNG web service.

Note: You need a API key to use this ###Macros

The target salvador.tinypng.macros provides the macros:
salvador.tinypng.compress: It takes a 'src' and 'destfile' attribute, sends the provided image to TinyPNG and saves the result. Make sure you set the property salvador.tinypng.apikey, containing your API key.
salvador.tinypng.compress.dir: It takes two attributes 'refid' and 'todir'. It converts the fileSet given by the reference 'refid' file and writes the result to 'todir' using salvador.tinypng.compress.

##jtidy.xml This file provides integration of JTidy. ###Additional Tasks

The target salvador.jtidy.install provides the JTidy Ant Tasks.

##groovy.xml This file provides integration of of Groovy. ###Additional Tasks

The target salvador.groovy.install provides the Groovy Ant Tasks

##basex.xml This file provides integration of BaseX. ###Macros

The macro salvador.basex.download downloads BaseX and sets a classpath for integration.

#Examples This section provides some examples.

Running a web application with Jetty and Compass

Make sure you import the required modules:

<import file="./build/modules/jetty.xml"/>

<import file="./build/modules/compass.xml"/>

#Development In the test directory is a test.xml which can by run by

ant -f test/test.xml

It calls every target configured, if you add a module create a target for it in test.xml, to be sure that every target works without any arguments.

GitHub 1C Enterprise Daily Trending

Anttwo/SuGaR

SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering

Abstract

BibTeX

Updates and To-do list

Overview

Installation

0. Requirements

1. Clone the repository

2. Install the required Python packages

3. Install the Gaussian Splatting rasterizer

Quick Start

Data and initial 3D Gaussian Splatting optimization

SuGaR optimization

Mesh extraction

SuGaR and mesh refinement (Hybrid representation)

(Optional) Parameters for traditional textured mesh extraction

(Optional) Default configurations

Tips for using SuGaR on your own data and obtain better reconstructions

1. Capture images or videos that cover the entire surface of the scene

2. Estimate camera poses with COLMAP

3. Density or SDF? Choose a regularization method that fits your scene

4. (Optional) Adapt the scale and the bounding box of the scene

Rendering, composition and animation

Evaluation

babaohuang/GeminiProChat

Solana Program Library

Deployments

Audits

Program Packages

CLI Packages

JavaScript Packages

Development

Environment Setup

Build

Build on-chain programs

Build clients

Test

Common Issues

Clippy

Coverage

MacOS

Release Process

Bump Version

Create Github tag

Publish Github release

Publish to Crates.io

Disclaimer

solidjs/solid-start

SolidStart

Start has just entered a new Beta Phase

Features

Getting started

Development

Monorepo & project.json "workspace" support

Credits

Dyalog/library-core

Dyalog/dyalog-tube-ws

EbolaMan-YT/IPTOOLKIT

rsm28/lethal_company_batch_files

SebLague/Tiny-Chess-Bot-Challenge-Results

Krypteria/AtlasLdr

Features

Usage

Compilation

Future work

Disclaimer

SJTU-IPADS/PowerInfer

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU

TL;DR

Demo 🔥

Abstract

Features

Getting Started

Setup and Installation

Get the Code

Build

Model Weights

Download PowerInfer GGUF via Hugging Face

Monorepo & `project.json` `"workspace"` support

说明（部署前请仔细阅读完）