GitHub 1C Enterprise Daily Trending

SAP/abap-file-formats

2024-12-08T01:29:08Z

File formats that define and specify the file representation for ABAP development objects

sqlfluff/sqlfluff

2024-12-07T01:28:46Z

A modular SQL linter and auto-formatter with support for multiple dialects and templated code.

The SQL Linter for Humans

SQLFluff is a dialect-flexible and configurable SQL linter. Designed with ELT applications in mind, SQLFluff also works with Jinja templating and dbt. SQLFluff will auto-fix most linting errors, allowing you to focus your time on what matters.

Dialects Supported

Although SQL is reasonably consistent in its implementations, there are several different dialects available with variations of syntax and grammar. SQLFluff currently supports the following SQL dialects (though perhaps not in full):

ANSI SQL - this is the base version and on occasion may not strictly follow the ANSI/ISO SQL definition
Athena
BigQuery
ClickHouse
Databricks (note: this extends the sparksql dialect with Unity Catalog syntax).
Db2
DuckDB
Exasol
Greenplum
Hive
Impala
Materialize
MySQL
Oracle
PostgreSQL (aka Postgres)
Redshift
Snowflake
SOQL
SparkSQL
SQLite
StarRocks
Teradata
Transact-SQL (aka T-SQL)
Trino
Vertica

We aim to make it easy to expand on the support of these dialects and also add other, currently unsupported, dialects. Please raise issues (or upvote any existing issues) to let us know of demand for missing support.

Pull requests from those that know the missing syntax or dialects are especially welcomed and are the question way for you to get support added. We are happy to work with any potential contributors on this to help them add this support. Please raise an issue first for any large feature change to ensure it is a good fit for this project before spending time on this work.

Templates Supported

SQL itself does not lend itself well to modularity, so to introduce some flexibility and reusability it is often templated as discussed more in our modularity documentation.

SQLFluff supports the following templates:

Jinja (aka Jinja2)
SQL placeholders (e.g. SQLAlchemy parameters)
Python format strings
dbt (requires plugin)

Again, please raise issues if you wish to support more templating languages/syntaxes.

VS Code Extension

We also have a VS Code extension:

Getting Started

To get started, install the package and run sqlfluff lint or sqlfluff fix.

$ pip install sqlfluff
$ echo "  SELECT a  +  b FROM tbl;  " > test.sql
$ sqlfluff lint test.sql --dialect ansi
== [test.sql] FAIL
L:   1 | P:   1 | LT01 | Expected only single space before 'SELECT' keyword.
                       | Found '  '. [layout.spacing]
L:   1 | P:   1 | LT02 | First line should not be indented.
                       | [layout.indent]
L:   1 | P:   1 | LT13 | Files must not begin with newlines or whitespace.
                       | [layout.start_of_file]
L:   1 | P:  11 | LT01 | Expected only single space before binary operator '+'.
                       | Found '  '. [layout.spacing]
L:   1 | P:  14 | LT01 | Expected only single space before naked identifier.
                       | Found '  '. [layout.spacing]
L:   1 | P:  27 | LT01 | Unnecessary trailing whitespace at end of file.
                       | [layout.spacing]
L:   1 | P:  27 | LT12 | Files must end with a single trailing newline.
                       | [layout.end_of_file]
All Finished 📜 🎉!

Alternatively, you can use the Official SQLFluff Docker Image or have a play using SQLFluff online.

For full CLI usage and rules reference, see the SQLFluff docs.

Documentation

For full documentation visit docs.sqlfluff.com. This documentation is generated from this repository so please raise issues or pull requests for any additions, corrections, or clarifications.

Releases

SQLFluff adheres to Semantic Versioning, so breaking changes should be restricted to major versions releases. Some elements (such as the python API) are in a less stable state and may see more significant changes more often. For details on breaking changes and how to migrate between versions, see our release notes. See the changelog for more details. If you would like to join in, please consider contributing.

New releases are made monthly. For more information, visit Releases.

SQLFluff on Slack

We have a fast-growing community on Slack, come and join us!

SQLFluff on Twitter

Contributing

We are grateful to all our contributors. There is a lot to do in this project, and we are just getting started.

If you want to understand more about the architecture of SQLFluff, you can find more here.

If you would like to contribute, check out the open issues on GitHub. You can also see the guide to contributing.

Sponsors

The turnkey analytics stack, find out more at Datacoves.com.

imputnet/cobalt

2024-12-07T01:28:46Z

best way to save what you love

best way to save what you love
cobalt.tools

💬 community discord server 🐦 twitter

cobalt is a media downloader that doesn't piss you off. it's friendly, efficient, and doesn't have ads, trackers, paywalls or other nonsense.

paste the link, get the file, move on. that simple, just how it should be.

cobalt monorepo

this monorepo includes source code for api, frontend, and related packages:

it also includes documentation in the docs tree:

thank you

cobalt is sponsored by royalehosting.net and the main processing servers are hosted on their network. we really appreciate their kindness and support!

ethics

cobalt is a tool that makes downloading public content easier. it takes zero liability. the end user is responsible for what they download, how they use and distribute that content. cobalt never caches any content, it works like a fancy proxy.

cobalt is in no way a piracy tool and cannot be used as such. it can only download free & publicly accessible content. same content can be downloaded via dev tools of any modern web browser.

contributing

thank you for considering making a contribution to cobalt! please check the contributing guidelines here before making a pull request.

licenses

for relevant licensing information, see the api and web READMEs. unless specified otherwise, the remainder of this repository is licensed under AGPL-3.0.

QuivrHQ/MegaParse

2024-12-07T01:28:46Z

File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.

MegaParse - Your Parser for every type of documents

MegaParse is a powerful and versatile parser that can handle various types of documents with ease. Whether you're dealing with text, PDFs, Powerpoint presentations, Word documents MegaParse has got you covered. Focus on having no information loss during parsing.

Key Features 🎯

Versatile Parser: MegaParse is a powerful and versatile parser that can handle various types of documents with ease.
No Information Loss: Focus on having no information loss during parsing.
Fast and Efficient: Designed with speed and efficiency at its core.
Wide File Compatibility: Supports Text, PDF, Powerpoint presentations, Excel, CSV, Word documents.
Open Source: Freedom is beautiful, and so is MegaParse. Open source and free to use.

Support

Files: ✅ PDF ✅ Powerpoint ✅ Word
Content: ✅ Tables ✅ TOC ✅ Headers ✅ Footers ✅ Images

Example

https://github.com/QuivrHQ/MegaParse/assets/19614572/1b4cdb73-8dc2-44ef-b8b4-a7509bc8d4f3

Installation

pip install megaparse

Usage

Add your OpenAI or Anthropic API key to the .env file
Install poppler on your computer (images and PDFs)
Install tesseract on your computer (images and PDFs)
If you have a mac, you also need to install libmagic brew install libmagic

from megaparse import MegaParse
from langchain_openai import ChatOpenAI
from megaparse.parser.unstructured_parser import UnstructuredParser

parser = UnstructuredParser()
megaparse = MegaParse(parser)
response = megaparse.load("./test.pdf")
print(response)
megaparse.save("./test.md")

Use MegaParse Vision

Change the parser to MegaParseVision

from megaparse import MegaParse
from langchain_openai import ChatOpenAI
from megaparse.parser.megaparse_vision import MegaParseVision

model = ChatOpenAI(model="gpt-4o", api_key=os.getenv("OPENAI_API_KEY"))  # type: ignore
parser = MegaParseVision(model=model)
megaparse = MegaParse(parser)
response = megaparse.load("./test.pdf")
print(response)
megaparse.save("./test.md")

Note: The model supported by MegaParse Vision are the multimodal ones such as claude 3.5, claude 4, gpt-4o and gpt-4.

(Optional) Use LlamaParse for Improved Results

Create an account on Llama Cloud and get your API key.
Change the parser to LlamaParser

from megaparse import MegaParse
from langchain_openai import ChatOpenAI
from megaparse.parser.llama_parser import LlamaParser

parser = LlamaParser(api_key = os.getenv("LLAMA_CLOUD_API_KEY"))
megaparse = MegaParse(parser)
response = megaparse.load("./test.pdf")
print(response)
megaparse.save("./test.md") #saves the last processed doc in md format

Use as an API

There is a MakeFile for you, simply use : make dev at the root of the project and you are good to go.

See localhost:8000/docs for more info on the different endpoints !

BenchMark

Parser	similarity_ratio
megaparse_vision	0.87
unstructured_with_check_table	0.77
unstructured	0.59
llama_parser	0.33

Higher the better

Note: Want to evaluate and compare your Megaparse module with ours ? Please add your config in evaluations/script.py and then run python evaluations/script.py. If it is better, do a PR, I mean, let's go higher together .

In Construction 🚧

Improve table checker
Create Checkers to add modular postprocessing ⚙️
Add Structured output, let's get computer talking 🤖

Star History

MrOtherGuy/firefox-csshacks

2024-12-08T01:36:42Z

Collection of userstyles affecting the browser

Collection of random CSS hacks for Firefox

This repository contains various styles to modify appearance of Mozilla Firefox. These stylesheets are mostly self-contained and can be mixed with each other somewhat freely, but there are no promises about compatibility with third-party styles.

In the case that a particular style relies on another style, the fact will be noted at the start of the file that requires so.

Stylesheets in this repository are tested only on Windows 10 and to a lesser amount on Linux. Most of them should also work on OSX and Windows7, but there may be wrong behavior especially when native widgets such as window titlebar or window control buttons are being styled.

Setup

As an overview, you will make Firefox load two special stylesheets - userChrome.css and userContent.css. Doing so requires setting a specific preference (see below) and then creating those files inside your Firefox user profile.

The setup is quite straightforward with the exception of how to find the profile folder so pay attention to that.

Set the pref to load stylesheets

Go to about:config and set the pref toolkit.legacyUserProfileCustomizations.stylesheets to true

After you set this pref, Firefox will try to load userChrome.css and userContent.css - but those files don't exist yet so now let's create them.

Setting up files

Find the profile folder

First, find your profile folder. While Firefox is running, go to about:support and find a Profile folder row near the top - there should also be a button labeled "Open folder" next to it. Clicking that button should open the folder in your file manager.

NOTE: On some Firefox versions clicking that button may open the profiles folder which houses all your profiles. In that case, navigate into the specific folder you wish to modify. about:support should still show the correct folder name so refer to that if you need to figure out the what folder you need to open.

The real profile folder should have files like prefs.js and places.sqlite If you see those two files in the folder, then great! You found the profile folder! Now lets actually create those stylesheet files.

Creating the stylesheet files

Note: only userChrome.css is mentioned in this section for brevity, but everything regarding that will also apply to userContent.css

Firefox loads userChrome.css from <profileFolder>/chrome/userChrome.css. That chrome-folder or the stylesheet files do not exist by default.

Set up files manually

Manually copying individual styles directly into userChrome.css is a simple way to do things for better and for worse.

Create a new folder into the profile folder and name it chrome
Create userChrome.css inside that newly created chrome-folder
Copy-paste contents of individual .css files from this repository into your userChrome.css file (and save it of course!)
If Firefox is running, restart Firefox so that the changes take effect

Pay attention to the filename of userChrome.css - the file extension must be .css and if your file manager is hiding file extensions then you might accidentally create a file named userChrome.css.txt and Firefox will not load that.

In the end you should have a folder structure like this:

<profile_folder>
|_ chrome
|   |_ userChrome.css
|   |_ userContent.css
|_ extensions
|_ prefs.js
...
all other profile folders and files
...

Set up files using git

Preferred way to do things, since it makes updates easier and makes organizing multiple styles easier.

Assumes that you have a git client installed, and that you do not already have a chrome folder in your profile.

Open a command prompt / console / terminal and cd into the profile folder
Clone this repository into the profile folder
- (NOTE: if you already have "chrome" folder, then rename it before cloning. After clone is complete, just copy the contents of the old folder into the new chrome folder)
- git clone https://github.com/MrOtherGuy/firefox-csshacks.git chrome on command-line
- This should create a new folder "chrome" into your profile folder with the contents of this repository
(Optional) Make a copy of userChrome_example.css and rename the copy to userChrome.css
@import individual style files into your userChrome.css
- Notice that any @imports must be placed before anything else in whatever file you are using them
- Check userChrome_example.css for how it uses @import
If Firefox is running, restart Firefox so that the changes take effect

Afterwards, you can just use git pull in the "chrome" folder and it will replace your copies with up-to-date versions. git pull won't replace your userChrome.css file so you can safely put your own custom rules into userChrome.css directly and those won't be overwritten when you update.

Style categories

The files themselves are only separated to chrome and content sub-folders. Files have a one or more tag applied to them as listed in tags.csv file.

You can browse the tag-categorized files by using this UI

Usage

Stylesheets are divided in to chrome and content folders. Firefox loads userChrome.css into the browser UI and it loads userContent.css into the content documents like web pages and built-in or extension pages.

Use stylesheets under "chrome" in userChrome.css

Use stylesheets under "content" in userContent.css

The above is not a technical requirement but the particular styles generally won't do anything when loaded in wrong context.

You can import the stylesheets with @-rule import like this:

@import url("path/filename.css");

A good habit would be to load each separate style without modifications using @import statements, and then apply your own modifications in userChrome.css after all imports. This makes it easier for you to update the files from the repository since your modifications will be preserved.

Example userChrome.css:

@import url(chrome/tab_close_button_always_on_hover.css);
@import url(chrome/tab_loading_progress_throbber.css);
@import url(chrome/button_effect_scale_onclick.css);

:root{
  --toolbar-bgcolor: rgb(36,44,59) !important;
  --uc-menu-bkgnd: var(--toolbar-bgcolor);
  --arrowpanel-background: var(--toolbar-bgcolor) !important;
  --autocomplete-popup-background: var(--toolbar-bgcolor) !important;
  --uc-menu-disabled: rgb(90,90,90) !important;
  --lwt-toolbar-field-focus: rgb(36,44,59) !important;
}

Important!

Note that all @import rules need to be placed before any other rules in the file, including @namespace rules. Additionally, the order of imported files is just as important as the order of rules within one file.

I would strongly advice using @import to include styles instead of copying contents directly to userChrome.css even with just a few file "components". The technical reason for this is that some files rely on @namespace rules and those only apply on file level such that a @namespace applies to every selector in that file (and in that file only). On top of that, @imports make managing multiple files much easier.

Further miscallaneous notes

Import any *_patch.css files after their base stylesheet. Import the shared window_control_support.css before other stylesheets.

Additionally, you are advised to import theme_ files before any other modules.

Theme

** NOTE ** Theme files are mostly out-of-date as of 2020-05-22

Stylesheets prefixed with theme_ require theme_color_variables.css to be imported.

Example userChrome.css resulting in rather complete dark blueish-grey UI:

@import url(theme_color_variables.css);
@import url(theme_sidebar.css);
@import url(theme_toolbars.css);
@import url(theme_popups_and_menus.css);

/* Your other rules here */

You can use individual modules from theme such as to only include popups_and_menus. But it would still be required that you import the theme_color_variables.css or you'll have to manually edit all the colors.

sdmg15/Best-websites-a-programmer-should-visit

2024-12-08T01:36:42Z

🔗 Some useful websites for programmers.

Best-websites-a-programmer-should-visit

Some useful websites for programmers.

When learning CS, there are some useful sites you must know to get always informed to do your technologies even better and learn new things. Here is a non-exhaustive list of some sites you should visit. This list will get updated as soon as I can get another link, but you can also contribute by adding those you know 😉

Note : Chinese Version

📖 Index

❓ When you get stuck

Codementor : A mentorship community to learn from fellow developers via live 1:1 help and more.
devRant : Community where you can rant and release your stress
Google : A search engine for anything you get stuck with.
Learn Anything : Community curated knowledge graph of best paths for learning anything
Quora : A place to share knowledge and better understand the world
Stack Overflow : subscribe to their weekly newsletter and any other topic which you find interesting
Stack Exchange : Stack Exchange is the combination of Stack Overflow and 170+ sites where you can get any kinds of help
Coderanch : A friendly place for programming greenhorns. Jump straight into any of our topics and light hearted discussions. Ranging from Java, Databases, Android, Programmer certification, Programming jobs and much more...

↥ Back To Top

📰 News

ACM TechNews : TechNews
Stratechery : Stratechery by Ben Thompson
AlternativeTo : Crowdsourced software recommendations
Ars Technica : Posts unique quality articles on technology, science, politics and society.
Better Dev Links : Weekly links to help you become a better developer
Better Programming : Featured articles for modern developers available every day
CNET : Your guide to a better future! An excellent source of up-to-date tech news.
Daily.Dev : A Great Chrome extension which provides recent and relevant tech articles and news
DevOpsLinks : An online community of thousands of developers and IT experts interested in DevOps
Frontend Focus Newsletter : once-weekly roundup of the frontend news, articles and tutorials
GSMArena.com : news related to the latest mobile phones and android.
Hacker News Digest : curated automatically, delivered as frequently as you want
Hacker News : news aggregator for programmers where civility is king, try a newsletter to get top news to your inbox:
Hacker Newsletter : curated by hand, delivered weekly
Hacker Noon : How hackers start their afternoons.
High Scalability : Success stories of various companies on their apps, infra scaling.
Lobsters : Lobsters is a technology-focused community centered around link aggregation and discussion.
product hunt : Discover your next favorite thing
Recode : Tech news that focuses on the business of Silicon Valley
Reddit.com/r/programming : Subreddit with aggregated tech news and articles
Slashdot : News, Social news (Originally "News for Nerds. Stuff that Matters")
StackShare : See what tools are popular with developers and companies and read about their technology decisions
TechCrunch : dedicated to obsessively profiling startups, reviewing new Internet products, and breaking tech news
The DEV Community : Where programmers share ideas and help each other grow.
The Verge : More product oriented tech news
WCCF Tech : curated news related to Computer Hardware, Software, and everything else in the Tech World.
XDA : Android Developers News
Hashnode : Hashnode is a friendly and inclusive community for software developers
Rekt : A journalism related to Defi, Web3, Blockchain and security on them.
Hackaday : Hacking is an art form that uses something in a way in which it was not originally intended.

↥ Back To Top

📚 Magazines

MIT Technology Review : MIT's tech review magazine.
Nautilus : NewYorker for tech.
LWN : Weekly news coverage of opensource technologies, programming, etc. ( Originally Linux Weekly News).

↥ Back To Top

👶 Coding practice for beginners

Codedex - start your coding adventure : Gamify your learning experience and have fun learning how to code!
freeCodeCamp : Learn to code and build projects for nonprofits. Build your full stack web development portfolio today
Blackbird School : Learn to code online with our beginner-friendly platform.
Glitch : Great place to play around with code in an easy-to-set-up environment.
Reddit.com/r/dailyprogrammer : interesting programming challenges where you can learn from looking at other's code, even if you are not able to solve code you can look at how others solved.
Programming by Doing : very good site for those who want to start with absolute basics
CodeAbbey - a place where everyone can master programming : Best place to begin with problems that start at the easiest and gradually increase difficulty with each problem.
Exercism.io : Download and solve practice problems in over 50 different languages, and share your solution with others.
InterviewBit/Coding Interview Questions : Gamifies the experience of practicing for your interview and includes lots of sample problems to solve.
karan/Projects-Solutions : Solutions to most of the problems in the link above
Lod - Cloud : The Linking Open Data cloud diagram
Cave of programming : Learn to program, Upgrade your skills.
Vim adventures : Learn VIM by playing
Learn Vimscript the Hard Way : Learn Vim Script from beginning to expert.
Programming Tasks : Large collection of small programs.
Pramp : It's your turn to be the interviewer. when done, click on the swap roles button on the left
Paqmind : Guides and challenges to learn programming
Codecademy : Learn to code interactively, for free.
Codegym : It contains a Java tutorial with 1200 Java practise tasks.
treehouse : The fast, easy, and affordable way to build your skills.
GeeksforGeeks : Learn to code, Study for placement, Do competitive coding.
edabit : Practice coding via various small challenges.
Learn Git Branching : Learn and practice git commands in an interactive way
Oh Shit, Git!?! : Cheat sheet to get yourself out of bad git situations
The Odin Project : The Odin Project provides a free open source coding curriculum that can be taken entirely online.
Udemy : Whether you want to learn or to share what you know, you’ve come to the right place. As a global destination for online learning, we connect people through knowledge.
Scaler : A curated interactive space for aspiring programmers to learn and master their coding skills. Also help in job interview.
SQL Zoo : Learn SQL interactively in stages, from basic queries to complex operations.
SQLTest.online : Boost Your SQL Proficiency with Challenging Problems.

↥ Back To Top

💰 CryptoCurrency

Blockchain Basics : Great introduction to blockchain
Blockchain demo : A visual demo of Blockchain technology
Build a blockchain in Python : Learn Blockchains by Building One
Coin demo : CryptoCurrency demo
GitCoin : Gitcoin is the easiest way to monetize or incentivize work in Open Source Software.
Learn About Bitcoin and Lightning Protocol: Complete 4 weeks seminar ciricullum for learning about Bitcoin.
Learn Me A Bitcoin: Bitcoin, Cryptocurrencies and Blockchain explained in plain English
Learn Web3 DAO: Learn to become a Web3 Developer for free.
Lite Paper : Cryptocurrencies & Blockchain made effortless
Lopp Bitcoin Resources : Some Bitcoin Resources
Mastering Bitcoin: Introduction to Bitcoin and tutorials to operate a full node
Mempool: Bitcoin block explorer, mempool visualizer, transaction tracker, and fee estimator

↥ Back To Top

💡 For those who want to start a small project but can't find the ideas

freeCodeCamp/React project ideas : 27 fun app ideas you can build while learning React.
karan/Projects : a large collection of small projects for beginners with
Wrong "big projects" for beginners : How to choose where to start
vicky002/1000-Projects : Mega List of practical projects that one can solve in any programming language!
reddit.com/r/AppIdeas : A place to discuss ideas for applications, for bored developers.
reddit.com/r/SomebodyMakeThis : A home for ideas by people who lack time, money, or skills.
florinpop17/app-ideas : A Collection of application ideas which can be used to improve your coding skills.

↥ Back To Top

🗣️ General Coding advice

How to Begin With Competitive Programming
10-ways-to-be-a-better-developer : Ways to become a better dev!
Code Review Best Practices : Kevin London's blog
Design Patterns : Design Patterns explained in detail with examples.
Game Programming Patterns : Game Programming Patterns is a collection of patterns Robert Nystrom found in games that make code cleaner, easier to understand, and faster
How to become a programmer or the art of Googling well : How to become a programmer or the art of Googling well
How to escape tutorial purgatory as a new developer — or at any time in your career : How to escape tutorial purgatory
JS Project Guidelines : A set of best practices for JavaScript projects.
Learn to Code With Me : A comprehensive site resource by Laurence Bradford for developers who aims to build a career in the tech world
Lessons From A Lifetime Of Being A Programmer : The Codist Header Lessons From A Lifetime Of Being A Programmer
MITRE - Top 25 Most Dangerous Software Weaknesses (2022) : The currently most common and impactful software weaknesses.
Software Architecture Guide : A site by Martin Fowler about Software Architecture patterns and best practices to help building software effectively.
Software design pattern : The entire collection of Design Patterns.
Things I Wish Someone Had Told Me When I Was Learning How to Code — Free Code Camp : What I’ve learned from teaching others
TeachYourselfCS : If you’re a self-taught engineer or bootcamp grad, you owe it to yourself to learn computer science. Thankfully, you can give yourself a world-class CS education without investing years and a small fortune in a degree program
What every computer science major should know : The Principles of Good Programming
Working as a Software Developer : Henrik Warne's blog
The Open Web Application Security Project (OWASP) : OWASP is an open community dedicated to enabling organizations to conceive, develop, acquire, operate, and maintain applications that can be trusted.

↥ Back To Top

🎨 Coding Style

Airbnb JS Style Guide : A mostly reasonable approach to JavaScript
Airbnb Ruby Style Guide : A Ruby style guide by Airbnb
Ruby coding style guide : A community-driven Ruby coding style guide
Angular 1 Style Guide : Officially endorsed style guide by John Pappa
CS 106B Coding Style Guide : must see for those who create spaghetti
Debugging Faqs : Check out how to debug your program
Directory of CS Courses (many with online lectures) : Another online CS courses
Directory of Online CS Courses : Free online CS courses
Good C programming habits. • /r/C_Programming : C programming habits to adopt
Google C++ Style Guide
How to Report Bugs Effectively : Want to report a bug but you don't know how? Check out this post
What are some bad coding habits you would recommend a beginner avoid getting into? : Bad habits to avoid when you get started
PEP8 - Style Guide for Python Code : Style Guide for Python Code
Standard JS Style Guide : JavaScript style guide, with linter & automatic code fixer
The Hitchhiker's Guide to Python : Best Practices for Python Development
Google Python Style Guide : Google Python Style Guide
Aurelia Style Guide : An Aurelia style guide by Behzad Abbasi(Behzad888)
Source Making : Design Patterns & Refactoring
Refactoring Guru: Refactoring And Design Patterns
Google Java Style Guide: Google Java Style Guide
Rust Coding style: Rust code formatting RFCs and coding style guides
Google C# Style Guide: Google C# Style Guide
Uber Go Style Guide: Uber Go Style Guide

↥ Back To Top

🛠️ General Tools

CoderPad : Quickly Conduct Coding Interviews and Phone Screen Interviews.
CodePen : Front End Developer Playground & Code Editor in the Browser
Crontab Guru : Quick and simple editor for cron schedule expressions
Devicons : Cheatsheet for devs icons
Diagrams.net : Drawing tools to make design and uml easily. Old draw.io
FreeFor.Dev : A huge list of free resources and tools
Hotkey Cheatsheet : A comprehensive hotkey cheatsheet for popular software and applications
Imgur : Online image sharing and image hosting service.
Kody Tools: 100+ dev tools including code converters, formatters, and minifiers.
Pyrexp : Online regex tester and visualizer for Python.
regex101 : Online regex tester and debugger: PHP, PCRE, Python, Golang and JavaScript
regexr : Another online tool to learn, build & test Regular Expressions
Prodia AI Image API : API for open sourced image models
Wit AI : Natural Language for Developers
SaaS Design : Collection of open source Figma UI design templates for your next project.
Seymour : Live Programming for the Classroom
Code share : Share code in real-time with other developers
Solid Tools for Developers : Online debugging tools for developers and system administrators
OS Query : Easily ask questions about your Linux, Windows, and macOS infrastructure
LaunchPad : Appollo launchepad for testing GraphQl queries
GraphOnline : Useful tool for visualizing Graphs
Data Structure Visualization : Perfect website for visually learning Algorithms
IDE Onlang : Write in one language and get the same result in other languages.
JSON Crack : An online open-source tool designed for visualizing data in various languages such as JSON, YAML, CSV, and more.
Pad.new : Free cloud-based IDE to run code and databases in almost any language
PullRequest : Code review as a service from vetted, professional reviewers
Python Visualizer : Watch the execution of basic Python, Java, C++, etc. code step-by-step. Recommended for new programmers and the Canadian Computing Competition.
Extends Class : Online developer tools: REST and SOAP clients, SQLite browser, testers (Regex, XPath, JSONPath) and other tools (Encoders, Converters and formatters)
Sourcegraph : Online tool for searching millions of open source repositories.
SVG Path Editor : Online and open source SVG Path editor.
EmailDrop: Emaildrop is a free disposable email provider.
Repl.it: Accessible prototyping tool for various needs.
KeyBr (Typing Practice): Easy to use typing practice app.
Svgator: Animate svg graphically. Its like a video editor but for svg.
Webhook.site: Useful tool for test and debug webhooks.
kandi: Jumpstart Application Development by finding the right Open Source resource
Svix Play: Webhook tester & debugger. Test webhooks directly from your test suite.
Typeracer: Increase your typing speed while racing against others.
Typerush: Increase typing speed while racing.
IT-Tools: Collection of handy online tools for developers.

↥ Back To Top

🐚 Bash and Shell scripting

Advanced Bash-Scripting Guide : An in-depth exploration of the art of shell scripting
Bash Guide for Beginners : Bash Guide for Beginners Machtelt Garrels
Bash Programming : by Mike G mikkey at dynamo.com.ar
Bash Reference Manual : Bash Reference Manual
BashGuide : BashGuide - Greg's Wiki
Conquering the Command Line : Unix and Linux Commands for Developers
Airborn OS : Private Google Docs Alternative
Commandlinefu : An extensive collection of Shell oneliners that can save your day on many occasions
Pure Bash Bible : A collection of pure bash alternatives to external processes.
25 Common Linux Bash Script Examples To Get You Started

↥ Back To Top

🎤 Interview Preparation

10 Frequently asked SQL Query Interview Questions
A Collection of Quant Riddles With Answers
A Developer's Guide to Working with Recruiters
Algorithm design canvas
Aptitude Questions and Answers : Quant and aptitude preparation
A site for technical interview questions, brain teasers, puzzles, quizzles : whatever the heck those are) and other things that make you think!
Big collection of interview preparation links • /r/cscareerquestions
Big-O Algorithm Complexity Cheat Sheet
BIG O Misconceptions
Bitwise tricks
ChiperSoft/InterviewThis : questions to ask during on a interview to know more about the company.
Code Project : For those who code!
Coderbytes : A platform for coding challenges and interview prep.
C PUZZLES, Some interesting C problems
Core Java Interview Questions
CS9: Problem-Solving for the CS Technical Interview
Delightful Puzzles
Determining the big-O runtimes of these different loops? : really good stackoverflow question that covers basics of calculating runtime complexity.
FAANGPath Resume template : Maximize your chances to get your resume in the hands on HR/HM.
five-essential-phone-screen-questions - steveyegge2
Freshers Interviews
Gainlo : Mock interview from professionals
GeeksforGeeks/A computer science portal for geeks : also subscribe to their feeds to get links to their new articles.
github.com/odino/interviews : list of important questions for interview
Give your résumé a face lift
Google Interview Warmup : Google Interview Warmup is an AI-based tool that helps individuals practise for their interviews.
Here's How to Prepare for Tech Interviews • /r/cscareerquestions
How to Answer "Tell Me a Little About Yourself"/The Art of Manliness
How to Break Into the Tech Industry - a Guide to Job Hunting and Tech Interviews
How to interview
How to prepare for an interview - 1
IIT Delhi Placement Experience
samwincott/Internship-Guide : An aggregate of links related to internships.
Interviewing At Jane Street : Interviewing At Jane Street
Interview Archives - Java Honk
Interview Cake : Free and Paid course options offering high quality technical interview practice.
Job Interview: How to Ace a Job Interview/The Art of Manliness
Job interviews News, Videos, Reviews and Gossip - Lifehacker
Job Interview Questions and Best Answers
kimberli/interviews : study sheet for Interview
LeetCode : A new way to learn. Here you can prepare for your interview.
Learnersbucket: Data Structures and Algorithms in Javascript.
Mission-peace/interview problems : A large collection of coding interview problems
NeetCode: Prepare topic wise coding questions which are most frequently asked in MAANG companies.
Pramp/A free on demand interview practice platform for Software Engineers : Practice coding interviews with real peers
Problems/LeetCode OJ : Coding practice for interviews
Programmer And Software Interview Questions Answers
Reddit.com/user/ashish2199/m/puzzles : Logic Puzzles
SQL interview questions : great SQL test
SQL Joins explained using venn diagram
svozniuk/java-interviews : Java interview questions
Techie Delight : Leading platform for technical interview preparation.
The 25 most difficult HR questions
Top 10 Algorithms for Coding Interview : Algorithms for Coding Interview
Unix / Linux Commands Cheat Sheet
Unsolicited_advice_for_job_seekers_and_employers
Visualising Data Structures and Algorithms through Animation
We Help Coders Get Hired : website offering courses on system design, interview strategies, soft skills etc
what_are_your_goto_questions_for_the_do_you_have?
Why You Make Less Money • /r/cscareerquestions
wu :: riddles(hard) : logic puzzles and riddles
What I Learned from Interviewing at multiple AI Companies and Start-Ups -Tech Interview Handbook: Guide on cracking tech interviews

↥ Back To Top

🎬 Documentaries

Breaking the Code : Biography of Alan Turing
Cracking The Code Interview : Cracking the Code Interview
Cracking the Coding Interview : Cracking the Coding Interview, Fullstack Speaker Series
Harvard CS50 - Asymptotic Notation (video) : Asymptotic Notation explained by Harvard
Machine Code Instructions (video) : Code instructions
Machine that Changed the World - a very good documentary about the history of computers
- Part 1 is unavailable for free streaming due to widespread copyright claims.
- Part 2: Inventing the Future
- Part 3: The Paperback Computer
- Part 4: The Thinking Machine
- Part 5: The World at Your Fingertips
Mechanical Computer (All Parts) : a very good video from the 1950s explaining how mechanical computers used to work without all the modern-day electronics.
Teach Yourself Computer Science : Teach Yourself Computer Science
The Code : Story of Linux documentary
The Internet's Own Boy : The Story of Aaron Swartz

↥ Back To Top

🎓 MOOCs for learning something new

Boot.dev : Interactive online course to learn modern backend programming in Python, Javascript, Go, and more.
Class Central : a directory of 100,000+ student reviews of thousands of MOOCs.
Classpert : a website that gathers MOOCs and online courses from several providers, focusing on computer science and data science courses.
Computer Science Resources : list of MOOCs for autodidacts
Coursera.org : Take the world's best courses, online.
CS50 : A set of goods tutorials from cs50
edX : Free Online Courses, Advance Your Career, Improve Your Life.
Kadenze/Creative Programming : Programming courses focused on art and creativity
MIT OCW Electrical Engineering and Computer Science
MOOC.fi : Free online courses from the University of Helsinki
NPTEL : Free online courses by IIT with certificates
prakhar1989/awesome-CS-courses : List containing large amount of CS courses
Pluralsight : An online learning and workforce development platform that helps businesses and individuals adjust to changing technology.

↥ Back To Top

🧑‍💻 Sites related to your preferred programming language (For me C++)

Best books for learning java must read : Get basics of Java
Bjarne Stroustrup's C++ Style and Technique FAQ : The C++ FAQ
Bjarne Stroustrup's FAQ : The C++ FAQ
C++11 - the new ISO C++ standard : The C++11 FAQ
Compilers (video) : A set of videos on how the GC works
Deep Dive Java : Garbage Collection is Good!
Free Online Chapters of Inside the Java Virtual Machine by Bill Venners : Java Corner
How Garbage Collection Works : Java memory management
Implementation of Algorithms and Data Structures, Interview Questions and Answers
IntelliJ Keyboard Shortcuts : Keyboard shortcuts to enhance your productivity when working in IntelliJ.
Java Corner at Artima.com : Java Corner at Artima.com
Java Lecture Notes : Java Student's Resource
Java Off Heap : Java Off the Heap house
Java Revisited : good for learning about Java Language and interview preparation.
Java-source : Java source
Java Visualizer : helps visualize references, values of variables, etc
JournalDev - Java, Java EE, Android, Web Development Tutorials : Java, Java EE, Android, Web Development Tutorials
Learning Java : a free online textbook for learning Java
Netbeans Keyboard Shortcuts : Keyboard shortcuts to enhance your productivity when working in Netbeans.
Official Qt Documentation : Documentation for different Qt versions, languages, tools and platforms
Search Open Source Java API : view source of java library and learn how things are implemented.
The C++ Programming Language : The C++ Programming Language.
The Java Memory Model: The Java Memory Model
The Java™ Tutorials : The best tutorials for Java.
Understanding JVM Internals : Understanding JVM Internals
what-is-garbage-collection : Demystify the garbage collection
JavaWorld : Welcome to Javaworld
JavatPoint : Best website to get a basic Java programming tutorial
The Rust Programming Language Book : Explains the Rust programming language
Rust Cookbook : Quickly get an overview of the capabilities of the Rust crate ecosystem
Getting start with python : A-Z python programming concepts and methods.
Rust by Example : A collection of runnable examples that illustrate various Rust concepts and standard libraries

↥ Back To Top

🤖 Learn AI

aima : The leading textbook in Artificial Intelligence (4th most cited publication of the century). Includes Github repositories and more AI resources!
fast.ai : Free practical deep learning course for coders without grad-level maths!
TypeDB : A Strongly-typed Database
Robots that learn : Robots that Learn
Unsupervised Sentiment Neuron : Unsupervised Sentiment Neuron
What's the difference between AI- DP and ML? : Difference artificial intelligence, machine-learning, deep-learning-ai
TensorFlow : An open-source software library for Machine Intelligence
Scikit-learn : A Python module for machine learning build on top of SciPy
DeepLearning.ai : Deep Learning course by Andrew Ng, Founder of Coursera
Coding the History of Deep Learning : Coding the History of Deep Learning
Serpent AI : Game Agent Framework. Helping you create AIs / Bots to play any game you own! BETA
Blog Floydhub : Colorizing B&W Photos with Neural Networks
MLCOURSE.AI : Open Machine Learning course by OpenDataScience
Elements of AI : A free course for AI basics by Reaktor and University of Helsinki
Machine Learning Mastery : A comprehensive blog that contains guidance, tutorials, and e-book for mastering ML
Machine Learning Crash Course : A jumpstart AI course from Google
Machine Learning Tutorial: A Step-by-Step Guide for Beginners : A one-stop playlist for all the basics of Machine Learning simplified, from Logistic Regression to Reinforcement Learning.
MLU-EXPLAIN : Machine Learning University (MLU) is an education initiative from Amazon designed for visual explanations of core machine learning concepts.

↥ Back To Top

📢 Seminar, research writing, talks, etc

Advice on Research and Writing : A collection of advice about how to do research and how to communicate effectively (primarily for computer scientists).
PHD MS Articles : articles and views
Seminar and reports : Everyone must read this tiny book before writing the seminar report
Latex reference : Arbitrary reference
Begin Latex in minutes : Brief Intro to LaTeX for beginners that helps you use LaTeX with the ease
Lshort : The Not So Short Introduction to LATEX2ε

↥ Back To Top

📦 Everything in one place

AI Tools : Explore a curated selection of AI tools and resources
API Documentation : A one-place well-known API Documentation with a searchable interface
Baeldung : Step-by-step guides for Spring, rest, Java, security, persistence, Jackson, HTTP client-side and Kotlin
BtechBasics : Fundamental concepts of Computer Science Engineering using hands-on exercises
Branition Colors : Collection of hand-curated color palettes best fitted for branding.
cheat.sh : curl cheat.sh — the only cheat sheet you need — instant answers on programming questions with curl
Developer Roadmaps : Step by step guides and paths to learn different tools or technologies
DevURLs : Developer news aggregator
Kaggle : All-in-one Machine Learning and Data Science Community – access free GPUs and a huge repository of community published data & code.
MDN Web Docs : A place with all the documentation of the web standards
Rico's cheatsheets : A set of good cheatsheets
Programming Subreddits : a multisubreddit of all subreddits of topics related to computer science and programming.
Websites a programmer should visit : Response on Quora by ashish2199
gitignore : A collection of useful .gitignore templates for your project. Select from 442 Operating System, IDE, and Programming Language
Hidden Tools : Discover a wide collection of tools made by the community - for you. ✨
Coolors : Create the perfect palette or get inspired by thousands of beautiful color schemes.
Tailwind CSS Page Builder : The perfect Tailwind CSS Page Builder
LottieFiles : The world’s largest online platform for the world’s smallest animation format for designers, developers, and more. Access Lottie animation tools and plugins for Android, iOS, and Web.
UI Design Daily :Weekly FREE UI resources straight to your inbox
Iconscout : Over 2.2 Million+ Design Assets, Curated SVGs, Vector Icons, Illustrations, 3D graphics, and Lottie Animations. Over 3000+ assets added every day. Integrated plugins, tools, editors, and more.
Json API App : Single repository for everything you need to build better products as a developer. API, DB, Queue, Server, Webhooks, Bin, Tools, Podcasts etc. Everything you need to build super apps that our world needs.
Library or micro code solutions : Community library of micro code pieces for popular issues.

↥ Back To Top

📺 YouTube Channels

C++Now (BoostCon) : C++Now (previously BoostCon) conference
code::dive conference : code::dive conference organized by NOKIA Wrocław Technology Center
Coding Blocks : Tutorials, how to's, tips and tricks
Computerphile : Must watch for every CS student
ComputerHistory : for those who like to know how we reached where we are.
CppCon : C++ Conference
Facebook Developers
Google Developers
GoogleTechTalks : videos on trending topics and cool stuff happening in the tech industry.
Gynvael Coldwin : Awesome reverse engineering and hacking(CTF) videocasts. Every Wednesday is new live streams.
HowToBecomeTV : contains good interviews of developers and people related to the tech industry.
Java : talks related to java
JavaOne : Java Conference
javidx9 : Game and graphics tutorials
Meeting C++ YT Kanalseite : Talks on C++
MIT OpenCourseWare : MIT OpenCourseWare for learning in-depth algorithms, data structures, and computer engineering
Murtaza's OpenCV Robotics and AI: OpenCV, Self Driving, Robotics and AI tutorials.
Netflix UI Engineering : great videos to watch for web developers, mobile developers and those interested in some of Netflix's tech stack
O'Reilly : interviews and talks of the world's best technical writers.
Placement Grid : Interview and campus placement experience
Scott Meyers: Past Talks
Siraj Raval : Artificial Intelligence and deep learning tutorials videos
ThinMatrix : blogs and tutorials developer making a 3d game in Java using OpenGL
thoughtbot : talks on various topics
Traversy Media :Web development and programming
yegor256
GOTO Conference : tech talks from the GOTO Conference by Developers for Developers
freeCodeCamp : freecodecamp youtube channel
Bo Qian : Learn advanced c++
Geeksforgeeks : geeksforgeeks youtube
Hacker earth : Hacker earth youtube
Hak5 : Put together by a band of IT ninjas, security professionals, and hardcore gamers, Hak5 isn't your typical tech show. We take on hacking in the old-school sense.
Khan Academy : Khan Academy youtube
LearnCode.academy : 100% FREE Web Development tutorials, web site design tutorials, and more. Including, but not limited to: HTML, CSS, JavaScript, CSS Layouts, Responsive Design, React.js, Node.js, Angular.js, Docker, Dev
Rachit Jain : competitive programming
sentdex : Python Programming tutorials, going further than just the basics. Learn about machine learning, finance, data analysis, robotics, web development, game development, and more.
Steve Griffith : The videos on this channel are largely about web design & development with a good dose of mobile development thrown in just for fun.
The Coding Train : In this YouTube channel I publish "creative coding" video tutorials every week. Subjects covered range from the basics of programming languages like JavaScript (with p5.js) and Java (with Processing) to generative algorithms like physics simulation, computer vision, and data visualization.
TheHappieCat : Videos on technology, storytelling, and being happie.
Tutorials Point (India) Pvt. Ltd. : Tutorials Point originated from the idea that there exists a class of readers who respond better to online content and prefer to learn new skills at their own pace from the comforts of their drawing rooms. We bring simple to understand Text and Video tutorials at zero cost and cover many subjects including IT, Engineering, MBA, MCA, Management, Various Exams Preparation, and Academic subjects. If you are not extravagant and willing to save your money and time then we recommend you to try our tutorials.experience
Udacity : Udacity, a pioneer in online education, is building "University by Silicon Valley", a new type of online university
V. Anton Spraul : Videos for people who want to understand the software better, including explanations of how the software performs some of its most important jobs and tools programmers can use to solve programming problems.
xdadevelopers : You have found the world-famous XDA-Developers.com YouTube Channel, known as XDA TV. We cover Android and Windows Mobile from HTC, Samsung, Motorola, Sony, and more. Learn about rooting your Android, how to build applications, and more.
KudVenkat : Microsoft .NET technologies and SQL Server tutorials.
The Net Ninja: Web development tutorials
Web Dev Simplified: Tutorials based on Web development from basic to advanced.
Take You Forward: Tutorials on Data structures and Algorithms in Java and C++.

↥ Back To Top

✍️ Good Articles

14 Things I Wish I’d Known When Starting with MongoDB
40 Keys Computer Science Concepts Explained In Layman’s Terms
A Gentle Introduction To Graph Theory
A programmer-friendly language that compiles to Lua.
A Software Developer’s Reading List : Some good books and links in there.
Code a TCP/IP stack : Let's code a TCP/IP stack, 5: TCP Retransmission
Codewords.recurse : The language of choice
Learn Data structure and Algorithms : List of some algorithms and data structures and learning resources.
Dive into the byte code
Expectations of a Junior Developer
Getting Started with MongoDB – An Introduction
Linux Inside
List of algorithms
Step by Step Guide to Database Normalization: A guide to database normalization.
The Key To Accelerating Your Coding Skills
Unicode
We are reinventing the retail industry through innovative technology
What every programmer absolutely, positively needs to know about encodings and character sets to work with text
What every programmer should know about memory - PDF
qotoqot - improving-focus : How I got to 200 productive hours a month
Pixel Beat - Unix : Parallel processing with Unix tools
Learning Vim : What I Wish I Knew
Write a Kernel : Kernel 101 – Let’s write a Kernel
Learning JavaScript Design Patterns : the online version of the Learning JavaScript Design Patterns published by O'Reilly, released by the author Addy Osmani under CC BY-NC-ND 3.0
Working with Webhooks : a comprehensive guide on webhooks
How I got TensorFlow Developer Certified : Step By Step guide to pass Tensorflow Developer Certification

↥ Back To Top

🎧 Podcasts

Coding Blocks : A podcast covering topics such as best programming practices, design patterns, coding for performance, object-oriented coding, database design and implementation, tips, tricks and a whole lot of other things.
Developer On Fire : A podcast that shares the humanity of developers and tells stories of some of the amazing people in software, hosted by Dave Rael.
Developer Tea : A podcast for developers designed to fit inside your tea break.
Front End Happy Hour : A podcast featuring a panel of Software Engineers from Netflix, Evernote, Atlassian & LinkedIn talking over drinks about all things Front End development.
Full Stack Radio : Everything from product design and user experience to unit testing and system administration.
Groovy Podcast : A podcast dedicated to the Groovy programming language and its ecosystem.
IPhreaks : A weekly group discussion about iOS development and related technology by development veterans. We discuss Apple, tools, practices, and code.
JavaScript Jabber : A weekly discussion about JavaScript, front-end development, community, careers, and frameworks.
Learn To Code With Me Podcast : A Season by season of tech podcast episodes by Laurence Bradford with topics ranging from Career in Tech to lessons in doing tech business
LispCast : A podcast by Eric Normand, a functional programming expert talking about FP concepts.
MS Dev Show : Jason Young and Carl Schweitzer talk about the latest in developer news covering topics such as the Azure cloud, Windows, Windows Phone, Visual Studio, and cross-platform development using the Microsoft platform.
React Native Radio : A weekly discussion of the tools, techniques, and technologies used to build mobile applications with JavaScript and React.
ShopTalk Show : A weekly podcast about just building websites from Dave Rupert and Chris Coyier.
Soft Skills Engineering : A weekly advice podcast for software developers about non-technical topics.
Software Engineering Daily : A daily technical interview about software topics.
Software Engineering Radio : A podcast targeted at the professional software developer. The goal is to be a lasting educational resource, not a newscast.
Syntax : A Tasty Treats Podcast for Web Developers by Wes Bos & Scott Tolinski.
The Bike Shed : Guests discuss their development experience and challenges with Ruby, Rails, JavaScript, and others.
The Changelog : A weekly conversation that gets to the heart of open source technologies and the people who create them.
The Cynical Developer : A podcast that aims to help you to improve your development knowledge and career, through explaining the latest and greatest in development technology and providing you with what you need to succeed as a developer. Covering Desktop, web, and mobile development, mainly around the .Net Stack but often looking into other software and frameworks.
The Real Python Podcast : A weekly Python podcast hosted by Christopher Bailey with interviews, coding tips, and conversation with guests from the Python community.
Blockchain Insider by 11:FS : Podcast to learn about the Blockchain Technology
Unchained Podcast to learn about the Blockchain Technology
Talk python to me Podcast to learn about Python through interviews and discussions
Python bytes Podcast to learn about the latest happenings and trends in Python

↥ Back To Top

🔄 Building a Simple Compiler/Interpreter

⛄ Possibly the smallest compiler ever : This is an ultra-simplified example of all the major pieces of a modern compiler written in easy to read JavaScript.
Awesome Compilers : Curated list of awesome resources on Compilers, Interpreters, and Runtimes.
Growing a compiler : Learn how to grow a compiler
Let’s Build A Simple Interpreter. Part 1. : Try to demystify compilers by building one
Resources for Amateur Compiler Writers : Resources for Amateur Compiler Writers
Structure and Interpretation of Computer Programs : Structure and Interpretation of Computer Programs
Writing My First Compiler : Write out your first compiler
An Intro to Compilers : How to Speak to Computers, Pre-Siri
Write your own compiler : How to write your compiler
Crafting Interpreters A handbook for writing interpreters, first implementing a tree walking interpreter and later a bytecode virtual machine
Writing a C Compiler : The first post in a series by Nora Sandler on writing your own C compiler

↥ Back To Top

🧑‍🏫 Tutorials

A Hacker's Guide to Git : for those wanting to learn git with a solid foundation
A Byte of Python : a free beginner introduction to python
Best Of - Gustavo Duarte : contains articles on various topics
CMSI 281: Data Structures : lightweight introduction to DS
Collecting all the cheat sheets : cheat sheets for lots of programming languages
C Programming
CryptoHack : Learn cryptography through challenges and tutorials. Has a leaderboard and new challenges are added every few months.
Programming Community Curated C++ Resources : resources recommended by developers
Deep C : very good presentation on C language
Design Patterns: Elements of Reusable Object-Oriented Software : aka the "Gang Of Four" book, or GOF
Dynamic programming - PrismoSkills : very good resource if want to learn how to solve DP problems.
Flexbox Froggy : a game that teaches you how to use CSS flexbox properties
Git from the inside out
Head First Design Patterns
How to Program in C++ : Good resource for revising C++ topics and STL
http://www.mysqltutorial.org/
indradhanush tutotials : Writing a Unix Shell
Introduction to C Programming
Learn UNIX in 10 minutes
Learning the shell.
Linux Journey : good site for learning Linux
Linux Tutorial : good resource for learning Linux
Missing Semester : The missing semester of your computer science education
More about Github-flavored markdown
MySQL Essentials
Open Data Structures : Excellent resource for learning about DS and algos, provides code in various languages C++, Java, and pseudocode.
OS Course Notes : Chapter-wise course notes according to Galvin's book
Programming, Web Development, and DevOps news, tutorials, and tools for beginners to experts
Prompt engineering : Prompting Guide AI is an online resource that helps users learn and craft effective prompts for AI models to generate better and more relevant outputs.
Stanford Programming Course : Full, free CS course created by Stanford
SQL (Structured Query Language) in one page : SQL.SU : a very good SQL cheat sheet
Subtle/Poor Man's CI : Learn how continuous integration platforms work under the hood, by building one of your own on top of git with Node.js
TCP/IP Illustrated Series
The Bash Guide : a very good guide for learning the Bash Shell
The Descent to C : for those moving to C from some higher programming language like java or python.
The Linux Command Line: A Complete Introduction
The Unix Programming Environment
TopCoder Tutorials
Tutorialspoint : Text and Video Tutorials for UPSC, IAS, PCS, Civil Services, Banking, Aptitude, Questions, Answers, Explanation, Interview, Entrance, Exams, Solutions
UNIX and Linux System Administration Handbook, 4th Edition
VimTutor+ : Learn VIM from the browser.
W3Schools Online Web Tutorials
Unix Shell : Unix shell scripting with ksh/bash
Snap SVG : The JavaScript SVG library for the modern web
vim.rtorr : Vim Cheat Sheet
Open Vim : Interactive Vim tutorials
Algorithm Using Dynamic Programming and A : Designing a Tree Diff Algorithm Using Dynamic Programming and A*
Learn Python : Free Interactive Python Tutorial
C++17 : A guide of C++17
The Bash Academy : The Bash Academy is an initiative to promote the bash shell language and educate people on its use.
Learn Shell Programming : This website is intended for everyone who wishes to learn programming with Unix/Linux shell interpreters.
Java tutorial : A programming community & a great place to find the best online programming courses and tutorials.
Explain Shell: Match command-line arguments to their help text
Speaking io : Tips for public speaking
Javascript.info : The Modern JavaScript Tutorial

↥ Back To Top

👀 Watch others code

Education Ecosystem : screencast of people building applications, websites, games, etc.
Twitch.tv : The programming community of twitch.

↥ Back To Top

🧠 What should a programmer know

Can I use : A website that provides up-to-date browser support tables for support of front-end web technologies on desktop and mobile web browsers.
GitHub.com Build software better, together : Place to showcase your project and collaborate with others. (Must know Git to use it effectively)
GitLab : An alternative to GitHub that offers free unlimited (private) repositories and unlimited collaborators.
Programmer Competency Matrix : article for knowing what our level as a programmer is.

↥ Back To Top

⚔️ Competitive programming

Topic Wise Problem For Competitive Programmer : Topic wise Practise Problem
Advent of Code : An Advent calendar of small programming puzzles
Archived Problems - Project Euler : Problems Archives
Art of Problem Solving : Is math class too easy for you? You've come to the right place!
AtCoder : One of the best places to build competitive programming skills for beginners to experts.
CodeChef : The only programming contests Web 2.0 platform
CodeSignal : Test your coding skills
CodeEval : Notepad for notes and code snippets, stored locally in the browser
Codeforces : Programming Competition,Programming Contest,Online Computer Programming
Codewars : Rank up by completing code kata
Codility : Verify and improve coding skills
Codingame : Learn coding through games and challenges!
Facebook Hacker Cup : Facebook's Programming Contest, past problems solutions and FAQ
Google Coding Competitions Archive : past contest problems for practice
HackerEarth - Programming challenges and Developer jobs
HackerRank : Practice coding. Compete. Find jobs.
LightOJ : Practicing at lightoj is so good for beginners as it is categorized and have also chat room and forum which helps to communicate with others about any problem.
PKU ACM ICPC Practice problems : Judge online for ACMACPC
Sphere Online Judge (SPOJ) : Become a true programming master Learn how to code and build efficient algorithms
Topcoder : Deliver Faster through Crowdsourcing
URI Online Judge : Practice coding, Compete and be a better coder.
UVa Online Judge : hundreds of problems supporting multiple languages.
WakaTime : leaderboards of coding metrics collected via editor plugins
PrepBytes : Topic and level wise proper arrange problems
A2OJ Ladders : Practice codeforces problems based on your proficiency and difficulty

↥ Back To Top

📖 Computer Books

Become a Programmer, Motherfucker (list of books) : Exhaustive list of books from Zed A. Shaw.
Best books for GATE CSE
cses.fi/book.html
github.com/vhf/free-programming-books : More than 500 free ebooks on almost any language you can think of
GitBook : GitBook helps your team write, collaborate, and publish content online.
Data Science course : Python Data Science Handbook
Goal Kicker : Programming Notes for Professionals books
The GraphQL Guide : The complete guide to GraphQL, the new REST ✨
Eloquent JavaScript : A book about JavaScript, programming, and the wonders of the digital.
programmingbooks.dev : An Ordered and Curated Reading List for Software Craftsmanship Growth.

↥ Back To Top

🔴 Video Tutorials

Aditya Verma: Algorithm tutorials playlists by an Indian youtuber Aditya verma.
codedamn : front end web dev tutorials
Code School : A PluralSight Company and an Interactive learning destination for aspiring and experienced Developers
CodingMadeEasy : C++ tutorials
CS1: Higher Computing - Richard Buckland UNSW : a very good introductory CS course
Derek Banas : good quality tutorials
Design and Analysis of Algorithms
DevTips : web dev tutorials
FreeCourses : Free courses about programming
Kathryn Hodge : Has good videos for beginners
Kunal Kushwaha : An amazing guide to Git and GitHub for beginners
mycodeschool : Data structures and algorithms tutorials
Pluralsight : Learn Software Development, DevOps and Data Science through multiple short courses
thenewboston : good but with too much talk as compared to actual content
Tushar Roy : Algorithm and Data structure tutorial by an Indian Youtuber.
Vim Tutorial Videos - Flarfnoogins : good video tutorial for learning vim
XDA-University - Helping You Learn Android Development
Khan Academy : learn about computer science for free
Functional programming : John Carmack on Functional Programming (2013)
Video about vims : A serie of tutorials about Vim
Mastering Next.js : A free serie of videos to learn Next.js

↥ Back To Top

💻 Online Compiler and Sharing Code snippets

C9.io : Your development environment, in the cloud
Carbon : Create pretty looking images of your code snippets.
Codeframe : Online pastebin-like playground for building websites using HTML, CSS, and JavaScript
CodePad : Code editor to try, test and run 25+ languages
Coder : A Web-based development environment using Visual Studio Code as a code editor
Codesandbox.io : CodeSandbox makes it easier to create, share, and reuse React projects with others.
Github Codespaces : Integrated cloud-based IDE directly to your browser.
Github Gist : Instantly share code, notes, and snippets.
Godbolt.org : Excellent tool for exploring the assembly output of different compilers with and without optimization.
Ideone.com : online compiler and debugging tool for more than 60 programming languages
JSFiddle : Test your JavaScript, CSS, HTML or CoffeeScript with online code editor
JSBin : Front end playground, Output is not framed, so it allows you to share those snippets that will break inside an iframe.
Judge0 IDE : Online compiler with 40+ interpreters and compilers.
Pastebin.com : Pastebin can store texts like code, notes, and snippets online for a set time which can be shared instantly.
PlayCode : Online Javascript playground with a built-in console and support for npm packages.
RunJS : Online JavaScript playground with instant live feedback
StackBlitz : Instant Dev environments with support of nodejs and npm packages.
Wandbox: Online compiler with bleeding edge C++ and 40 other languages.
PHPize.online: Online PHP compiler with SQL support.

↥ Back To Top

📝 Blogs of Developers

Algo-Geeks : Programming Puzzles, Math Tricks, Algorithms, etc
Amit Merchant : Tutorials, tips & tricks, and rants about programming and design.
Andy Heathershaw : Personal website and blog of software developer Andy Heathershaw
Antonio081014's Algorithms Codes : The world is under the RULE.
Archives — Ask a Manager : HR-related stuff
Armin Ronacher's Thoughts and Writings : blog on Python and open source
blog.might.net : the blog of might dot net
Brendon Gregg - Linux Kernel Dev : the blog of Brendon D. Gregg
Clean Coder Blog : a blog of the author of the book "Clean Code"
CodeAhoy : Blog on software and human factors. 100% Tested on Humans.
CoderGears Blog Insights from : the CoderGears Team
Coding Geek - A blog about IT, programming and Java : A blog about IT, programming and Java
Coding Horror : one the best coding blog
CSE Blog : quant, math, computer science puzzles
CSS Tricks : about building websites and all that entails, mostly from a front-end perspective
Daedtech.com : Stories about software
Dan Dreams of Coding
Daniel Lemire's Blog : Daniel Lemire's blog
Eli Bendersky : everything from Python to LLVM
Geek Land : My precious collectibles
HackerEarth Blog : The HackerEarth blog
IT Enthusiast : IT Enthusiast
Joel on Software : The blog of the CEO of StackOverflow
Late Developer : Random thoughts of an old C++ guy
Linux Forums : A Friendly Linux Forum
1ucasvb's laboriginal math and physics visualization : Lucas Vieira Barbosa's lab original math and physics visualization
Math ∩ Programming : Math ∩ Programming
My Tech Interviews : PREPARE FOR A TECHNICAL INTERVIEW
Paul Graham Essays : Paul Grahan Essays
Programming Blog : programming blog of Yegor Bugayenko
Programming in the 21st Century : programming in the twenty-first century
rudhakar Rayavaram : Sudhakar Rayavaram Blog's
Runhe Tian Coding Practice : Technical interview questions from Apple, Google, Facebook, Amazon, and Microsoft
Small Programming Challenges and Puzzles : Project Nayuki
stevehanov.ca : I know how to make and sell software online, and I can share my tips with you.
Takipi Blog : mainly focuses on Java and JVM languages
The Pragmatic Engineer : Software. People. Problems. Ideas. Engineering.
XDA - Android Developer Forum : Android Open Source Developers Forum
The Net Ninja: Web development tutorials

↥ Back To Top

🗣️ For improving your English

Englishclub.com/learn-english
Guide to Grammar and Writing : for those who want to improve their English language skills
Punctuation and Capitalization Rules
Purdue University Online Writing Lab (OWL)
Quia - English
AntiMoon Immersion Approach: Immersion-based learning of English, can be used by people on different levels.

↥ Back To Top

🧘‍♂️ When you get bored from CS related stuff

Barcroft TV : Daily short documentaries about the incredible variety of people that make up the world
Big Think : Expert driven, actionable, educational content, featuring experts ranging from Bill Clinton to Bill Nye
ColdFusion : Past, present, and future of technology
CrashCourse : small courses on various subjects
Every Frame a Painting : High-quality analysis of films and filmmaking
National Geographic : High volume of high-quality content from all over the world
r/ProgrammerHumor : Subreddit dedicated to exactly what it sounds like
Reddit the front page of the internet : Where free time goes to die
Ridddle : A youtube channel about science, mainly, but not only, the universe and space.
Regex Crossword : A simple crossword game where clues are regex that must be matched.
SciShow : Answers to interesting questions that you've always wondered about
SmarterEveryDay : Lots of amazing scientific information about the world around us, usually captured with a high-speed camera
TED : Great talks about technology, entertainment, and design
TestTube News : Interesting information about news from around the world
How to live for a long time ? : The secret of longevity
Vsauce : The best youtube channel
Ox A cuk : How to live a happy life
lonelyspeck : Expose to the Right for Astrophotography in Light Pollution – Palos Verdes, Los Angeles, California

↥ Back To Top

🔓 Open Source Websites

N-O-D-E : Everything open-source and hacker culture - news, zines, and projects
Open Hatch : OpenHatch is a non-profit dedicated to matching prospective free software contributors with communities, tools, and education.
Source Forge : SourceForge hosts nearly 280,000 projects (at last count). It serves more than 2 million downloads a day and includes apps and tools in a wide variety of categories.
Google Code : Google offers free hosting for open source projects using the Subversion or Mercurial version control systems. It offers 2 GB of storage, integrated code review tools, a wiki, and an issue tracker. The Google Code site also provides links to Google's many publicly available APIs and other developer tools.
Launch Pad : Maintained by Canonical, LaunchPad is particularly targeted at projects that run on Ubuntu. It provides hosting for more than 21,000 projects that use the Bazaar version control system.
Google Open Source : Google Open Source
Red Hat Developer : The world's leading provider of open source solutions
Open Source : Open Source
Google Summer of Code : Google Summer of Code is a global program focused on bringing more student developers into open source software development. Students work with an open-source organization on a 3-month programming project during their break from school.
Open Source Web Design : Open Source Web Design is a platform for sharing standards-compliant free web design templates. We give web publishers a voice through good design.
Mozilla Winter of Security : The Winter of Security (MWOS) is a program organized by Mozilla's Security teams to involve students with Security projects. Students who have to perform a semester project as part of their university curriculum can apply to one of the MWOS projects.
Bit Bucket : Like GitHub, BitBucket hosts both public and private projects. On this site, open-source projects and private projects with fewer than five users are free. It hosts more than 48,000 repositories, many of which are searchable on the site.
Media Wiki : MediaWiki is a free software open source wiki package written in PHP, originally for use on Wikipedia. It is now also used by several other projects of the non-profit Wikimedia Foundation and by many other wikis, including this website, the home of MediaWiki.
Code Curiosity : CodeCuriosity is a platform that encourages contributions to open source. Everyone is rewarded for their efforts, no matter how big or small they are.
Code Triage : Help out your favorite open-source projects and become a better developer while doing it.
Issue Hub : Contribute to Open Source. Search issue labels to find the right project for you
Up for Grabs : This is a list of projects which have curated tasks specifically for new contributors. These are a great way to get started with a project or to help share the load of working on open source projects.
First Timers Only : Contributing to open source for the first time can be scary and a little overwhelming. Perhaps you’re a Code Newbie or maybe you’ve been coding for a while but haven’t found a project you felt comfortable contributing to.
Your First PR : Your First PR helps you get started contributing to Open Source by showcasing great starter issues on GitHub and elsewhere.
Awesome First PR Opportunities : An awesome repository for finding beginner-friendly projects in different programming languages.
EddieHub Open source community : A Supportive community for people who are interested or already contributing in Open source.
MLH Fellowship prgramme:A fully remote, 12-week internship alternative where participants earn a stipend and learn to collaborate on real open source projects with peers and engineers from top companies.

↥ Back To Top

🌱 Internships

Chegg : It is an awesome resource for finding internships, scholarships, tutors, etc.
Internshala : You can search for internships here according to your skill sets for your interested location. It also helps you in getting a good PPO offer from the company.
Letsintern : Get a smart and challenging internship for you from the LetsIntern.
PerfectIntern: Get help finding a paid internship, resume prep, interview prep, and more!

↥ Back To Top

💼 Jobs

Wellfound : Wellfound (formerly known as Angellist) is a website for startups, angel investors, and job-seekers looking to work at startups.
CareerBuilder : CareerBuilder is one of the largest job boards, providing job listings, resume posting, and career advice and resources to job seekers.
Dice : Dice is the leading site for tech job seekers. You can search by company, job title, keyword, employment type, and location.
Devsnap : Devsnap is a job aggregator for developer jobs.
DevITjobs UK : Transparent job board for UK based developers
Glassdoor : Find the job that's right for you. Search all the open positions on the web. Get your own personalized salary estimate.
IAESTE : Find more and amazing international internships and jobs.
Indeed : Find international jobs on Indeed and get a market salary for your dedication and devotion.
Instahyre : Your dream job is here. Showcase yourself to a curated list of top companies. Complete privacy and no spam.
JobsCollider : Tens of thousands of remote jobs from over 10,000 companies and startups worldwide.
Mentat : Get your dream job 10x faster. Never apply for a job ever again, talk directly to decision-makers and hiring managers.
The Muse : Find everything you need to succeed from dream jobs to career advice. You can do a lot here on The Muse like exploring companies, browsing jobs, career advice, discover careers, career coaching. Try it.
Paysa : Paysa helps you in finding new and interesting jobs according to your wish.
SimplyHired : Simply Hired is a free job search engine (and mobile app) that takes the hassle out of getting hired and provides you with all the information you need to make a sound career move.
SwissDev Jobs : Tech job board for Software Engineers that want to work in Switzerland.
Undercover Recruiter : Become Recruiter, Candidate or Employer. You can get your job easily here.
Who is Hiring : An awesome resource for searching, filtering, and finding new and attractive jobs according to your needs and interests.
ZipRecruiter : The Smartest Way to Get Hired.
RemoteML : Remote Machine Learning jobs.
Linkedin jobs : A very nice research tool for programming jobs
GermanTech Jobs : Dedicated job board for tech roles in Germany - with salary brackets
We Work Remotely : The largest remote work community in the world.
OkJob : 4 day week job board.

↥ Back To Top

🌟 Special Thanks

Please consider a GitHub star if you find this useful and/or consider contributing.
A special thanks to Ashish Padalkar (@ashish2199) for contributing a great amount of data and structure to the initial repository Original Post.

Guovin/iptv-api

2024-12-08T01:36:42Z

📺IPTV电视直播源更新工具🚀：✨央视、📡卫视、☘️广东及各省份地方台、🌊港·澳·台、🎬电影、🎥咪咕、🏀体育、🪁动画、🎮游戏、🎵音乐、🏛经典剧场；支持IPv4/IPv6；支持自定义增加频道；支持组播源、酒店源、订阅源、关键字搜索；每天自动更新两次，结果可用于TVBox等播放软件；支持工作流、Docker(amd64/arm64/arm v7)、命令行、GUI运行方式 | IPTV live TV source update tool

IPTV-API

自定义频道，自动获取直播源接口，测速验效后生成可用的结果

默认结果包含：📺央视频道、💰央视付费频道、📡卫视频道、🏠广东频道、🌊港·澳·台频道、🎬电影频道、🎥咪咕直播、🏀体育频道、🪁动画频道、🎮游戏频道、🎵音乐频道、🏛经典剧场

具体频道

📺央视频道: CCTV-1, CCTV-2, CCTV-3, CCTV-4, CCTV-5, CCTV-5+, CCTV-6, CCTV-7, CCTV-8, CCTV-9, CCTV-10, CCTV-11, CCTV-12, CCTV-13, CCTV-14, CCTV-15, CCTV-16, CCTV-17, CETV1, CETV2, CETV4, CETV5

💰央视付费频道: 文化精品, 央视台球, 风云音乐, 第一剧场, 风云剧场, 怀旧剧场, 女性时尚, 高尔夫网球, 风云足球, 电视指南, 世界地理, 兵器科技

📡卫视频道: 广东卫视, 香港卫视, 浙江卫视, 湖南卫视, 北京卫视, 湖北卫视, 黑龙江卫视, 安徽卫视, 重庆卫视, 东方卫视, 东南卫视, 甘肃卫视, 广西卫视, 贵州卫视, 海南卫视, 河北卫视, 河南卫视, 吉林卫视, 江苏卫视, 江西卫视, 辽宁卫视, 内蒙古卫视, 宁夏卫视, 青海卫视, 山东卫视, 山西卫视, 陕西卫视, 四川卫视, 深圳卫视, 三沙卫视, 天津卫视, 西藏卫视, 新疆卫视, 云南卫视

☘️广东频道: 广东珠江, 广东体育, 广东新闻, 广东民生, 广东卫视, 大湾区卫视, 广州综合, 广州影视, 广州竞赛, 江门综合, 江门侨乡生活, 佛山综合, 深圳卫视, 汕头综合, 汕头经济, 汕头文旅, 茂名综合, 茂名公共

☘️各省份地方台

🌊港·澳·台: 翡翠台, 明珠台, 凤凰中文, 凤凰资讯, 凤凰香港, 凤凰卫视, TVBS亚洲, 香港卫视, 纬来体育, 纬来育乐, J2, Viutv, 三立台湾, 无线新闻, 三立新闻, 东森综合, 东森超视, 东森电影, Now剧集, Now华剧, 靖天资讯, 星卫娱乐, 卫视卡式

🎬电影频道: CHC家庭影院, CHC动作电影, CHC高清电影, 淘剧场, 淘娱乐, 淘电影, NewTV惊悚悬疑, NewTV动作电影, 黑莓电影, 纬来电影, 靖天映画, 靖天戏剧, 星卫娱乐, 艾尔达娱乐, 经典电影, IPTV经典电影, 天映经典, 无线星河, 星空卫视, 私人影院, 东森电影, 龙祥电影, 东森洋片, 东森超视

🎥咪咕直播: 咪咕直播1-45

🏀体育频道: CCTV-5, CCTV-5+, 广东体育, 纬来体育, 五星体育, 体育赛事, 劲爆体育, 爱体育, 超级体育, 精品体育, 广州竞赛, 深圳体育, 福建体育, 辽宁体育, 山东体育, 成都体育, 天津体育, 江苏体育, 安徽综艺体育, 吉林篮球, 睛彩篮球, 睛彩羽毛球, 睛彩广场舞, 风云足球, 足球频道, 魅力足球, 天元围棋, 快乐垂钓, JJ斗地主

🪁动画频道: 少儿动画, 卡酷动画, 动漫秀场, 新动漫, 青春动漫, 爱动漫, 中录动漫, 宝宝动画, CN卡通, 优漫卡通, 金鹰卡通, 睛彩少儿, 黑莓动画, 炫动卡通, 24H国漫热播, 浙江少儿, 河北少儿科教, 七龙珠, 火影忍者, 海绵宝宝, 中华小当家, 斗破苍穹玄幻剧, 猫和老鼠, 经典动漫, 蜡笔小新, 漫画解说

🎮游戏频道: 游戏风云, 游戏竞技, 电竞游戏, 海看电竞, 电竞天堂, 爱电竞

🎵音乐频道: CCTV-15, 风云音乐, 音乐现场, 音乐之声, 潮流音乐, 天津音乐, 音乐广播, 音乐调频广播

🏛经典剧场: 笑傲江湖, 天龙八部, 鹿鼎记, 仙剑奇侠传, 西游记, 三国演义, 水浒传, 新白娘子传奇, 天龙八部, 济公游记, 封神榜, 闯关东, 上海滩, 射雕英雄传

English | 中文

特点

✅ 自定义模板，生成您想要的频道
✅ 支持多种获取源方式：组播源、酒店源、订阅源、关键字搜索
✅ 接口测速验效，响应时间、分辨率优先级，过滤无效接口
✅ 偏好设置：IPv6、接口来源排序优先级与数量配置、接口白名单
✅ 定时执行，北京时间每日 6:00 与 18:00 执行更新
✅ 支持多种运行方式：工作流、命令行、GUI 软件、Docker(amd64/arm64/arm v7)
✨ 更多功能请见配置参数

配置

配置参数

快速上手

方式一：工作流

Fork 本项目并开启工作流更新，具体步骤请见详细教程

方式二：命令行

pip install pipenv

pipenv install --dev

启动更新：

pipenv run dev

启动服务：

pipenv run service

方式三：GUI 软件

下载IPTV-API 更新软件，打开软件，点击更新，即可完成更新
或者在项目目录下运行以下命令，即可打开 GUI 软件：

pipenv run ui

方式四：Docker

iptv-api（完整版本）：性能要求较高，更新速度较慢，稳定性、成功率高；修改配置 open_driver = False 可切换到 Lite 版本运行模式（推荐酒店源、组播源、关键字搜索使用此版本）
iptv-api:lite（精简版本）：轻量级，性能要求低，更新速度快，稳定性不确定（推荐订阅源使用此版本）

拉取镜像：

iptv-api：

docker pull guovern/iptv-api:latest

iptv-api:lite：

docker pull guovern/iptv-api:lite

运行容器：

iptv-api：

docker run -d -p 8000:8000 guovern/iptv-api

iptv-api:lite：

docker run -d -p 8000:8000 guovern/iptv-api:lite

卷挂载参数（可选）：实现宿主机文件与容器文件同步，修改模板、配置、获取更新结果文件可直接在宿主机文件夹下操作

以宿主机路径/etc/docker 为例：

iptv-api：

docker run -v /etc/docker/config:/iptv-api/config -v /etc/docker/output:/iptv-api/output -d -p 8000:8000 guovern/iptv-api

iptv-api:lite：

docker run -v /etc/docker/config:/iptv-api-lite/config -v /etc/docker/output:/iptv-api-lite/output -d -p 8000:8000 guovern/iptv-api:lite

更新结果：

接口地址：ip:8000
M3u 接口：ip:8000/m3u
Txt 接口：ip:8000/txt
接口内容：ip:8000/content
测速日志：ip:8000/log

更新日志

赞赏

开发维护不易，请我喝杯咖啡☕️吧~

支付宝	微信

关注

微信公众号搜索 Govin，或扫码，接收更新推送、学习更多使用技巧：

免责声明

本项目仅供学习交流用途，接口数据均来源于网络，如有侵权，请联系删除

许可证

PSAppDeployToolkit/PSAppDeployToolkit

2024-12-07T01:29:27Z

Project Homepage & Forums

Enterprise App Deployment, Simplified.

PSAppDeployToolkit is a versatile solution that streamlines and standardizes the process of software deployment, making it easier than ever to manage your IT environment. It has a comprehensive set of features, such as dynamic logging, user interaction capabilities, and customizable functionality.

Standardize and enhance every software deployment

PSAppDeployToolkit allows you to encapsulate a typical Windows Installer MSI or Setup executable to provide it with enhanced capabilities.

Validate prerequisites such as dependencies on minimum software versions
Ensure that in-use applications are closed and prevent reopening during the deployment
Check with the user if now is a good time to start an install and allow them to defer
Uninstall existing applications and perform clean up operations
Capture any important settings that may be required for an upgrade or migration
Run the installation silently and capture logs in the event of an issue
Run post-installation configuration tasks to customize for your environment
Prompt the user to restart their computer if required, immediately, on a timer and with a deadline

Getting Started

-> System Requirements -> Downloading

PSAppDeployToolkit Links

-> Homepage -> Documentation -> Function & Variable References -> Download Latest Release -> News

Community Links

-> Discourse Forum -> Discord Chat -> Reddit

License

The PowerShell App Deployment Tool is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

nuke-build/nuke

2024-12-08T01:37:28Z

🏗 The AKEless Build System for C#/.NET

The AKEless Build System for C#/.NET

Elevator Pitch

Solid and scalable CI/CD pipelines are an essential pillar for being competitive and creating a great product. But why are most of us a little afraid of touching YAML files and don't even dare to look at build scripts? Much of this is because C# developers are spoiled with a great language and smart IDEs, and they don't like missing their buddy for code-completion, ease of debugging, refactorings, and code formatting.

NUKE brings your build automation to an even level with every other .NET project. How? It's a regular console application allowing all the OOP goodness! Besides, it solves many common problems in build automation, like parameter injection, path separator abstraction, access to solution and project models, and build step sharing across repositories. NUKE can also generate CI/CD configurations (YAML, etc.) that automatically parallelize build steps on multiple agents to optimize throughput!

For more information check out our documentation or visit our community ...

Example

Build Status

NUKE builds and tests itself on several CI/CD services, which helps to ensure a working integration with those systems. At the same time, the individual configuration files serve as examples for the generation experience:

Build Server	Platform	Configuration
TeamCity	Win	settings.kts
GitHub Actions	Win / Ubuntu / macOS	continuous.yml
GitLab CI	Ubuntu	.gitlab-ci.yml
Azure Pipelines	Win / Linux / macOS	azure-pipelines.yml
AppVeyor	Win / Ubuntu	appveyor.yml

In Action

Sponsors

Thanks to all companies, organizations, and individuals who are sponsoring the further development of this project. Your support means a lot! 💙

Technology Sponsors

Thanks to JetBrains for providing licenses for Rider and access to a TeamCity Cloud instance, which both make open-source development a real pleasure!

Thanks to SignPath for providing free code signing service and to the SignPath Foundation for a free code signing certificate.

PSAppDeployToolkit/PSAppDeployToolkit

2024-12-08T01:37:28Z

Project Homepage & Forums

Enterprise App Deployment, Simplified.

Standardize and enhance every software deployment

PSAppDeployToolkit allows you to encapsulate a typical Windows Installer MSI or Setup executable to provide it with enhanced capabilities.

Validate prerequisites such as dependencies on minimum software versions
Ensure that in-use applications are closed and prevent reopening during the deployment
Check with the user if now is a good time to start an install and allow them to defer
Uninstall existing applications and perform clean up operations
Capture any important settings that may be required for an upgrade or migration
Run the installation silently and capture logs in the event of an issue
Run post-installation configuration tasks to customize for your environment
Prompt the user to restart their computer if required, immediately, on a timer and with a deadline

Getting Started

-> System Requirements -> Downloading

PSAppDeployToolkit Links

-> Homepage -> Documentation -> Function & Variable References -> Download Latest Release -> News

Community Links

-> Discourse Forum -> Discord Chat -> Reddit

License

mgth/LittleBigMouse

2024-12-08T01:37:28Z

DPI Aware mouse move across screens

Little Big Mouse

Little Big Mouse (LBM) is an open-source software designed to enhance the multi-monitor experience on Windows 10 and 11 by providing accurate mouse screen crossover location within multi-DPI monitors environments. This is particularly useful for setups involving a 4K monitor alongside a full HD monitor.

Donations

If you find Little Big Mouse helpful, consider supporting the project with a donation. Your contribution helps us maintain and improve the software.

Presentation Video

Check the very nice video from Touble Chute (a very big thanks to him):

Features

Smooth Mouse Transitions: Ensures smooth and accurate mouse movement across screens with different DPI settings.
DPI Aware Mouse Movement: Adjusts mouse speed to remain consistent across monitors with different DPI values.
Infinite Mouse Scrolling: Enables seamless cursor movement between screens, either horizontally or vertically.
Display Size Adjustments: Allows for adjustments in the relative sizes of displays.
Border Resistance: Allow some resistance before crossing.
Display Color and Brightness Balancing: Offers control over color and brightness profiles of displays.
Access to Display Debugging Information: Provides detailed information from your displays and drivers.

Installation

Download the latest release from the Releases page.
Run the Little Big Mouse installer from your computer.
(Optional) Change the default installation path.
Follow the installation progress and complete the installation.
Access the program from the Start Menu.

Usage

Documentation : https://github.com/mgth/LittleBigMouse/wiki

Little Big Mouse provides a single-window interface with three main sections:

Top Panel: Access view tabs for display and display adapter information, changing relative sizes and positions of displays, and adjusting color and brightness profiles.
Center Panel: Displays information about your display devices, including makes and models, capabilities, adapters, and relative positions.
Bottom Panel: Offers options and operations, including copying config to clipboard, enabling/disabling LBM functionality, and adjusting speed, pointer size, and corner crossing.

Support

If you encounter any issues or have suggestions for improvements, please open an issue on our Issues page. We appreciate your feedback and are always looking to improve the tool.

We Welcome Contributions: Your help is invaluable! Whether it's reporting bugs, suggesting new features, or submitting pull requests, we encourage you to get involved. Your contributions can make a significant difference in the development and improvement of Little Big Mouse. First time? Check out This guide to get started.

Acknowledgements

HLab Project: Little Big Mouse depends on the HLab project for its functionality. Check out the HLab and HLab.Avalonia repositories for more information.
MouseKeyboardActivityMonitor: Inspired by the MouseKeyboardActivityMonitor project.
Task Scheduler Managed Wrapper: Utilizes the Task Scheduler Managed Wrapper for scheduling tasks.

Thank you for using Little Big Mouse!

XRPLF/rippled

2024-12-08T01:37:33Z

Decentralized cryptocurrency blockchain daemon implementing the XRP Ledger protocol in C++

The XRP Ledger

The XRP Ledger is a decentralized cryptographic ledger powered by a network of peer-to-peer nodes. The XRP Ledger uses a novel Byzantine Fault Tolerant consensus algorithm to settle and record transactions in a secure distributed database without a central operator.

XRP

XRP is a public, counterparty-free asset native to the XRP Ledger, and is designed to bridge the many different currencies in use worldwide. XRP is traded on the open-market and is available for anyone to access. The XRP Ledger was created in 2012 with a finite supply of 100 billion units of XRP.

rippled

The server software that powers the XRP Ledger is called rippled and is available in this repository under the permissive ISC open-source license. The rippled server software is written primarily in C++ and runs on a variety of platforms. The rippled server software can run in several modes depending on its configuration.

If you are interested in running an API Server (including a Full History Server), take a look at Clio. (rippled Reporting Mode has been replaced by Clio.)

Build from Source

Read the build instructions in BUILD.md
If you encounter any issues, please open an issue

Key Features of the XRP Ledger

Censorship-Resistant Transaction Processing: No single party decides which transactions succeed or fail, and no one can "roll back" a transaction after it completes. As long as those who choose to participate in the network keep it healthy, they can settle transactions in seconds.
Fast, Efficient Consensus Algorithm: The XRP Ledger's consensus algorithm settles transactions in 4 to 5 seconds, processing at a throughput of up to 1500 transactions per second. These properties put XRP at least an order of magnitude ahead of other top digital assets.
Finite XRP Supply: When the XRP Ledger began, 100 billion XRP were created, and no more XRP will ever be created. The available supply of XRP decreases slowly over time as small amounts are destroyed to pay transaction costs.
Responsible Software Governance: A team of full-time, world-class developers at Ripple maintain and continually improve the XRP Ledger's underlying software with contributions from the open-source community. Ripple acts as a steward for the technology and an advocate for its interests, and builds constructive relationships with governments and financial institutions worldwide.
Secure, Adaptable Cryptography: The XRP Ledger relies on industry standard digital signature systems like ECDSA (the same scheme used by Bitcoin) but also supports modern, efficient algorithms like Ed25519. The extensible nature of the XRP Ledger's software makes it possible to add and disable algorithms as the state of the art in cryptography advances.
Modern Features for Smart Contracts: Features like Escrow, Checks, and Payment Channels support cutting-edge financial applications including the Interledger Protocol. This toolbox of advanced features comes with safety features like a process for amending the network and separate checks against invariant constraints.
On-Ledger Decentralized Exchange: In addition to all the features that make XRP useful on its own, the XRP Ledger also has a fully-functional accounting system for tracking and trading obligations denominated in any way users want, and an exchange built into the protocol. The XRP Ledger can settle long, cross-currency payment paths and exchanges of multiple currencies in atomic transactions, bridging gaps of trust with XRP.

Source Code

Here are some good places to start learning the source code:

Read the markdown files in the source tree: src/ripple/**/*.md.
Read the levelization document to get an idea of the internal dependency graph.
In the big picture, the main function constructs an ApplicationImp object, which implements the Application virtual interface. Almost every component in the application takes an Application& parameter in its constructor, typically named app and stored as a member variable app_. This allows most components to depend on any other component.

Repository Contents

Folder	Contents
`./bin`	Scripts and data files for Ripple integrators.
`./Builds`	Platform-specific guides for building `rippled`.
`./docs`	Source documentation files and doxygen config.
`./cfg`	Example configuration files.
`./src`	Source code.

Some of the directories under src are external repositories included using git-subtree. See those directories' README files for more details.

Additional Documentation

trustwallet/wallet-core

2024-12-08T01:37:33Z

Cross-platform, cross-blockchain wallet library.

Trust Wallet Core is an open-source, cross-platform, mobile-focused library implementing low-level cryptographic wallet functionality for a high number of blockchains. It is a core part of the popular Trust Wallet, and some other projects. Most of the code is C++ with a set of strict C interfaces, and idiomatic interfaces for supported languages: Swift for iOS and Java (Kotlin) for Android.

Documentation

For comprehensive documentation, see developer.trustwallet.com.

Audit Reports

Security Audit reports can be found in the audit directory.

Supported Blockchains

Wallet Core supports more than 130 blockchains: Bitcoin, Ethereum, BNB, Cosmos, Solana, and most major blockchain platforms. The full list is here.

Building

For build instructions, see developer.trustwallet.com/wallet-core/building.

Using from your project

If you want to use wallet core in your project follow these instructions.

Android

Android releases are hosted on GitHub packages, you need to add GitHub access token to install it. Please check out this installation guide or build.gradle from our android sample

Don't forget replacing the version in the code with latest:

iOS

We currently support Swift Package Manager and CocoaPods (will discontinue in the future).

SPM

Download latest Package.swift from GitHub Releases and put it in a local WalletCore folder.

Add this line to the dependencies parameter in your Package.swift:

.package(name: "WalletCore", path: "../WalletCore"),

Or add remote url + master branch, it points to recent (not always latest) binary release.

.package(name: "WalletCore", url: "https://github.com/trustwallet/wallet-core", .branchItem("master")),

Then add libraries to target's dependencies:

.product(name: "WalletCore", package: "WalletCore"),
.product(name: "SwiftProtobuf", package: "WalletCore"),

CocoaPods

Add this line to your Podfile and run pod install:

pod 'TrustWalletCore'

NPM (beta)

npm install @trustwallet/wallet-core

Go (beta)

Please check out the Go integration sample.

Kotlin Multipleplatform (beta)

Please check out the Kotlin Multiplatform sample

Projects

Projects using Trust Wallet Core. Add yours too!

Community

There are a few community-maintained projects that extend Wallet Core to some additional platforms and languages. Note this is not an endorsement, please do your own research before using them:

Flutter binding https://github.com/weishirongzhen/flutter_trust_wallet_core
Python binding https://github.com/phuang/wallet-core-python
Wallet Core on Windows https://github.com/kaetemi/wallet-core-windows

Contributing

The best way to submit feedback and report bugs related to WalletCore is to open a GitHub issue. If the bug is not related to WalletCore but to the TrustWallet app, please create a Customer Support ticket. If you want to contribute code please see Contributing. If you want to add support for a new blockchain also see Adding Support for a New Blockchain, make sure you have read the requirements section.

Thanks to all the people who contribute.

Disclaimer

The Wallet Core project is led and managed by Trust Wallet with a large contributor community and actively used in several projects. Our goal at Wallet Core is to give other wallets an easy way to add chain support.

Trust Wallet products leverage wallet core, however, they may or may not leverage all the capabilities, features, and assets available in wallet core due to their own product requirements.

License

Trust Wallet Core is available under the Apache 2.0 license. See the LICENSE file for more info.

microsoft/BitNet

2024-12-08T01:37:33Z

Official inference framework for 1-bit LLMs

bitnet.cpp

bitnet.cpp is the official inference framework for 1-bit LLMs (e.g., BitNet b1.58). It offers a suite of optimized kernels, that support fast and lossless inference of 1.58-bit models on CPU (with NPU and GPU support coming next).

The first release of bitnet.cpp is to support inference on CPUs. bitnet.cpp achieves speedups of 1.37x to 5.07x on ARM CPUs, with larger models experiencing greater performance gains. Additionally, it reduces energy consumption by 55.4% to 70.0%, further boosting overall efficiency. On x86 CPUs, speedups range from 2.37x to 6.17x with energy reductions between 71.9% to 82.2%. Furthermore, bitnet.cpp can run a 100B BitNet b1.58 model on a single CPU, achieving speeds comparable to human reading (5-7 tokens per second), significantly enhancing the potential for running LLMs on local devices. Please refer to the technical report for more details.

The tested models are dummy setups used in a research context to demonstrate the inference performance of bitnet.cpp.

Demo

A demo of bitnet.cpp running a BitNet b1.58 3B model on Apple M2:

https://github.com/user-attachments/assets/7f46b736-edec-4828-b809-4be780a3e5b1

What's New:

11/08/2024 BitNet a4.8: 4-bit Activations for 1-bit LLMs
10/21/2024 1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
10/17/2024 bitnet.cpp 1.0 released.
03/21/2024 The-Era-of-1-bit-LLMs__Training_Tips_Code_FAQ
02/27/2024 The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
10/17/2023 BitNet: Scaling 1-bit Transformers for Large Language Models

Acknowledgements

This project is based on the llama.cpp framework. We would like to thank all the authors for their contributions to the open-source community. Also, bitnet.cpp's kernels are built on top of the Lookup Table methodologies pioneered in T-MAC. For inference of general low-bit LLMs beyond ternary models, we recommend using T-MAC.

Supported Models

❗️We use existing 1-bit LLMs available on Hugging Face to demonstrate the inference capabilities of bitnet.cpp. These models are neither trained nor released by Microsoft. We hope the release of bitnet.cpp will inspire the development of 1-bit LLMs in large-scale settings in terms of model size and training tokens.

Model	Parameters	CPU	Kernel
I2_S	TL1	TL2
bitnet_b1_58-large	0.7B	x86	✔	✘	✔
ARM	✔	✔	✘
bitnet_b1_58-3B	3.3B	x86	✘	✘	✔
ARM	✘	✔	✘
Llama3-8B-1.58-100B-tokens	8.0B	x86	✔	✘	✔
ARM	✔	✔	✘

Installation

Requirements

python>=3.9
cmake>=3.22
clang>=18
- For Windows users, install Visual Studio 2022. In the installer, toggle on at least the following options(this also automatically installs the required additional tools like CMake):
  - Desktop-development with C++
  - C++-CMake Tools for Windows
  - Git for Windows
  - C++-Clang Compiler for Windows
  - MS-Build Support for LLVM-Toolset (clang)
- For Debian/Ubuntu users, you can download with Automatic installation script
  
  bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
conda (highly recommend)

Build from source

[!IMPORTANT] If you are using Windows, please remember to always use a Developer Command Prompt / PowerShell for VS2022 for the following commands

Clone the repo

git clone --recursive https://github.com/microsoft/BitNet.git
cd BitNet

Install the dependencies

# (Recommended) Create a new conda environment
conda create -n bitnet-cpp python=3.9
conda activate bitnet-cpp

pip install -r requirements.txt

Build the project

# Download the model from Hugging Face, convert it to quantized gguf format, and build the project
python setup_env.py --hf-repo HF1BitLLM/Llama3-8B-1.58-100B-tokens -q i2_s

# Or you can manually download the model and run with local path
huggingface-cli download HF1BitLLM/Llama3-8B-1.58-100B-tokens --local-dir models/Llama3-8B-1.58-100B-tokens
python setup_env.py -md models/Llama3-8B-1.58-100B-tokens -q i2_s

usage: setup_env.py [-h] [--hf-repo {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens}] [--model-dir MODEL_DIR] [--log-dir LOG_DIR] [--quant-type {i2_s,tl1}] [--quant-embd]
                    [--use-pretuned]

Setup the environment for running inference

optional arguments:
  -h, --help            show this help message and exit
  --hf-repo {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens}, -hr {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens}
                        Model used for inference
  --model-dir MODEL_DIR, -md MODEL_DIR
                        Directory to save/load the model
  --log-dir LOG_DIR, -ld LOG_DIR
                        Directory to save the logging info
  --quant-type {i2_s,tl1}, -q {i2_s,tl1}
                        Quantization type
  --quant-embd          Quantize the embeddings to f16
  --use-pretuned, -p    Use the pretuned kernel parameters

Usage

Basic usage

# Run inference with the quantized model
python run_inference.py -m models/Llama3-8B-1.58-100B-tokens/ggml-model-i2_s.gguf -p "Daniel went back to the the the garden. Mary travelled to the kitchen. Sandra journeyed to the kitchen. Sandra went to the hallway. John went to the bedroom. Mary went back to the garden. Where is Mary?\nAnswer:" -n 6 -temp 0

# Output:
# Daniel went back to the the the garden. Mary travelled to the kitchen. Sandra journeyed to the kitchen. Sandra went to the hallway. John went to the bedroom. Mary went back to the garden. Where is Mary?
# Answer: Mary is in the garden.

usage: run_inference.py [-h] [-m MODEL] [-n N_PREDICT] -p PROMPT [-t THREADS] [-c CTX_SIZE] [-temp TEMPERATURE]

Run inference

optional arguments:
  -h, --help            show this help message and exit
  -m MODEL, --model MODEL
                        Path to model file
  -n N_PREDICT, --n-predict N_PREDICT
                        Number of tokens to predict when generating text
  -p PROMPT, --prompt PROMPT
                        Prompt to generate text from
  -t THREADS, --threads THREADS
                        Number of threads to use
  -c CTX_SIZE, --ctx-size CTX_SIZE
                        Size of the prompt context
  -temp TEMPERATURE, --temperature TEMPERATURE
                        Temperature, a hyperparameter that controls the randomness of the generated text

Benchmark

We provide scripts to run the inference benchmark providing a model.

usage: e2e_benchmark.py -m MODEL [-n N_TOKEN] [-p N_PROMPT] [-t THREADS]  
   
Setup the environment for running the inference  
   
required arguments:  
  -m MODEL, --model MODEL  
                        Path to the model file. 
   
optional arguments:  
  -h, --help  
                        Show this help message and exit. 
  -n N_TOKEN, --n-token N_TOKEN  
                        Number of generated tokens. 
  -p N_PROMPT, --n-prompt N_PROMPT  
                        Prompt to generate text from. 
  -t THREADS, --threads THREADS  
                        Number of threads to use.

Here's a brief explanation of each argument:

-m, --model: The path to the model file. This is a required argument that must be provided when running the script.
-n, --n-token: The number of tokens to generate during the inference. It is an optional argument with a default value of 128.
-p, --n-prompt: The number of prompt tokens to use for generating text. This is an optional argument with a default value of 512.
-t, --threads: The number of threads to use for running the inference. It is an optional argument with a default value of 2.
-h, --help: Show the help message and exit. Use this argument to display usage information.

For example:

python utils/e2e_benchmark.py -m /path/to/model -n 200 -p 256 -t 4

This command would run the inference benchmark using the model located at /path/to/model, generating 200 tokens from a 256 token prompt, utilizing 4 threads.

For the model layout that do not supported by any public model, we provide scripts to generate a dummy model with the given model layout, and run the benchmark on your machine:

python utils/generate-dummy-bitnet-model.py models/bitnet_b1_58-large --outfile models/dummy-bitnet-125m.tl1.gguf --outtype tl1 --model-size 125M

# Run benchmark with the generated model, use -m to specify the model path, -p to specify the prompt processed, -n to specify the number of token to generate
python utils/e2e_benchmark.py -m models/dummy-bitnet-125m.tl1.gguf -p 512 -n 128

aleksey-hoffman/sigma-file-manager

2024-12-08T01:37:22Z

"Sigma File Manager" is a free, open-source, quickly evolving, modern file manager (explorer / browser) app for Windows and Linux.

[!NOTE]

Sigma File Manager (current page): development is paused
Sigma File Manager v2.0: development is active

Latest news

Version 2.0.0-alpha.1 was released.
Version 1.7.0 was released. Video review: https://www.youtube.com/watch?v=ZQy754HOM7Y
Started creating Sigma File Manager Next (v2.0). Rewriting the app codebase with newer, better technologies (Rust, Vue 3, Typescript), as well as improving code quality and moving away from Electron.js. Once finished, the app will become faster, more capable, and better in every other way.
Reddit community r/SigmaFileManager was created. This is where I will post all the updates, polls, progress, image and video previews of new / modified features, tips, etc. Check it out and post some cool ideas!

What's new in `v1.7`

4 new features
9 improvements
5 fixes

SEE RELEASE NOTES →

Sigma File Manager

"Sigma File Manager" is a free, open-source, quickly evolving, modern file manager (explorer / finder) app for Windows and Linux.

Designed, developed, and maintained by Aleksey Hoffman

Community links

Discord | Reddit | YouTube | X (Twitter) | Telegram
Thanks to everyone who has been sharing this project with others!

Goal

Create the best existing intelligent file manager & productivity app that will significantly speed up your workflow, provide you with advanced tools, and keep your data safe.

Done 21 of 100 major features.

Features

The list of major features:

SMART GLOBAL SEARCH: feature allows you to find any file / directory on your computer in just a few seconds. It also implements a typo correction system, allowing you to find files even when the search query includes typos, wrong case, wrong word order, missing words, missing symbols, and missing file extension.
TABS: feature allows you to quickly switch between multiple directories with a click or a shortcut.
WORKSPACES: feature allows you to create separate workspaces with their own tabs and pre-defined actions, allowing you to quickly switch between them. Actions can perform different operations, e.g. open a website, run a script, open a file in a specific program, etc.
ADVANCED WIRELESS FILE SHARING: share a directory or stream any file to your local devices wirelessly without installing any apps. It works on any device with a modern web browser (smart TVs, mobile devices, laptops, virtual machines, etc).
ADDRESS BAR: lets you quickly navigate directories with your keyboard. It autocompletes file names and automatically opens directories as you type.
ITEM FILTER: allows you to filter out items from the list to quickly find the one you need. It supports advanced glob patterns and property prefixes.
ADVANCED FILE DOWNLOADER: stream / download any file from the internet just by dragging it (or its URL link) onto the app. It supports all file types, including videos from video services like Youtube and video streams (with or without auth tokens / URL timestamps).
SMART DRAG & DROP: allows you to easily download files from the internet, and provides a comfortable way of copying / moving local files.
SHORTCUTS: most actions within the app can be performed with shortcuts. You can also create global (system wide) shortcuts for some actions like opening / hiding the app or opening the app and instantly creating a new note.
CUSTOMIZABLE HOME PAGE BANNER: The home page of the app has a unique, recognizable feature - a customizable media background (banner). You can set a custom image / video or choose one of the built-in artworks (kindly provided to me by different artists) and personalize the app to your liking.
NOTES: Advanced note editor in this app allows you to create featureful notes and store them in one convenient, quickly accessible place.
DASHBOARD: Dashboard page provides quick access to pinned / protected / tagged items, and the timeline.
FILE PROTECTION: This feature allows you to protect any file / directory / note from being modified / renamed / moved / deleted.
INFO PANEL: displays useful information about currently selected file / directory and the file preview for media files: audio, video, images (including animated gifs, apng, webp, and many other formats).
ARCHIVER: built-in archiver allows you to compress files into archives and extract existing archives. It supports all most popular formats. See the list of formats on the "features" wiki page
AUTO UPDATES: the app updates itself automatically. It will let you decide if and when you want to download and install an update.
COMMUNITY PARTICIPATION: unlike closed-sourced apps, you participate in the development of this app (request features, vote which features should be prioritized, participate in discussions, modify code of the app yourself).
QUICK VIEW: allows you to quickly open selected files in a preview window, rather than an external program by pressing Space button. It supports images, videos, audio, PDF, most plain text formats.
INFUSIVE DESIGN: this app implements my custom, modern, beautiful, minimalistic, customizable, design system called "Infusive design".
WINDOW TRANSPARENCY EFFECTS: feature allows you to customize the app design and create literally infinite amount of custom, beautiful, modern UI designs by allowing you to beautifully infuse any image / video into every element of the app UI.
LOCALIZATION: the app interface can be displayed in any language. The languages can be edited, and new languages can be added by the community.

MANY MORE FEATURES ARE COMING

Several more major features / improvements are already in development and will be added in the next updates.

MORE INFO

DETAILED FEATURE DESCRIPTION (WIKI) →

FEATURES IN DEVELOPMENT →

Supporters

Consider supporting the project on Patreon:

See other methods

Sponsors

Hover is a popular service where you buy domain names for your website.
Use my Hover link and we'll both get $2 off their domains.

Houston's #1 Photo Restoration Service
photoancestry.com

Supporters

Andy Pfister

Alex Ivantsov

See the full list of supporters →

The funding will be used for funding the development of this and my other big projects. I'm also working in collaboration with a few universities on a project that will help scientists speed up development of new medications and treatments for diseases, reducing the time need to find a new medication from 10 years (current average) to just a few months, and hopefully help humanity get rid of diseases (the project will be open-sourced later);

Download

⚡ Requirements:

Memory (RAM): minimum: ~100 MB, average: ~400 MB;
Storage: minimum: ~370 MB;
OS: 64-bit: Windows, Linux;

📎 Download links:

You can install the app in one of these ways:

Operating system	Download link (latest version)
Windows (via winget)	`winget install --id=AlekseyHoffman.Sigma-File-Manager --source winget`
Windows (via Microsoft store)
Windows (unsigned installable)	Installable file (.exe)
Debian (Ubuntu) portable app \| experimental	AppImage
If there's no compiled binary for your Linux dist, you can build the app from the source code yourself, using `Node v15.14.0`	`npm run build`

Info

This app is primarily developed for Windows. The Linux builds get a lot less development time. They are less stable and have more issues. With more supporters, I will be able to spend more time on Linux builds.

Windows | Installation notes

It's recommended to install the app via the Microsoft store. If you decide to install the Installable file (.exe) instead, you might see a security warning during installation. If you see the warning prompt, click "More Info" and then "Run Anyway".

Ubuntu 22.04 LTS | Installation notes

Running AppImage applications in Ubuntu 22.04 LTS

⚠ Notes:

If you're running the app on the integrated GPU, the RAM usage will increase during video playback. Videos playing in the background (e.g. on the home page) are the main reason for high memory usage. You can reduce memory usage by choosing a static image for the "Windows transparency effect" and the home banner.
The app is still in early development (prototyping) stage. It is safe to use, it should not delete or damage any files by itself, but it's recommended to backup (copy) your important data to an external backup drive or to your cloud storage, just for good practice.
Expect to see some errors and performance issues in the first versions of the app. I will be fixing the problems gradually as they get reported by the users.
The app has not been optimized yet, RAM usage can spike and go over 1 GB during some operations. So, for now, avoid using the app on low-tier computers (i.e. computers with 4 GB of RAM or less).

💬 Get notified

Community links: Reddit | YouTube | X (Twitter) | Telegram

If you can also click the "watch" button on top of the page if you want to get notified when I publish a new release, or someone opens a new discussion / issue.

💡 Hidden treasures

There's a fun secret hidden somewhere in the app. See if you can find it. Hint: only the dinosaur knows about it.

Project links

Milestones

✅ v1.0.0 (May 25 2021): publish the app;

✅ by v1.1.0 (May 27 2021): new features, improvements, and fixes;

✅ by v1.2.0 (June 2021): new features, improvements, and fixes;

✅ by v1.3.0 (August 2021): new features, improvements, and fixes;

✅ by v1.4.0 (September 2021): new features, improvements, and fixes;

✅ by v1.5.0: new features, improvements, and fixes.

✅ by v1.6.0: new features, improvements, and fixes, extensions support, localize the app to 10+ most popular languages;

⬜ ...

⬜ by v2.0.0: Reach the 20 major features count;

⬜ End goal: improve the app to the point of becoming the best file manager and one the most useful productivity apps. This milestone will be considered to be achieved when at least 3 reputable independent sources (reviewers / platforms) decide that the app has achieved all of the following titles:

⬜ (0/3 sources) the best productivity impact;
⬜ (0/3 sources) the best UI design;
⬜ (0/3 sources) the best feature set;
⬜ (0/3 sources) the fastest improvement rate;
⬜ (0/3 sources) the most loved file manager app;
⬜ (0/3 sources) the most intelligent file manager app;
⬜ (0/3 sources) is one of top 3 desktop productivity apps;

Links to reviews: none yet.

Become my student

If you want to quickly learn how to build web and desktop apps like Sigma File Manager, I can teach you.

See more details on my Github profile page

Contributors - open-content creators

Artworks used in the app

Click on an image to visit the artist's page. If you like their art, consider supporting their work. You can find their support links in the app on the artwork cards in the background manager (app home page > banner menu > background manager).


Julien Gauthier	Ahmed Teilab	Johannes Plenio	Suzy Hazelwood	Dana Franklin	Kevin Lanceplaine

Klaus Pillon	Klaus Pillon	Taryn Elliott	Marcel van Vuuren	Vadim Sadovski	Andreas

Darius Kalinauskas	Alena Aenami	Alena Aenami	Alena Aenami	Wang Jie	Vladimir Manyukhin

Laury Guintrand	Sweeper3d

Special thanks

Open-source projects:

Electron: Makes it possible to create cross-platform apps for desktop platforms.
Vue.js: A wonderful JS framework that speeds up web development.
VSCode: One of the best code editors that significantly simplifies the development process.
Vuetify: Material components library that speeds up UI development.
Templarian/MaterialDesign: Provides beautiful, community created, free icons.
yt-dlp: Allows the app to download videos from a URL link using Python.
FFMPEG: Allows the app to convert and edit audio/video/images and get the media info.
7-zip.org: Allows the app to work with archives (compress/decompress directories and files).
xxoo/node-fswin: Windows OS native API bindings for Node.js.

For developers

Developers, feel free to optimize and improve the app. Make sure to create a new issue or a discussion before implementing changes, so we can discuss the changes first.

See CONTRIBUTING.md file for more info.

License

Sigma file manager is licensed under GNU GPLv3 or later.

astral-sh/uv-docker-example

2024-12-08T01:38:25Z

An example of using uv in Docker images

uv-docker-example

An example project for using uv in Docker images, with a focus on best practices for developing with the project mounted in the local image.

See the uv Docker integration guide for more background.

Trying it out

A run.sh utility is provided for quickly building the image and starting a container. This script demonstrates best practices for developing using the container, using bind mounts for the project and virtual environment directories.

To build and run the web application in the container using docker run:

$ ./run.sh

Then, check out http://localhost:8000 to see the website.

A Docker compose configuration is also provided to demonstrate best practices for developing using the container with Docker compose. Docker compose is more complex than using docker run, but has more robust support for various workflows.

To build and run the web application using Docker compose:

docker compose up --watch

By default, the image is set up to start the web application. However, a command-line interface is provided for demonstration purposes as well.

To run the command-line entrypoint in the container:

$ ./run.sh hello

Project overview

Dockerfile

The Dockerfile defines the image and includes:

Installation of uv
Installing the project dependencies and the project separately for optimal image build caching
Placing environment executables on the PATH
Running the web application

The multistage.Dockerfile example extends the Dockerfile example to use multistage builds to reduce the final size of the image.

Dockerignore file

The .dockerignore file includes an entry for the .venv directory to ensure the .venv is not included in image builds. Note that the .dockerignore file is not applied to volume mounts during container runs.

Run script

The run.sh script includes an example of invoking docker run for local development, mounting the source code for the project into the container so that edits are reflected immediately.

Docker compose file

The compose.yml file includes a Docker compose definition for the web application. It includes a watch directive for Docker compose, which is a best-practice method for updating the container on local changes.

Application code

The Python application code for the project is at src/uv_docker_example/__init__.py — there's a command line entrypoint and a basic FastAPI application — both of which just display "hello world" output.

Project definition

The project at pyproject.toml includes Ruff as an example development dependency, includes FastAPI as a dependency, and defines a hello entrypoint for the application.

Useful commands

To check that the environment is up-to-date after image builds:

$ ./run.sh uv sync --frozen
Audited 2 packages ...

To enter a bash shell in the container:

$ ./run.sh /bin/bash

To build the image without running anything:

$ docker build .

To build the multistage image:

$ docker build . --file multistage.Dockerfile

emacs-jupyter/jupyter

2024-12-08T01:30:53Z

An interface to communicate with Jupyter kernels.

An interface to communicate with Jupyter kernels in Emacs.

#+BEGIN_HTML #+END_HTML

Table of Contents :TOC:

[[#what-does-this-package-do][What does this package do?]]
[[#how-do-i-install-this-package][How do I install this package?]]
- [[#using-melpa][Using MELPA]]
- [[#manual-installation][Manual installation]]
- [[#building-the-widget-support-experimental][Building the widget support (EXPERIMENTAL)]]
[[#related-packages][Related packages]]
- [[#ob-ipython][=ob-ipython=]]
- [[#emacs-ipython-notebook-ein][=emacs-ipython-notebook= (=ein=)]]
[[#how-do-i-use-the-built-in-frontends][How do I use the built-in frontends?]]
- [[#repl][REPL]]
- [[#org-mode-source-blocks][=org-mode= source blocks]]
- [[#kernelnotebook-server][Kernel/notebook server]]
- [[#customizable-variables-available-for-all-frontends][Customizable variables available for all frontends]]

What does this package do?

Provides REPL and =org-mode= source block frontends to Jupyter kernels.
Kernel interactions integrated with Emacs's built-in features. For example
- Inspecting a piece of code under =point= will display the information for that symbol in the =Help= buffer. You can re-visit inspection requests made to the kernel by calling =help-go-back= or =help-go-forward= while in the =Help= buffer.
- Uses the =completion-at-point= interface for code completion.
- Kernel requests for user input entered through the minibuffer.
- You can search through REPL history using =isearch=.

How do I install this package?

** Using MELPA

NOTE: This package relies on the =emacs-zmq= package which means your Emacs needs to have been built with module support. See the README of that package for more information.

You can install this package with any package manager that allows you to install MELPA packages. For Emacs's built-in package manager:

Ensure MELPA is in =package-archives=

#+BEGIN_SRC elisp (add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/")) #+END_SRC
Ensure the latest versions of MELPA packages are available

=M-x package-refresh-contents RET=
Install Jupyter

=M-x package-install RET jupyter RET=

** Manual installation

For a manual installation you can add the repository directory to your =load-path= and ensure the following dependencies are installed:

markdown-mode (optional) :: https://jblevins.org/projects/markdown-mode/
company-mode (optional) :: http://company-mode.github.io/
emacs-websocket :: https://github.com/ahyatt/emacs-websocket
simple-httpd :: https://github.com/skeeto/emacs-web-server
zmq :: http://github.com/nnicandro/emacs-zmq

#+BEGIN_SRC elisp (add-to-list 'load-path "~/path/to/jupyter") (require 'jupyter) #+END_SRC ** Building the widget support (EXPERIMENTAL) :PROPERTIES: :ID: 59559FA3-59AD-453F-93E7-113B43F85493 :END:

There is limited support for interacting with Jupyter widgets through an external browser. In this case, Emacs acts as a relay for passing messages between the kernel and the browser.

To try it out, install =node= (https://nodejs.org/en/) then run the following shell command from the top-level directory of this project.

#+BEGIN_SRC shell make widgets #+END_SRC

After, launch Emacs, connect to a kernel (e.g. through a REPL), and run some code that creates a widget.

How do I run the tests?

You must have [[https://github.com/doublep/eldev][Eldev]] installed to be able to run the tests. Once Eldev is installed, then in the top level directory of this project you can run one of the following from the command line

#+begin_src shell

Run the whole set of tests

make test

Run tests tagged with org

make test TAGS=org

Run tests tagged with org and babel

make test TAGS=org,babel

Run tests whose name match a pattern

make test PATTERN=font-lock #+end_src

Related packages

** =ob-ipython=

The =org-mode= source block frontend in =emacs-jupyter= is similar to what is offered by [[https://github.com/gregsexton/ob-ipython][ob-ipython]] (and also the [[https://github.com/jkitchin/scimax][scimax]] version).

** =emacs-ipython-notebook= (=ein=)

[[https://github.com/millejoh/emacs-ipython-notebook][ein]] is a complete Jupyter notebook interface in Emacs with many powerful features for Python kernels. There is some overlap in the features provided by =emacs-jupyter= and =ein=, but I have never used =ein= so I cannot speak very much about any similarities/differences.

How do I use the built-in frontends? ** REPL

=M-x jupyter-run-repl= launches a new local kernel and displays a REPL buffer.

=M-x jupyter-connect-repl= connects to an existing kernel using the kernel's [[https://jupyter-client.readthedocs.io/en/stable/kernels.html#connection-files][connection file]], which is supplied by the user, and displays a REPL buffer.

The REPL supports some of the [[https://ipython.readthedocs.io/en/stable/interactive/plotting.html#rich-outputs][rich output]] that a kernel may send to a client, e.g. images, LaTeX, and HTML.

*** Rich kernel output

Below is a table of the supported output mimetypes and their dependencies. If a dependency is not available for a particular mimetype, a mimetype of lower priority gets displayed instead.

For widgets, before attempting to open one, you also need to run the shell command =make widgets= in the top-level directory of this project to build some JavaScript files.

| Mimetype | Dependencies | |------------------------------------------+---------------------------| | =application/vnd.jupyter.widget-view+json= | [[https://github.com/ahyatt/emacs-websocket][websocket]], [[https://github.com/skeeto/emacs-web-server][simple-httpd]] | | =text/html= | Emacs built with libxml2 | | =text/markdown= | [[https://jblevins.org/projects/markdown-mode/][markdown-mode]] | | =text/latex= | [[https://orgmode.org/][org-mode]] | | =image/svg+xml= | Emacs built with librsvg2 | | =image/png= | none | | =text/plain= | none | *** Inspection

To inspect the code around =point= press =M-i=.

*** Completion

Completion is implemented through the =completion-at-point= interface and should just work.

In addition to completing symbols in the REPL buffer, completion also works in buffers [[id:DA597E05-E9A9-4DCE-BBD7-6D25238638C5][associated]] with a REPL. For =org-mode= users, there is even completion in the =org-mode= buffer when editing the contents of a Jupyter source code block. *** REPL history

To navigate the REPL history: =M-n= and =M-p=.

To search the REPL history: =C-s= and =C-s C-r=.

*** Associating buffers with a REPL (=jupyter-repl-interaction-mode=) :PROPERTIES: :ID: DA597E05-E9A9-4DCE-BBD7-6D25238638C5 :END:

=M-x jupyter-repl-associate-buffer= sets the =jupyter-current-client= of the current buffer to an existing REPL client and enables =jupyter-repl-interaction-mode=, allowing you to, for example, send the current line for evaluation by the client's kernel.

When =jupyter-repl-interaction-mode= is enabled, the following keybindings are available

| Key binding | Command | |-------------+-------------------------------| | =C-M-x= | =jupyter-eval-defun= | | =M-i= | =jupyter-inspect-at-point= | | =C-c C-b= | =jupyter-eval-buffer= | | =C-c C-c= | =jupyter-eval-line-or-region= | | =C-c C-i= | =jupyter-repl-interrupt-kernel= | | =C-c C-r= | =jupyter-repl-restart-kernel= | | =C-c C-s= | =jupyter-repl-scratch-buffer= | | =C-c C-o= | =jupyter-eval-remove-overlays= | | =C-c M-:= | =jupyter-eval-string= |

**** Integration with =emacsclient=

If =emacsclient= is set as the =EDITOR= and evaluated code opens a file in a =major-mode= compatible with the client that sent the code, the opened file will automatically be associated with the client and have =jupyter-repl-interaction-mode= enabled.

This feature probably wont work correctly when there are multiple competing clients sending requests to their underlying kernels that want to open files or if the underlying kernel takes longer than =jupyter-long-timeout= seconds to open a file.

See =jupyter-server-mode-set-client= for more details.

*** =jupyter-repl-maximum-size=

A variable that determines the maximum number of lines a REPL buffer can have before being truncated.

*** =jupyter-repl-allow-RET-when-busy=

A variable that determines whether to allow insertion of newlines in a REPL cell when a kernel is busy or not. See the variable documentation for more details.

*** =jupyter-repl-echo-eval-p=

A variable that determines whether code evaluated with the =jupyter-eval-*= commands gets copied over to a REPL input cell or not. You can set this variable to =t= if you prefer having the history of all evaluated code visible in the REPL.

** =org-mode= source blocks

To enable support for Jupyter based source code blocks, add =jupyter= to =org-babel-load-languages=. Ensure the =jupyter= entry is added last since loading =ob-jupyter= depends on the value of variables such as =org-src-lang-modes= and =org-babel-tangle-lang-exts=.

#+BEGIN_SRC elisp (org-babel-do-load-languages 'org-babel-load-languages '((emacs-lisp . t) (julia . t) (python . t) (jupyter . t))) #+END_SRC

After loading, source code blocks with names like =jupyter-LANG= will be available for use. =LANG= can be any one of the kernel languages found on your system. See =jupyter-available-kernelspecs=.

The =:session= parameter is required for all Jupyter based source code blocks.

#+BEGIN_SRC org ,#+BEGIN_SRC jupyter-python :session py x = 'foo' y = 'bar' x + ' ' + y ,#+END_SRC #+END_SRC
By default, source blocks are executed synchronously. To execute a source block asynchronously set the =:async= parameter to =yes=:

#+BEGIN_SRC org ,#+BEGIN_SRC jupyter-python :session py :async yes x = 'foo' y = 'bar' x + ' ' + y ,#+END_SRC #+END_SRC
To change the kernel, set the =:kernel= parameter.

#+BEGIN_SRC org ,#+BEGIN_SRC jupyter-python :session py :async yes :kernel python2 x = 'foo' y = 'bar' x + ' ' + y ,#+END_SRC #+END_SRC

Note, the same session name can be used for different values of =:kernel= since the underlying REPL buffer's name is based on both =:session= and =:kernel=.
Any of the default parameters for a language can be changed by setting =org-babel-default-header-args:jupyter-LANG= to an appropriate value. For example to change the defaults for the =julia= kernel, you can set =org-babel-default-header-args:jupyter-julia= to something like

#+BEGIN_SRC elisp (setq org-babel-default-header-args:jupyter-julia '((:async . "yes") (:session . "jl") (:kernel . "julia-1.0"))) #+END_SRC *** Note on the language name provided by a kernelspec

Some kernelspecs use spaces in the name of the kernel language. Those get replaced by dashes in the language name you need to use for the corresponding source blocks, e.g. =Wolfram Language= has the source block language =jupyter-Wolfram-Language=.

*** Integration with =ob-async=

If you have =ob-async= installed and are getting errors when your source block specifies the =:async= header argument, try putting something like the following in your configuration:

#+BEGIN_SRC elisp (setq ob-async-no-async-languages-alist '("jupyter-python" "jupyter-julia")) #+END_SRC

See [[https://github.com/astahlman/ob-async#ob-async-no-async-languages-alist][ob-async-no-async-languages-alist]] for more details.

*** Issues with =ob-ipython=

If both =ob-ipython= and this package are installed, you may experience issues such as [[https://github.com/dzop/emacs-jupyter/issues/133#issuecomment-502444999][this one]], causing =Search failed= errors. To avoid such errors, remove =ipython= from =org-babel-do-load-languages= and restart your Emacs.

*** Overriding built-in src-block languages

Instead of having to specify =jupyter-LANG= as a source block name, you can have =LANG= source blocks use the Jupyter machinery. To do so, place a call to =org-babel-jupyter-override-src-block= somewhere in your config (after the call to =org-babel-do-load-languages=).

#+BEGIN_SRC elisp (org-babel-jupyter-override-src-block "python") #+END_SRC

After calling the above function, all =python= source blocks are effectively aliases of =jupyter-python= source blocks and the variable =org-babel-default-header-args:python= will be set to the value of =org-babel-default-header-args:jupyter-python=.

Note, =org-babel-default-header-args:python= will not be an alias of =org-babel-default-header-args:jupyter-python=, the value of the former is merely set to the value of the latter after calling =org-babel-jupyter-override-src-block=.

You can restore the original behavior by calling =org-babel-jupyter-restore-src-block=.

#+BEGIN_SRC elisp (org-babel-jupyter-restore-src-block "python") #+END_SRC

*** Rich kernel output

The supported display mimetypes ordered by priority are:

text/org
image/svg+xml, image/jpeg, image/png
text/html
text/markdown
text/latex
text/plain

**** A note on using the =:results= header argument

There are some cases where the normal result insertion mechanism may not be wanted. To control result insertion somewhat, use the =:results= header argument:

Insert unwrapped LaTeX :: Normally LaTeX results are wrapped in a =BEGIN_EXPORT= block, in order to insert LaTeX unwrapped, specify =:results raw=.
Suppress table creation :: Whenever a result can be converted into an =org-mode= table, e.g. when it look like =[1, 2 , 3]=, it is automatically converted into a table. To suppress this behavior you can specify =:results scalar=.

**** Fixing the file name of images with the =:file= argument

Whenever an image result is returned, a random image file name is generated and the image is written to =org-babel-jupyter-resource-directory=. To specify your own file name for the image, set the =:file= header argument.

If no file extension is specified in the provided =:file=, then one will be inferred from the returned output. This can be useful in scenarios where the file resulting from the src-block can have different types depeneding on the code, e.g. if the image type returned can be either =png= or =svg= depending on certain settings, you can specify =:file = output= which will be converted into =output.png= or =output.svg= depending on the MIME type return by the executed src-block.

**** Changing the mime-type priority with the =:display= argument

The priority of mimetypes used to display results can be overwritten using the =:display= option. If instead of displaying HTML results we'd wish to display plain text, the argument =:display text/plain text/html= would prioritize plain text results over html ones. The following example displays plain text instead of HTML: #+BEGIN_SRC org ,#+BEGIN_SRC jupyter-python :session py :display plain import pandas as pd data = [[1, 2], [3, 4]] pd.DataFrame(data, columns=["Foo", "Bar"]) ,#+END_SRC #+END_SRC

**** Image output without the =:file= header argument

For images sent by the kernel, if no =:file= parameter is provided to the code block, a file name is automatically generated based on the image data and the image is written to file in =org-babel-jupyter-resource-directory=. This is great for quickly generating throw-away plots while you are working on your code. Once you are happy with your results you can specify the =:file= parameter to fix the file name. **** =org-babel-jupyter-resource-directory=

This variable is similar to =org-preview-latex-image-directory= but solely for any files created when Jupyter code blocks are run, e.g. automatically generated image file names.

***** Deletion of generated image files

Whenever you run a code block multiple times and replace its results, before the results are replaced, any generated files will be deleted to reduce the clutter in =org-babel-jupyter-resource-directory=. **** Convert rich kernel output with the =:pandoc= header argument

By default html, markdown, and latex results are wrapped in a =BEGIN_EXPORT= block. If the header argument =:pandoc t= is set, they are instead converted to org-mode format with [[https://pandoc.org/][pandoc]]. You can control which outputs get converted with the custom variable =jupyter-org-pandoc-convertable=.

*** Editing the contents of a code block

When editing a Jupyter code block's contents, i.e. by pressing =C-c '= when at a code block, =jupyter-repl-interaction-mode= is automatically enabled in the edit buffer and the buffer will be associated with the REPL session of the code block (see =jupyter-repl-associate-buffer=).

You may also bind the command =org-babel-jupyter-scratch-buffer= to an appropriate key in =org-mode= to display a scratch buffer in the code block's =major-mode= and connected to the code block's session. *** Connecting to an existing kernel

To connect to an existing kernel, pass the kernel's connection file as the value of the =:session= parameter. The name of the file must have a =.json= suffix for this to work. **** Remote kernels

If the connection file is a [[https://www.gnu.org/software/emacs/manual/html_node/emacs/Remote-Files.html][remote file name]], i.e. has a prefix like =/method:host:=, the kernel's ports are assumed to live on =host=. Before attempting to connect to the kernel, =ssh= tunnels for the connection are created. So if you had a remote kernel on a host named =ec2= whose connection file is =/run/user/1000/jupyter/kernel-julia-0.6.json= on that host, you could specify the =:session= like

#+BEGIN_SRC org ,#+BEGIN_SRC jupyter-julia :session /ssh:ec2:/run/user/1000/jupyter/kernel-julia-0.6.json ... ,#+END_SRC #+END_SRC

Note, the kernel on the remote host needs to have the ZMQ socket ports exposed. This means that starting a kernel using

#+BEGIN_SRC shell jupyter notebook --no-browser #+END_SRC

currently doesn't work since the notebook server does not allow communication with a kernel using ZMQ sockets. You will have to use the connection file created from using something like

#+BEGIN_SRC shell jupyter kernel --kernel=python #+END_SRC

***** Password handling for remote connections Currently there is no password handling, so if your =ssh= connection requires a password I suggest you instead use [[https://www.ssh.com/ssh/keygen/][key-based authentication]]. Or if you are connecting to a server using a =pem= file add something like

#+BEGIN_SRC conf Host ec2 User HostName IdentityFile .pem #+END_SRC

to your =~/.ssh/config= file. *** Starting a remote kernel

If =:session= is a remote file name that doesn't end in =.json=, e.g. =/ssh:ec2:jl=, then a kernel on the remote host =/ssh:ec2:= is started using the =jupyter kernel= command on the host. The local part of the session name serves to distinguish different remote sessions on the same host.

*** Communicating with kernel (notebook) servers

If =:session= is a TRAMP file name like =/jpy:localhost#8888:NAME= it is interpreted as corresponding to a connection to a kernel through a Jupyter notebook server located at =http://localhost:8888=.

If =NAME= is a kernel ID corresponding to an existing kernel on a server, e.g. =/jpy::161b2318-180c-497a-b4bf-de76176061d9=, then a connection to an existing kernel with the corresponding ID will be made. Otherwise, a new kernel will be launched on the server and =NAME= will be used as an identifier for the session.

When a new kernel is launched, =NAME= will also be associated with the kernel's ID, see =jupyter-server-kernel-names=. This is useful to distinguish Org mode =:session= kernels from other ones in the buffer shown by =jupyter-server-list-kernels=.

When connecting to an existing kernel, i.e. when =NAME= is the ID of a kernel, the =:kernel= header argument must match the name of the kernel's kernelspec.

To connect to a kernel behind an =HTTPS= connection, use a TRAMP file name that looks like =/jpys:...= instead.

*** Standard output, displayed data, and code block results

In contrast to non-Jupyter code blocks, the kernel of Jupyter code block can request extra data, other than stdout or a code block's result, be displayed (see [[https://jupyter-client.readthedocs.io/en/stable/messaging.html#display-data][display_data messages]]).

To account for this, Jupyter code blocks do not go through the normal =org-mode= result insertion mechanism (see =org-babel-insert-result=), instead providing its own result insertion. The downside is that, compared to normal =org-mode= code blocks, only a small subset of the header arguments are supported. The upside is that all forms of results produced by a kernel can be inserted into the buffer similar to a Jupyter notebook.

*** =jupyter-org-interaction-mode=

A minor mode that enables completion and custom keybindings when =point= is inside a Jupyter code block. This mode is enabled by default in =org-mode= buffers, but only has an effect when =point= is inside a Jupyter code block.

**** Custom keybindings inside Jupyter code blocks

You can define new keybindings that are enabled when =point= is inside a Jupyter code block by using the function =jupyter-org-define-key=. These bindings are added to =jupyter-org-interaction-mode-map= and are only active when =jupyter-org-interaction-mode= is enabled.

By default the following keybindings from =jupyter-repl-interaction-mode= are available when =jupyter-org-interaction-mode= is enabled

| Key binding | Command | |-------------+---------------------------------| | =C-M-x= | =jupyter-eval-defun= | | =M-i= | =jupyter-inspect-at-point= | | =C-x C-e= | =jupyter-eval-line-or-region= | | =C-c C-i= | =jupyter-repl-interrupt-kernel= | | =C-c C-r= | =jupyter-repl-restart-kernel= |

*** Disable automatic connections to a source block session

When typing into the region of a Jupyter source block, under certain conditions, an attempt at connecting to the source block's session is made if not already connected.

This behavior can be suppressed by setting =jupyter-org-auto-connect= to =nil=. In this case, a connection is attempted upon executing a source block, for example.

*** Enable client-side queuing of requests

If the customizable variable =jupyter-org-queue-requests= is non-nil, then perform client side queuing of source block execute requests. This means that when multiple requests are made, for example by executing a subtree, the requests are queued locally in Emacs instead of sending all the requests immediately to the kernel as would happen when =:async yes= is specified on all the source blocks. It is only when one request finishes that the next is sent. In addition, if any request fails all the queued requests that are meant to come after it are aborted and do not get sent to the kernel.

To turn client side queuing on or off you can =M-x jupyter-org-toggle-request-queuing=.

** Kernel/notebook server *** Managing live kernels

The main entry point for working with a kernel server is the =jupyter-server-list-kernels= command which shows a list of all live kernels from the server URL that you provide when first calling the command. Any subsequent calls to the command will use the same URL as the first call. To change server URLs give a prefix argument, =C-u M-x jupyter-server-list-kernels=. This will then set the current server URL for future calls to the one you provide. See the =jupyter-current-server= command for more details.

From the buffer shown by =jupyter-server-list-kernels= you can launch new kernels (=C-RET=), connect a REPL to an existing kernel (=RET=), interrupt a kernel (=C-c TAB=), kill a kernel (=C-c C-d= or =d=), refresh the list of kernels (=g=) etc. See the =jupyter-server-kernel-list-mode= for all the available key bindings.

Note, the =default-directory= of the =jupyter-server-kernel-list-mode= buffer will be the root directory of the kernel server (so that =dired-jump= will show a =dired= listing of the directory). See the section on TRAMP integration below.

*** Naming kernels

From the =jupyter-server-list-kernels= buffer one can also name (or rename) a kernel (=R=) so that it has an identifier other than its ID. Naming a kernel adds the name to the =jupyter-server-kernel-names= global variable in a form suitable for persisting across Emacs sessions. See its documentation for more details about persisting its value.

*** TRAMP integration

There is also integration with the Jupyter notebook contents API in the form of a TRAMP backend. This means that reading/writing the contents of directories the notebook server has access to can be done using normal Emacs file operations using file names with TRAMP syntax. Two new TRAMP file name methods are defined, =jpy= for HTTP connections and =jpys= for HTTPS connections. So suppose you have a local notebook server at http://localhost:8888, then to access its directory contents you can type

#+begin_example M-x dired RET /jpy:localhost#8888:/ #+end_example

Note =localhost= is the default host and =8888= is the default port so =/jpy::= is equivalent to =/jpy:localhost#8888:=. You can change the defaults by modifying the =jpy= or =jpys= methods in the variable =tramp-methods= and =tramp-default-host-alist=.

*** =jupyter-api-authentication-method=

Authentication method used for new notebook server connections. By default, when connecting to a new notebook server you will be asked if either a password or a token should be used for authentication. If you only use tokens for authentication you can change this variable to avoid being asked on every new connection.

** Customizable variables available for all frontends

*** =jupyter-eval-use-overlays=

When non-nil, display the =text/plain= representation of evaluation results inline using overlays. All other representations are displayed in the usual way. This only works with the =jupyter-eval-*= commands like =jupyter-eval-line-or-region=.

You can control the appearance of the overlay, see =jupyter-eval-overlay-prefix= and the =jupyter-eval-overlay= face.

To clear all overlays from the buffer, bind =jupyter-eval-remove-overlays= to some key. Its bound to =C-c C-o= when =jupyter-repl-interaction-mode= is enabled. Individual overlays are removed whenever the text in the region that was evaluated is modified.

For multi-line overlays you can fold/unfold the overlay by pressing =S-RET= when =point= is inside the region of code that caused the overlay to be created. See =jupyter-eval-overlay-keymap=.

*** =jupyter-eval-short-result-max-lines=

If the number of lines of an evaluation result is smaller than this variable, the function stored in =jupyter-eval-short-result-display-function= is used to display a result.

jrblevin/markdown-mode

2024-12-08T01:38:45Z

Emacs Markdown Mode

markdown-mode is a major mode for editing Markdown-formatted text. The latest stable version is markdown-mode 2.6, released on Aug 30, 2023. See the release notes for details. markdown-mode is free software, licensed under the GNU GPL, version 3 or later.

Documentation

The primary documentation for Markdown Mode is available below, and is generated from comments in the source code. For a more in-depth treatment, the Guide to Markdown Mode for Emacs covers Markdown syntax, advanced movement and editing in Emacs, extensions, configuration examples, tips and tricks, and a survey of other packages that work with Markdown Mode. Finally, Emacs is also a self-documenting editor. This means that the source code itself contains additional documentation: each function has its own docstring available via C-h f (describe-function), individual keybindings can be investigated with C-h k (describe-key), and a complete list of keybindings is available using C-h m (describe-mode).

Installation

Note: To use all of the features of markdown-mode, you'll need to install the Emacs package itself and also have a local Markdown processor installed (e.g., Markdown.pl, MultiMarkdown, or Pandoc). The external processor is not required for editing, but will be used for rendering HTML for preview and export. After installing the Emacs package, be sure to configure markdown-command to point to the preferred Markdown executable on your system. See the Customization section below for more details.

The recommended way to install markdown-mode is to install the package from MELPA Stable using package.el. First, configure package.el and the MELPA Stable repository by adding the following to your .emacs, init.el, or equivalent startup file:

(require 'package)
(add-to-list 'package-archives
             '("melpa-stable" . "https://stable.melpa.org/packages/"))
(package-initialize)

Then, after restarting Emacs or evaluating the above statements, issue the following command: M-x package-install RET markdown-mode RET. When installed this way, the major modes markdown-mode and gfm-mode will be autoloaded and markdown-mode will be used for file names ending in .md, .markdown, .mkd, .mdown, .mkdn, .mdwn.

Alternatively, if you manage loading packages with use-package then you can automatically install and configure markdown-mode by adding a declaration such as this one to your init file (as an example; adjust settings as desired):

(use-package markdown-mode
  :ensure t
  :mode ("README\\.md\\'" . gfm-mode)
  :init (setq markdown-command "multimarkdown")
  :bind (:map markdown-mode-map
         ("C-c C-e" . markdown-do)))

Direct Download

Alternatively you can manually download and install markdown-mode. First, download the latest stable version and save the file where Emacs can find it (i.e., a directory in your load-path). You can then configure markdown-mode and gfm-mode to load automatically by adding the following to your init file:

(autoload 'markdown-mode "markdown-mode"
   "Major mode for editing Markdown files" t)
(add-to-list 'auto-mode-alist
             '("\\.\\(?:md\\|markdown\\|mkd\\|mdown\\|mkdn\\|mdwn\\)\\'" . markdown-mode))

(autoload 'gfm-mode "markdown-mode"
   "Major mode for editing GitHub Flavored Markdown files" t)
(add-to-list 'auto-mode-alist '("README\\.md\\'" . gfm-mode))

(with-eval-after-load 'markdown-mode
  (define-key markdown-mode-map (kbd "C-c C-e") #'markdown-do))

Development Version

To follow or contribute to markdown-mode development, you can browse or clone the Git repository on GitHub:

git clone https://github.com/jrblevin/markdown-mode.git

If you prefer to install and use the development version, which may become unstable at some times, you can either clone the Git repository as above or install markdown-mode from MELPA.

If you clone the repository directly, then make sure that Emacs can find it by adding the following line to your startup file:

(add-to-list 'load-path "/path/to/markdown-mode/repository")

Packaged Installation

markdown-mode is also available in several package managers. You may want to confirm that the package you install contains the latest stable version first (and please notify the package maintainer if not).

Debian Linux: elpa-markdown-mode and emacs-goodies-el
Ubuntu Linux: elpa-markdown-mode and emacs-goodies-el
RedHat and Fedora Linux: emacs-goodies
NetBSD: textproc/markdown-mode
MacPorts: markdown-mode.el
FreeBSD: textproc/markdown-mode.el

Dependencies

To enable editing of code blocks in indirect buffers using C-c ', you will need to install the edit-indirect package.

Usage

Keybindings are grouped by prefixes based on their function. For example, the commands for styling text are grouped under C-c C-s and toggle commands begin with C-c C-x. The primary commands in each group will are described below. You can obtain a list of all keybindings by pressing C-c C-h. Movement and shifting commands tend to be associated with paired delimiters such as M-{ and M-} or C-c < and C-c >. Outline navigation keybindings the same as in org-mode. Finally, commands for running Markdown or doing maintenance on an open file are grouped under the C-c C-c prefix. The most commonly used commands are described below.

Links and Images: C-c C-l and C-c C-i

C-c C-l (markdown-insert-link) is a general command for inserting new link markup or editing existing link markup. This is especially useful when markup or URL hiding is enabled, so that URLs can't easily be edited directly. This command can be used to insert links of any form: either inline links, reference links, or plain URLs in angle brackets. The URL or [reference] label, link text, and optional title are entered through a series of interactive prompts. The type of link is determined by which values are provided:
- If both a URL and link text are given, insert an inline link: [text](url).
- If both a [reference] label and link text are given, insert a reference link: [text][reference].
- If only link text is given, insert an implicit reference link: [text][].
- If only a URL is given, insert a plain URL link: <url>.
Similarly, C-c C-i (markdown-insert-image) is a general command for inserting or editing image markup. As with the link insertion command, through a series interactive prompts you can insert either an inline or reference image:
- If both a URL and alt text are given, insert an inline image: ![alt text](url).
- If both a [reference] label and alt text are given, insert a reference link: ![alt text][reference].
If there is an existing link or image at the point, these command will edit the existing markup rather than inserting new markup. Otherwise, if there is an active region, these commands use the region as either the default URL (if it seems to be a URL) or link text value otherwise. In that case, the region will be deleted and replaced by the link.

Note that these functions can be used to convert links and images from one type to another (inline, reference, or plain URL) by selectively adding or removing properties via the interactive prompts.

If a reference label is given that is not yet defined, you will be prompted for the URL and optional title and the reference will be inserted according to the value of markdown-reference-location. If a title is given, it will be added to the end of the reference definition and will be used to populate the title attribute when converted to HTML. In addition, it is possible to have the markdown-link-make-text-function function, if non-nil, define the default link text before prompting the user for it.

If markdown-disable-tooltip-prompt is non-nil, the user will not be prompted to add or modify a tooltip text.

Images associated with image links may be displayed inline in the buffer by pressing C-c C-x C-i (markdown-toggle-inline-images). This is a toggle command, so pressing this once again will remove inline images. By default, only local images are displayed. However, remote images will also be downloaded and displayed if markdown-display-remote-images is non-nil.

Large images may be scaled down to fit in the buffer using markdown-max-image-size, a cons cell of the form (max-width . max-height). Resizing requires Emacs to be built with ImageMagick support.
Text Styles: C-c C-s

C-c C-s i inserts markup to make a region or word italic. If there is an active region, make the region italic. If the point is at a non-italic word, make the word italic. If the point is at an italic word or phrase, remove the italic markup. Otherwise, simply insert italic delimiters and place the point in between them. Similarly, use C-c C-s b for bold, C-c C-s c for inline code, and C-c C-s k for inserting <kbd> tags.

C-c C-s q inserts a blockquote using the active region, if any, or starts a new blockquote. C-c C-s Q is a variation which always operates on the region, regardless of whether it is active or not (i.e., when transient-mark-mode is off but the mark is set). The appropriate amount of indentation, if any, is calculated automatically given the surrounding context, but may be adjusted later using the region indentation commands.

C-c C-s p behaves similarly for inserting preformatted code blocks (with C-c C-s P being the region-only counterpart) and C-c C-s C inserts a GFM style backquote fenced code block.
Headings: C-c C-s

To insert or replace headings, there are two options. You can insert a specific level heading directly or you can have markdown-mode determine the level for you based on the previous heading. As with the other markup commands, the heading insertion commands use the text in the active region, if any, as the heading text. Otherwise, if the current line is not blank, they use the text on the current line. Finally, the setext commands will prompt for heading text if there is no active region and the current line is blank.

C-c C-s h inserts a heading with automatically chosen type and level (both determined by the previous heading). C-c C-s H behaves similarly, but uses setext (underlined) headings when possible, still calculating the level automatically. In cases where the automatically-determined level is not what you intended, the level can be quickly promoted or demoted (as described below). Alternatively, a C-u prefix can be given to insert a heading promoted (lower number) by one level or a C-u C-u prefix can be given to insert a heading demoted (higher number) by one level.

To insert a heading of a specific level and type, use C-c C-s 1 through C-c C-s 6 for atx (hash mark) headings and C-c C-s ! or C-c C-s @ for setext headings of level one or two, respectively. Note that ! is S-1 and @ is S-2.

If the point is at a heading, these commands will replace the existing markup in order to update the level and/or type of the heading. To remove the markup of the heading at the point, press C-c C-k to kill the heading and press C-y to yank the heading text back into the buffer.
Horizontal Rules: C-c C-s -

C-c C-s - inserts a horizontal rule. By default, insert the first string in the list markdown-hr-strings (the most prominent rule). With a C-u prefix, insert the last string. With a numeric prefix N, insert the string in position N (counting from 1).
Footnotes: C-c C-s f

C-c C-s f inserts a footnote marker at the point, inserts a footnote definition below, and positions the point for inserting the footnote text. Note that footnotes are an extension to Markdown and are not supported by all processors.
Wiki Links: C-c C-s w

C-c C-s w inserts a wiki link of the form [[WikiLink]]. If there is an active region, use the region as the link text. If the point is at a word, use the word as the link text. If there is no active region and the point is not at word, simply insert link markup. Note that wiki links are an extension to Markdown and are not supported by all processors.
Markdown and Maintenance Commands: C-c C-c

Compile: C-c C-c m will run Markdown on the current buffer and show the output in another buffer. Preview: C-c C-c p runs Markdown on the current buffer and previews, stores the output in a temporary file, and displays the file in a browser. Export: C-c C-c e will run Markdown on the current buffer and save the result in the file basename.html, where basename is the name of the Markdown file with the extension removed. Export and View: press C-c C-c v to export the file and view it in a browser. Open: C-c C-c o will open the Markdown source file directly using markdown-open-command. Live Export: Press C-c C-c l to turn on markdown-live-preview-mode to view the exported output side-by-side with the source Markdown. For all export commands, the output file will be overwritten without notice. markdown-live-preview-window-function can be customized to open in a browser other than eww. If you want to force the preview window to appear at the bottom or right, you can customize markdown-split-window-direction.
```
;; Set custom markdown preview function
(setq markdown-live-preview-window-function #'my-markdown-preview-function)

;; always open the preview window at the right
(setq markdown-split-window-direction 'right)
;; always open the preview window at the bottom
(setq markdown-split-window-direction 'below)

;; delete exported HTML file after markdown-live-preview-export is called
(setq markdown-live-preview-delete-export 'delete-on-export)
```
To summarize:
- C-c C-c m: markdown-command > *markdown-output* buffer.
- C-c C-c p: markdown-command > temporary file > browser.
- C-c C-c e: markdown-command > basename.html.
- C-c C-c v: markdown-command > basename.html > browser.
- C-c C-c w: markdown-command > kill ring.
- C-c C-c o: markdown-open-command.
- C-c C-c l: markdown-live-preview-mode > *eww* buffer.
C-c C-c c will check for undefined references. If there are any, a small buffer will open with a list of undefined references and the line numbers on which they appear. In Emacs 22 and greater, selecting a reference from this list and pressing RET will insert an empty reference definition at the end of the buffer. Similarly, selecting the line number will jump to the corresponding line.

C-c C-c u will check for unused references. This will also open a small buffer if any are found, similar to undefined reference checking. The buffer for unused references will contain X buttons that remove unused references when selected.

C-c C-c n renumbers any ordered lists in the buffer that are out of sequence.

C-c C-c ] completes all headings and normalizes all horizontal rules in the buffer.
Following Links: C-c C-o

Press C-c C-o when the point is on an inline or reference link to open the URL in a browser. When the point is at a wiki link, open it in another buffer (in the current window, or in the other window with the C-u prefix). Use M-p and M-n to quickly jump to the previous or next link of any type.
Doing Things: C-c C-d

Use C-c C-d to do something sensible with the object at the point:
- Jumps between reference links and reference definitions. If more than one link uses the same reference label, a window will be shown containing clickable buttons for jumping to each link. Pressing TAB or S-TAB cycles between buttons in this window.
- Jumps between footnote markers and footnote text.
- Toggles the completion status of GFM task list items (checkboxes).
- Re-aligns table columns.
Promotion and Demotion: C-c C-- and C-c C-=

Headings, horizontal rules, and list items can be promoted and demoted, as well as bold and italic text. For headings, "promotion" means decreasing the level (i.e., moving from <h2> to <h1>) while "demotion" means increasing the level. For horizontal rules, promotion and demotion means moving backward or forward through the list of rule strings in markdown-hr-strings. For bold and italic text, promotion and demotion means changing the markup from underscores to asterisks. Press C-c C-- or C-c LEFT to promote the element at the point if possible.

To remember these commands, note that - is for decreasing the level (promoting), and = (on the same key as +) is for increasing the level (demoting). Similarly, the left and right arrow keys indicate the direction that the atx heading markup is moving in when promoting or demoting.
Completion: C-c C-]

Complete markup is in normalized form, which means, for example, that the underline portion of a setext header is the same length as the heading text, or that the number of leading and trailing hash marks of an atx header are equal and that there is no extra whitespace in the header text. C-c C-] completes the markup at the point, if it is determined to be incomplete.
Editing Lists: M-RET, C-c UP, C-c DOWN, C-c LEFT, and C-c RIGHT

New list items can be inserted with M-RET or C-c C-j. This command determines the appropriate marker (one of the possible unordered list markers or the next number in sequence for an ordered list) and indentation level by examining nearby list items. If there is no list before or after the point, start a new list. As with heading insertion, you may prefix this command by C-u to decrease the indentation by one level. Prefix this command by C-u C-u to increase the indentation by one level.

Existing list items (and their nested sub-items) can be moved up or down with C-c UP or C-c DOWN and indented or outdented with C-c RIGHT or C-c LEFT.
Editing Subtrees: C-c UP, C-c DOWN, C-c LEFT, and C-c RIGHT

Entire subtrees of ATX headings can be promoted and demoted with C-c LEFT and C-c RIGHT, which are the same keybindings used for promotion and demotion of list items. If the point is in a list item, the operate on the list item. Otherwise, they operate on the current heading subtree. Similarly, subtrees can be moved up and down with C-c UP and C-c DOWN.

These commands currently do not work properly if there are Setext headings in the affected region.

Please note the following "boundary" behavior for promotion and demotion. Any level-six headings will not be demoted further (i.e., they remain at level six, since Markdown and HTML define only six levels) and any level-one headings will promoted away entirely (i.e., heading markup will be removed, since a level-zero heading is not defined).
Shifting the Region: C-c < and C-c >

Text in the region can be indented or outdented as a group using C-c > to indent to the next indentation point (calculated in the current context), and C-c < to outdent to the previous indentation point. These keybindings are the same as those for similar commands in python-mode.
Killing Elements: C-c C-k

Press C-c C-k to kill the thing at point and add important text, without markup, to the kill ring. Possible things to kill include (roughly in order of precedece): inline code, headings, horizontal rules, links (add link text to kill ring), images (add alt text to kill ring), angle URIs, email addresses, bold, italics, reference definitions (add URI to kill ring), footnote markers and text (kill both marker and text, add text to kill ring), and list items.
Outline Navigation: C-c C-n, C-c C-p, C-c C-f, C-c C-b, and C-c C-u

These keys are used for hierarchical navigation in lists and headings. When the point is in a list, they move between list items. Otherwise, they move between headings. Use C-c C-n and C-c C-p to move between the next and previous visible headings or list items of any level. Similarly, C-c C-f and C-c C-b move to the next and previous visible headings or list items at the same level as the one at the point. Finally, C-c C-u will move up to the parent heading or list item.
Movement by Markdown paragraph: M-{, M-}, and M-h

Paragraphs in markdown-mode are regular paragraphs, paragraphs inside blockquotes, individual list items, headings, etc. These keys are usually bound to forward-paragraph and backward-paragraph, but the built-in Emacs functions are based on simple regular expressions that fail in Markdown files. Instead, they are bound to markdown-forward-paragraph and markdown-backward-paragraph. To mark a paragraph, you can use M-h (markdown-mark-paragraph).
Movement by Markdown block: C-M-{, C-M-}, and C-c M-h

Markdown blocks are regular paragraphs in many cases, but contain many paragraphs in other cases: blocks are considered to be entire lists, entire code blocks, and entire blockquotes. To move backward one block use C-M-{ (markdown-beginning-block) and to move forward use C-M-} (markdown-end-of-block). To mark a block, use C-c M-h (markdown-mark-block).
Movement by Defuns: C-M-a, C-M-e, and C-M-h

The usual Emacs commands can be used to move by defuns (top-level major definitions). In markdown-mode, a defun is a section. As usual, C-M-a will move the point to the beginning of the current or preceding defun, C-M-e will move to the end of the current or following defun, and C-M-h will put the region around the entire defun.
Table Editing:

Markdown Mode includes support for editing tables, which have the following basic format:
```
| Right | Left | Center | Default |
|------:|:-----|:------:|---------|
|    12 | 12   | 12     | 12      |
|   123 | 123  | 123    | 123     |
|     1 | 1    | 1      | 1       |
```
The first line contains column headers. The second line contains a separator line between the headers and the content. Each following line is a row in the table. Columns are always separated by the pipe character. The colons indicate column alignment.

A table is re-aligned automatically each time you press TAB or RET inside the table. TAB also moves to the next field (RET to the next row) and creates new table rows at the end of the table or before horizontal separator lines. The indentation of the table is set by the first line. Column centering inside Emacs is not supported.

Beginning pipe characters are required for proper detection of table borders inside Emacs. Any line starting with |- or |: is considered as a horizontal separator line and will be expanded on the next re-align to span the whole table width. No padding is allowed between the beginning pipe character and header separator symbol. So, to create the above table, you would only type
```
|Right|Left|Center|Default|
|-
```
and then press TAB to align the table and start filling in cells.

Then you can jump with TAB from one cell to the next or with S-TAB to the previous one. RET will jump to the to the next cell in the same column, and create a new row if there is no such cell or if the next row is beyond a separator line.

You can also convert selected region to a table. Basic editing capabilities include inserting, deleting, and moving of columns and rows, and table re-alignment, sorting, transposition:
- C-c UP or C-c DOWN - Move the current row up or down.
- C-c LEFT or C-c RIGHT - Move the current column left or right.
- C-c S-UP - Kill the current row.
- C-c S-DOWN - Insert a row above the current row. With a prefix argument, row line is created below the current one.
- C-c S-LEFT - Kill the current column.
- C-c S-RIGHT - Insert a new column to the left of the current one.
- C-c C-d - Re-align the current table (markdown-do).
- C-c C-c ^ - Sort the rows of a table by a specified column. This command prompts you for the column number and a sort method (alphabetical or numerical, optionally in reverse).
- C-c C-c | - Convert the region to a table. This function attempts to recognize comma, tab, and space separated data and then splits the data into cells accordingly.
- C-c C-c t - Transpose table at point.
The table editing functions try to handle markup hiding correctly when calculating column widths, however, columns containing hidden markup may not always be aligned properly.

C-c C-s t (markdown-insert-table) is a general command for inserting new table. The command prompts for table size and column alignment and inserts an empty pipe table at point.
Viewing Modes:

Read-only viewing modes, markdown-view-mode and gfm-view-mode are provided for viewing Markdown content. These modes provide simplified keybindings for navigating the buffer. Many of these are like help-mode and view-mode, such as SPC, DEL, <, and > for scrolling, q for quitting, and ? or h for help. Other keys are provided that mirror the outline navigation commands when editing: n, p, f, b, and u. Both of these modes enable markup hiding by default, but this can be customized by setting markdown-hide-markup-in-view-modes.
Miscellaneous Commands:

When the edit-indirect package is installed, C-c ' (markdown-edit-code-block) can be used to edit a code block in an indirect buffer in the native major mode. Press C-c C-c to commit changes and return or C-c C-k to cancel. You can also give a prefix argument to the insertion command, as in C-u C-c C-s C, to edit the code block in an indirect buffer upon insertion.

As noted, many of the commands above behave differently depending on whether Transient Mark mode is enabled or not. When it makes sense, if Transient Mark mode is on and the region is active, the command applies to the text in the region (e.g., C-c C-s b makes the region bold). For users who prefer to work outside of Transient Mark mode, since Emacs 22 it can be enabled temporarily by pressing C-SPC C-SPC. When this is not the case, many commands then proceed to look work with the word or line at the point.

When applicable, commands that specifically act on the region even outside of Transient Mark mode have the same keybinding as their standard counterpart, but the letter is uppercase. For example, markdown-insert-blockquote is bound to C-c C-s q and only acts on the region in Transient Mark mode while markdown-blockquote-region is bound to C-c C-s Q and always applies to the region (when nonempty).

Note that these region-specific functions are useful in many cases where it may not be obvious. For example, yanking text from the kill ring sets the mark at the beginning of the yanked text and moves the point to the end. Therefore, the (inactive) region contains the yanked text. So, C-y followed by C-c C-s Q will yank text and turn it into a blockquote.

markdown-mode attempts to be flexible in how it handles indentation. When you press TAB repeatedly, the point will cycle through several possible indentation levels corresponding to things you might have in mind when you press RET at the end of a line or TAB. For example, you may want to start a new list item, continue a list item with hanging indentation, indent for a nested pre block, and so on. Outdenting is handled similarly when backspace is pressed at the beginning of the non-whitespace portion of a line.

markdown-mode supports outline-minor-mode as well as org-mode-style visibility cycling for atx- or hash-style headings. There are two types of visibility cycling: Pressing S-TAB cycles globally between the table of contents view (headings only), outline view (top-level headings only), and the full document view. Pressing TAB while the point is at a heading will cycle through levels of visibility for the subtree: completely folded, visible children, and fully visible. Note that mixing hash and underline style headings will give undesired results.

Customization

Although no configuration is necessary there are a few things that can be customized. The M-x customize-mode command provides an interface to all of the possible customizations:

markdown-command - the command used to run Markdown (default: markdown). This variable may be customized to pass command-line options to your Markdown processor of choice. We recommend you to use list of strings if you want to set command line options like. '("pandoc" "--from=markdown" "--to=html5"). It can also be a function; in this case markdown will call it with three arguments or four arguments, depending on markdown-command-needs-filename. The first three arguments are: the beginning and end of the region to process, and a buffer to write the output to. When markdown-command-needs-filename is t, the fourth argument is set to the name of the file.
markdown-command-needs-filename - set to t if markdown-command does not accept standard input (default: nil). When nil, markdown-mode will pass the Markdown content to markdown-command using standard input (stdin). When set to t, markdown-mode will pass the name of the file as the final command-line argument to markdown-command. Note that in the latter case, you will only be able to run markdown-command from buffers which are visiting a file.
markdown-open-command - the command used for calling a standalone Markdown previewer which is capable of opening Markdown source files directly (default: nil). This command will be called with a single argument, the filename of the current buffer. A representative program is the Mac app Marked 2, a live-updating Markdown previewer which can be called from a simple shell script. This variable can also be a function; in this case markdown-open will call it without arguments to preview the current buffer.
markdown-open-image-command - the command used for opening image link (default: nil) via markdown-follow-* commands. This variable can also be a function, in this case it is called with a single argument, image-link. If this value is nil, markdown-mode opens image links by find-file.
markdown-hr-strings - list of strings to use when inserting horizontal rules. Different strings will not be distinguished when converted to HTML--they will all be converted to <hr/>--but they may add visual distinction and style to plain text documents. To maintain some notion of promotion and demotion, keep these sorted from largest to smallest.
markdown-bold-underscore - set to a non-nil value to use two underscores when inserting bold text instead of two asterisks (default: nil).
markdown-italic-underscore - set to a non-nil value to use underscores when inserting italic text instead of asterisks (default: nil).
markdown-asymmetric-header - set to a non-nil value to use asymmetric header styling, placing header characters only on the left of headers (default: nil).
markdown-header-scaling - set to a non-nil value to use a variable-pitch font for headings where the size corresponds to the level of the heading (default: nil).
markdown-header-scaling-values - list of scaling values, relative to baseline, for headers of levels one through six, used when markdown-header-scaling is non-nil (default: (2.0 1.7 1.4 1.1 1.0 1.0)).
markdown-marginalize-headers - put opening atx header markup in the left margin when non-nil (default: nil).
markdown-marginalize-headers-margin-width - width of margin used for marginalized headers (default: 6).
markdown-list-indent-width - depth of indentation for lists when inserting, promoting, and demoting list items (default: 4).
markdown-indent-function - the function to use for automatic indentation (default: markdown-indent-line).
markdown-indent-on-enter - Set to a non-nil value to automatically indent new lines when RET is pressed. Set to indent-and-new-item to additionally continue lists when RET is pressed (default: t).
markdown-enable-wiki-links - syntax highlighting for wiki links (default: nil). Set this to a non-nil value to turn on wiki link support by default. Wiki link support can be toggled later using the function markdown-toggle-wiki-links."
markdown-wiki-link-alias-first - set to a non-nil value to treat aliased wiki links like [[link text|PageName]] (default: t). When set to nil, they will be treated as [[PageName|link text]].
markdown-uri-types - a list of protocol schemes (e.g., "http") for URIs that markdown-mode should highlight.
markdown-enable-math - font lock for inline and display LaTeX math expressions (default: nil). Set this to t to turn on math support by default. Math support can be toggled interactively later using C-c C-x C-e (markdown-toggle-math).
markdown-enable-html - font lock for HTML tags and attributes (default: t).
markdown-css-paths - CSS files to link to in XHTML output (default: nil). These can be either local files (relative or absolute) or URLs.
markdown-content-type - used to set to the http-equiv attribute to be included in the XHTML <head> block (default: "text/html"). Set to an alternate value application/xhtml+xml if needed, or set to an empty string to remove the attribute. See also: markdown-coding-system.
markdown-coding-system - used for specifying the character set identifier in the http-equiv attribute when included (default: nil). See markdown-content-type, which must be set for this variable to have any effect. When set to nil, buffer-file-coding-system will be used to automatically determine the coding system string (falling back to utf-8 when unavailable). Common settings are iso-8859-1 and iso-latin-1.
markdown-xhtml-header-content - additional content to include in the XHTML <head> block (default: "").
markdown-xhtml-body-preamble - additional content to include in the XHTML block, before the output (default: ""). This is useful for enclosing additional elements around the Markdown output.
markdown-xhtml-body-epilogue - additional content to include in the XHTML block, after the output (default: ""). This is useful for enclosing additional elements around the Markdown output.
markdown-xhtml-standalone-regexp - a regular expression which markdown-mode uses to determine whether the output of markdown-command is a standalone XHTML document or an XHTML fragment (default: "^\$<\\?xml\\|<!DOCTYPE\\|<html\$"). If this regular expression not matched in the first five lines of output, markdown-mode assumes the output is a fragment and adds a header and footer.
markdown-link-space-sub-char - a character to replace spaces when mapping wiki links to filenames (default: "_"). For example, use an underscore for compatibility with the Python Markdown WikiLinks extension. In gfm-mode, this is set to "-" to conform with GitHub wiki links.
markdown-reference-location - where to insert reference definitions (default: header). The possible locations are the end of the document (end), after the current block (immediately), the end of the current subtree (subtree), or before the next header (header).
markdown-footnote-location - where to insert footnote text (default: end). The set of location options is the same as for markdown-reference-location.
markdown-nested-imenu-heading-index - Use nested imenu heading instead of a flat index (default: t). A nested index may provide more natural browsing from the menu, but a flat list may allow for faster keyboard navigation via tab completion.
markdown-add-footnotes-to-imenu - Add footnote definitions to the end of the imenu index (default: t).
comment-auto-fill-only-comments - variable is made buffer-local and set to nil by default. In programming language modes, when this variable is non-nil, only comments will be filled by auto-fill-mode. However, comments in Markdown documents are rare and the most users probably intend for the actual content of the document to be filled. Making this variable buffer-local allows markdown-mode to override the default behavior induced when the global variable is non-nil.
markdown-gfm-additional-languages, - additional languages to make available, aside from those predefined in markdown-gfm-recognized-languages, when inserting GFM code blocks (default: nil). Language strings must have be trimmed of whitespace and not contain any curly braces. They may be of arbitrary capitalization, though.
markdown-gfm-use-electric-backquote - use markdown-electric-backquote for interactive insertion of GFM code blocks when backquote is pressed three times (default: t).
markdown-make-gfm-checkboxes-buttons - Whether GitHub Flavored Markdown style task lists (checkboxes) should be turned into buttons that can be toggled with mouse-1 or RET. If non-nil (default), then buttons are enabled. This works in markdown-mode as well as gfm-mode.
markdown-hide-urls - Determines whether URL and reference labels are hidden for inline and reference links (default: nil). When non-nil, inline links will appear in the buffer as [link](∞) instead of [link](http://perhaps.a/very/long/url/). To change the placeholder (composition) character used, set the variable markdown-url-compose-char. URL hiding can be toggled interactively using C-c C-x C-l (markdown-toggle-url-hiding) or from the Markdown | Links & Images menu.
markdown-hide-markup - Determines whether all possible markup is hidden or otherwise beautified (default: nil). The actual buffer text remains unchanged, but the display will be altered. Brackets and URLs for links will be hidden, asterisks and underscores for italic and bold text will be hidden, text bullets for unordered lists will be replaced by Unicode bullets, and so on. Since this includes URLs and reference labels, when non-nil this setting supersedes markdown-hide-urls. Markup hiding can be toggled using C-c C-x C-m (markdown-toggle-markup-hiding) or from the Markdown | Show & Hide menu.

Unicode bullets are used to replace ASCII list item markers. The list of characters used, in order of list level, can be specified by setting the variable markdown-list-item-bullets. The placeholder characters used to replace other markup can be changed by customizing the corresponding variables: markdown-blockquote-display-char, markdown-hr-display-char, and markdown-definition-display-char.
markdown-fontify-code-blocks-natively - Whether to fontify code in code blocks using the native major mode. This only works for fenced code blocks where the language is specified where we can automatically determine the appropriate mode to use. The language to mode mapping may be customized by setting the variable markdown-code-lang-modes. This can be toggled interactively by pressing C-c C-x C-f (markdown-toggle-fontify-code-blocks-natively).
markdown-gfm-uppercase-checkbox - When non-nil, complete GFM task list items with [X] instead of [x] (default: nil). This is useful for compatibility with org-mode, which doesn't recognize the lowercase variant.
markdown-translate-filename-function - A function to be used to translate filenames in links.
markdown-unordered-list-item-prefix - When non-nil, markdown-insert-list-item inserts enumerated numbers for ordered list marker. While nil, it always inserts 1..
markdown-enable-highlighting-syntax - font lock for highlighting syntax like Obsidian, Quilt(default: nil).
markdown-fontify-whole-heading-line - font lock for highlighting the whole line for headings.(default: nil)
markdown-special-ctrl-a/e - set to non-nil to behave specially in headlines and items. When t, C-a will bring back the cursor to the beginning of the headline text. In an item, this will be the position after bullet and check-box, if any. C-e will jump to the end of the headline, ignoring the presence of closing tags in the headline. When set to the symbol reversed, the first C-a or C-e works normally, going to the true line boundary first. Only a directly following, identical keypress will bring the cursor to the special positions (default: nil).

Additionally, the faces used for syntax highlighting can be modified to your liking by issuing M-x customize-group RET markdown-faces or by using the "Markdown Faces" link at the bottom of the mode customization screen.

Extensions

Besides supporting the basic Markdown syntax, Markdown Mode also includes syntax highlighting for [[Wiki Links]]. This can be enabled by setting markdown-enable-wiki-links to a non-nil value. Wiki links may be followed by pressing C-c C-o when the point is at a wiki link. Use M-p and M-n to quickly jump to the previous and next links (including links of other types). Aliased or piped wiki links of the form [[link text|PageName]] are also supported. Since some wikis reverse these components, set markdown-wiki-link-alias-first to nil to treat them as [[PageName|link text]]. If markdown-wiki-link-fontify-missing is also non-nil, Markdown Mode will highlight wiki links with missing target file in a different color. By default, Markdown Mode only searches for target files in the current directory. You can control search type by setting markdown-wiki-link-search-type. This value type is a symbol list. Possible values are

sub-directories : search in sub directories
parent-directories : search in parent directories
project : search under project root

SmartyPants support is possible by customizing markdown-command. If you install SmartyPants.pl at, say, /usr/local/bin/smartypants, then you can set markdown-command to "markdown | smartypants". You can do this either by using M-x customize-group markdown or by placing the following in your .emacs file:

(setq markdown-command "markdown | smartypants")

Syntax highlighting for mathematical expressions written in LaTeX (only expressions denoted by $..$ , $$..$$, or \[..\]) can be enabled by setting markdown-enable-math to a non-nil value, either via customize or by placing (setq markdown-enable-math t) in .emacs, and then restarting Emacs or calling markdown-reload-extensions.

GitHub Flavored Markdown (GFM)

A GitHub Flavored Markdown mode, gfm-mode, is also available. The GitHub implementation differs slightly from standard Markdown in that it supports things like different behavior for underscores inside of words, automatic linking of URLs, strikethrough text, and fenced code blocks with an optional language keyword.

The GFM-specific features above apply to README.md files, wiki pages, and other Markdown-formatted files in repositories on GitHub. GitHub also enables additional features for writing on the site (for issues, pull requests, messages, etc.) that are further extensions of GFM. These features include task lists (checkboxes), newlines corresponding to hard line breaks, auto-linked references to issues and commits, wiki links, and so on. To make matters more confusing, although task lists are not part of GFM proper, since 2014 they are rendered (in a read-only fashion) in all Markdown documents in repositories on the site. These additional extensions are supported to varying degrees by markdown-mode and gfm-mode as described below.

URL autolinking: Both markdown-mode and gfm-mode support highlighting of URLs without angle brackets.
Multiple underscores in words: You must enable gfm-mode to toggle support for underscores inside of words. In this mode variable names such as a_test_variable will not trigger emphasis (italics).
Fenced code blocks: Code blocks quoted with backquotes, with optional programming language keywords, are highlighted in both markdown-mode and gfm-mode. They can be inserted with C-c C-s C. If there is an active region, the text in the region will be placed inside the code block. You will be prompted for the name of the language, but may press enter to continue without naming a language.

In addition, in gfm-mode, GFM code blocks can be inserted via the option markdown-gfm-use-electric-backquote. If the option markdown-code-block-braces is set to t, code blocks inserted with C-c C-s C or electric backquotes will include braces ("{}") around the language attributes.
Strikethrough: Strikethrough text is supported in both markdown-mode and gfm-mode. It can be inserted (and toggled) using C-c C-s s.
Task lists: GFM task lists will be rendered as checkboxes (Emacs buttons) in both markdown-mode and gfm-mode when markdown-make-gfm-checkboxes-buttons is set to a non-nil value (and it is set to t by default). These checkboxes can be toggled by clicking mouse-1, pressing RET over the button, or by pressing C-c C-d (markdown-do) with the point anywhere in the task list item. A normal list item can be turned to a check list item by the same command, or more specifically C-c C-s [ (markdown-insert-gfm-checkbox).
Wiki links: Generic wiki links are supported in markdown-mode, but in gfm-mode specifically they will be treated as they are on GitHub: spaces will be replaced by hyphens in filenames and the first letter of the filename will be capitalized. For example, [[wiki link]] will map to a file named Wiki-link with the same extension as the current file. If a file with this name does not exist in the current directory, the first match in a subdirectory, if any, will be used instead.
Newlines: Neither markdown-mode nor gfm-mode do anything specifically with respect to newline behavior. If you use gfm-mode mostly to write text for comments or issues on the GitHub site--where newlines are significant and correspond to hard line breaks--then you may want to enable visual-line-mode for line wrapping in buffers. You can do this with a gfm-mode-hook as follows:
```
;; Use visual-line-mode in gfm-mode
(defun my-gfm-mode-hook ()
  (visual-line-mode 1))
(add-hook 'gfm-mode-hook 'my-gfm-mode-hook)
```
Preview: GFM-specific preview can be powered by setting markdown-command to use Docter. This may also be configured to work with Marked 2 for markdown-open-command.

Acknowledgments

markdown-mode has benefited greatly from the efforts of the many volunteers who have sent patches, test cases, bug reports, suggestions, helped with packaging, etc. Thank you for your contributions! See the contributors graph for details.

Bugs

markdown-mode is developed and tested primarily for compatibility with GNU Emacs 27.1 and later. If you find any bugs in markdown-mode, please construct a test case or a patch and open a ticket on the GitHub issue tracker. See the contributing guidelines in CONTRIBUTING.md for details on creating pull requests.

History

markdown-mode was written and is maintained by Jason Blevins. The first version was released on May 24, 2007.

2007-05-24: Version 1.1
2007-05-25: Version 1.2
2007-06-05: Version 1.3
2007-06-29: Version 1.4
2007-10-11: Version 1.5
2008-06-04: Version 1.6
2009-10-01: Version 1.7
2011-08-12: Version 1.8
2011-08-15: Version 1.8.1
2013-01-25: Version 1.9
2013-03-24: Version 2.0
2016-01-09: Version 2.1
2017-05-26: Version 2.2
2017-08-31: Version 2.3
2020-05-30: Version 2.4
2022-02-12: Version 2.5
2023-08-30: Version 2.6

TanklesXL/gladvent

2024-12-07T01:31:01Z

A Gleam framework for running Advent of Code solutions

henrygd/beszel

2024-12-08T01:39:16Z

Lightweight server monitoring hub with historical data, docker stats, and alerts.

Beszel

A lightweight server resource monitoring hub with historical data, docker stats, and alerts.

Features

Lightweight: Smaller and less resource-intensive than leading solutions.
Simple: Easy setup, no need for public internet exposure.
Docker stats: Tracks CPU, memory, and network usage history for each container.
Alerts: Configurable alerts for CPU, memory, disk, bandwidth, temperature, and system status.
Multi-user: Each user manages their own systems. Admins can share systems across users.
OAuth / OIDC: Supports multiple OAuth2 providers. Password authentication can be disabled.
Automatic backups: Save and restore data from disk or S3-compatible storage.
REST API: Use or update your data in your own scripts and applications.

Introduction

Beszel consists of two main components: the hub and the agent.

Hub: A web application that provides a dashboard for viewing and managing connected systems. Built on PocketBase.
Agent: Runs on each system you want to monitor, creating a minimal SSH server to communicate system metrics to the hub.

Getting started

If not using docker, skip steps 4-5 and run the agent using the binary.

Start the hub (see installation).
Open http://localhost:8090 and create an admin user.
Click "Add system." Enter the name and host of the system you want to monitor.
Click "Copy docker compose" to copy the agent's docker-compose.yml file to your clipboard.
On the agent system, create the compose file and run docker compose up to start the agent.
Back in the hub, click the "Add system" button in the dialog to finish adding the system.

If all goes well, the system should flip to green. If it turns red, check the Logs page and refer to troubleshooting tips.

Tutoriel en français

Pour le tutoriel en français, consultez https://belginux.com/installer-beszel-avec-docker/

Installation

You can install the hub and agent as single binaries or using Docker.

Docker

Hub: See the example docker-compose.yml file.

Agent: The hub provides compose content for the agent, but you can also reference the example docker-compose.yml file.

The agent uses host network mode to access network interface stats, which automatically exposes the port. Change the port using an environment variable if needed.

If you don't need network stats, remove that line from the compose file and map the port manually.

Binary

[!TIP] If using Linux, see guides/systemd.md for a script to install the hub or agent as a system service. This is also built into the web UI.

Download and run the latest binaries from the releases page or use the commands below.

Hub

curl -sL "https://github.com/henrygd/beszel/releases/latest/download/beszel_$(uname -s)_$(uname -m | sed 's/x86_64/amd64/' | sed 's/armv7l/arm/' | sed 's/aarch64/arm64/').tar.gz" | tar -xz -O beszel | tee ./beszel >/dev/null && chmod +x beszel && ls beszel

Running the hub directly:

./beszel serve

Agent

curl -sL "https://github.com/henrygd/beszel/releases/latest/download/beszel-agent_$(uname -s)_$(uname -m | sed 's/x86_64/amd64/' | sed 's/armv7l/arm/' | sed 's/aarch64/arm64/').tar.gz" | tar -xz -O beszel-agent | tee ./beszel-agent >/dev/null && chmod +x beszel-agent && ls beszel-agent

Running the agent directly:

PORT=45876 KEY="{PASTE_YOUR_KEY}" ./beszel-agent

Updating

Use ./beszel update and ./beszel-agent update to update to the latest version.

Environment variables

Hub

Name	Default	Description
`CSP`	unset	Adds a Content-Security-Policy header with this value.
`DISABLE_PASSWORD_AUTH`	false	Disables password authentication.

Agent

Name	Default	Description
`DOCKER_HOST`	unset	Overrides the docker host (docker.sock) if using a proxy.[^socket]
`EXTRA_FILESYSTEMS`	unset	See Monitoring additional disks, partitions, or remote mounts
`FILESYSTEM`	unset	Device, partition, or mount point to use for root disk stats.
`KEY`	unset	Public SSH key to use for authentication. Provided in hub.
`LOG_LEVEL`	info	Logging level. Valid values: "debug", "info", "warn", "error".
`MEM_CALC`	unset	Overrides the default memory calculation.[^memcalc]
`NICS`	unset	Whitelist of network interfaces to monitor for bandwidth chart.
`PORT`	45876	Port or address:port to listen on.
`SENSORS`	unset	Whitelist of temperature sensors to monitor.
`SYS_SENSORS`	unset	Overrides sys path for sensors. See #160.

[^socket]: Beszel only needs access to read container information. For linuxserver/docker-socket-proxy you would set CONTAINERS=1. [^memcalc]: The default value for used memory is based on gopsutil's Used calculation, which should align fairly closely with free. Set MEM_CALC to htop to align with htop's calculation.

OAuth / OIDC Setup

Beszel supports OpenID Connect and many OAuth2 authentication providers (see list below).

Visit the "Auth providers" page to enable your provider. The redirect / callback URL should be <your-beszel-url>/api/oauth2-redirect.

Supported provider list

Apple
Bitbucket
Discord
Facebook
Gitea
Gitee
GitHub
GitLab
Google
Instagram
Kakao
LiveChat
mailcow
Microsoft
OpenID Connect
Patreon (v2)
Spotify
Strava
Twitch
Twitter
VK
Yandex

Monitoring additional disks, partitions, or remote mounts

The method for adding additional disks differs depending on your deployment method.

Use lsblk to find the names and mount points of your partitions. If you have trouble, check the agent logs.

Note: The charts will use the name of the device or partition if available, and fall back to the folder name. You will not get I/O stats for network mounted drives.

Docker

Mount a folder from the target filesystem in the container's /extra-filesystems directory. For example:

volumes:
  - /mnt/disk1/.beszel:/extra-filesystems/sdb1:ro
  - /dev/mmcblk0/.beszel:/extra-filesystems/mmcblk0:ro

Binary

Set the EXTRA_FILESYSTEMS environment variable to a comma-separated list of devices, partitions, or mount points to monitor. For example:

EXTRA_FILESYSTEMS="sdb,sdc1,mmcblk0,/mnt/network-share"

REST API

Because Beszel is built on PocketBase, you can use the PocketBase web APIs and client-side SDKs to read or update data from outside Beszel itself.

Security

The hub and agent communicate over SSH, so they don't need to be exposed to the internet. Even if you place an external auth gateway, such as Authelia, in front of the hub, it won't disrupt or break the connection between the hub and agent.

When the hub is started for the first time, it generates an ED25519 key pair.

The agent's SSH server is configured to accept connections using this key only. It does not provide a pseudo-terminal or accept input, so it's impossible to execute commands on the agent even if your private key is compromised.

User roles

Admin

Admins have access to additional links in the hub, such as backups, SMTP settings, etc. The first user created is automatically an admin and can log into PocketBase.

Changing a user's role does not create a PocketBase admin account for them. To do that, go to Settings > Admins in PocketBase and add them manually.

User

Users can create their own systems and alerts. Links to PocketBase settings are not shown in the hub.

Read only

Read-only users cannot create systems but can view any system shared with them by an admin and create alerts.

FAQ / Troubleshooting

Agent is not connecting

Assuming the agent is running, the connection is probably being blocked by a firewall. You have two options:

Add an inbound rule to the agent system's firewall(s) to allow TCP connections to the port. Check any active firewalls, like iptables, and your cloud provider's firewall settings if applicable.
Alternatively, use software like Cloudflare Tunnel, WireGuard, or Tailscale to securely bypass your firewall.

You can test connectivity by running telnet <agent-ip> <port>.

Connecting the hub and agent on the same system using Docker

If using host network mode for the agent but not the hub, add your system using the hostname host.docker.internal, which resolves to the internal IP address used by the host. See the example docker-compose.yml.

If using host network mode for both, you can use localhost as the hostname.

Otherwise, use the agent's container_name as the hostname if both are in the same Docker network.

Finding the correct filesystem

Specify the filesystem/device/partition for root disk stats using the FILESYSTEM environment variable.

If not set, the agent will try to find the partition mounted on / and use that. This may not work correctly in a container, so it's recommended to set this value. Use one of the following methods to find the correct filesystem:

Run lsblk and choose an option under "NAME."
Run df -h and choose an option under "Filesystem."
Run sudo fdisk -l and choose an option under "Device."

Docker container charts are empty or missing

If container charts show empty data or don't appear at all, you may need to enable cgroup memory accounting. To verify, run docker stats. If that shows zero memory usage, follow this guide to fix the issue:

https://akashrajpurohit.com/blog/resolving-missing-memory-stats-in-docker-stats-on-raspberry-pi/

Docker Containers Are Not Populating Reliably

Try upgrading your Docker version on the agent system. This issue was observed on a machine running version 24 and was resolved by upgrading to version 27.

Month / week records are not populating reliably

Records for longer time periods are created by averaging stats from shorter periods. The agent must run uninterrupted for a full set of data to populate these records.

Pausing/unpausing the agent for longer than one minute will result in incomplete data, resetting the timing for the current interval.

Compiling

Both the hub and agent are written in Go, so you can easily build them yourself, or cross-compile for different platforms. Please install Go first if you haven't already.

Using Makefile

Run make in /beszel. This creates a build directory containing the binaries.

cd beszel && make

You can also build for different platforms:

make OS=freebsd ARCH=arm64

See a list of valid options by running go tool dist list.

Manual compilation

Prepare dependencies

cd beszel && go mod tidy

Agent

Go to beszel/cmd/agent and run the following command to create a binary in the current directory:

CGO_ENABLED=0 go build -ldflags "-w -s" .

Hub

The hub embeds the web UI in the binary, so you must build the website first. I use Bun, but you may use Node.js if you prefer:

cd beszel/site
bun install
bun run build

Then in beszel/cmd/hub:

CGO_ENABLED=0 go build -ldflags "-w -s" .

Cross-compiling

You can cross-compile for different platforms using the GOOS and GOARCH environment variables.

For example, to build for FreeBSD ARM64:

GOOS=freebsd GOARCH=arm64 CGO_ENABLED=0 go build -ldflags "-w -s" .

See a list of valid options by running go tool dist list.

Contributing

Contributions are welcome, but it's a good idea to check with us first in a discussion / issue if you plan on doing anything significant.

We use Crowdin to manage translations. New languages or improvements to existing translations are appreciated!

We'll have more helpful information about contributing to Beszel in the near future.

License

Beszel is licensed under the MIT License. See the LICENSE file for more details.

nezhahq/nezha

2024-12-08T01:39:16Z

Self-hosted, lightweight server and website monitoring and O&M tool

LOGO designed by 熊大 .

) Nezha Monitoring: Self-hostable, lightweight, servers and websites monitoring and O&M tool.

Supports monitoring system status, HTTP (SSL certificate change, upcoming expiration, expired), TCP, Ping and supports push alerts, run scheduled tasks and web terminal.

>> Telegram Channel: 哪吒监控（中文通知频道）

>> Telegram Group: Nezha Monitoring Global (English Only), 哪吒监控（中文群组）

>> Use Cases | 我们的用户 (Google)

User Guide

Screenshots

用户前台 @hamster1963	管理后台 @uubulb

hamster1963/nezha-dash	nezhahq/admin-frontend

Supported Languages

English
中文
Español

You can change the dashboard language in the settings page (/setting) after the dashboard is installed.

Contributors

Special Thanks

IPInfo for providing an accurate GeoIP Database.

Star History

qdm12/gluetun

2024-12-08T01:39:16Z

VPN client in a thin Docker container for multiple VPN providers, written in Go, and using OpenVPN or Wireguard, DNS over TLS, with a few proxy servers built-in.

Gluetun VPN client

Lightweight swiss-knife-like VPN client to multiple VPN service providers

Quick links

Setup
Features
Problem?
- Check the Wiki common errors and faq
- Start a discussion
- Fix the Unraid template
Suggestion?
- Create an issue
Happy?
- Sponsor me on github.com/sponsors/qdm12
- Donate to paypal.me/qmcgaw
- Drop me an email
Want to add a VPN provider? check the development page and add a provider page
Video:
Substack Console interview

Features

Based on Alpine 3.20 for a small Docker image of 35.6MB
Supports: AirVPN, Cyberghost, ExpressVPN, FastestVPN, Giganews, HideMyAss, IPVanish, IVPN, Mullvad, NordVPN, Perfect Privacy, Privado, Private Internet Access, PrivateVPN, ProtonVPN, PureVPN, SlickVPN, Surfshark, TorGuard, VPNSecure.me, VPNUnlimited, Vyprvpn, WeVPN, Windscribe servers
Supports OpenVPN for all providers listed
Supports Wireguard both kernelspace and userspace
- For AirVPN, FastestVPN, Ivpn, Mullvad, NordVPN, Perfect privacy, ProtonVPN, Surfshark and Windscribe
- For Cyberghost, Private Internet Access, PrivateVPN, PureVPN, Torguard, VPN Unlimited, VyprVPN and WeVPN using the custom provider
- For custom Wireguard configurations using the custom provider
- More in progress, see #134
DNS over TLS baked in with service provider(s) of your choice
DNS fine blocking of malicious/ads/surveillance hostnames and IP addresses, with live update every 24 hours
Choose the vpn network protocol, udp or tcp
Built in firewall kill switch to allow traffic only with needed the VPN servers and LAN devices
Built in Shadowsocks proxy server (protocol based on SOCKS5 with an encryption layer, tunnels TCP+UDP)
Built in HTTP proxy (tunnels HTTP and HTTPS through TCP)
Connect other containers to it
Connect LAN devices to it
Compatible with amd64, i686 (32 bit), ARM 64 bit, ARM 32 bit v6 and v7, and even ppc64le 🎆
Custom VPN server side port forwarding for Perfect Privacy, Private Internet Access, PrivateVPN and ProtonVPN
Possibility of split horizon DNS by selecting multiple DNS over TLS providers
Can work as a Kubernetes sidecar container, thanks @rorph

Setup

🎉 There are now instructions specific to each VPN provider with examples to help you get started as quickly as possible!

Go to the Wiki!

🐛 Found a bug in the Wiki?!

Here's a docker-compose.yml for the laziest:

version: "3"
services:
  gluetun:
    image: qmcgaw/gluetun
    # container_name: gluetun
    # line above must be uncommented to allow external containers to connect.
    # See https://github.com/qdm12/gluetun-wiki/blob/main/setup/connect-a-container-to-gluetun.md#external-container-to-gluetun
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun:/dev/net/tun
    ports:
      - 8888:8888/tcp # HTTP proxy
      - 8388:8388/tcp # Shadowsocks
      - 8388:8388/udp # Shadowsocks
    volumes:
      - /yourpath:/gluetun
    environment:
      # See https://github.com/qdm12/gluetun-wiki/tree/main/setup#setup
      - VPN_SERVICE_PROVIDER=ivpn
      - VPN_TYPE=openvpn
      # OpenVPN:
      - OPENVPN_USER=
      - OPENVPN_PASSWORD=
      # Wireguard:
      # - WIREGUARD_PRIVATE_KEY=wOEI9rqqbDwnN8/Bpp22sVz48T71vJ4fYmFWujulwUU=
      # - WIREGUARD_ADDRESSES=10.64.222.21/32
      # Timezone for accurate log times
      - TZ=
      # Server list updater
      # See https://github.com/qdm12/gluetun-wiki/blob/main/setup/servers.md#update-the-vpn-servers-list
      - UPDATER_PERIOD=

🆕 Image also available as ghcr.io/qdm12/gluetun

License

grails-plugins/grails-resources

2024-12-08T01:39:25Z

Resources framework for Grails 2.0

Flomp/wanderer

2024-12-08T01:31:46Z

wanderer is a self-hosted trail database. Save your adventures!

The trail catalogue that makes your GPS data searchable

wanderer is a self-hosted trail database. You can upload your recorded tracks or create new ones and add various metadata to build an easily searchable catalogue.

Core Features

Manage your trails
Plan new routes
Extensive map integration and visualization
Share trails with other people and explore theirs
Advanced filter and search functionality
Create custom lists to organize your trails further

Getting started

The recommended and quickest way to install wanderer is using docker compose:

# download the docker compose file
wget https://raw.githubusercontent.com/Flomp/wanderer/main/docker-compose.yml

# build and launch via docker compose
docker compose up -d

The first startup can take up to 90 seconds after which you can access the frontend at localhost:3000.

ℹ️ if you are not hosting wanderer at http://localhost:3000 make sure to change ORIGIN variable. Otherwise you will run into CORS errors.

⚠️ if you are using wanderer in a production environment make sure to change the MEILI_MASTER_KEY variable.

You can also run wanderer on bare-metal. Check out the documentation for a detailed how-to guide.

Support wanderer

If you like wanderer and would like to give something back you can donate over here.

Documentation

Please check the website for the complete documentation.

Contributing

Help is welcome at any time. If you are not sure where you can start check the roadmap for features in the backlog. If you would like to contribute a translation, you can do so here.

License

This project is licensed under the AGPLv3 License. See the LICENSE file for the full license text.

wdndev/llm_interview_note

2024-12-08T01:39:36Z

主要记录大语言大模型（LLMs）算法（应用）工程师相关的知识及面试题

LLMs 相关知识及面试题

简介

本仓库为大模型面试相关概念，由本人参考网络资源整理，欢迎阅读，如果对你有用，麻烦点一下 🌟 star，谢谢！

为了在低资源情况下，学习大模型，进行动手实践，创建 tiny-llm-zh仓库，旨在构建一个小参数量的中文大语言模型，该项目已部署，可以在如下网站上体验：ModeScope Tiny LLM。

其他学习资源推荐：

llama3-from-scratch-zh : 从零实现 llama3, 可加载 meta 官方权重，可在本地笔记本（16G内存）调试运行
tiny-rag : 实现一个简单的RAG系统，支持多路召回、重排等功能，快速了解搜索相关内容；
AI 工程师八股 : 包含深度学习、机器学习、推荐系统、搜索系统等通用知识

在线阅读

在线阅读链接：LLMs Interview Note

注意：

相关答案为自己撰写，若有不合理地方，请指出修正，谢谢！

欢迎关注微信公众号，会不定期更新LLM内容，以及一些面试经验：

microsoft/Security-101

2024-12-08T01:39:36Z

8 Lessons, Kick-start Your Cybersecurity Learning.

🚀 Cybersecurity for Beginners – a curriculum

In this rapidly changing age of AI technology adoption, it is even more critical to understand how to secure IT systems. This course is designed to teach you fundamental cyber security concepts to kick-start your security learning. It is vendor agnostic and is divided into small lessons that should take around 30-60 mins to complete. Each lesson has a small quiz and links to further reading if you want to dive into the topic a bit more.

What this course covers 📚

🔐 Basic cyber security concepts such as the CIA triad, the differences between risks, threats, etc,
🛡️ Understanding what a security control is and what forms they take.
🌐 Understanding what zero trust is and why this is important in modern cyber security.
🔑 Understanding key concepts and themes across identity, networking, security operations, infrastructure and data security.
🔧 Giving some examples of tools used to implement security controls.

What this course does not cover 🙅‍♂️

🚫 How to use specific security tools.
🚫 How to "hack" or do red teaming/offensive security.
🚫 Learning about specific compliance standards.

When you have finished this course, you can move onto some of our Microsoft Learn modules. We recommend you continue your learning with Microsoft Security, Compliance, and Identity Fundamentals.

Ultimately, you could consider taking the Exam SC-900: Microsoft Security, Compliance, and Identity Fundamentals exam.

💁 If you have any feedback or suggestions on this course and any content we're missing, we'd love to hear from you!

Modules Overview 📝

Module number	Module name	Concepts taught	Learning objectives
1.1	Basic security concepts	The CIA triad	Learn about confidentiality, availability and integrity. Also authenticity and also nonrepudiation and privacy.
1.2	Basic security concepts	Common cyber security threats	Learn about the common cyber security threats facing individuals and organizations.
1.3	Basic security concepts	Understanding risk management	Learn about assessing and understanding risk – impact/likelihood and implementing controls.
1.4	Basic security concepts	Security practices and documentation	Learn about the difference between policies, procedures, standards and regulations/laws.
1.5	Basic security concepts	Zero trust	Learn about what is zero trust and how does it affect architecture? What is defense in depth?
1.6	Basic security concepts	The shared responsibility model	What is the shared responsibility model and how does it affect cyber security?
1.7	End of module quiz
2.1	Identity & access management fundamentals	IAM key concepts	Learn about the principle of least privilege, segregation of duties, how IAM supports zero trust.
2.2	Identity & access management fundamentals	IAM zero trust architecture	Learn about how identity is the new perimeter for modern IT environments and the threats it mitigates.
2.3	Identity & access management fundamentals	IAM capabilities	Learn about IAM capabilities and controls to secure identities
2.4	End of module quiz
3.1	Network security fundamentals	Networking key concepts	Learn about networking concepts (IP addressing, port numbers, encryption, etc.)
3.2	Network security fundamentals	Networking zero trust architecture	Learn about how networking contributes to an E2E ZT architecture and the threats it mitigates.
3.3	Network security fundamentals	Network security capabilities	Learn about network security tooling – firewalls, WAF, DDoS protection, etc.
3.4	End of module quiz
4.1	Security operations fundamentals	SecOps key concepts	Learn about why security operations are important and how it differs from normal IT ops teams.
4.2	Security operations fundamentals	SecOps zero trust architecture	Learn about how SecOps contributes to an E2E ZT architecture and the threats it mitigates.
4.3	Security operations fundamentals	SecOps capabilities	Learn about SecOps tooling – SIEM, XDR, etc.
4.4	End of module quiz
5.1	Application security fundamentals	AppSec key concepts	Learn about AppSec concepts such as secure by design, input validation, etc.
5.2	Application security fundamentals	AppSec capabilities	Learn about AppSec tooling: pipeline security tools, code scanning, secret scanning, etc.
5.3	End of module quiz
6.1	Infrastructure security fundamentals	Infrastructure security key concepts	Learn about hardening systems, patching, security hygiene, container security.
6.2	Infrastructure security fundamentals	Infrastructure security capabilities	Learn about tooling that can assist with infrastructure security e.g. CSPM, container security, etc.
6.3	End of module quiz
7.1	Data security fundamentals	Data security key concepts	Learn about data classification and retention and why this is important to an organization.
7.2	Data security fundamentals	Data security capabilities	Learn about data security tooling – DLP, inside risk management, data governance, etc.
7.3	End of module quiz
8.1	AI security fundamentals	AI security key concepts	Learn about the differences and similarities between traditional security and AI security.
8.2	AI security fundamentals	AI security capabilities	Learn about AI security tooling and the controls that can be used to secure AI.
8.3	AI security fundamentals	Responsible AI	Learn about what responsible AI is and AI specific harms that security professionals need to be aware of.
8.4	End of module quiz

🎒 Other Courses

Our team produces other courses! Check out:

datarhei/restreamer

2024-12-08T01:39:36Z

The Restreamer is a complete streaming server solution for self-hosting. It has a visually appealing user interface and no ongoing license costs. Upload your live stream to YouTube, Twitch, Facebook, Vimeo, or other streaming solutions like Wowza. Receive video data from OBS and publish it with the RTMP and SRT server.

Restreamer

A really nice and free alternative for handling live streams.

Try live demo

Self-hosting solution to stream live to your website and publish to many like YouTube-Live, Twitter, Twitch, Vimeo, and other platforms or services. Our Docker-Image is easy to install and runs on Linux environments (MacOS/Windows by Docker Desktop). Moreover, combine the Restreamer with single-board computers like Raspberry Pi or GPU powered systems for Video-Encoding.

Features

Simplified User-Interface
Easy wizard configuration
Multiple audio/video inputs, outputs, protocols, and codecs
ReStreaming to platforms such as YouTube-Live, to software such as Wowza Media Server, and others based on protocols such as RTMP, SRT, ...
Option to mux a separate audio channel to the video
Build-in VideoJS-Player for your Website
Configurable publication website for streaming without player embedding
Content license with Creative Commons
HTTP/S- (HLS), RTMP/S- & SRT-Streaming Server
Automatic Let's Encrypt HTTPS certification
Viewer/Bandwidth Monitoring and limiting
Raspberry Pi (MMAL/OMX), Nvidia Cuda, Intel VAAPI support
Support for Hardware- and Virtual-Devices
FFmpeg Video-Processing (as native as possible)
REST-API (JSON) and 100% Swagger documented
Resource Monitoring (optionally by Prom-Metrics)
Server- and Process-Logging
GDPR compliant without third-party providers and does not save audience data

Quick setup

AMD64/ARMv7/ARM64:

docker run -d --restart=always --name restreamer \
   -v /opt/restreamer/config:/core/config -v /opt/restreamer/data:/core/data \
   -p 8080:8080 -p 8181:8181 \
   -p 1935:1935 -p 1936:1936 \
   -p 6000:6000/udp \
   datarhei/restreamer:latest

--privileged just for local devices like usb cameras.
Try --security-opt seccomp=unconfined if no network source can be reached.

ARMv7/ARM64 Raspberry Pi:

docker run -d --restart=always --name restreamer \
   -v /opt/restreamer/config:/core/config -v /opt/restreamer/data:/core/data \
   --privileged \
   -p 8080:8080 -p 8181:8181 \
   -p 1935:1935 -p 1936:1936 \
   -p 6000:6000/udp \
   datarhei/restreamer:rpi-latest

--privileged just for local devices like usb cameras.
Try --security-opt seccomp=unconfined if no network source can be reached.

AMD64 Nvidia Cuda:

docker run -d --restart=always --name restreamer \
   -v /opt/restreamer/config:/core/config -v /opt/restreamer/data:/core/data \
   --runtime=nvidia --privileged \
   -p 8080:8080 -p 8181:8181 \
   -p 1935:1935 -p 1936:1936 \
   -p 6000:6000/udp \
   datarhei/restreamer:cuda-latest

--privileged just for local devices like usb cameras.
Try --security-opt seccomp=unconfined if no network source can be reached.

AMD64 Intel VAAPI:

docker run -d --restart=always --name restreamer \
   -v /opt/restreamer/config:/core/config -v /opt/restreamer/data:/core/data \
   -v /dev/dri:/dev/dri --privileged \
   -p 8080:8080 -p 8181:8181 \
   -p 1935:1935 -p 1936:1936 \
   -p 6000:6000/udp \
   datarhei/restreamer:vaapi-latest

--privileged just for local devices like usb cameras.
Try --security-opt seccomp=unconfined if no network source can be reached.

For external access (http/s, rtmp/s, srt), port forwarding from your Internet-Router to the Restreamer's internal IP address may need to be set up.

Documentation

Documentation is available on docs.datarhei.com/restreamer. We give many pieces of information, from setting up a camera, embedding your player upon your website, and streaming to services like, e.g., YouTube-Live, and many more.

Development

Create a custom image (bundle):

Restreamer FFmpeg:

$ git clone github.com/datarhei/ffmpeg
$ cd ffmpeg
$ docker build -f Dockerfile.alpine -t myffmpeg .

Restreamer backend (Golang):

$ git clone github.com/datarhei/core
$ cd core
$ docker build -t mycore .

Restreamer interface (React):

$ git clone github.com/datarhei/restreamer-ui
$ cd restreamer-ui
$ docker build -t myrsui .

Restreamer bundle:

$ git clone github.com/datarhei/restreamer
$ cd restreamer
$ docker build --build-arg FFMPEG_IMAGE=myffmpeg --build-arg CORE_IMAGE=mycore --build-arg RESTREAMER_UI_IMAGE=myrsui -t myrestreamer .
$ docker run -it --rm -p 8080:8080 myrestreamer

To add/fix translations in the Restreamer interface:

The Restreamer interface is currently translated in different languages, such as German, French, Italian, Spanish, and more. If you find errors in the translations or have better suggestions for some sentences, you can become a translation contributor on poeditor.com. There you can also start a translation into a language that is not yet available in the Restreamer interface.

Contribute to the translations on: https://poeditor.com/join/project/ogATl3F48K

Community support

For general help using Restreamer, please refer to the official documentation. For additional support, you can use Github to ask a question (Bug reports, Contributions, Features).

License

See the LICENSE file for licensing information.

Business inquiries

We provide support for commercial requirements with professional support, agile software development, and consulting. If you have a commercial request, be it a bug or a feature enhancement, please contact us directly at support@datarhei.com.

rems-project/sail

2024-12-08T01:39:57Z

Sail architecture definition language

infinilabs/analysis-ik

2024-12-08T01:40:04Z

🚌 The IK Analysis plugin integrates Lucene IK analyzer into Elasticsearch and OpenSearch, support customized dictionary.

IK Analysis for Elasticsearch and OpenSearch

The IK Analysis plugin integrates Lucene IK analyzer, and support customized dictionary. It supports major versions of Elasticsearch and OpenSearch. Maintained and supported with ❤️ by INFINI Labs.

The plugin comprises analyzer: ik_smart , ik_max_word, and tokenizer: ik_smart , ik_max_word

How to Install

You can download the packaged plugins from here: https://release.infinilabs.com/,

or you can use the plugin cli to install the plugin like this:

For Elasticsearch

bin/elasticsearch-plugin install https://get.infini.cloud/elasticsearch/analysis-ik/8.4.1

For OpenSearch

bin/opensearch-plugin install https://get.infini.cloud/opensearch/analysis-ik/2.12.0

Tips: replace your own version number related to your elasticsearch or opensearch.

Getting Started

1.create a index

curl -XPUT http://localhost:9200/index

2.create a mapping

curl -XPOST http://localhost:9200/index/_mapping -H 'Content-Type:application/json' -d'
{
        "properties": {
            "content": {
                "type": "text",
                "analyzer": "ik_max_word",
                "search_analyzer": "ik_smart"
            }
        }

}'

3.index some docs

curl -XPOST http://localhost:9200/index/_create/1 -H 'Content-Type:application/json' -d'
{"content":"美国留给伊拉克的是个烂摊子吗"}
'

curl -XPOST http://localhost:9200/index/_create/2 -H 'Content-Type:application/json' -d'
{"content":"公安部：各地校车将享最高路权"}
'

curl -XPOST http://localhost:9200/index/_create/3 -H 'Content-Type:application/json' -d'
{"content":"中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"}
'

curl -XPOST http://localhost:9200/index/_create/4 -H 'Content-Type:application/json' -d'
{"content":"中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"}
'

4.query with highlighting

curl -XPOST http://localhost:9200/index/_search  -H 'Content-Type:application/json' -d'
{
    "query" : { "match" : { "content" : "中国" }},
    "highlight" : {
        "pre_tags" : ["<tag1>", "<tag2>"],
        "post_tags" : ["</tag1>", "</tag2>"],
        "fields" : {
            "content" : {}
        }
    }
}
'

Result

{
    "took": 14,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 2,
        "hits": [
            {
                "_index": "index",
                "_type": "fulltext",
                "_id": "4",
                "_score": 2,
                "_source": {
                    "content": "中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"
                },
                "highlight": {
                    "content": [
                        "<tag1>中国</tag1>驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首 "
                    ]
                }
            },
            {
                "_index": "index",
                "_type": "fulltext",
                "_id": "3",
                "_score": 2,
                "_source": {
                    "content": "中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"
                },
                "highlight": {
                    "content": [
                        "均每天扣1艘<tag1>中国</tag1>渔船 "
                    ]
                }
            }
        ]
    }
}

Dictionary Configuration

Config file IKAnalyzer.cfg.xml can be located at {conf}/analysis-ik/config/IKAnalyzer.cfg.xml or {plugins}/elasticsearch-analysis-ik-*/config/IKAnalyzer.cfg.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
	<entry key="ext_dict">custom/mydict.dic;custom/single_word_low_freq.dic</entry>
	<entry key="ext_stopwords">custom/ext_stopword.dic</entry>
	<entry key="remote_ext_dict">location</entry>
	<entry key="remote_ext_stopwords">http://xxx.com/xxx.dic</entry>
</properties>

Hot-reload Dictionary

The current plugin supports hot reloading dictionary for IK Analysis, through the configuration mentioned earlier in the IK configuration file.

	<entry key="remote_ext_dict">location</entry>
	<entry key="remote_ext_stopwords">location</entry>

Among which location refers to a URL, such as http://yoursite.com/getCustomDict. This request only needs to meet the following two points to complete the segmentation hot update.

The HTTP request needs to return two headers, one is Last-Modified, and the other is ETag. Both of these are of string type, and if either changes, the plugin will fetch new segmentation to update the word library.
The content format returned by the HTTP request is one word per line, and the newline character is represented by \n.

Meeting the above two requirements can achieve hot word updates without the need to restart the ES instance.

You can place the hot words that need to be automatically updated in a .txt file encoded in UTF-8. Place it under nginx or another simple HTTP server. When the .txt file is modified, the HTTP server will automatically return the corresponding Last-Modified and ETag when the client requests the file. You can also create a separate tool to extract relevant vocabulary from the business system and update this .txt file.

FAQs

Why isn't the custom dictionary taking effect?

Please ensure that the text format of your custom dictionary is UTF8 encoded.

What is the difference between ik_max_word and ik_smart?

ik_max_word: Performs the finest-grained segmentation of the text. For example, it will segment "中华人民共和国国歌" into "中华人民共和国,中华人民,中华,华人,人民共和国,人民,人,民,共和国,共和,和,国国,国歌", exhaustively generating various possible combinations, suitable for Term Query.

ik_smart: Performs the coarsest-grained segmentation of the text. For example, it will segment "中华人民共和国国歌" into "中华人民共和国,国歌", suitable for Phrase queries.

Note: ik_smart is not a subset of ik_max_word.

Community

Fell free to join the Discord server to discuss anything around this project:

https://discord.gg/4tKTMkkvVX

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

checkstyle/checkstyle

2024-12-08T01:40:04Z

Checkstyle is a development tool to help programmers write Java code that adheres to a coding standard. By default it supports the Google Java Style Guide and Sun Code Conventions, but is highly configurable. It can be invoked with an ANT task and a command line program.

Checkstyle - Java Code Quality Tool

Checkstyle is a tool that ensures adherence to a code standard or a set of best practices.

The latest release version can be found at GitHub releases or at Maven repo.

Each-commit builds of maven artifacts can be found at Maven Snapshot repository.

Documentation is available in HTML format, see https://checkstyle.org/checks.html .

Quick Start

Download our Latest Release from GitHub or Add Checkstyle to your build from Maven Central.
Read our Documentation for usage and configuration.

$ cat config.xml
<?xml version="1.0"?>
<!DOCTYPE module PUBLIC
          "-//Puppy Crawl//DTD Check Configuration 1.3//EN"
          "https://checkstyle.org/dtds/configuration_1_3.dtd">
<module name="Checker">
  <module name="TreeWalker">
    <module name="FallThrough"/>
  </module>
</module>

$ cat Test.java
class Test {
  public void foo() {
    int i = 0;
    while (i >= 0) {
      switch (i) {
        case 1:
        case 2:
          i++;
        case 3: // violation 'fall from previous branch of the switch'
          i++;
      }
    }
  }
}

$ java -jar checkstyle-10.18.1-all.jar -c config.xml Test.java
Starting audit...
[ERROR] Test.java:9:9: Fall through from previous branch of switch statement [FallThrough]
Audit done.
Checkstyle ends with 1 errors.

Contributing

Thanks for your interest in contributing to CheckStyle! Please see the Contribution Guidelines for information on how to contribute to the project. This includes creating issues, submitting pull requests, and setting up your development environment.

Build Instructions

Please see the CheckStyle Documentation for information on how to build the project.

Feedback and Support

Visit our Discussions Page, where you can ask questions and discuss the project with other users and contributors. This is our preferred method of communication for topics like usage and configuration questions, debugging, and other feedback.
Stack Overflow is another place to ask questions about Checkstyle usage.
If you are interested in contributing to the project, you can join our Discord Contributors Chat.
Our Google Groups Forum is a mailing list for discussion and support; however, we may be slow to respond there.

Javadoc

Take a look at our javadoc to see our API documentation.

Sponsor Checkstyle

Checkstyle is an open-source project that is developed and maintained by volunteers. If you find Checkstyle useful, please consider sponsoring the project. Your support helps us to maintain and improve Checkstyle.

Licensing

Checkstyle is licensed under the GNU LGPL v2.1 License. Checkstyle uses libraries:

freeok/so-novel

2024-12-08T01:40:04Z

so-novel

前言

适合用户：既想免费看正版付费新书，又想获得最佳的阅读体验（主要针对网络小说）

适用场景：国内网上 98% 的 txt 、epub 等格式的小说都是完本。如果想看新书，要么去正版平台付费，要么笔趣阁等一类网站，要么用”阅读“（仅限安卓）等一类 APP。其实这两个方案足够多数人用了，但总会有众口难调的情况：嫌弃 UI 难看的、吐槽功能的、受限于平台的。这时阅读器的优势就体现出来了：DIY。这个工具最大的意义就是能把连载的新书下载为 epub 等电子书格式，这样就能导入自己喜爱的阅读器

对于完本小说，本工具依然可以搜索下载。若出现错别字、排版等问题，建议自行搜索下载对应精校版

介绍

交互式小说下载器，Windows、macOS、Linux 解压即用

可根据书名、作者搜索并下载小说

支持导出格式：epub、txt、html（支持翻页）

结合以下电子书阅读器使用更佳

电脑：koodo-reader
手机：Apple Books、Moon+ Reader、 ~~微信读书~~（2024.4 更新后非付费会员每月最多导入三本书，先培养用户习惯，后开始收割）

[!WARNING]

iOS 16 Apple Books 目录不能定位到当前章节，如果看到很多章了，就需要从上一直向下划，十分不便

iOS 17 修复了此 BUG

效果

保留部分下载日志

使用

普通安装

下载最新版 https://github.com/freeok/so-novel/releases
根据 readme.txt 使用

🍨 Scoop 安装

# 若搜不到 so-novel，请先执行下面这一行命令
scoop bucket add spc https://ghp.ci/https://github.com/lzwme/scoop-proxy-cn
scoop install so-novel

🍺 Homebrew 安装

brew tap ownia/homebrew-ownia
brew install so-novel

[!TIP]

部分书源会屏蔽国外 IP，建议关闭代理后使用

由于书源 URL 会变动，旧版可能无法使用，请使用最新版

若最新版的书源无法使用，请 New issue，将尽快修复

欢迎在 issue 推荐高质量书源（无盾、能搜到起点人气榜单的书、错别字少、排版不错乱、直连可访问）

常见问题

报错：SocketTimeoutException (https://github.com/freeok/so-novel/issues/37)

浏览器是否可以打开书源网站
关闭代理
更换网络
多次尝试，或稍后再试

报错：UnknownHostException (https://github.com/freeok/so-novel/issues/25)

检查 DNS 解析 (nslookup 命令)
更换 DNS

WPS 打不开 epub，提示：不支持打开该类型文件或文件已损坏

使用专业阅读器
下载格式改为 txt

macOS & Linux 运行 .sh 失败，尝试以下命令

dos2unix macos-run.sh
或
dos2unix linux-run.sh

Star History

netptop/siteproxy

2024-12-07T01:31:52Z

reverse proxy, online proxy, 反向代理,免翻墙访问Youtube/twitter/Google, 支持github和telegram web登录(请注意不要通过不信任的代理进行登录)。支持DuckDuckGo AI Chat(可免费访问chatGPT3.5和Claude3)

SiteProxy 2.0

English ver

SiteProxy 是一个功能强大的在线代理工具，采用了最新的技术，提升了代理的稳定性和兼容性。我们致力于提供 简单、高效、安全 的代理服务，为用户提供最佳的互联网访问体验。

超高速性能：采用 Hono 替代传统的Express 服务器，性能提升 4 倍，带来更流畅的使用体验。
云端部署：完美支持 Cloudflare Worker 部署，快速且高效。
AI 智能聊天：集成 DuckDuckGo AI Chat，免费提供 GPT-3.5 和 Claude 3，让你的代理服务更加智能。
高级安全保护：支持密码控制代理，只有授权用户才能访问，大幅提升安全性。
零配置使用：用户无需进行任何客户端配置，只需访问代理网址即可畅游全球互联网。
便捷登录：全面支持 GitHub 和 Telegram Web 登录，操作简单快捷。
强力加密：采用 RSA + AES 双重加密技术，保护用户登录密码，防止中间人攻击。
隐私保护：通过代理网址访问全球互联网，同时隐藏用户真实 IP，保护隐私。
无缝体验：无需任何软件安装和浏览器配置，即可立即使用，提供极致便利的用户体验。

查看原理

                                                 +----> google/youtube
                             +----------------+  |
                             |                |  |
user browser +-------------->+ siteproxy      +-------> wikipedia
                             |                |  |
                             +----------------+  |
                                                 +----> chinese forums

[!CAUTION] 严禁将本项目用于任何非法用途，否则后果自负

[!WARNING] 由于支持多个网站的 Login，为了减少钓鱼风险，Siteproxy 在 2.0 版本对代码进行了混淆，同时禁止了默认主页网址的修改。

展示

来自 Telegram 的网友的展示 Siteproxy 部署后，加上优选IP的速度测试：

部署到 Cloudflare Pages

确保域名管理：
- 确保你的域名已经在 Cloudflare 名下进行管理。
克隆仓库：
- 执行命令：git clone https://github.com/netptop/siteproxy.git
编辑配置文件：
- 使用文本编辑器打开 siteproxy/build/cf_page/_worker.js 文件。
- 搜索并替换字符串 http://localhost:5006 为你的代理服务器域名, 必须替换为HTTPS，例如 https://your-proxy-domain.com（请使用 https）。
- 搜索并替换字符串 user22334455 为你想设置的访问密码。如果密码为空，则表示不需要密码即可访问。保存文件。
登录 Cloudflare：
- 进入 Workers 和 Pages 部分，选择 使用直接上传创建 一个 Page，上传 siteproxy/build/cf_page 目录进行部署。
配置自定义域：
- 在 Workers & Pages 页面，打开刚才部署的 Page。
- 点击顶部的 自定义域，然后选择 添加自定义域，设置为你的代理域名并激活域名。
访问代理服务：
- 现在可以通过 https://your-proxy-domain.com/your-password/ 访问代理服务（确保最后的斜杠存在）。注意将域名和密码替换为你自己的。

部署到 Cloudflare Workers

确保域名管理：
- 确保你的域名已经在 Cloudflare 名下进行管理。
下载并编辑 Worker 文件：
- 下载 build/worker.js 文件：链接，并使用文本编辑器打开。
- 搜索并替换字符串 http://localhost:5006 为你的代理服务器域名，必须替换为HTTPS，例如 https://your-worker-domain.com（请使用 https）。
- 搜索并替换字符串 user22334455 为你想设置的访问密码。如果密码为空，则表示不需要密码即可访问。
创建 Worker：
- 登录 Cloudflare，进入 Workers 和 Pages 部分，创建一个 Worker。
- 编辑刚才创建的 Worker，将编辑过的 worker.js 文件内容复制粘贴到 Worker 内部，保存并部署。
配置自定义域：
- 在 Workers & Pages 页面，打开刚才保存的 Worker。
- 点击顶部的 设置 -> 触发器，然后选择 添加自定义域，设置为你的代理域名。自定义域名设置成功后，DNS 页面应显示对应的 DNS 类型为 Worker。
访问代理服务：
- 现在可以通过 https://your-worker-domain.com/your-password/ 访问代理服务（确保最后的斜杠存在，并替换为你自己的域名和密码）。

部署到 VPS 或者云服务器

创建 SSL 网站：
- 使用 certbot 和 nginx 创建 SSL 网站。具体用法可以 Google 搜索。
- 配置 nginx，确保 /etc/nginx/conf.d/default.conf 文件包含以下内容：
```
server {
   server_name your-proxy.domain.name; #请替换为你的实际域名
   location / {
     proxy_pass http://localhost:5006;
   }
}
```
重启 nginx：
- 执行命令：sudo systemctl restart nginx

安装 Node.js v21 或更高版本：

执行以下命令：

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.bashrc
nvm install v21

克隆仓库：
- 执行命令：git clone https://github.com/netptop/siteproxy.git
进入项目目录：
- 执行命令：cd siteproxy
测试运行：
- 执行命令：node bundle.js
- 如果没有错误，按 Ctrl+C 结束程序。

配置文件修改：

打开并修改 config.json 文件，内容如下：

{
   "proxy_url": "https://your-proxy.domain.name", // 替换为HTTPS加你的代理服务器域名，确保使用 https
   "token_prefix": "/user-SetYourPasswordHere/",  // 设置网站密码，用于防止非法访问，保留首尾的斜杠。为空表示不设置密码
   "local_listen_port": 5006, // 不要修改，以确保与 nginx 配置一致
   "description": "注意：token_prefix 相当于网站密码，请谨慎设置。 proxy_url 和 token_prefix 合起来就是访问网址。"
}

安装 Forever：
- 执行命令：npm install -g forever
启动应用：
- 执行命令：forever stopall && forever start bundle.js
访问代理服务：
- 在浏览器中访问你的域名，网址为 proxy_url 加 token_prefix。
使用 Cloudflare 加速（可选）：
- 参考 Cloudflare 的官方说明进行设置。

现在，你的代理服务已经成功部署并可以通过浏览器访问。

Docker 部署

配置 SSL 证书和 Nginx：
- 配置域名对应的 SSL 证书和 Nginx，将其指向本地的 5006 端口。
克隆仓库：
- 执行命令：git clone https://github.com/netptop/siteproxy.git

编辑配置文件：

打开并修改 config.json 文件，内容如下：

{
   "proxy_url": "https://your-proxy-domain.com", // 替换为你申请到的代理服务器域名
   "token_prefix": "/user-SetYourPasswordHere/",  // 设置网站密码，用于防止非法访问，保留首尾的斜杠
   "description": "注意：token_prefix 相当于网站密码，请谨慎设置。 proxy_url 和 token_prefix 合起来就是访问网址。"
}

保存文件。

启动 Docker 容器：
- 进入 docker-node 子目录。
- 执行命令：sudo docker compose up
访问代理服务：
- 现在可以通过 https://your-proxy-domain.com/user-your-password/ 访问代理服务。请将域名和密码替换为你自己的域名和密码。

感谢

netptop.com 默认主页由 Telgram 网友 SenZyo 设计, 感谢贡献！
文档由 LAGSNES 编写

联系方式

Telegram群: https://siteproxy.t.me E-mail: netptop@gmail.com

InternLM/MindSearch

2024-12-08T01:40:10Z

🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)

📃 Paper | 💻 Demo

English | 简体中文

https://github.com/user-attachments/assets/44ffe4b9-be26-4b93-a77b-02fed16e33fe

✨ MindSearch: Mimicking Human Minds Elicits Deep AI Searcher

📅 Changelog

2024/11/05: 🥳 MindSearch is now deployed on Puyu! 👉 Try it 👈
- Refactored the agent module based on Lagent v0.5 for better performance in concurrency.
- Improved the UI to embody the simultaneous multi-query search.

⚽️ Build Your Own MindSearch

Step1: Dependencies Installation

git clone https://github.com/InternLM/MindSearch
cd MindSearch
pip install -r requirements.txt

Step2: Setup Environment Variables

Before setting up the API, you need to configure environment variables. Rename the .env.example file to .env and fill in the required values.

mv .env.example .env
# Open .env and add your keys and model configurations

Step3: Setup MindSearch API

Setup FastAPI Server.

python -m mindsearch.app --lang en --model_format internlm_server --search_engine DuckDuckGoSearch --asy

--lang: language of the model, en for English and cn for Chinese.
--model_format: format of the model.
- internlm_server for InternLM2.5-7b-chat with local server. (InternLM2.5-7b-chat has been better optimized for Chinese.)
- gpt4 for GPT4. if you want to use other models, please modify models
--search_engine: Search engine.
- DuckDuckGoSearch for search engine for DuckDuckGo.
- BingSearch for Bing search engine.
- BraveSearch for Brave search web api engine.
- GoogleSearch for Google Serper web search api engine.
- TencentSearch for Tencent search api engine.
Please set your Web Search engine API key as the WEB_SEARCH_API_KEY environment variable unless you are using DuckDuckGo, or TencentSearch that requires secret id as TENCENT_SEARCH_SECRET_ID and secret key as TENCENT_SEARCH_SECRET_KEY.
--asy: deploy asynchronous agents.

Step4: Setup MindSearch Frontend

Providing following frontend interfaces,

React

First configurate the backend URL for Vite proxy.

HOST="127.0.0.1"  # modify as you need
PORT=8002
sed -i -r "s/target:\s*\"\"/target: \"${HOST}:${PORT}\"/" frontend/React/vite.config.ts

# Install Node.js and npm
# for Ubuntu
sudo apt install nodejs npm

# for windows
# download from https://nodejs.org/zh-cn/download/prebuilt-installer

# Install dependencies

cd frontend/React
npm install
npm start

Details can be found in React

Gradio

python frontend/mindsearch_gradio.py

Streamlit

streamlit run frontend/mindsearch_streamlit.py

🌐 Change Web Search API

To use a different type of web search API, modify the searcher_type attribute in the searcher_cfg located in mindsearch/agent/__init__.py. Currently supported web search APIs include:

GoogleSearch
DuckDuckGoSearch
BraveSearch
BingSearch
TencentSearch

For example, to change to the Brave Search API, you would configure it as follows:

BingBrowser(
    searcher_type='BraveSearch',
    topk=2,
    api_key=os.environ.get('BRAVE_API_KEY', 'YOUR BRAVE API')
)

🐞 Using the Backend Without Frontend

For users who prefer to interact with the backend directly, use the backend_example.py script. This script demonstrates how to send a query to the backend and process the response.

python backend_example.py

Make sure you have set up the environment variables and the backend is running before executing the script.

🐞 Debug Locally

python -m mindsearch.terminal

📝 License

This project is released under the Apache 2.0 license.

Citation

If you find this project useful in your research, please consider cite:

@article{chen2024mindsearch,
  title={MindSearch: Mimicking Human Minds Elicits Deep AI Searcher},
  author={Chen, Zehui and Liu, Kuikun and Wang, Qiuchen and Liu, Jiangning and Zhang, Wenwei and Chen, Kai and Zhao, Feng},
  journal={arXiv preprint arXiv:2407.20183},
  year={2024}
}

Our Projects

Explore our additional research on large language models, focusing on LLM agents.

Lagent: A lightweight framework for building LLM-based agents
AgentFLAN: An innovative approach for constructing and training with high-quality agent datasets (ACL 2024 Findings)
T-Eval: A Fine-grained tool utilization evaluation benchmark (ACL 2024)

carbon-design-system/carbon

2024-12-08T01:40:10Z

A design system built by IBM

Carbon Design System

Carbon is an open-source design system built by IBM. With the IBM Design Language as its foundation, the system consists of working code, design tools and resources, human interface guidelines, and a vibrant community of contributors.

Getting started

If you're just getting started, check out @carbon/react or @carbon/web-components.

We also have community-contributed components for the following technologies:

If you're trying to find something specific, here's a full list of packages that we support!

Package name	Description
`@carbon/react`	React components and styles
`@carbon/web-components`	Web Components
`@carbon/styles`	Sass styles for components
`@carbon/elements`	IBM Design Language elements like colors, type, iconography, and more
`@carbon/colors`	Work with IBM Design Language colors
`@carbon/grid`	Build layouts using the new 16 column grid system
`@carbon/icons`	Iconography assets. We also offer support in: React, Angular, Vue, and Svelte
`@carbon/pictograms`	Pictogram assets. We also offer support in: React and Svelte
`@carbon/layout`	Layout-based units and spacing scale
`@carbon/motion`	Productive and expressive motion curves
`@carbon/themes`	Color tokens available in the Carbon Design System, like `$interactive-01`
`@carbon/type`	New type tokens used alongside IBM Plex

📚 Documentation

See our documentation site here for full how-to docs and guidelines
Contributing: Guidelines for making contributions to this repo.
🏃‍♀️ Migration Guides

🙌 Contributing

We're always looking for contributors to help us fix bugs, build new features, or help us improve the project documentation. If you're interested, definitely check out our Contributing Guide and our Developer Guide! 👀

Contributors

_{Taylor Jones} 💻 📖 ️️️️♿️	_{TJ Egan} 💻 📖 🚇 👀 ️️️️♿️	_{Alessandra Davila} 💻 📖 👀 ️️️️♿️	_{Andrea N. Cardona} 💻 📖 👀 ️️️️♿️	_{Francine Lucca} 💻 📖 👀 ️️️️♿️	_{Scott Strubberg} 💻 📖 👀 ️️️️♿️	_{Alison Joseph} 💻 📖 👀 ️️️️♿️
_{Anna Gonzales} 🎨 👀	_{Lauren Rice} 🎨 👀	_{Josh Black} 💻 📖 🚇 👀 ️️️️♿️	_DAK 💻 📖 👀 ️️️️♿️	_{Josefina Mancilla} 💻 📖 👀 ️️️️♿️	_emyarod 💻 📖 👀 ️️️️♿️	_{Eric Marcoux} 💻
_{Vince Picone} 💻 📖 👀 ️️️️♿️	_{jillianhowarth} 🖋 🎨 👀	_{Ricardo Henriquez} 💻 📖 👀 ️️️️♿️	_johnbister 🎨 👀	_{Dominik Brugger} 💻 🚧	_{Jan Hassel} 💻 📖 ️️️️♿️ 🎨	_{Alexander Lyon} 💻
_{Rosie Z} 💻	_{Nishith P} 📖	_{Eric Charpentier} 💻	_{Luiza Mendes} 💻 🚧	_{Akmal Hakimi Mohd Zamri} 💻	_sanjitbauli 📖	_{Laszlo Moczo} 💻
_LMapes 🖋 📖	_conradennis 🎨	_{Eric Liu} 💻 📖	_{Richard Všianský} 💻	_{Lee Chase} 💻 📖	_Anton 💻	_{Panpan Lin} 📖
_{Ashley Harrison} 💻	_{Jen Downs} 💻 📖 ️️️️♿️	_{Abdul Rehman} 💻	_{MIchael Dudley} 🎨	_{David Bradshaw} 💻	_{Simon Finney} 💻 ️️️️♿️	_{Attila Bartha} 💻
_배하람 💻	_{Yohanna Gadelrab} 📖	_{Akira Sudoh} 💻 📖 ️️️️♿️ 🚇	_{Oyinkan Oyetunmibi} 📖	_pbenson322 📖	_{Abbey Hart} 💻 📖 ️️️️♿️	_Lucas 💻
_{Dylan Klohr} 📖	_{Gilli Sigurdsson} 🎨	_kennylam 💻 ️️️️♿️	_Sébastien 💻	_{Dusan Milko} 💻	_{Taras Polovyi} 💻	_{Chris Connors} 🎨 📖
_{David Conner} 💻 ️️️️♿️	_{Harish Mohanani} 💻	_{Frivalszky-Mayer Péter} 💻 ️️️️♿️	_s100 💻	_{Taranveer Virk} 💻	_{Niall Cargill} 📖	_{Matt Chapman} 💻
_{Boston Cartwright} 💻	_DavidSCChen 💻	_molyholy 💻	_{Scott Dickerson} 💻	_{Evgeniy Podgaetskiy} 💻	_{CassidyJensen} 💻 ️️️️♿️	_{Zsolt Lattmann} 💻
_{Conrad Schmidt} 💻	_{Ignacio Becerra} 💻	_{Swapnil Patil} 📖	_{Fei Z} 💻	_{Ignas Ausiejus} 📖	_{Ryan O. Mackey} 💻	_{Dávid Halász} 💻
_{Bill Guigue} 💻	_{szymonbrandys} 💻	_{Adam Alston} 📖	_{Krithika S Udupa} 📖	_{Eshin Griffith} 💻	_{@RianTavaresOn} 💻 🎨	_ColbyJohnIBM 💻
_Håkon 💻 📖	_{Tanner Summers} 💻	_{Zhen Wang} 💻 📖	_{Cathal Kenneally} 💻	_{Joel Humberto Gómez Paredes} 💻	_{James Nash} 💻 📖	_{Jakub Faliszewski} 💻
_{Nick Gong} 💻	_{Hannele Valtanen} 💻	_Llam4u 💻 🐛	_{G. Torres} 💻	_Fiona 💻	_kindoflew 💻	_{Mario Gueyraud} 💻
_{Dongjoon Lee} 💻	_{ShankarV-CodeJunkie} 💻	_{dario platania} 💻	_{Mateusz Krzyżanowski} 💻	_jpsorensen 💻	_{jae kaplan} 🚇	_{Sierra Wetmore} 💻
_kcprevatt 💻	_{Marcin Lewandowski} 💻	_remolueoend 💻	_{Jesse Hull} 💻 ️️️️♿️	_{Ashvin Warrier} 💻	_GalvinGao 💻	_{Bianca Sparxs} 💻
_{Mahmoud Abdulazim} 💻	_{Dave Steinberg} 💻	_{Seong-Hyun Ryoo} 💻	_{Pratik Karad} 💻 ️️️️♿️	_Gerzon 💻	_{Guilherme Datilio Ribeiro} 💻 📖 ️️️️♿️ 👀	_{Josef Kubíček} 💻
_{Sunny Johal} 💻	_{Steven Black} 💻 ️️️️♿️	_{Mark Judy} 💻	_{Anton Tsymuk} 💻	_{Mohammed Aslam P. A.} 💻 📖	_{Tony ZL} 💻	_{Petr Kadlec} 💻
_{David Ragsdale} 💻	_{Hao Cheng} 💻	_cordesmj 💻	_{Aziz Chebbi} 💻	_{Michał Konopski} 💻	_{Omkar Ajagunde} 💻 📖	_{Aman Lajpal} 💻 📖
_{Niraj Sah} 💻	_{David Padilla} 💻	_{Allison Ishida} 💻	_{Alex Lewitt} 💻	_Tresau 💻	_{Daniel Castillo} 💻	_HaRuki 💻 📖
_{Matej Ocovsky} 💻	_{SamChinellato} 💻	_{stevenpatrick009} 💻	_HunterXalc 💻	_{Onur Özkardeş} 💻	_{Matias Borghi} 💻	_{Alexandr Ovchinnikov} 💻
_{J Thomas} 💻	_{Garrett Dawson} 💻 📖	_{Daniel Adebonojo} 📖	_{Anjana M R} 💻	_{Joseph Schultz} 💻	_anjaly0606 💻	_jesnajoseijk 💻
_{Jawahar S} 💻	_{Holly Springsteen} 💻	_{Nikhil Tomar} 💻	_{Anina Antony} 💻	_{Ahmed Semih Erkan} 💻	_{Yael Chavoya} 💻 📖	_{Kilian Collender} 💻
_{nandininarayanofficial} 💻	_{Andrea DG} 💻	_Luis 💻	_{Luke Harrison} 💻	_{Ahmed Alsinan} 💻	_{Nevan Tan} 💻	_{Ali Al Dobyan} 💻
_{Adam Shea} 💻	_{Md Nabeel Ayubee} 💻	_{Patan Amrulla Khan} 💻 📖	_{Noah Sgorbati} 💻 📖	_{Divya G} 💻	_{Soumya Raju} 💻	_{Mohammed Mikdhad C} 💻
_{IRFAD KP} 💻	_{ziyadzulfikar} 💻	_{IRFAD KP} 💻	_Mariat 💻	_{Thamjith Thaha} 💻	_{Zoë Gathercole} 💻	_{Zach Tindall} 💻
_{Ashna Thomas} 💻

This project follows the all-contributors specification. Contributions of any kind welcome!

📝 License

Licensed under the Apache 2.0 License.

overleaf/overleaf

2024-12-08T01:40:10Z

A web-based collaborative LaTeX editor

An open-source online real-time collaborative LaTeX editor.

Wiki • Server Pro • Contributing • Mailing List • Authors • License

Figure 1: A screenshot of a project being edited in Overleaf Community Edition.

Community Edition

Overleaf is an open-source online real-time collaborative LaTeX editor. We run a hosted version at www.overleaf.com, but you can also run your own local version, and contribute to the development of Overleaf.

Enterprise

If you want help installing and maintaining Overleaf in your lab or workplace, we offer an officially supported version called Overleaf Server Pro. It also includes more features for security (SSO with LDAP or SAML), administration and collaboration (e.g. tracked changes). Find out more!

Keeping up to date

Installation

We have detailed installation instructions in the Overleaf Toolkit.

Upgrading

If you are upgrading from a previous version of Overleaf, please see the Release Notes section on the Wiki for all of the versions between your current version and the version you are upgrading to.

Overleaf Docker Image

This repo contains two dockerfiles, Dockerfile-base, which builds the sharelatex/sharelatex-base image, and Dockerfile which builds the sharelatex/sharelatex (or "community") image.

The Base image generally contains the basic dependencies like wget, plus texlive. We split this out because it's a pretty heavy set of dependencies, and it's nice to not have to rebuild all of that every time.

The sharelatex/sharelatex image extends the base image and adds the actual Overleaf code and services.

Use make build-base and make build-community from server-ce/ to build these images.

We use the Phusion base-image (which is extended by our base image) to provide us with a VM-like container in which to run the Overleaf services. Baseimage uses the runit service manager to manage services, and we add our init-scripts from the server-ce/runit folder.

Contributing

Please see the CONTRIBUTING file for information on contributing to the development of Overleaf.

Authors

The Overleaf Team

License

The code in this repository is released under the GNU AFFERO GENERAL PUBLIC LICENSE, version 3. A copy can be found in the LICENSE file.

JuliaGeometry/MeshIntegrals.jl

2024-12-07T01:32:04Z

Numerical integration over Meshes.jl geometry domains

ed-donner/llm_engineering

2024-12-08T01:40:26Z

Repo to accompany my mastering LLM engineering course

LLM Engineering - Master AI and LLMs

Your 8 week journey to proficiency starts today

I'm so happy you're joining me on this path. We'll be building immensely satisfying projects in the coming weeks. Some will be easy, some will be challenging, many will ASTOUND you! The projects build on each other so you develop deeper and deeper expertise each week. One thing's for sure: you're going to have a lot of fun along the way.

A note before you begin

I'm here to help you be most successful with your learning! If you hit any snafus, or if you have any ideas on how I can improve the course, please do reach out in the platform or by emailing me direct (ed@edwarddonner.com). It's always great to connect with people on LinkedIn to build up the community - you'll find me here:
https://www.linkedin.com/in/eddonner/

Resources to accompany the course, including the slides and useful links, are here:
https://edwarddonner.com/2024/11/13/llm-engineering-resources/

Instant Gratification instructions for Week 1, Day 1

We will start the course by installing Ollama so you can see results immediately!

Download and install Ollama from https://ollama.com
On a PC, start a Command prompt / Powershell (Press Win + R, type cmd, and press Enter). On a Mac, start a Terminal (Applications > Utilities > Terminal).
Run ollama run llama3.2 or for smaller machines try ollama run llama3.2:1b
If this doesn't work, you may need to run ollama serve in another Powershell (Windows) or Terminal (Mac), and try step 3 again
And if that doesn't work on your box, I've set up this on the cloud. This is on Google Colab, which will need you to have a Google account to sign in, but is free: https://colab.research.google.com/drive/1-_f5XZPsChvfU1sJ0QqCePtIuc55LSdu?usp=sharing

Any problems, please contact me!

Then, Setup instructions

After we do the Ollama quick project, and after I introduce myself and the course, we get to work with the full environment setup.

Hopefully I've done a decent job of making these guides bulletproof - but please contact me right away if you hit roadblocks:

PC people please follow the instructions in SETUP-PC.md
Mac people please follow the instructions in SETUP-mac.md
Linux people, the Mac instructions should be close enough!

An important point on API costs (which are optional! No need to spend if you don't wish)

During the course, I'll suggest you try out the leading models at the forefront of progress, known as the Frontier models. I'll also suggest you run open-source models using Google Colab. These services have some charges, but I'll keep cost minimal - like, a few cents at a time. And I'll provide alternatives if you'd prefer not to use them.

Please do monitor your API usage to ensure you're comfortable with spend; I've included links below. There's no need to spend anything more than a couple of dollars for the entire course. Some AI providers such as OpenAI require a minimum credit like $5 or local equivalent; we should only spend a fraction of it, and you'll have plenty of opportunity to put it to good use in your own projects. During Week 7 you have an option to spend a bit more if you're enjoying the process - I spend about $10 myself and the results make me very happy indeed! But it's not necessary in the least; the important part is that you focus on learning.

I'll also show you an alternative if you'd rather not spend anything on APIs.

How this Repo is organized

There are folders for each of the "weeks", representing modules of the class, culminating in a powerful autonomous Agentic AI solution in Week 8 that draws on many of the prior weeks.
Follow the setup instructions above, then open the Week 1 folder and prepare for joy.

The most important part

The mantra of the course is: the best way to learn is by DOING. I don't type all the code during the course; I execute it for you to see the results. You should work along with me or after each lecture, running each cell, inspecting the objects to get a detailed understanding of what's happening. Then tweak the code and make it your own. There are juicy challenges for you throughout the course. I'd love it if you wanted to push your code so I can follow along with your progress, and I can make your solutions available to others so we share in your progress. While the projects are enjoyable, they are first and foremost designed to be educational, teaching you business skills that can be put into practice in your work.

Starting in Week 3, we'll also be using Google Colab for running with GPUs

You should be able to use the free tier or minimal spend to complete all the projects in the class. I personally signed up for Colab Pro+ and I'm loving it - but it's not required.

Learn about Google Colab and set up a Google account (if you don't already have one) here

The colab links are in the Week folders and also here:

For week 3 day 1, this Google Colab shows what colab can do
For week 3 day 2, here is a colab for the HuggingFace pipelines API
For week 3 day 3, here's the colab on Tokenizers
For week 3 day 4, we go to a colab with HuggingFace models
For week 3 day 5, we return to colab to make our Meeting Minutes product
For week 7, we will use these Colab books: Day 1 | Day 2 | Days 3 and 4 | Day 5

Monitoring API charges

You can keep your API spend very low throughout this course; you can monitor spend at the dashboards: here for OpenAI, here for Anthropic and here for Google Gemini.

The charges for the exercsies in this course should always be quite low, but if you'd prefer to keep them minimal, then be sure to always choose the cheapest versions of models:

For OpenAI: Always use model gpt-4o-mini in the code instead of gpt-4o
For Anthropic: Always use model claude-3-haiku-20240307 in the code instead of the other Claude models
During week 7, look out for my instructions for using the cheaper dataset

Please do message me or email me at ed@edwarddonner.com if this doesn't work or if I can help with anything. I can't wait to hear how you get on.

Other resources

I've put together this webpage with useful resources for the course. This includes links to all the slides.
https://edwarddonner.com/2024/11/13/llm-engineering-resources/
Please keep this bookmarked, and I'll continue to add more useful links there over time.

openvinotoolkit/openvino_notebooks

2024-12-08T01:40:26Z

📚 Jupyter notebook tutorials for OpenVINO™

English | 简体中文

📚 OpenVINO™ Notebooks

A collection of ready-to-run Jupyter notebooks for learning and experimenting with the OpenVINO™ Toolkit. The notebooks provide an introduction to OpenVINO basics and teach developers how to leverage our API for optimized deep learning inference.

🚀 Checkout interactive GitHub pages application for navigation between OpenVINO™ Notebooks content: OpenVINO™ Notebooks at GitHub Pages

List of all notebooks is available in index file.

📝 Installation Guide

OpenVINO Notebooks require Python and Git. To get started, select the guide for your operating system or environment:

Windows	Ubuntu	macOS	Red Hat	CentOS	Azure ML	Docker	Amazon SageMaker

🚀 Getting Started

Explore Jupyter notebooks using this page, select one related to your needs or give them all a try. Good Luck!

NOTE: The main branch of this repository was updated to support the new OpenVINO 2024.5 release. To upgrade to the new release version, please run pip install --upgrade -r requirements.txt in your openvino_env virtual environment. If you need to install for the first time, see the Installation Guide section below. If you wish to use the previous release version of OpenVINO, please checkout the 2024.4 branch. If you wish to use the previous Long Term Support (LTS) version of OpenVINO check out the 2023.3 branch.

If you need help, please start a GitHub Discussion.

If you run into issues, please check the troubleshooting section, FAQs or start a GitHub discussion.

Notebooks with and buttons can be run without installing anything. Binder and Google Colab are free online services with limited resources. For the best performance, please follow the Installation Guide and run the notebooks locally.

⚙️ System Requirements

The notebooks run almost anywhere — your laptop, a cloud VM, or even a Docker container. The table below lists the supported operating systems and Python versions.

Supported Operating System	Python Version (64-bit)
Ubuntu 20.04 LTS, 64-bit	3.9 - 3.12
Ubuntu 22.04 LTS, 64-bit	3.9 - 3.12
Red Hat Enterprise Linux 8, 64-bit	3.9 - 3.12
CentOS 7, 64-bit	3.9 - 3.12
macOS 10.15.x versions or higher	3.9 - 3.12
Windows 10, 64-bit Pro, Enterprise or Education editions	3.9 - 3.12
Windows Server 2016 or higher	3.9 - 3.12

💻 Run the Notebooks

To Launch a Single Notebook

If you wish to launch only one notebook, like the Monodepth notebook, run the command below (from the repository root directory):

jupyter lab notebooks/vision-monodepth/vision-monodepth.ipynb

To Launch all Notebooks

Launch Jupyter Lab with index README.md file opened for easier navigation between notebooks directories and files. Run the following command from the repository root directory:

jupyter lab notebooks/README.md

Alternatively, in your browser select a notebook from the file browser in Jupyter Lab using the left sidebar. Each tutorial is located in a subdirectory within the notebooks directory.

🧹 Cleaning Up

Shut Down Jupyter Kernel

To end your Jupyter session, press Ctrl-c. This will prompt you to Shutdown this Jupyter server (y/[n])? enter y and hit Enter.

Deactivate Virtual Environment

To deactivate your virtualenv, simply run deactivate from the terminal window where you activated openvino_env. This will deactivate your environment.

To reactivate your environment, run source openvino_env/bin/activate on Linux or openvino_env\Scripts\activate on Windows, then type jupyter lab or jupyter notebook to launch the notebooks again.

Delete Virtual Environment (Optional)

To remove your virtual environment, simply delete the openvino_env directory:

On Linux and macOS:
```
rm -rf openvino_env
```

On Windows:
```
rmdir /s openvino_env
```

Remove openvino_env Kernel from Jupyter
```
jupyter kernelspec remove openvino_env
```

⚠️ Troubleshooting

If these tips do not solve your problem, please open a discussion topic or create an issue!

To check some common installation problems, run python check_install.py. This script is located in the openvino_notebooks directory. Please run it after activating the openvino_env virtual environment.
If you get an ImportError, double-check that you installed the Jupyter kernel. If necessary, choose the openvino_env kernel from the Kernel->Change Kernel menu in Jupyter Lab or Jupyter Notebook.
If OpenVINO is installed globally, do not run installation commands in a terminal where setupvars.bat or setupvars.sh are sourced.
For Windows installation, it is recommended to use Command Prompt (cmd.exe), not PowerShell.

📚 Additional Resources

OpenVINO Blog - a collection of technical articles with OpenVINO best practices, interesting use cases and tutorials.
Awesome OpenVINO - a curated list of OpenVINO based AI projects.
OpenVINO GenAI Samples - collection of OpenVINO GenAI API samples.
Edge AI Reference Kit - pre-built components and code samples designed to accelerate the development and deployment of production-grade AI applications across various industries, such as retail, healthcare, and manufacturing.
Open Model Zoo demos - console applications that provide templates to help implement specific deep learning inference scenarios. These applications show how to preprocess and postprocess data for model inference and organize processing pipelines.
oneAPI-samples repository demonstrates the performance and productivity offered by oneAPI and its toolkits such as oneDNN in a multiarchitecture environment. OpenVINO™ toolkit takes advantage of the discrete GPUs using oneAPI, an open programming model for multi-architecture programming.

🧑‍💻 Contributors

Made with contrib.rocks.

❓ FAQ

* Other names and brands may be claimed as the property of others.

Human Rights Information: “Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles at https://www.intel.com/content/dam/www/central-libraries/us/en/documents/policy-human-rights.pdf. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

DerGoogler/MMRL

2024-12-08T01:40:33Z

MMRL is an Android app that helps manage your own modules repository.

MMRL

MMRL is an Android app that helps manage your own modules repository.

Features

Wide root manager support (Magisk, KernelSU & APatch)
WebUI support with features like Edge-to-Edge support and dynamic app theme
Full supported bulk install for online and local install
Support for module dependencies to make it for the end-user easier to install your developed module system
Providing Magisk Modules Repositories for everyone from everyone
A blacklist for user transparency
Customizable module pages (categories, screenshots, icon, cover and more)
Transparency about module included files (APKs, service files and more)
Analytics on the homepage

Preview

Supported Versions

Android 8.0 ~ 14
Magisk 24.0 ~ latest
KernelSU 0.5.1 ~ latest
APatch 10253 ~ latest

Modules Repository

Googlers-Repo/magisk-modules-repo-util: A command line interface for building modules repository

Credits

dragonflyoss/helm-charts

2024-12-08T01:43:43Z

Dragonfly Helm Charts

onfido/onfido-ios-sdk

2024-12-08T01:43:59Z

A Swift client for the Onfido API and a collection of UI elements to capture photos of documents.

Onfido Smart Capture iOS SDK

Overview

The Onfido Smart Capture SDKs provide a set of screens and functionalities that enable applications to implement user identity verification flows. Each SDK contains:

Carefully designed UX to guide your customers through the different photo or video capture processes
Modular design to help you seamlessly integrate the different photo or video capture processes into your application's flow
Advanced image quality detection technology to ensure the quality of the captured images meets the requirement of the Onfido identity verification process, guaranteeing the best success rate
Direct image upload to the Onfido service, to simplify integration
A suite of advanced fraud detection signals to protect against malicious users

All Onfido Smart Capture SDKs are orchestrated using Onfido Studio workflows, with only minor customization differences between the available platforms.

Environments and testing with the SDK

Two environments exist to support the Onfido SDK integrations:

'sandbox' - to be used for testing with sample documents
'live' - to be used only with real documents and in production apps

The environment being used is determined by the API token that is used to generate the necessary SDK token.

Going Live

Once you are satisfied with your integration and are ready to go live, please contact Onfido's Customer Support to obtain a live API token. You will have to replace the sandbox token in your code with the live token.

Check that you have entered correct billing details inside your Onfido Dashboard, before going live.

Adding the SDK dependency

The iOS SDK supports:

iOS 13+
Xcode 15+ (because of Swift minimum version)
The SDK supports the following presentation styles:
- Only full screen style for iPhones
- Full screen and form sheet styles for iPads

Note: The latest SDK version to support Xcode 11.5-12 is iOS SDK version 22, Xcode 14+ is iOS SDK version 29. There is a workaround for older versions of Xcode if required. Please contact Onfido's Customer Support team for more information.

Note: The iOS SDK requires CoreNFC to run (regardless of whether you use NFC or not). Since Xcode 12, there is a bug where libnfshared.dylib is missing from simulators. Refer to Stack Overflow for a solution to this problem.

Note: In the event that you disable the NFC feature, Apple might ask you to provide a video to demonstrate NFC usage because NFC-related code is part of the SDK binary, regardless of runtime configuration. While we're working on a permanent solution for this problem, you can contact Onfido's Customer Support in the meantime to obtain a video.

App permissions

The SDK makes use of a user's device camera (for document and face capture) and microphone (for video and motion capture). You're required to have the following keys in your application's Info.plist file:

NSCameraUsageDescription
NSMicrophoneUsageDescription

<key>NSCameraUsageDescription</key>
<string>Required for document and face capture</string>
<key>NSMicrophoneUsageDescription</key>
<string>Required for video capture</string>

Note: All keys will be required for app submission.

Adding the dependency

Using Swift Package Manager

The SDK is available with Swift Package Manager, and you can include it in your project by adding the following package repository URL:

Swift

dependencies: [
    .package(url: "https://github.com/onfido/onfido-ios-sdk.git", .branch("master"))
]

Using CocoaPods

The SDK is also available on CocoaPods, and you can include it in your project by adding the following to your Podfile:

pod 'Onfido'

Run pod install to get the SDK.

Manual installation

The SDK is available in the GitHub Releases tab, where you can download the compressed framework. You can find the latest release here.

Download the compressed zip file containing the Onfido.xcframework
Uncompress the zip file and then move the Onfido.xcframework artefact into your project folder
Open your app's project file in Xcode. Then select your app's target under target list
Add Onfido.xcframework located within your project to the Embedded binaries section in the General tab of your iOS app target

⚠️ Do not add the xcframework as resources to your app target, as only few files are required that xcode will automatically take during build.

Non-Swift apps

If your app is not Swift based, then you must create a new Swift file inside of your project. This file is required to force Xcode to package Swift runtime libraries required for the Onfido iOS SDK to run.

Create a Swift file with the following contents:

import Foundation
import AVFoundation
import CoreImage
import UIKit
import Vision

func fixLibSwiftOnoneSupport() {
    // from https://stackoverflow.com/a/54511127/2982993
    print("Fixes dyld: Library not loaded: @rpath/libswiftSwiftOnoneSupport.dylib")
}

Set Always Embed Swift Standard Libraries to Yes in your project configuration.

Initializing the SDK

⚠️ The following SDK initialization documentation applies to identity verification workflows orchestrated using Onfido Studio. For integrations where the verification steps are manually defined and configured, please refer to the Advanced flow customization section below.

The iOS SDK has multiple initialization and customization options that provide flexibility to your integration, while remaining easy to integrate.

Defining a workflow

Onfido Studio is the platform used to create highly reusable identity verification workflows for use with the Onfido SDKs. For an introduction to working with workflows, please refer to our Getting Started guide, or the Onfido Studio product guide.

SDK sessions are orchestrated by a session-specific workflow_run_id, itself derived from a workflow_id, the unique identifier of a given workflow.

For details on how to generate a workflow_run_id, please refer to the POST /workflow_runs/ endpoint definition in the Onfido API reference.

Note that in the context of the SDK, the workflow_run_id property is referred to as workflowRunId.

Applicant ID reuse

When defining workflows and creating identity verifications, we highly recommend saving the applicant_id against a specific user for potential reuse. This helps to keep track of users should you wish to run multiple identity verifications on the same individual, or in scenarios where a user returns to and resumes a verification flow.

SDK authentication

The SDK is authenticated using SDK tokens. Onfido Studio generates and exposes SDK tokens in the workflow run payload returned by the API when a workflow run is created.

SDK tokens for Studio can only be used together with the specific workflow run they are generated for, and remain valid for a period of five weeks.

Note: You must never use API tokens in the frontend of your application as malicious users could discover them in your source code. You should only use them on your server.

Build a configuration object

To use the SDK, you need to obtain an instance of the client object, using your generated SDK token and workflow run ID.

Swift

let workflowConfiguration = WorkflowConfiguration(workflowRunId: "<WORKFLOW_RUN_ID>", sdkToken: "<YOUR_SDK_TOKEN>")

Start the flow

Swift

let onfidoRun = OnfidoFlow(workflowConfiguration: orchestrationConfig)
customerViewController.present(try onfidoRun.run(), animated: true, completion: nil)
// listen for the result

NFC capture

Recent passports, national identity cards and residence permits contain a chip that can be accessed using Near Field Communication (NFC). The Onfido SDKs provide a set of screens and functionalities to extract this information, verify its authenticity and provide the resulting verification as part of a Document report.

From version 29.1.0 onwards of the Onfido iOS SDK, NFC is enabled by default and offered to end users when both the document and the device support NFC.

For more information on how to configure NFC and the list of supported documents, please refer to the NFC for Document Report guide.

Pre-requisites

This feature requires Near Field Communication Tag Reading capability in your app target. If you haven't added it before, please follow the steps in Apple's documentation.
To support NFC PACE documents, you need to edit app entitlements:
- Add a new entry nested under the Near Field Communication Tag Reader Session Formats key
- Select Password Authenticated Connection Establishment (PACE) from the dropdown list
- Alternatively you can also edit your entitlements, with the following entries:
```
<key>com.apple.developer.nfc.readersession.formats</key>
<array>
    <string>PACE</string>
    <string>TAG</string>
</array>
```
You're required to have the following key in your application's Info.plist file:

<key>NFCReaderUsageDescription</key>
<string>Required to read ePassports</string>

You have to include the entries below in your app target's Info.plist file to be able to read NFC tags properly.

<key>com.apple.developer.nfc.readersession.felica.systemcodes</key>
<array>
  <string>12FC</string>
</array>
<key>com.apple.developer.nfc.readersession.iso7816.select-identifiers</key>
<array>
  <string>A0000002471001</string>
  <string>A0000002472001</string>
  <string>00000000000000</string>
  <string>D2760000850101</string>
</array>

Configuring NFC

To configure NFC using Onfido Studio, you can use one of the following options in the workflow builder:

Off: NFC reading will not be asked to the end-users
If Possible (optional): NFC reading will be attempted, if possible
Required: NFC reading will be enforced, not allowing end-users to finish the flow without a successful reading

To configure NFC in code, call the withNFC() function while configuring OnfidoConfig (see the Advanced flow customization section below) using the above options.

⚠️ When NFC is configured as required with code, unlike with Studio, SDK will not filter out document types that are not NFC capable, for end-user best experience, expose only document types that are NFC capable as listed here, or prefer using Studio

Styling customization

The iOS SDK supports the customization of colors, fonts and strings used in the SDK flow.

UI customization

Appearance and Colors

To customize the look and feel of the SDK, you can pass the required CSS values to the Appearance object, in the root of the WorkflowConfiguration object.

Swift

let appearance = Appearance()
appearance.primaryColor = <DESIRED_UI_COLOR_HERE>
appearance.primaryTitleColor = <DESIRED_UI_COLOR_HERE>

Objective-C

ONAppearance *appearance = [[ONAppearance alloc] init];
appearance.primaryColor = <DESIRED_UI_COLOR_HERE>;
appearance.primaryTitleColor = <DESIRED_UI_COLOR_HERE>;

Please refer to the SDK customization documentation for details of the supported UI options that can be set in this property.

Dark theme

The iOS SDK supports dark theme customization. By default, the user's active device theme will be automatically applied to the Onfido SDK. However, you can opt out from dynamic theme switching at run time and instead set a theme statically at the build time as shown below. In this case, the flow will always be displayed in the selected theme regardless of the user's device theme.

interfaceStyle allows you to force light or dark mode via .dark and .light respectively. By default, it is set to .unspecified, which will follow the system's interface style.

Note: The previous attribute supportDarkMode is now deprecated. Please use interfaceStyle instead.

For example, to set the interface style to .dark, you can use the code below:

Swift

let appearance = Appearance()
appearance.setUserInterfaceStyle(.dark)

Objective-C

ONAppearance *appearance = [ONAppearance new];
[appearance setUserInterfaceStyle:UIUserInterfaceStyleDark];

Applying the Appearance object

To apply the appearance you can use the methods below:

Swift

let configBuilder = OnfidoConfig.builder()
configBuilder.withAppearance(appearance)

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withAppearance:appearance];

Co-branding

The Onfido SDK allows for two co-branding options that affect the display of the Onfido logo at the bottom of the Onfido screens.

Text co-branding

cobrand {Object} - optional

The most effective way to add your brand to the footer watermark is by use of the cobrand property under enterpriseFeatures. This property takes a text parameter.

Swift

let companyName = "MyCompany"
let enterpriseFeatures = EnterpriseFeatures.builder()
    .withCobrandingText(companyName)
    .build()

Objective-C

NSString *companyName = @"MyCompany";
ONEnterpriseFeaturesBuilder *enterpriseFeatures = [ONEnterpriseFeatures builder];
[enterpriseFeatures withCobrandingText: companyName];
[enterpriseFeatures build];

Please note: Text co-branding must be enabled by Onfido. Please contact your Solutions Engineer or Customer Success Manager to activate the feature.

Logo co-branding

logoCobrand {Object} - optional

As an alternative to cobrand, you may specify a set of images to be defined in the logoCobrand property under enterpriseFeatures. You must provide the path to an image for use in 'dark' mode and a separate image for 'light' mode. Both images must have a resolution of 144x32.

Swift

let onfidoEnterpriseFeatures = EnterpriseFeatures.builder()
    .withCobrandingLogo(
        UIImage(named: "imageName_for_lightmode")!,
        cobrandingLogoDarkMode: UIImage(named: "imageName_for_darkmode")!
     )
    .build()

Objective-C

ONEnterpriseFeaturesBuilder *enterpriseFeatures = [ONEnterpriseFeatures builder];
[enterpriseFeatures withCobrandingLogo:
  [UIImage imageNamed:@"onfido-logo-white"] cobrandingLogoDarkMode:[UIImage imageNamed:@"onfido-logo-grey"]
];
[enterpriseFeatures build];

Please note: Logo co-branding must be enabled by Onfido. Please contact your Solutions Engineer or Customer Success Manager to activate the feature.

Add co-branding to OnfidoConfig

To apply co-branding, add the enterprise features object to OnfidoConfig:

Swift

let configBuilder = OnfidoConfig.builder()
configBuilder.withEnterpriseFeatures(enterpriseFeatures)

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withEnterpriseFeatures: enterpriseFeatures];

Language localization

The Onfido SDK supports and maintains translations for over 40 languages.

The strings used within the SDK can be customized by having a Localizable.strings in your app for the desired language and by configuring the flow using the withCustomLocalization() method on the configuration builder.

Swift

- (void)withCustomLocalization {
    [self.configBuilder withCustomLocalization]; // will look for localizable strings in your Localizable.strings file
}

For the list of languages supported by Onfido, please refer to our SDK customization documentation.

Note: If no language is selected, the SDK will detect and use the end user's device language setting. If the device's language is not supported, the SDK will default to English (en_US).

Custom languages

The SDK can also be displayed in a custom language for locales that Onfido does not currently support. You can supply full or partial translations. For any key without a translation, the supported language default will be used.

When adding custom translations, you must add the whole set of keys included in the Localizable_EN.strings file (reference).

You can name the strings file with the translated keys as you desire but the name of the file will have to be provided to the SDK as a parameter to the withCustomLocalization() method:

withCustomLocalization(andTableName: "MY_CUSTOM_STRINGS_FILE") (Swift)
[configBuilder withCustomLocalizationWithTableName:@"MY_CUSTOM_STRINGS_FILE"]; (Objective-C)

Additionally you can specify the bundle from which to read the strings file:

withCustomLocalization(andTableName: "MY_CUSTOM_STRINGS_FILE", in: myBundle) (Swift)
[configBuilder withCustomLocalizationWithTableName:@"MY_CUSTOM_STRINGS_FILE" in: myBundle]; (Objective-C)

Note:

Any string translation change will result in a MINOR version release. Any custom translations you have should not be impacted by this if they have been implemented according to the guidance above
You are responsible for ensuring the correct layout of any custom translations

To request a new language translation, or offer feedback or suggestions on the translations provided, you can get in touch with Onfido's Customer Support

Completing a session

Handling callbacks

When the Onfido SDK session concludes, a range of callback functions may be triggered.

For advanced callbacks used for user analytics and returning submitted media, please refer to the Advanced Callbacks section of this document.

To receive the result from a completed workflow, you should pass a callback to the instance of OnfidoFlow. The following code is provided as an example:

Swift

onfidoRun.with(responseHandler: { (response: OnfidoResponse) in
    switch response {
        case .success:
        // User completed the flow

        case .cancel(let cancellationReason):
        // Flow cancelled by user
        print(cancellationReason)

        case .error(let error):
        // Error occurred
        print(error)

    }
},
    dismissFlowOnCompletion: true)
    // Dismiss the whole flow when the user completes it, and return back to the integrator view

ATTRIBUTE	NOTES
.success	Callback that fires when all interactive tasks in the workflow have been completed. On success, if you have configured webhooks, a notification will be sent to your backend confirming the workflow run has finished. You do not need to create a check using your backend as this is handled directly by the workflow
.error(Error)	Callback that fires when an error occurs
.cancel	Callback that fires when the workflow was exited prematurely by the user. The reason can be `.userExit` or `.consentDenied`

Error handling

The Error object returned as part of OnfidoResponse.error(Error) is of type OnfidoFlowError. It's an enum with multiple cases depending on the error type.

switch response {
  case let OnfidoResponse.error(error):
    switch error {
      case OnfidoFlowError.cameraPermission:
        // This happens if the user denies permission to the SDK during the flow
      case OnfidoFlowError.failedToWriteToDisk:
        // This happens when the SDK tries to save capture to disk, maybe due to a lack of space
      case OnfidoFlowError.microphonePermission:
        // This happens when the user denies permission for microphone usage by the app during the flow
      case OnfidoFlowError.upload(let OnfidoApiError):
        // This happens when the SDK receives an error from an API call.
        // See https://documentation.onfido.com/api/latest#errors for more information
      case OnfidoFlowError.exception(withError: let error, withMessage: let message):
        // This happens when an unexpected error occurs.
        // Please email [Customer support](mailto:supportonfido.com) when this happens
      case OnfidoFlowError.versionInsufficient:
        // This happens when you are using an older version of the iOS SDK and trying
        // to access a new functionality from workflow. You can fix this by updating the SDK

      default: // necessary because of Swift
    }
}

Generating verification reports

While the SDK is responsible for capturing and uploading the user's media and data, identity verification reports themselves are generated based on workflows created using Onfido Studio.

For a step-by-step walkthrough of creating an identity verification using Onfido Studio and our SDKs, please refer to our Quick Start Guide.

If your application initializes the Onfido iOS SDK using the options defined in the Advanced customization section of this document, you may create checks and retrieve report results manually using the Onfido API. You may also configure webhooks to be notified asynchronously when the report results have been generated.

Advanced flow customization

This section on 'Advanced customization' refers to the process of initializing the Onfido iOS SDK without the use of Onfido Studio. This process requires a manual definition of the verification steps and their configuration.

These flow step parameters are mutually exclusive with workflowRunId, requiring an alternative method of instantiating the client and starting the flow.

Note that this initialization process is not recommended as the majority of new features are exclusively released for Studio workflows.

Manual SDK authentication

The SDK is authenticated using SDK tokens. As each SDK token must be specific to a given applicant and session, a new token must be generated each time you initialize the Onfido iOS SDK.

Parameter	Notes
`applicant_id`	required Specifies the applicant for the SDK instance.
`application_id`	required The application ID (for iOS "application bundle ID") that was set up during development. For iOS, this is usually in the form `com.your-company.app-name`. Make sure to use a valid `application_id` or you'll receive a 401 error.

For details on how to manually generate SDK tokens, please refer to POST /sdk_token/ definition in the Onfido API reference.

Note: You must never use API tokens in the frontend of your application as malicious users could discover them in your source code. You should only use them on your server.

Managing SDK Token Expiry with `expireHandler`

When manually generating SDK tokens, it's important to note that they expire after 90 minutes.

With this in mind, we recommend you use the optional expireHandler parameter in the SDK token configuration function to generate and pass a new SDK token when it expires. This ensures the SDK continues its flow even after an SDK token has expired.

For example:

Swift

func getSDKToken(_ completion: @escaping (String) -> Void) {
    // Your network request logic to retrieve SDK token goes here
    completion(myNewSDKtoken)
}

let config = try OnfidoConfig.builder()
    .withSDKToken("<YOUR_SDK_TOKEN>", expireHandler: getSDKToken)

Objective-C

-(void) getSDKToken: (void(^)(NSString *)) handler {
   // <Your network request logic to retrieve SDK token goes here>
   handler(sdkToken);
}

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withSdkToken:@"YOUR_SDK_TOKEN" expireHandler:^(void (^ handler)(NSString *  expireHandler)) {
    [self getSDKToken:handler];
}];

Create the SDK configuration

Once you have added the SDK dependency, and you have an applicant ID, you can manually configure the SDK flow steps:

Swift

let config = try OnfidoConfig.builder()
    .withSDKToken("<YOUR_SDK_TOKEN>")
    .withWelcomeStep()
    .withDocumentStep()
    .withProofOfAddressStep()
    .withFaceStep(ofVariant: .photo(withConfiguration: nil))
    .build()

let onfidoFlow = OnfidoFlow(withConfiguration: config)
    .with(responseHandler: { results in
        // Callback when flow ends
    })

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];

[configBuilder withSdkToken:@"YOUR_SDK_TOKEN"];
[configBuilder withWelcomeStep];
[configBuilder withDocumentStep];
[configBuilder withProofOfAddressStep];

NSError *variantConfigError = NULL;
Builder *variantBuilder = [ONFaceStepVariantConfig builder];
[variantBuilder withPhotoCaptureWithConfig: NULL];
[configBuilder withFaceStepOfVariant: [variantBuilder buildAndReturnError: &variantConfigError]];

if (variantConfigError == NULL) {
  NSError *configError = NULL;
  ONFlowConfig *config = [configBuilder buildAndReturnError:&configError];

  if (configError == NULL) {
      ONFlow *onFlow = [[ONFlow alloc] initWithFlowConfiguration:config];
      [onFlow withResponseHandler:^(ONFlowResponse *response) {
          // Callback when flow ends
      }];
  }
}

Start the flow

Swift

try onfidoRun.run(from: yourViewController, animated: true)

Objective-C

NSError *runError = NULL;
[onFlow runFrom:yourViewController animated:YES error:&runError completion:nil];

if (runError != NULL) {
    // do fallback logic
}

Advanced UI customization

Appearance and Colors

To customize the look and feel of the SDK, you can pass the required CSS values to the Appearance object in the root of the OnfidoConfig.builder() object.

Swift

let appearance = Appearance()
appearance.primaryColor = <DESIRED_UI_COLOR_HERE>
appearance.primaryTitleColor = <DESIRED_UI_COLOR_HERE>

Objective-C

ONAppearance *appearance = [[ONAppearance alloc] init];
appearance.primaryColor = <DESIRED_UI_COLOR_HERE>;
appearance.primaryTitleColor = <DESIRED_UI_COLOR_HERE>;

To apply the appearance, you can use the methods below:

Swift

let configBuilder = OnfidoConfig.builder()
configBuilder.withAppearance(appearance)

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withAppearance:appearance];

Please refer to the SDK customization documentation for details of the supported UI options that can be set in this property.

Flow customization

You can customize the flow of the SDK by adding steps to the SDK flow.

The possible steps include:

Step	Description
`withWelcomeStep`	Welcome screen shown to the user with preliminary instructions. Customization options include modification to the text elements and instructions shown to the user.
`withDocumentStep`	Set of screens that control the capture via photo or upload of the user's document. Numerous customization options are available to define the document list presented to the user and the overall capture experience.
`withFaceStep`	Set of screens that control the capture of a selfie, video or motion capture of the user. The customization options allow the selection of the capture variant.
`withProofOfAddressStep`	Screen where the user selects the issuing country and type of document to verify their address.

Welcome step

This step is the introduction screen of the SDK. It introduces the process and prepares the user for the steps they will need to complete.

While this screen is optional, we only recommend its removal if you already have your own identity verification welcome screen in place.

You can show the welcome screen by calling configBuilder.withWelcomeStep() in Swift or [configBuilder withWelcomeStep] in Objective-C.

Swift

let config = try OnfidoConfig.builder()
    .withWelcomeStep()
    ...
    .build()

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withSdkToken:@"YOUR_SDK_TOKEN_HERE"];
...
[configBuilder withWelcomeStep];

NSError *configError = NULL;
ONFlowConfig *config = [configBuilder buildAndReturnError:&configError];

if (configError) {
    // Handle config build error
} else {
    // use config
}

Consent step

This step contains the consent language required when you offer your service to US users, as well as links to Onfido's policies and terms of use. For applicants created with a location parameter value of the United States, consent collection is mandatory.

The user must click "Accept" to move past this step and continue with the flow. The content is available in English only, and is not translatable.

Note: This step does not automatically inform Onfido that the user has given their consent:

When creating checks using API v3.3 or lower, you need to set the value of the API parameter privacy_notices_read_consent_given (now deprecated) at the end of the SDK flow when creating a check
From API v3.4 onwards, user consent is confirmed when creating or updating an applicant using the consents parameter

If you choose to disable Onfido’s SDK Consent step, you must still incorporate the required consent language and links to Onfido's policies and terms of use into your own application's flow before your users start interacting with the Onfido SDK.

For more information about this step, and how to collect user consent, please visit Onfido Privacy Notices and Consent.

Document step

In the Document Capture step, an end user can select the issuing country and document type before capture. In a very limited number of cases, the end user may also be asked if they have a card or paper version of their document.

This information is used to optimize the capture experience, as well as inform the end user about which documents they are allowed to use.

This selection screen is dynamic, and will be automatically hidden where the end user is not required to indicate which document will be captured.

By default, the country selection will be pre-populated based on the end user’s primary SIM, but the end user can select another country from the list where allowed. The selection will default to empty when no SIM is present.

You can specify allowed issuing countries and document types for the document capture step in one of three ways:

Onfido Studio: If you are using Onfido Studio, this is configured within the Document Capture task, as documented in the Studio Product Guide
Otherwise, the recommended approach is to apply this configuration globally in the Dashboard under Accounts \ Supported Documents. This option also ensures that the list is enforced as part of the Document Report validation. Any document that has been uploaded by an end user against your guidance will result in a Document Report sub-result of "rejected" and be flagged as Image Integrity > Supported Document.

Country and document type selection by Dashboard

Configuring the issuing country and document type selection step using your Dashboard is the recommended method of customization (available from iOS SDK version 28.0.0 and Android SDK version 16.0.0 onwards), as this configuration is also applied to your Document Reports.

We will be rolling out Dashboard-based configuration of allowed documents soon. In the meantime, contact support@onfido.com or your Customer Support Manager to request access to this feature.

Open the Accounts tab on your Dashboard then click Supported Documents
You will be presented with a list of all available countries and their associated supported documents. Make your selection, then click Save Change

Please note the following SDK behaviour:

Hard coding any document type and issuing country configuration in your SDK integration will fully override the Dashboard-based settings
Currently, only passport, national ID card, driving licence and residence permit are visible for document selection by the end user in the SDK. If you nominate other document types in your Dashboard (visa, for example), these will not be displayed in the user interface
If you need to add other document types to the document selection screen, you can mitigate this limitation in the near-term, using the Custom Document feature
If for any reason the configuration fails or is not enabled, the SDK will fallback to display the selection screen for the complete list of documents supported within the selection screens

Country and document type selection - SDK integration code

If you want to use your own custom document selection UI instead of displaying the Onfido document selection screen, you will need to specify the document details during SDK initialization.

The document selection screen will be skipped automatically when the single document type is specified.

The SDK will accept the following:

The Document Type is required. This controls fundamental SDK document capture behaviour
The Country is optional, but recommended. This enables any optimizations the SDK may have for this specific document issued by this country
The Document Format is optional, and only accepted for French driving licence, Italian national identity card and South African national identity card. This defaults to Card, representing modern forms of these documents. If the end user indicates that they have an older, paper version of one of these documents, use Folded to ensure an optimized capture experience

Note: You may still wish to configure the Dashboard-based approach to ensure that the Document Report also rejects any document that has been uploaded by an end user against your guidance.

Document type

The list of document types visible for the user to select can be shown or hidden using this option. Each document type has its own configuration class. While configuring document type, you can optionally pass a configuration object along with the document type.

The following document types are supported:

Document Type	Configuration Class	Configurable Properties
passport	PassportConfiguration	- country
drivingLicence	DrivingLicenceConfiguration	- country - documentFormat
nationalIdentityCard	NationalIdentityConfiguration	- country - documentFormat
residencePermit	ResidencePermitConfiguration	- country
visa	VisaConfiguration	- country
workPermit	WorkPermitConfiguration	- country
generic	GenericDocumentConfiguration	- title - subtitle - country - pages

Note: Generic document type doesn't offer an optimized capture experience for a desired document type.

Note: If only one document type is specified, users will not see the selection screen and will be taken directly to the capture screen.

Document country

Country configuration allows you to specify the country of origin of the document. This is optional, but recommended. This enables any optimizations the SDK may have for this specific document issued by this country.

You'll need to pass the corresponding ISO 3166-1 alpha-3 country code to the SDK.

Note: You can specify country for all document types except passport. This is because passports have the same format worldwide so the SDK does not require this additional information.

Note:: The SDK will throw a OnfidoConfigError.invalidCountryCode (ONFlowConfigErrorInvalidCountryCode) error if an invalid country code is provided.

For example, to only capture UK driving licenses:

Swift

let config = try OnfidoConfig.builder()
    .withSDKToken("<YOUR_SDK_TOKEN_HERE>")
    .withDocumentStep(ofType: .drivingLicence(config: DrivingLicenceConfiguration(country: "GBR")))
    .build()

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withSdkToken:@"YOUR_SDK_TOKEN_HERE"];

NSError *documentConfigError = NULL;
DocumentConfigBuilder *documentConfigBuilder = [ONDocumentTypeConfig builder];
[documentConfigBuilder withDrivingLicenceWithConfig:[[DrivingLicenceConfiguration alloc] initWithCountry: @"GBR"]];
ONDocumentTypeConfig *documentTypeConfig = [documentConfigBuilder buildAndReturnError: documentConfigError];

if (documentConfigError) {
  // Handle variant config error
} else {
  NSError *configError = NULL;
  [configBuilder withDocumentStepOfType:documentTypeConfig];
  ONFlowConfig *config = [configBuilder buildAndReturnError:&configError];
}

Document format

The Document Format is optional, and only accepted for French driving licence, Italian national identity card and South African national identity card. This defaults to Card, representing modern forms of these documents. If the end user indicates that they have an older, paper version of one of these documents, use Folded to ensure an optimized capture experience.

If Folded is configured, the SDK will show a specific template overlay during document capture.

The following document formats are supported for each document type:

Document Type/ Document Format	card	folded
drivingLicence	ALL COUNTRIES	Only France (Country code "FRA")
nationalIdentityCard	ALL COUNTRIES	Italy (Country code "ITA") South Africa (Country code "ZAF")

Document Type/ Document Format
passport	NOT CONFIGURABLE
residencePermit	NOT CONFIGURABLE
visa	NOT CONFIGURABLE
workPermit	NOT CONFIGURABLE
generic	NOT CONFIGURABLE

Note: If you configure the SDK with an unsupported document format, the SDK will throw an OnfidoConfigError.invalidDocumentFormatAndCountryCombination (ONFlowConfigErrorInvalidDocumentFormatAndCountryCombination in Objective-C) error during runtime.

For example, for a folded Italian national identity card:

Swift

let config = try OnfidoConfig.builder()
    .withSDKToken("YOUR_SDK_TOKEN_HERE")
    .withDocumentStep(ofType: .nationalIdentityCard(config: NationalIdentityConfiguration(documentFormat: .folded, country: "ITA")))
    .build()

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withSdkToken:@"YOUR_SDK_TOKEN_HERE"];

NSError *documentConfigError = NULL;
DocumentConfigBuilder *documentConfigBuilder = [ONDocumentTypeConfig builder];
[documentConfigBuilder withNationalIdentityCardWithConfig:[[NationalIdentityConfiguration alloc] initWithDocumentFormat:DocumentFormatFolded country: @"ITA"]];
ONDocumentTypeConfig *documentTypeConfig = [documentConfigBuilder buildAndReturnError: documentConfigError];

if (documentConfigError) {
  // Handle variant config error
} else {
  NSError *configError = NULL;
  [configBuilder withDocumentStepOfType:documentTypeConfig];
  ONFlowConfig *config = [configBuilder buildAndReturnError:&configError];
}

Customize the issuing country and document type selection screen with pre-selected documents

You can also customize the screen to display only a limited list of document types, using the configuration function to specify the ones you want to show.

Currently you can only include passport, identityCard, drivingLicence, residencePermit in the list.

For example, to show only the passport and drivingLicence document types:

Swift

let config = try OnfidoConfig.builder()
    .withDocumentStep(ofSelectableTypes: [.passport, .drivingLicence])

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withDocumentStepWithSelectableDocumentTypes: @[@(SelectableDocumentTypePassport), @(SelectableDocumentTypeDrivingLicence)]];

Adding custom documents by SDK integration code You can also capture custom documents that Onfido does not fully support in the SDK by using the Generic document type.

This can be useful for running the Document Report on documents that are supported in the Onfido platform, but are not yet present in the SDK, or for changing the default name of the document type presented to the user or the default number of sides/pages the user will be asked to capture.

The Generic document type supports limited on-device image quality checks. Use the standard document types where available for an improved capture experience.

A custom document consists of the following information:

Title (required): Displayed in the document selection screen
Subtitle (optional): Displayed in the document selection screen below the title
Country (required): To support multiple countries, create a custom document for each country
Pages (required): Tells the SDK The number of document pages to be captured. Currently supporting one or two sides/pages

It is recommended to provide pre-translated title and subtitle strings when you initialize the SDK, as these custom strings are not translated internally.

Here are several options when adding custom documents:

Adding the custom documents per country on top of the documents we support. You can add more than one custom document. If you require documents for multiple countries, you will have to add each document for each country.

Swift

let config = try! OnfidoConfig.builder()
    .withSDKToken("YOUR_SDK_TOKEN_HERE")
    .withGenericDocuments(additionalTypes: [
        .generic(config: GenericDocumentConfiguration(title: "Tax ID", country: "PHL", pages: .frontAndBack))
        .generic(config: GenericDocumentConfiguration(title: "Voter's ID", subtitle: "your voting ID", country: "PHL", pages: .single))
    ])
    .build()

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withSdkToken:@"YOUR_SDK_TOKEN_HERE"];

NSError *documentConfigError = NULL;
DocumentConfigBuilder * documentConfigBuilder = [ONDocumentTypeConfig builder];
[documentConfigBuilder withGenericWithConfig:[[GenericDocumentConfiguration alloc] initWithTitle: @"Tax ID" subtitle: nil country: @"PHL" pages: ONDocumentPagesFrontAndBack]];
[documentConfigBuilder withGenericWithConfig:[[GenericDocumentConfiguration alloc] initWithTitle: @"Voter's ID" subtitle: @"your voting ID" country: @"PHL" pages: ONDocumentPagesSingle]];
ONDocumentTypeConfig *documentTypeConfig = [documentConfigBuilder buildAndReturnError: documentConfigError];

if (documentConfigError) {
  // Handle variant config error
} else {
  NSError *configError = NULL;
  [configBuilder withGenericDocumentsWithAdditionalTypes:documentTypeConfig];
  ONFlowConfig *config = [configBuilder buildAndReturnError:&configError];
}

Preselect documents to show in the document selection screen and include custom documents for a specific country (custom documents will only be shown if the country configured is the same as the country selected).

Swift

let config = try! OnfidoConfig.builder()
    .withSDKToken("YOUR_SDK_TOKEN_HERE")
    .withDocumentStep(ofSelectableTypes: [.passport, .drivingLicence])
    .withGenericDocuments(additionalTypes: [
        .generic(config: GenericDocumentConfiguration(title: "Tax ID", country: "PHL", pages: .frontAndBack))
        .generic(config: GenericDocumentConfiguration(title: "Voter's ID", subtitle: "your voting ID", country: "PHL", pages: .single))
    ])
    .build()

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withSdkToken:@"YOUR_SDK_TOKEN_HERE"];

NSError *documentConfigError = NULL;
DocumentConfigBuilder *documentConfigBuilder = [ONDocumentTypeConfig builder];
[documentConfigBuilder withDocumentStepWithSelectableDocumentTypes: @[@(SelectableDocumentTypePassport), @(SelectableDocumentTypeDrivingLicence)]];
[documentConfigBuilder withGenericWithConfig:[[GenericDocumentConfiguration alloc] initWithTitle: @"Tax ID" subtitle: nil country: @"PHL" pages: ONDocumentPagesFrontAndBack]];
[documentConfigBuilder withGenericWithConfig:[[GenericDocumentConfiguration alloc] initWithTitle: @"Voter's ID" subtitle: @"your voting ID" country: @"PHL" pages: ONDocumentPagesSingle]];
ONDocumentTypeConfig *documentTypeConfig = [documentConfigBuilder buildAndReturnError: documentConfigError];

if (documentConfigError) {
  // Handle variant config error
} else {
  NSError *configError = NULL;
  [configBuilder withGenericDocumentsWithAdditionalTypes:documentTypeConfig];
  ONFlowConfig *config = [configBuilder buildAndReturnError:&configError];
}

Add a custom document and skip the document/country selection screen. This can only be done when adding only 1 document.

Swift

let config = try! OnfidoConfig.builder()
    .withSDKToken("YOUR_SDK_TOKEN_HERE")
    .withDocumentStep(ofType:.generic(config:
        GenericDocumentConfiguration(title: "Voter's ID", subtitle: "your voting ID", country: "PHL", pages: .single))
    )
    .build()

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withSdkToken:@"YOUR_SDK_TOKEN_HERE"];

NSError *documentConfigError = NULL;
DocumentConfigBuilder *documentConfigBuilder = [ONDocumentTypeConfig builder];
[documentConfigBuilder withGenericWithConfig:[[GenericDocumentConfiguration alloc] initWithTitle: @"Tax ID" subtitle: nil country: @"PHL" pages: ONDocumentPagesSingle]];
ONDocumentTypeConfig *documentTypeConfig = [documentConfigBuilder buildAndReturnError: documentConfigError];

if (documentConfigError) {
  // Handle variant config error
} else {
  NSError *configError = NULL;
  [configBuilder withDocumentStepOfType:documentTypeConfig];
  ONFlowConfig *config = [configBuilder buildAndReturnError:&configError];
}

Note:

Adding an empty title will throw an error OnfidoConfigError.invalidDocumentTitle
Adding duplicate documents (same title and country) will also throw an error OnfidoConfigError.duplicateGenericDocument
Custom Documents are not currently compatible with the Dashboard-based configuration method or with Studio
When using a Custom Document to force the single-sided capture of a document that Onfido would usually capture two sides for, the Document Report may return a Caution for Image Integrity > Conclusive Document Quality > missing_back) as the back of certain documents contain expected data points or fraud features. You may wish to customize your result processing logic to accommodate this

Face step

In the Face step, a user can use the front camera of their device to capture a live photo of their face (photo), a live video (video) or a motion capture using the Motion product (Motion).

Objective-C

For the Objective-C interface, you should use ONFaceStepVariantConfig as shown below.

To configure for a live photo:

NSError * error;
Builder * variantBuilder = [ONFaceStepVariantConfig builder];
[variantBuilder withPhotoCaptureWithConfig: [[PhotoStepConfiguration alloc] initWithShowSelfieIntroScreen: YES]]];
[configBuilder withFaceStepOfVariant: [variantBuilder buildAndReturnError: &error]];

To configure for a live video:

NSError * error;
Builder * variantBuilder = [ONFaceStepVariantConfig builder];
[variantBuilder withVideoCaptureWithConfig:
   [[VideoStepConfiguration alloc] initWithShowIntroVideo: YES manualLivenessCapture: NO]];
[configBuilder withFaceStepOfVariant: [variantBuilder buildAndReturnError: &error]];

To configure for Motion:

NSError * error;
Builder * variantBuilder = [ONFaceStepVariantConfig builder];
[variantBuilder withMotionWithConfig: NULL];
[configBuilder withFaceStepOfVariant: [variantBuilder buildAndReturnError: &error]];

To configure for Motion with audio recording:

NSError * error;
Builder * variantBuilder = [ONFaceStepVariantConfig builder];
[variantBuilder withMotionWithConfig: [[MotionStepConfiguration alloc] initWithRecordAudio: YES]];
[configBuilder withFaceStepOfVariant: [variantBuilder buildAndReturnError: &error]];

To configure for Motion without intro screen:

The intro screens are designed to give end-users clear context on what actions they need to take and why these actions are necessary. Removing this screen could lead to confusion and higher drop-off rates, as users might not understand what is required of them and choose to abandon the process. It is crucial, especially during onboarding, to provide a similar level of guidance if the intro screen is removed, ensuring users are still well-informed. However, when re-verifying returning users who are already familiar with the flow, skipping the intro screen may enhance the user experience by streamlining the process and eliminating unnecessary steps.

NSError * error;
Builder * variantBuilder = [ONFaceStepVariantConfig builder];
[variantBuilder withMotionWithConfig: [[MotionStepConfiguration alloc] initWithShowIntro: NO]];
[configBuilder withFaceStepOfVariant: [variantBuilder buildAndReturnError: &error]];

Swift

The Face step has 3 variants for the Swift interface:

FaceStepVariant.photo(with: PhotoStepConfiguration?)
FaceStepVariant.video(with: VideoStepConfiguration?)
FaceStepVariant.motion(with: MotionStepConfiguration?)

To configure for a live photo:

let config = try OnfidoConfig.builder()
    .withSDKToken("<YOUR_SDK_TOKEN_HERE>")
    .withWelcomeStep()
    .withDocumentStep()
    .withFaceStep(ofVariant: .photo(withConfiguration: PhotoStepConfiguration(showSelfieIntroScreen: true)))
    .build()

To configure for a live video:

let config = try OnfidoConfig.builder()
    .withSDKToken("<YOUR_SDK_TOKEN_HERE>")
    .withWelcomeStep()
    .withDocumentStep()
    .withFaceStep(ofVariant: .video(withConfiguration: VideoStepConfiguration(showIntroVideo: true, manualLivenessCapture: false)))
    .build()

To configure for Motion:

let config = try OnfidoConfig.builder()
    .withSDKToken("<YOUR_SDK_TOKEN_HERE>")
    .withWelcomeStep()
    .withDocumentStep()
    .withFaceStep(ofVariant: .motion(withConfiguration: nil))
    .build()

To configure for Motion with audio recording:

let config = try OnfidoConfig.builder()
    .withSDKToken("<YOUR_SDK_TOKEN_HERE>")
    .withWelcomeStep()
    .withDocumentStep()
    .withFaceStep(ofVariant: .motion(withConfiguration: MotionStepConfiguration(recordAudio: true)))
    .build()

To configure for Motion without intro screen:

let config = try OnfidoConfig.builder()
    .withSDKToken("<YOUR_SDK_TOKEN_HERE>")
    .withWelcomeStep()
    .withDocumentStep()
    .withFaceStep(ofVariant: .motion(withConfiguration: MotionStepConfiguration(showIntro: false)))
    .build()

Proof of Address step

In the Proof of Address step, a user picks the issuing country and type of document that proves their address before capturing the document with their device camera or uploading it.

Swift

let config = try OnfidoConfig.builder()
    .withSDKToken("<YOUR_SDK_TOKEN_HERE>")
    .withProofOfAddressStep()
    .build()

Objective-C

ONFlowConfigBuilder *configBuilder = [ONFlowConfig builder];
[configBuilder withSdkToken:@"YOUR_SDK_TOKEN_HERE"];
[configBuilder withProofOfAddressStep];

NSError *configError = NULL;
ONFlowConfig *config = [configBuilder buildAndReturnError:&configError];

if (configError) {
    // Handle config build error
} else {
    // use config
}

Advanced callbacks

Handling callbacks

Swift

To receive the result from the flow, you should pass a callback to the instance of OnfidoFlow.

The result object passed to the callback may include the following attributes:

let responseHandler: (OnfidoResponse) -> Void = { response in
    switch response {
    case .error(let error):
    case .success(let results):
    case .cancel(let reason):
    }
}

Attribute	Notes
`.success([OnfidoResult])`	User completed the flow. You can now create a check on your backend server.
`.error(Error)`	Callback that fires when an error occurs.
`.cancel`	Flow was cancelled by the user. The reason can be `.userExit` (when the user taps back button on the first screen) or `.deniedConsent` (when the user denies consent on the consent screen) or `.requiredNFCFlowNotCompleted` (when the NFC step is required and the user did not complete it).

Objective-C

To receive the result from the flow, you should pass a callback to the instance of ONFlow.

An instance of ONFlowResponse is passed back to the callback with 3 properties:

(^responseHandlerBlock)(ONFlowResponse *response) {
    if (response.userCanceled) {
    } else if (response.results) {
    } else if (response.error) {
    }
}

Properties	Notes
`results`	User completed the flow. You can now create a check on your backend server.
`error`	Callback that fires when an error occurs.
`userCanceled`	Flow was cancelled by the user. You can check why the user cancelled using `response.userCanceled.reason`. When `userCanceled` is false, then `results` or `error` properties will be set.

Success handling

Success is when the user has reached the end of the flow.

Swift

[OnfidoResult] is a list with multiple results. The results are different enum values, each with its own associated value (also known as payload). This enum, OnfidoResult, can have the following values:

OnfidoResult.document and OnfidoResult.face: Its payload is relevant in case you want to manipulate or preview the captures in some way.

Capture result payload

You shouldn't need to inspect the results of the document and face captures as the SDK handles file uploads. However, if you want to see further information, you can access the result object.

Example for a document capture:

let document: Optional<OnfidoResult> = results.first { result in
    if case OnfidoResult.document = result { return true }
    return false
}

if let document,
   case OnfidoResult.document(let documentResult) = document {
    print(document.front.id)
}

To access the result object for a face capture, input the case as OnfidoResult.face.

Objective-C

[ONFlowResult] is a list with multiple results. The result is an instance of ONFlowResult containing 2 properties:

type, which is an enum with values ONFlowResultTypeDocument, ONFlowResultTypeFace
result, whose instance type can be of ONDocumentResult or ONFaceResult. The result type can be derived by the type property

Capture result payload

You shouldn't need to inspect the results of the document and face captures as the SDK handles file uploads. However, if you want to see further information, you can access the result object.

Example for a document capture:

NSPredicate *documentResultPredicate = [NSPredicate predicateWithBlock:^BOOL(id flowResult, NSDictionary *bindings) {
    if (((ONFlowResult *)flowResult).type == ONFlowResultTypeDocument) {
        return YES;
    } else {
        return NO;
    }
}];
NSArray *flowWithDocumentResults = [results filteredArrayUsingPredicate:documentResultPredicate];

if (flowWithDocumentResults.count > 0) {
    ONDocumentResult *documentResult = ((ONFlowResult *)flowWithDocumentResults[0]).result;
    NSLog(@"%@", documentResult.front.id);
}

Sample of instance returned by a flow with FlowStep.document, FlowStep.face and FlowStep.proofOfAddress:

Document:
        Front: DocumentSideResult(id=document_id, side=FRONT, type=DRIVING_LICENCE, issuingCounfry=GBR)
        Back: DocumentSideResult(id=document_id, side=BACK, type=DRIVING_LICENCE, issuingCounfry=GBR)
        Type: DRIVING_LICENCE

Face:
        Face(id=face_id, variant=PHOTO)

Proof of address:
        ProofOfAddress(type=UTILITY_BILL, front=(id=front_side_id, type=(optional)), back=(id=back_side_id, type=(optional)))

To access the result object for a face capture, change the type to ONFlowResultTypeFace.

Error handling

Swift

Response Handler Errors

The Error object returned as part of OnfidoResponse.error(Error) is of type OnfidoFlowError. It's an enum with multiple cases depending on the error type.

switch response {
case let OnfidoResponse.error(error):
    switch error {
    case OnfidoFlowError.cameraPermission:
        // Occurs if the user denies permission to the SDK during the flow
    case OnfidoFlowError.microphonePermission:
        // Occurs when the user denies permission for microphone usage by the app during the flow
    case OnfidoFlowError.failedToWriteToDisk:
        // Occurs when the SDK tries to save capture to disk, maybe due to a lack of space
    case OnfidoFlowError.upload(let OnfidoApiError):
        // Occurs when the SDK receives an error from an API call, see [https://documentation.onfido.com/api/latest#errors](https://documentation.onfido.com/api/latest#errors) for more information
    case OnfidoFlowError.exception(withError: let error, withMessage: let message):
        // Returned when an unexpected error occurs, please contact [support](mailto:support@onfido.com) when this happens
    case OnfidoFlowError.invalidImageData:
        // Occurs when the SDK tries to save capture to disk, but the image failed to compress to JPEG data
    case OnfidoFlowError.versionInsufficient:
        // Occurs when the workflow version is insufficient
    default: // necessary because swift
    }
}

Note: Not all errors will be passed to OnfidoResponse.error. Run Exceptions and Configuration errors will be returned as an exception.

Run exceptions

When initiating the SDK there can be an exception.

You can handle run exceptions in Swift with a do/catch as shown below:

do {
    try onfidoFlow.run(from: yourViewController, animated: true)
} catch let error {
    switch error {
    case OnfidoFlowError.cameraPermission:
        // do something about it here
    case OnfidoFlowError.microphonePermission:
        // do something about it here
    default:
        // should not happen, so if it does, log it and let us know
    }
}

Configuration errors

You must provide the following when configuring the Onfido iOS SDK:

SDK token
applicant
at least one capture step

Otherwise, you may encounter the following errors when calling the build() function on the OnfidoConfig.Builder instance:

Error	Notes
`OnfidoConfigError.missingSDKToken`	When no token is provided or the token is an empty string.
`OnfidoConfigError.invalidSDKToken`	When an invalid token is provided.
`OnfidoConfigError.missingSteps`	When no step is provided.
`OnfidoConfigError.invalidDocumentFormatAndCountryCombination`	When it is an unsupported document format for the specified country provided. See Document Type Configuration to check supported combinations.
`OnfidoConfigError.invalidCountryCode`	When an invalid country code is provided.

Objective-C

Response Handler Errors

The error property of the ONFlowResponse is of type NSError.

You can identify the error by comparing the code property of the NSError instance with ONFlowError, i.e. response.code == ONFlowErrorCameraPermission. You can also print or log the userInfo property of the NSError instance.

The NSError contained within the ONFlowResponse's error property can be handled as follows:

switch (error.code) {
case ONFlowErrorCameraPermission:
    // Occurs if the user denies permission to the SDK during the flow
    break;
case ONFlowErrorMicrophonePermission:
    // Occurs when the user denies permission for microphone usage by the app during the flow
    break;
case ONFlowErrorFailedToWriteToDisk:
    // Occurs when the SDK tries to save capture to disk, maybe due to a lack of space
    break;
case ONFlowErrorUpload:
    // Occurs when the SDK receives an error from an API call, see [https://documentation.onfido.com/api/latest#errors](https://documentation.onfido.com/api/latest#errors) for more information
    // you can find out more by printing or logging userInfo from error
    break;
case ONFlowErrorException:
    // Returned when an unexpected error occurs, please contact Onfido's [Customer Support](mailto:support@onfido.com) when this happens
    break;
case ONFlowErrorInvalidImageData:
    // Occurs when the SDK tries to save capture to disk, but the image failed to compress to JPEG data
    break;
case ONFlowErrorVersionInsufficient:
    // Occurs when the workflow version is insufficient
    break;
}

Note: Not all errors which are part of ONFlowError will be passed to the response handler block. Run Exceptions and Configuration errors will be returned as an exception.

Run exceptions

You can handle run exceptions as shown below:

NSError *runError = NULL;
[onFlow runFrom:yourViewController animated:YES error:&runError completion:nil]; //`yourViewController` should be your view controller

if (runError) {
    switch (runError.code) {
        case ONFlowErrorCameraPermission:
            // do something about it here
            break;
        case ONFlowErrorMicrophonePermission:
            // do something about it here
            break;
        default:
            // do something about it here
            break;
    }
}

Configuration errors

You must provide the following when configuring the Onfido iOS SDK:

SDK token
applicant
at least one capture step

Otherwise you may encounter the following errors when calling the build() function on the ONFlowConfigBuilder:

Error	Notes
`ONFlowConfigErrorMissingToken`	When no token is provided or the token is an empty string.
`ONFlowConfigErrorMissingApplicant`	When no applicant instance is provided.
`ONFlowConfigErrorMissingSteps`	When no step is provided.
`ONFlowConfigErrorMultipleTokenTypes`	When both an SDK token and a Mobile token are provided.
`ONFlowConfigErrorApplicantProvidedWithSDKToken`	When both an SDK token and an applicant are provided.
`ONFlowConfigErrorInvalidDocumentFormatAndCountryCombination`	When it is an unsupported document format for the specified country provided. See Document Type Configuration to check supported combinations.
`ONFlowConfigErrorInvalidCountryCode`	When an invalid country code is provided.

Custom Media Callbacks

The following features must be enabled for your account before they can be used. For more information, please contact your Onfido Solution Engineer or Customer Success Manager.

Introduction

Onfido provides the possibility to integrate with our Smart Capture SDK, without the requirement of using this data only through the Onfido API. Media callbacks enable you to control the end user data collected by the SDK after the end user has submitted their captured media. As a result, you can leverage Onfido’s advanced on-device technology, including image quality validations, while still being able to handle end users’ data directly. This unlocks additional use cases, including compliance requirements and multi-vendor configurations, that require this additional flexibility.

This feature must be enabled for your account. Please contact your Onfido Solution Engineer or Customer Success Manager.

Implementation

To use this feature, use .withMediaCallback and provide the callbacks for MediaDocumentResult for documents and MediaFile for live photos and live videos.

Swift

final class SwiftDynamicFrameworkOnfidoRunner: OnfidoRunner, MediaCallback {
    func onMediaCaptured(result: MediaResult) {
        switch result {
        case let documentResult as MediaDocumentResult:
            // Your callback code here
        case let selfieResult as SelfieResult:
            // Your callback code here
        case let livenessResult as LivenessResult:
            // Your callback code here
        default:
            break
        }
    }

    init() {
        ...
        configBuilder.withMediaCallback(mediaCallback: self)
    }
}

User data

The callbacks return an object including the information that the SDK normally sends directly to Onfido. The callbacks are invoked when the end user confirms submission of their image through the SDK’s user interface.

Note: Currently, end user data will still automatically be sent to the Onfido backend, but you are not required to use Onfido to process this data.

The callback returns 3 possible objects:

For documents, the callback returns a MediaDocumentResult object:

class MediaDocumentResult {
    let metadata: DocumentMetadata
    let file: MediaFile
}

The DocumentMetadata object contains the metadata of the captured document:

class DocumentMetadata {
    let side: String
    let type: String
    let issuingCountry: String?
}

Note: issuingCountry is optional based on end-user selection, and can be null.

Note: If a document was scanned using NFC, the callback will return the passport photo in file, but no additional data.

For live photos, the callback returns a SelfieResult object:

class SelfieResult {
    let fileData: MediaFile
}

For videos, the callback returns a LivenessResult object:

class LivenessResult {
    let fileData: MediaFile
}

And the MediaFile object has:

class MediaFile {
    let fileData: Data
    let fileName: String
    let fileType: String
}

User analytics

The SDK also allows you to track a user's journey through the Onfido verification process via a definable hook.

Overriding the hook

In order to expose a user's progress through the SDK, a hook method must be defined while creating the OnfidoFlow.swift instance using a .with(eventHandler: EventHandler) call. For example:

OnfidoFlow(withConfiguration: config)
    .with(eventHandler: {
        (event: Event) -> () in
        // Your code here
    })

The code inside of the defined method will now be called when a particular event is triggered, usually when the user reaches a new screen. For a full list of events see, tracked events.

The parameter being passed in is an OnfidoFlow.Event struct which contains the following:


`eventName`	string Indicates the type of event. This will always be returned as `"Screen"`, as each tracked event is a user visiting a screen.
`properties`	dictionary Contains the specific details of an event. For example, the name of the screen visited.

Using the data

You can use the data to keep track of how many users reach each screen in your flow. You can do this by storing the number of users that reach each screen and comparing that to the number of users who reached the Welcome screen.

Tracked events

Below is the list of potential events currently being tracked by the hook:

WELCOME - User reached the "Welcome" screen
USER_CONSENT - User reached "User consent" screen
DOCUMENT_CAPTURE - User reached the "Document capture" screen (for one-sided document)
DOCUMENT_CAPTURE_FRONT - User reached the "Document capture" screen for the front side (for two-sided document)
DOCUMENT_CAPTURE_BACK - User reached the "Document capture" screen for the back side (for two-sided document)
DOCUMENT_CAPTURE_CONFIRMATION - User reached the "Document confirmation" screen (for one-sided document)
DOCUMENT_CAPTURE_CONFIRMATION_FRONT - User reached the "Document confirmation" screen for the front side (for two-sided document)
DOCUMENT_CAPTURE_CONFIRMATION_BACK - User reached the "Document confirmation" screen for the back side (for two-sided document)
DOCUMENT_UPLOAD - User's document is uploading
FACIAL_INTRO - User reached the "Selfie intro" screen
FACIAL_CAPTURE - User reached the "Selfie capture" screen
FACIAL_CAPTURE_CONFIRMATION - User reached the "Selfie confirmation" screen
FACIAL_UPLOAD - User's selfie is uploading
VIDEO_FACIAL_INTRO - User reached the "Liveness intro" screen
VIDEO_FACIAL_CAPTURE - User reached the "Liveness video capture" screen
VIDEO_FACIAL_CAPTURE_STEP_1 - User reached the 1st challenge during "Liveness video capture", challenge_type can be found in eventProperties
VIDEO_FACIAL_CAPTURE_STEP_2 - User reached the 1st challenge during "Liveness video capture", challenge_type can be found in eventProperties
VIDEO_FACIAL_CAPTURE_CONFIRMATION - User reached the "Liveness video confirmation" screen
VIDEO_FACIAL_UPLOAD - User's liveness video is uploading
MOTION_FACIAL_INTRO - User reached the "Motion intro" screen
MOTION_FACIAL_ALIGNMENT - User reached the "Motion alignment" screen
MOTION_FACIAL_CAPTURE - User reached the "Motion capture" screen
MOTION_FACIAL_NO_FACE_DETECTED - User's face was not detected
MOTION_FACIAL_CAPTURE_ERROR_TIMEOUT - User's motion capture timed out
MOTION_FACIAL_CAPTURE_ERROR_TOO_FAST - User performed the motion head turn too fast
MOTION_FACIAL_UPLOAD - User's motion capture is uploading
MOTION_FACIAL_UPLOAD_COMPLETED - User's motion capture finished uploading
MOTION_FACIAL_CONNECTION_ERROR - User was presented the "Motion connection error" screen during upload

Custom biometric token storage

When using the authentication with local storage solution, by default the SDK manages biometric token storage. The SDK also allows you to have control over the token lifecycle by exposing an API to override the default implementation to read and write the token, so it can be stored on-device, in the cloud, in a keychain or on your servers.

Implementation

You need to provide a class that conforms to the EncryptedBiometricTokenHandler protocol. Note the customerUserHash parameter in both functions that need to be implemented. This is a unique identifier for the user that can be used as a key for token storage. Feel free to ignore it if you have your own identifier.

Example of a class handling the callbacks for when a token is generated and for when we request a token from you:

class CustomTokenHandlerClass: EncryptedBiometricTokenHandler {
    func onTokenGenerated(customerUserHash: String, encryptedBiometricToken: String) {
        // You store `customerUserHash` and `encryptedBiometricToken` however you choose to do so
    }

    func onTokenRequested(customerUserHash: String, completion: @escaping (String) -> Void) {
        // You use the `customerUserHash` to retrieve the encrypted biometric token you have stored and call `completion`, passing in this token
    }
}

Example of initialising a workflow with a class that handles tokens:

let workflowConfiguration = WorkflowConfiguration(workflowRunId: "<WORKFLOW_RUN_ID>", sdkToken: "<YOUR_SDK_TOKEN>")
workflowConfiguration.withEncryptedBiometricTokenHandler(handler: self) // `self` to be replaced with a different instance if you are using a different class conforming to `EncryptedBiometricTokenHandler`

Migrating

You can find the migration guide at MIGRATION.md

Security

Certificate pinning

Note: Certificate pinning works only on devices running on iOS 10.3 or above.

You can pin any communications between our SDK and server through the .withCertificatePinning() method in our OnfidoConfig.Builder configuration builder. This method accepts CertificatePinningConfiguration as a parameter, with sha-256 hashes of the certificate's public keys.

For more information about the hashes, please email Onfido's Customer Support.

Swift

let config = try OnfidoConfig.builder()
...
do {
    config.withCertificatePinning(try CertificatePinningConfiguration(hashes: ["<EXAMPLE_HASH>"]))
} catch {
    // handle CertificatePinningConfiguration initialisation failures. i.e Providing empty array causes initializer to be failed.
}
...
configBuilder.build()

Objective-C

ONFlowConfigBuilder * builder =[ONFlowConfig builder];
...
NSError * error = NULL;
ONCertificatePinningConfiguration * pinningConf = [[ONCertificatePinningConfiguration alloc] initWithHashes: @[@"<EXAMPLE_HASH>"] error: &error]];
if (error != NULL) {
    // handle ONCertificatePinningConfiguration initialisation failures. i.e Providing empty array causes initializer to be failed.
}
[builder withCertificatePinningConfiguration: pinningConf];

...

Handling certificate pinning error

To identify a certificate pinning error, check the message property of the OnfidoFlowError.exception object. It will return invalid_certificate for certificate pinning related errors.

let responseHandler: (OnfidoResponse) -> Void = { response in
  switch response {
    case let .error(error):
        // Some error happened
        if case OnfidoFlowError.exception(withError: _, withMessage: let optionalMessage) = error, let message = optionalMessage {
            if message == "invalid_certificate" {
                // HANDLE INVALID CERTIFICATE CASE HERE
            }
        }
    case let .success(results):
        // User completed the flow
        // You can create your check here
    case .cancel:
        // Flow cancelled by the user
  }
}

Accessibility

The Onfido SDKs are WCAG 2.1 compliant to level AA. They have been optimized to provide the following accessibility support by default:

Screen reader support: accessible labels for textual and non-textual elements available to aid VoiceOver navigation, including dynamic alerts
Dynamic font size support: all elements scale automatically according to the device's font size setting
Sufficient color contrast: default colors have been tested to meet the recommended level of contrast
Sufficient touch target size: all interactive elements have been designed to meet the recommended touch target size

Refer to our accessibility statement for more details.

Licensing

Due to API design constraints, and to avoid possible conflicts during the integration, we bundle some of our 3rd party dependencies. For those, we include the licensing information inside our bundle and also in this repo under licenses folder, with the file named onfido_licenses.json. This file contains a summary of our bundled dependencies and all the licensing information required, including links to the relevant license texts contained in the same folder. Integrators of our library are then responsible for keeping this information along with their integrations.

Example on how to access the licenses:

let onfidoBundle = Bundle(for: OnfidoFlow.self)
guard let licensesPath = onfidoBundle.path(forResource: "onfido_licenses", ofType: "json", inDirectory: nil),
      let licensesData = try? Data(contentsOf: URL(fileURLWithPath: licensesPath)),
      let licensesContent = String(data: licensesData, encoding: .utf8)
else {
    return
}

print(licensesContent)

guard let mitLicensePath = onfidoBundle.path(forResource: "onfido_licenses_mit", ofType: "txt", inDirectory: nil),
      let mitLicenseData = try? Data(contentsOf: URL(fileURLWithPath: mitLicensePath)),
      let mitLicenseFileContents = String(data: mitLicenseData, encoding: .utf8)
else {
    return
}

print(mitLicenseFileContents)

More Information

Sample App

We have included sample apps to show how to integrate with the Onfido iOS SDK using both Swift and Objective-C. See the SampleApp and SampleAppObjC directories for more information.

Support

Should you encounter any technical issues during integration, please contact Onfido’s Customer Support team via email, including the word ISSUE: at the start of the subject line.

Alternatively, you can search the support documentation available via the customer experience portal, public.support.onfido.com.

We recommend you update your SDK to the latest version release as frequently as possible. Customers on newer versions of the Onfido SDK consistently see better performance across user onboarding and fraud mitigation, so we strongly advise keeping your SDK integration up-to-date.

You can review our full SDK versioning policy here.

Kappa-Dev/KappaTools

2024-12-07T01:33:34Z

Tool suite for kappa models. Documentation and binaries can be found in the release section. Try it online at

nextcloud/richdocuments

2024-12-08T01:34:08Z

📑 Collabora Online for Nextcloud

Nextcloud Office

A Nextcloud app integrating Collabora Online into your Nextcloud!

Nextcloud Office supports editing your documents in real time with multiple other editors, showing high fidelity, WYSIWYG rendering and preserving the layout and formatting of your documents.

Users can insert and reply to comments and invite others without a Nextcloud account for anonymous editing of files with a public link shared folder.

Nextcloud Office supports dozens of document formats including DOC, DOCX, PPT, PPTX, XLS, XLSX + ODF, Import/View Visio, Publisher and many more…

Nextcloud Office is based on the Collabora Online Development Edition (CODE) and is available free and under heavy development, adding features and improvements all the time! Enterprise users have access to the more stable, scalable Collabora Online Enterprise based version through a Nextcloud support subscription.

Installation

Nextcloud Office is built on Collabora Online which requires a dedicated service running next to the Nextcloud webserver stack. There are several ways to run the coolwsd service. For full details, see the related section in the admin manual https://docs.nextcloud.com/server/latest/admin_manual/office/index.html

This repository covers only the Nextcloud integration app which requires a Collabora Online server to connect to.

Note: it is possible to use Collabora Online’s integration with re-compiled and/or re-branded backends. This app may work with other WOPI Edtiors such as LibreOffice Online but it is not tested.

Federated editing / Global Scale

Collaborative editing of federated documents requires richdocuments version 3.4 on all involved servers. Besides that the following conditions must be met:

Make sure the remote server is added as a trusted server
Allow any domain to embed Collabora Online in a frame: <frame_ancestors>https://*</frame_ancestors> must be set in coolwsd.xml

Note: Due to our content security policy we cannot open a document on a remote instance without reloading the page to allow Nextcloud embedding the remote Collabora Online instance in a frame.

Development setup

Just clone this repo into your apps directory (Nextcloud server installation needed). Additionally, npm and Node.js are needed for installing JavaScript dependencies and building the frontend code.

Once npm and Node.js are installed, this can be done by running:

npm ci
npm run dev

Support

Our Forum

Enterprise users who need a more reliable and scalable solution can take advantage of Nextcloud GmbH's optional support contract for Collabora Online. Find out more about Enterprise support for Collabora Online over here: https://nextcloud.com/enterprise/

Request a quote

SciPhi-AI/R2R

2024-12-07T01:34:21Z

A simple, easy-to-hack GraphRAG implementation

A simple, easy-to-hack GraphRAG implementation

😭 GraphRAG is good and powerful, but the official implementation is difficult/painful to read or hack.

😊 This project provides a smaller, faster, cleaner GraphRAG, while remaining the core functionality(see benchmark and issues ).

🎁 Excluding tests and prompts, nano-graphrag is about 1100 lines of code.

👌 Small yet portable(faiss, neo4j, ollama...), asynchronous and fully typed.

Install

Install from source (recommend)

# clone this repo first
cd nano-graphrag
pip install -e .

Install from PyPi

pip install nano-graphrag

Quick Start

[!TIP]

Please set OpenAI API key in environment: export OPENAI_API_KEY="sk-...".

[!TIP] If you're using Azure OpenAI API, refer to the .env.example to set your azure openai. Then pass GraphRAG(...,using_azure_openai=True,...) to enable.

[!TIP] If you're using Amazon Bedrock API, please ensure your credentials are properly set through commands like aws configure. Then enable it by configuring like this: GraphRAG(...,using_amazon_bedrock=True, best_model_id="us.anthropic.claude-3-sonnet-20240229-v1:0", cheap_model_id="us.anthropic.claude-3-haiku-20240307-v1:0",...). Refer to an example script.

[!TIP]

If you don't have any key, check out this example that using transformers and ollama . If you like to use another LLM or Embedding Model, check Advances.

download a copy of A Christmas Carol by Charles Dickens:

curl https://raw.githubusercontent.com/gusye1234/nano-graphrag/main/tests/mock_data.txt > ./book.txt

Use the below python snippet:

from nano_graphrag import GraphRAG, QueryParam

graph_func = GraphRAG(working_dir="./dickens")

with open("./book.txt") as f:
    graph_func.insert(f.read())

# Perform global graphrag search
print(graph_func.query("What are the top themes in this story?"))

# Perform local graphrag search (I think is better and more scalable one)
print(graph_func.query("What are the top themes in this story?", param=QueryParam(mode="local")))

Next time you initialize a GraphRAG from the same working_dir, it will reload all the contexts automatically.

Batch Insert

graph_func.insert(["TEXT1", "TEXT2",...])

Incremental Insert

nano-graphrag supports incremental insert, no duplicated computation or data will be added:

with open("./book.txt") as f:
    book = f.read()
    half_len = len(book) // 2
    graph_func.insert(book[:half_len])
    graph_func.insert(book[half_len:])

nano-graphrag use md5-hash of the content as the key, so there is no duplicated chunk.

However, each time you insert, the communities of graph will be re-computed and the community reports will be re-generated

Naive RAG

nano-graphrag supports naive RAG insert and query as well:

graph_func = GraphRAG(working_dir="./dickens", enable_naive_rag=True)
...
# Query
print(rag.query(
      "What are the top themes in this story?",
      param=QueryParam(mode="naive")
)

Async

For each method NAME(...) , there is a corresponding async method aNAME(...)

await graph_func.ainsert(...)
await graph_func.aquery(...)
...

Available Parameters

GraphRAG and QueryParam are dataclass in Python. Use help(GraphRAG) and help(QueryParam) to see all available parameters! Or check out the Advances section to see some options.

Components

Below are the components you can use:

Type	What	Where
LLM	OpenAI	Built-in
	Amazon Bedrock	Built-in
	DeepSeek	examples
	`ollama`	examples
Embedding	OpenAI	Built-in
	Amazon Bedrock	Built-in
	Sentence-transformers	examples
Vector DataBase	`nano-vectordb`	Built-in
	`hnswlib`	Built-in, examples
	`milvus-lite`	examples
	faiss	examples
Graph Storage	`networkx`	Built-in
	`neo4j`	Built-in(doc)
Visualization	graphml	examples
Chunking	by token size	Built-in
	by text splitter	Built-in

Built-in means we have that implementation inside nano-graphrag. examples means we have that implementation inside an tutorial under examples folder.
Check examples/benchmarks to see few comparisons between components.
Always welcome to contribute more components.

Advances

Some setup options

GraphRAG(...,always_create_working_dir=False,...) will skip the dir-creating step. Use it if you switch all your components to non-file storages.

Only query the related context

graph_func.query return the final answer without streaming.

If you like to interagte nano-graphrag in your project, you can use param=QueryParam(..., only_need_context=True,...), which will only return the retrieved context from graph, something like:

# Local mode
-----Reports-----
```csv
id,	content
0,	# FOX News and Key Figures in Media and Politics...
1, ...
```
...

# Global mode
----Analyst 3----
Importance Score: 100
Donald J. Trump: Frequently discussed in relation to his political activities...
...

You can integrate that context into your customized prompt.

Prompt

nano-graphrag use prompts from nano_graphrag.prompt.PROMPTS dict object. You can play with it and replace any prompt inside.

Some important prompts:

PROMPTS["entity_extraction"] is used to extract the entities and relations from a text chunk.
PROMPTS["community_report"] is used to organize and summary the graph cluster's description.
PROMPTS["local_rag_response"] is the system prompt template of the local search generation.
PROMPTS["global_reduce_rag_response"] is the system prompt template of the global search generation.
PROMPTS["fail_response"] is the fallback response when nothing is related to the user query.

Customize Chunking

nano-graphrag allow you to customize your own chunking method, check out the example.

Switch to the built-in text splitter chunking method:

from nano_graphrag._op import chunking_by_seperators

GraphRAG(...,chunk_func=chunking_by_seperators,...)

LLM Function

In nano-graphrag, we requires two types of LLM, a great one and a cheap one. The former is used to plan and respond, the latter is used to summary. By default, the great one is gpt-4o and the cheap one is gpt-4o-mini

You can implement your own LLM function (refer to _llm.gpt_4o_complete):

async def my_llm_complete(
    prompt, system_prompt=None, history_messages=[], **kwargs
) -> str:
  # pop cache KV database if any
  hashing_kv: BaseKVStorage = kwargs.pop("hashing_kv", None)
  # the rest kwargs are for calling LLM, for example, `max_tokens=xxx`
	...
  # YOUR LLM calling
  response = await call_your_LLM(messages, **kwargs)
  return response

Replace the default one with:

# Adjust the max token size or the max async requests if needed
GraphRAG(best_model_func=my_llm_complete, best_model_max_token_size=..., best_model_max_async=...)
GraphRAG(cheap_model_func=my_llm_complete, cheap_model_max_token_size=..., cheap_model_max_async=...)

You can refer to this example that use deepseek-chat as the LLM model

You can refer to this example that use ollama as the LLM model

Json Output

nano-graphrag will use best_model_func to output JSON with params "response_format": {"type": "json_object"}. However there are some open-source model maybe produce unstable JSON.

nano-graphrag introduces a post-process interface for you to convert the response to JSON. This func's signature is below:

def YOUR_STRING_TO_JSON_FUNC(response: str) -> dict:
  "Convert the string response to JSON"
  ...

And pass your own func by GraphRAG(...convert_response_to_json_func=YOUR_STRING_TO_JSON_FUNC,...).

For example, you can refer to json_repair to repair the JSON string returned by LLM.

Embedding Function

You can replace the default embedding functions with any _utils.EmbedddingFunc instance.

For example, the default one is using OpenAI embedding API:

@wrap_embedding_func_with_attrs(embedding_dim=1536, max_token_size=8192)
async def openai_embedding(texts: list[str]) -> np.ndarray:
    openai_async_client = AsyncOpenAI()
    response = await openai_async_client.embeddings.create(
        model="text-embedding-3-small", input=texts, encoding_format="float"
    )
    return np.array([dp.embedding for dp in response.data])

Replace default embedding function with:

GraphRAG(embedding_func=your_embed_func, embedding_batch_num=..., embedding_func_max_async=...)

You can refer to an example that use sentence-transformer to locally compute embeddings.

Storage Component

You can replace all storage-related components to your own implementation, nano-graphrag mainly uses three kinds of storage:

base.BaseKVStorage for storing key-json pairs of data

By default we use disk file storage as the backend.
GraphRAG(.., key_string_value_json_storage_cls=YOURS,...)

base.BaseVectorStorage for indexing embeddings

By default we use nano-vectordb as the backend.
We have a built-in hnswlib storage also, check out this example.
Check out this example that implements milvus-lite as the backend (not available in Windows).
GraphRAG(.., vector_db_storage_cls=YOURS,...)

base.BaseGraphStorage for storing knowledge graph

By default we use networkx as the backend.
We have a built-in Neo4jStorage for graph, check out this tutorial.
GraphRAG(.., graph_storage_cls=YOURS,...)

You can refer to nano_graphrag.base to see detailed interfaces for each components.

FQA

Check FQA.

Roadmap

See ROADMAP.md

Contribute

nano-graphrag is open to any kind of contribution. Read this before you contribute.

Benchmark

benchmark for English
benchmark for Chinese
An evaluation notebook on a multi-hop RAG task

Projects that used `nano-graphrag`

Medical Graph RAG: Graph RAG for the Medical Data
LightRAG: Simple and Fast Retrieval-Augmented Generation
fast-graphrag: RAG that intelligently adapts to your use case, data, and queries

Welcome to pull requests if your project uses nano-graphrag, it will help others to trust this repo❤️

Issues

nano-graphrag didn't implement the covariates feature of GraphRAG
nano-graphrag implements the global search different from the original. The original use a map-reduce-like style to fill all the communities into context, while nano-graphrag only use the top-K important and central communites (use QueryParam.global_max_consider_community to control, default to 512 communities).

qodo-ai/qodo-cover

2024-12-07T01:34:21Z

Installation

To get started with ESM, install the library using pip:

pip install esm

ESM C

ESM Cambrian is a parallel model family to our flagship ESM3 generative models. While ESM3 focuses on controllable generation of proteins for therapeutic and many other applications, ESM C focuses on creating representations of the underlying biology of proteins.

ESM C comes with major performance benefits over ESM2. The 300M parameter ESM C delivers similar performance to ESM2 650M with dramatically reduced memory requirements and faster inference. The 600M parameter ESM C rivals the 3B parameter ESM2 and approaches the capabilities of the 15B model, delivering frontier performance with far greater efficiency. The 6B parameter ESM C sets a new benchmark, outperforming the best ESM2 models by a wide margin.

ESM C models are available immediately for academic and commercial use under a new license structure designed to promote openness and enable scientists and builders. You can find the high level take-away of the license structure in the Licenses section of this page, and the full license structure in the LICENSE.md file.

You can use the following guides to start using ESM C models today, either running the model locally, the Forge API and AWS SageMaker.

Using ESM C 300M and 600M via GitHub

ESM C model weights are stored on the HuggingFace hub under https://huggingface.co/EvolutionaryScale/.

from esm.models.esmc import ESMC
from esm.sdk.api import ESMProtein, LogitsConfig

protein = ESMProtein(sequence="AAAAA")
client = ESMC.from_pretrained("esmc_300m").to("cuda") # or "cpu"
protein_tensor = client.encode(protein)
logits_output = client.logits(
   protein_tensor, LogitsConfig(sequence=True, return_embeddings=True)
)
print(logits_output.logits, logits_output.embeddings)

Using ESM C 6B via Forge API

ESM C models, including ESMC 6B, are accessible via EvolutionaryScale Forge. You can request access and utilize these models through forge.evolutionaryscale.ai, as demonstrated in the example below.

from esm.sdk.forge import ESM3ForgeInferenceClient
from esm.sdk.api import ESMProtein, LogitsConfig

# Apply for forge access and get an access token
forge_client = ESM3ForgeInferenceClient(model="esmc-6b-2024-12", url="https://forge.evolutionaryscale.ai", token="<your forge token>")
protein_tensor = forge_client.encode(protein)
logits_output = forge_client.logits(
   protein_tensor, LogitsConfig(sequence=True, return_embeddings=True)
)
print(logits_output.logits, logits_output.embeddings)

Using ESM C 6B via SageMaker

ESM C models are also available on Amazon SageMaker. They function similarly to the ESM3 model family, and you can refer to the sample notebooks provided in this repository for examples.

You'll need an admin AWS access to an AWS account to follow these instructions. To deploy, first we need to deploy the AWS package:

Find the ESM C model version you want to subscribe to. All of our offerings are visible here.
Click the name of the model version you are interested in, review pricing information and the end user license agreement (EULA), then click "Continue to Subscribe".
Once you have subscribed, you should be able to see our model under your marketplace subscriptions.
Click the product name and then from the "Actions" dropdown select "Configure".
You will next see the "Configure and Launch" UI. There are multiple deployment paths - we recommend using "AWS CloudFormation".
The default value for "Service Access" may or may not work. We recommend clicking "Create and use a new service role".
Click "Launch CloudFormation Template". This takes 15 to 25 minutes depending on model size.
On the "Quick create stack" page, ensure the stack name and endpoint names are not already used. You can check existing stack names here and existing endpoint names here.

The Sagemaker deployment of the model now lives on a dedicated GPU instance inside your AWS environment, and will be billed directly to your AWS account. Make sure to remember to shut down the instance after you stop using it. Find the CloudFormation stack you created here, select it, and then click "Delete" to clean up all resources.

After creating the endpoint, you can create a sagemaker client and use it the same way as a forge client. They share the same API.

Ensure that the code below runs in an environment that has AWS credentials available for the account which provisioned SageMaker resources. Learn more about general AWS credential options here.

from esm.sdk.sagemaker import ESM3SageMakerClient
from esm.sdk.api import ESMProtein, LogitsConfig

sagemaker_client = ESM3SageMakerClient(
   # E.g. "Endpoint-ESMC-6B-1"
   endpoint_name=SAGE_ENDPOINT_NAME,
   # E.g. "esmc-6b-2024-12". Same model names as in Forge.
   model=MODEL_NAME,
)

protein = ESMProtein(sequence="AAAAA")
protein_tensor = sagemaker_client.encode(protein)
logits_output = sagemaker_client.logits(
   protein_tensor, LogitsConfig(sequence=True, return_embeddings=True)
)
print(logits_output.logits, logits_output.embeddings)

ESM 3

ESM3 is a frontier generative model for biology, able to jointly reason across three fundamental biological properties of proteins: sequence, structure, and function. These three data modalities are represented as tracks of discrete tokens at the input and output of ESM3. You can present the model with a combination of partial inputs across the tracks, and ESM3 will provide output predictions for all the tracks.

ESM3 is a generative masked language model. You can prompt it with partial sequence, structure, and function keywords, and iteratively sample masked positions until all positions are unmasked. This iterative sampling is what the .generate() function does.

The ESM3 architecture is highly scalable due to its transformer backbone and all-to-all reasoning over discrete token sequences. At its largest scale, ESM3 was trained with 1.07e24 FLOPs on 2.78 billion proteins and 771 billion unique tokens, and has 98 billion parameters. Learn more by reading the blog post and the pre-print (Hayes et al., 2024).

Here we present esm3-open-small. With 1.4B parameters it is the smallest and fastest model in the family. ESM3-open is available under the Cambrian non-commercial license agreement, as outlined in LICENSE.md (note: updated with ESM C release). Visit our Discussions page to get in touch, provide feedback, ask questions or share your experience with ESM3!

Quickstart for ESM3-open

pip install esm

In order to download the weights, we require users to accept our non-commercial license. The weights are stored on HuggingFace Hub under HuggingFace/EvolutionaryScale/esm3. Please create an account and accept the license.

from huggingface_hub import login
from esm.models.esm3 import ESM3
from esm.sdk.api import ESM3InferenceClient, ESMProtein, GenerationConfig

# Will instruct you how to get an API key from huggingface hub, make one with "Read" permission.
login()

# This will download the model weights and instantiate the model on your machine.
model: ESM3InferenceClient = ESM3.from_pretrained("esm3-open").to("cuda") # or "cpu"

# Generate a completion for a partial Carbonic Anhydrase (2vvb)
prompt = "___________________________________________________DQATSLRILNNGHAFNVEFDDSQDKAVLKGGPLDGTYRLIQFHFHWGSLDGQGSEHTVDKKKYAAELHLVHWNTKYGDFGKAVQQPDGLAVLGIFLKVGSAKPGLQKVVDVLDSIKTKGKSADFTNFDPRGLLPESLDYWTYPGSLTTPP___________________________________________________________"
protein = ESMProtein(sequence=prompt)
# Generate the sequence, then the structure. This will iteratively unmask the sequence track.
protein = model.generate(protein, GenerationConfig(track="sequence", num_steps=8, temperature=0.7))
# We can show the predicted structure for the generated sequence.
protein = model.generate(protein, GenerationConfig(track="structure", num_steps=8))
protein.to_pdb("./generation.pdb")
# Then we can do a round trip design by inverse folding the sequence and recomputing the structure
protein.sequence = None
protein = model.generate(protein, GenerationConfig(track="sequence", num_steps=8))
protein.coordinates = None
protein = model.generate(protein, GenerationConfig(track="structure", num_steps=8))
protein.to_pdb("./round_tripped.pdb")

Congratulations, you just generated your first proteins with ESM3! Let's explore some more advanced prompting with the help of our notebooks and scripts.

generate.ipynb will walk through two prompting examples (scaffolding and secondary structure editing) using the open model:

gfp_design.ipynb will walk through the more complex generation procedure we used to design esmGFP:

We also provide example scripts that show common workflows under examples/:

local_generate.py shows how simple and elegant common tasks are: it shows folding, inverse folding and chain of thought generation, all by calling just model.generate() for iterative decoding.
seqfun_struct.py shows direct use of the model as a standard pytorch model with a simple model forward call.

Forge: Access to larger ESM3 models

You can apply for beta access to the full family of larger and higher capability ESM3 models at EvolutionaryScale Forge.

We encourage users to interact with the Forge API through the python esm library instead of the command line. The python interface enables you to interactively load proteins, build prompts, and inspect generated proteins with the ESMProtein and config classes used to interact with the local model.

In any example script try to replace a local ESM3 model with a Forge API client:

# Instead of loading the model locally on your machine:
model: ESM3InferenceClient = ESM3.from_pretrained("esm3_sm_open_v1").to("cuda") # or "cpu"
# just replace the line with this:
model: ESM3InferenceClient = esm.sdk.client("esm3-medium-2024-08", token="<your forge token>")
# and now you're interfacing with the model running on our remote servers.
...

and the exact same code will work. This enables a seamless transition from smaller and faster models, to our large 98B protein language models for protein design work.

Responsible Development

EvolutionaryScale is a public benefit company. Our mission is to develop artificial intelligence to understand biology for the benefit of human health and society, through partnership with the scientific community, and open, safe, and responsible research. Inspired by the history of our field as well as new principles and recommendations, we have created a Responsible Development Framework to guide our work towards our mission with transparency and clarity.

The core tenets of our framework are

We will communicate the benefits and risks of our research
We will proactively and rigorously evaluate the risk of our models before public deployment
We will adopt risk mitigation strategies and precautionary guardrails
We will work with stakeholders in government, policy, and civil society to keep them informed

With this in mind, we have performed a variety of mitigations for esm3-sm-open-v1, detailed in our paper

Licenses

The code and model weights of ESM3 and ESM C are available under a mixture of non-commercial and more permissive licenses, fully outlined in LICENSE.md.

FoundationVision/VAR

2024-12-08T01:44:48Z

[NeurIPS 2024 Oral][GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

VAR: a new visual generation method elevates GPT-style models beyond diffusion🚀 & Scaling laws observed📈

Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction

NeurIPS 2024 Oral

News

2024-12: We Release our Text-to-Image research based on VAR, please check Infinity.
2024-09: VAR is accepted as NeurIPS 2024 Oral Presentation.
2024-04: Visual AutoRegressive modeling is released.

🕹️ Try and Play with VAR!

We provide a demo website for you to play with VAR models and generate images interactively. Enjoy the fun of visual autoregressive modeling!

We also provide demo_sample.ipynb for you to see more technical details about VAR.

What's New?

🔥 Introducing VAR: a new paradigm in autoregressive visual generation✨:

Visual Autoregressive Modeling (VAR) redefines the autoregressive learning on images as coarse-to-fine "next-scale prediction" or "next-resolution prediction", diverging from the standard raster-scan "next-token prediction".

🔥 For the first time, GPT-style autoregressive models surpass diffusion models🚀:

🔥 Discovering power-law Scaling Laws in VAR transformers📈:

🔥 Zero-shot generalizability🛠️:

For a deep dive into our analyses, discussions, and evaluations, check out our paper.

VAR zoo

We provide VAR models for you to play with, which are on or can be downloaded from the following links:

model	reso.	FID	rel. cost	#params	HF weights🤗
VAR-d16	256	3.55	0.4	310M	var_d16.pth
VAR-d20	256	2.95	0.5	600M	var_d20.pth
VAR-d24	256	2.33	0.6	1.0B	var_d24.pth
VAR-d30	256	1.97	1	2.0B	var_d30.pth
VAR-d30-re	256	1.80	1	2.0B	var_d30.pth

You can load these models to generate images via the codes in demo_sample.ipynb. Note: you need to download vae_ch160v4096z32.pth first.

Installation

Install torch>=2.0.0.
Install other pip packages via pip3 install -r requirements.txt.

Prepare the ImageNet dataset

assume the ImageNet is in `/path/to/imagenet`. It should be like this:

/path/to/imagenet/:
    train/:
        n01440764: 
            many_images.JPEG ...
        n01443537:
            many_images.JPEG ...
    val/:
        n01440764:
            ILSVRC2012_val_00000293.JPEG ...
        n01443537:
            ILSVRC2012_val_00000236.JPEG ...

NOTE: The arg --data_path=/path/to/imagenet should be passed to the training script.

(Optional) install and compile flash-attn and xformers for faster attention computation. Our code will automatically use them if installed. See models/basic_var.py#L15-L30.

Training Scripts

To train VAR-{d16, d20, d24, d30, d36-s} on ImageNet 256x256 or 512x512, you can run the following command:

# d16, 256x256
torchrun --nproc_per_node=8 --nnodes=... --node_rank=... --master_addr=... --master_port=... train.py \
  --depth=16 --bs=768 --ep=200 --fp16=1 --alng=1e-3 --wpe=0.1
# d20, 256x256
torchrun --nproc_per_node=8 --nnodes=... --node_rank=... --master_addr=... --master_port=... train.py \
  --depth=20 --bs=768 --ep=250 --fp16=1 --alng=1e-3 --wpe=0.1
# d24, 256x256
torchrun --nproc_per_node=8 --nnodes=... --node_rank=... --master_addr=... --master_port=... train.py \
  --depth=24 --bs=768 --ep=350 --tblr=8e-5 --fp16=1 --alng=1e-4 --wpe=0.01
# d30, 256x256
torchrun --nproc_per_node=8 --nnodes=... --node_rank=... --master_addr=... --master_port=... train.py \
  --depth=30 --bs=1024 --ep=350 --tblr=8e-5 --fp16=1 --alng=1e-5 --wpe=0.01 --twde=0.08
# d36-s, 512x512 (-s means saln=1, shared AdaLN)
torchrun --nproc_per_node=8 --nnodes=... --node_rank=... --master_addr=... --master_port=... train.py \
  --depth=36 --saln=1 --pn=512 --bs=768 --ep=350 --tblr=8e-5 --fp16=1 --alng=5e-6 --wpe=0.01 --twde=0.08

A folder named local_output will be created to save the checkpoints and logs. You can monitor the training process by checking the logs in local_output/log.txt and local_output/stdout.txt, or using tensorboard --logdir=local_output/.

If your experiment is interrupted, just rerun the command, and the training will automatically resume from the last checkpoint in local_output/ckpt*.pth (see utils/misc.py#L344-L357).

Sampling & Zero-shot Inference

For FID evaluation, use var.autoregressive_infer_cfg(..., cfg=1.5, top_p=0.96, top_k=900, more_smooth=False) to sample 50,000 images (50 per class) and save them as PNG (not JPEG) files in a folder. Pack them into a .npz file via create_npz_from_sample_folder(sample_folder) in utils/misc.py#L344. Then use the OpenAI's FID evaluation toolkit and reference ground truth npz file of 256x256 or 512x512 to evaluate FID, IS, precision, and recall.

Note a relatively small cfg=1.5 is used for trade-off between image quality and diversity. You can adjust it to cfg=5.0, or sample with autoregressive_infer_cfg(..., more_smooth=True) for better visual quality. We'll provide the sampling script later.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Citation

If our work assists your research, feel free to give us a star ⭐ or cite us using:

@Article{VAR,
      title={Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction}, 
      author={Keyu Tian and Yi Jiang and Zehuan Yuan and Bingyue Peng and Liwei Wang},
      year={2024},
      eprint={2404.02905},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

stanfordnlp/dspy

2024-12-08T01:44:48Z

DSPy: The framework for programming—not prompting—language models

DSPy: Programming—not prompting—Foundation Models

Documentation: DSPy Docs

DSPy is the open-source framework for programming—rather than prompting—language models. It allows you to iterate fast on building modular AI systems and provides algorithms for optimizing their prompts and weights, whether you're building simple classifiers, sophisticated RAG pipelines, or Agent loops.

DSPy stands for Declarative Self-improving Python. Instead of brittle prompts, you write compositional Python code and use DSPy's tools to teach your LM to deliver high-quality outputs. This lecture is a good conceptual introduction. Meet the community, seek help, or start contributing via our GitHub repo here and our Discord server.

Documentation: dspy.ai

Please go to the DSPy Docs at dspy.ai

Installation

pip install dspy

To install the very latest from main:

pip install git+https://github.com/stanfordnlp/dspy.git

📜 Citation & Reading More

[Jun'24] Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs
[Oct'23] DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
[Jul'24] Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together
[Jun'24] Prompts as Auto-Optimized Training Hyperparameters
[Feb'24] Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models
[Jan'24] In-Context Learning for Extreme Multi-Label Classification
[Dec'23] DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines
[Dec'22] Demonstrate-Search-Predict: Composing Retrieval & Language Models for Knowledge-Intensive NLP

To stay up to date or learn more, follow @lateinteraction on Twitter.

The DSPy logo is designed by Chuyi Zhang.

If you use DSPy or DSP in a research paper, please cite our work as follows:

@inproceedings{khattab2024dspy,
  title={DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines},
  author={Khattab, Omar and Singhvi, Arnav and Maheshwari, Paridhi and Zhang, Zhiyuan and Santhanam, Keshav and Vardhamanan, Sri and Haq, Saiful and Sharma, Ashutosh and Joshi, Thomas T. and Moazam, Hanna and Miller, Heather and Zaharia, Matei and Potts, Christopher},
  journal={The Twelfth International Conference on Learning Representations},
  year={2024}
}
@article{khattab2022demonstrate,
  title={Demonstrate-Search-Predict: Composing Retrieval and Language Models for Knowledge-Intensive {NLP}},
  author={Khattab, Omar and Santhanam, Keshav and Li, Xiang Lisa and Hall, David and Liang, Percy and Potts, Christopher and Zaharia, Matei},
  journal={arXiv preprint arXiv:2212.14024},
  year={2022}
}

Guovin/iptv-api

2024-12-08T01:44:48Z

IPTV-API

自定义频道，自动获取直播源接口，测速验效后生成可用的结果

具体频道

📺央视频道: CCTV-1, CCTV-2, CCTV-3, CCTV-4, CCTV-5, CCTV-5+, CCTV-6, CCTV-7, CCTV-8, CCTV-9, CCTV-10, CCTV-11, CCTV-12, CCTV-13, CCTV-14, CCTV-15, CCTV-16, CCTV-17, CETV1, CETV2, CETV4, CETV5

💰央视付费频道: 文化精品, 央视台球, 风云音乐, 第一剧场, 风云剧场, 怀旧剧场, 女性时尚, 高尔夫网球, 风云足球, 电视指南, 世界地理, 兵器科技

☘️各省份地方台

🎥咪咕直播: 咪咕直播1-45

🎮游戏频道: 游戏风云, 游戏竞技, 电竞游戏, 海看电竞, 电竞天堂, 爱电竞

🎵音乐频道: CCTV-15, 风云音乐, 音乐现场, 音乐之声, 潮流音乐, 天津音乐, 音乐广播, 音乐调频广播

English | 中文

特点

✅ 自定义模板，生成您想要的频道
✅ 支持多种获取源方式：组播源、酒店源、订阅源、关键字搜索
✅ 接口测速验效，响应时间、分辨率优先级，过滤无效接口
✅ 偏好设置：IPv6、接口来源排序优先级与数量配置、接口白名单
✅ 定时执行，北京时间每日 6:00 与 18:00 执行更新
✅ 支持多种运行方式：工作流、命令行、GUI 软件、Docker(amd64/arm64/arm v7)
✨ 更多功能请见配置参数

配置

配置参数

快速上手

方式一：工作流

Fork 本项目并开启工作流更新，具体步骤请见详细教程

方式二：命令行

pip install pipenv

pipenv install --dev

启动更新：

pipenv run dev

启动服务：

pipenv run service

方式三：GUI 软件

下载IPTV-API 更新软件，打开软件，点击更新，即可完成更新
或者在项目目录下运行以下命令，即可打开 GUI 软件：

pipenv run ui

方式四：Docker

iptv-api（完整版本）：性能要求较高，更新速度较慢，稳定性、成功率高；修改配置 open_driver = False 可切换到 Lite 版本运行模式（推荐酒店源、组播源、关键字搜索使用此版本）
iptv-api:lite（精简版本）：轻量级，性能要求低，更新速度快，稳定性不确定（推荐订阅源使用此版本）

拉取镜像：

iptv-api：

docker pull guovern/iptv-api:latest

iptv-api:lite：

docker pull guovern/iptv-api:lite

运行容器：

iptv-api：

docker run -d -p 8000:8000 guovern/iptv-api

iptv-api:lite：

docker run -d -p 8000:8000 guovern/iptv-api:lite

卷挂载参数（可选）：实现宿主机文件与容器文件同步，修改模板、配置、获取更新结果文件可直接在宿主机文件夹下操作

以宿主机路径/etc/docker 为例：

iptv-api：

docker run -v /etc/docker/config:/iptv-api/config -v /etc/docker/output:/iptv-api/output -d -p 8000:8000 guovern/iptv-api

iptv-api:lite：

docker run -v /etc/docker/config:/iptv-api-lite/config -v /etc/docker/output:/iptv-api-lite/output -d -p 8000:8000 guovern/iptv-api:lite

更新结果：

接口地址：ip:8000
M3u 接口：ip:8000/m3u
Txt 接口：ip:8000/txt
接口内容：ip:8000/content
测速日志：ip:8000/log

更新日志

赞赏

开发维护不易，请我喝杯咖啡☕️吧~

支付宝	微信

关注

微信公众号搜索 Govin，或扫码，接收更新推送、学习更多使用技巧：

免责声明

本项目仅供学习交流用途，接口数据均来源于网络，如有侵权，请联系删除

许可证

mTvare6/hello-world.rs

2024-12-07T01:34:39Z

🚀Memory safe, blazing fast, configurable, minimal hello world written in rust(🚀) in a few lines of code with few(1092🚀) dependencies🚀

mTvare6/hello-world.rs

2024-12-08T01:45:03Z

🚀Memory safe, blazing fast, configurable, minimal hello world written in rust(🚀) in a few lines of code with few(1092🚀) dependencies🚀

rescript-lang/rescript

2024-12-08T01:45:04Z

ReScript is a robustly typed language that compiles to efficient and human-readable JavaScript.

slimm609/checksec

2024-12-08T01:45:09Z

Checksec

samizdatco/skia-canvas

2024-12-07T01:34:56Z

A GPU-accelerated 2D graphics environment for Node.js

Getting Started · Documentation · Release Notes · Discussion Forum

Skia Canvas is a browser-less implementation of the HTML Canvas drawing API for Node.js. It is based on Google’s Skia graphics engine and, accordingly, produces very similar results to Chrome’s <canvas> element. The library is well suited for use on desktop machines where you can render hardware-accelerated graphics to a window and on the server where it can output a variety of image formats.

While the primary goal of this project is to provide a reliable emulation of the standard API according to the spec, it also extends it in a number of areas to take greater advantage of Skia's advanced graphical features and provide a more expressive coding environment.

In particular, Skia Canvas:

is fast and compact since rendering takes place on the GPU and all the heavy lifting is done by native code written in Rust and C++
can render to windows using an OS-native graphics pipeline and provides a browser-like UI event framework
generates images in both raster (JPEG, PNG, & WEBP) and vector (PDF & SVG) formats
can save images to files, return them as Buffers, or encode dataURL strings
uses native threads in a user-configurable worker pool for asynchronous rendering and file I/O
can create multiple ‘pages’ on a given canvas and then output them as a single, multi-page PDF or an image-sequence saved to multiple files
can simplify, blunt, combine, excerpt, and atomize bézier paths using efficient boolean operations or point-by-point interpolation
provides 3D perspective transformations in addition to scaling, rotation, and translation
can fill shapes with vector-based Textures in addition to bitmap-based Patterns and supports line-drawing with custom markers
supports the full set of CSS filter image processing operators
offers rich typographic control including:
- multi-line, word-wrapped text
- line-by-line text metrics
- small-caps, ligatures, and other opentype features accessible using standard font-variant syntax
- proportional letter-spacing, word-spacing, and leading
- support for variable fonts and transparent mapping of weight values
- use of non-system fonts loaded from local files

Installation

If you’re running on a supported platform, installation should be as simple as:

npm install skia-canvas

This will download a pre-compiled library from the project’s most recent release.

Platform Support

The underlying Rust library uses N-API v8 which allows it to run on Node.js versions:

v12.22+
v14.17+
v15.12+
v16.0.0 and later

Pre-compiled binaries are available for:

Linux (x64 & arm64)
macOS (x64 & Apple silicon)
Windows (x64)

Nearly everything you need is statically linked into the library. A notable exception is the Fontconfig library which must be installed separately if you’re running on Linux.

Running in Docker

The library is compatible with Linux systems using glibc 2.28 or later as well as Alpine Linux (x64 & arm64) and the musl C library it favors. In both cases, Fontconfig must be installed on the system for skia-canvas to operate correctly.

If you are setting up a Dockerfile that uses node as its basis, the simplest approach is to set your FROM image to one of the (Debian-derived) defaults like node:lts, node:18, node:16, node:14-buster, node:12-buster, node:bullseye, node:buster, or simply:

FROM node

You can also use the ‘slim’ image if you manually install fontconfig:

FROM node:slim
RUN apt-get update && apt-get install -y -q --no-install-recommends libfontconfig1

If you wish to use Alpine as the underlying distribution, you can start with something along the lines of:

FROM node:alpine
RUN apk update && apk add fontconfig

Compiling from Source

If prebuilt binaries aren’t available for your system you’ll need to compile the portions of this library that directly interface with Skia.

Start by installing:

The Rust compiler and cargo package manager using rustup
A C compiler toolchain (either LLVM/Clang or MSVC)
Python 3 (used by Skia's build process)
The Ninja build system
On Linux: Fontconfig and OpenSSL

Detailed instructions for setting up these dependencies on different operating systems can be found in the ‘Building’ section of the Rust Skia documentation. Once all the necessary compilers and libraries are present, running npm run build will give you a usable library (after a fairly lengthy compilation process).

Multithreading

When rendering canvases in the background (e.g., by using the asynchronous saveAs or toBuffer methods), tasks are spawned in a thread pool managed by the rayon library. By default it will create up to as many threads as your CPU has cores. You can see this default value by inspecting any Canvas object's engine.threads property. If you wish to override this default, you can set the SKIA_CANVAS_THREADS environment variable to your preferred value.

For example, you can limit your asynchronous processing to two simultaneous tasks by running your script with:

SKIA_CANVAS_THREADS=2 node my-canvas-script.js

Example Usage

Generating image files

import {Canvas} from 'skia-canvas'

let canvas = new Canvas(400, 400),
    ctx = canvas.getContext("2d"),
    {width, height} = canvas;

let sweep = ctx.createConicGradient(Math.PI * 1.2, width/2, height/2)
sweep.addColorStop(0, "red")
sweep.addColorStop(0.25, "orange")
sweep.addColorStop(0.5, "yellow")
sweep.addColorStop(0.75, "green")
sweep.addColorStop(1, "red")
ctx.strokeStyle = sweep
ctx.lineWidth = 100
ctx.strokeRect(100,100, 200,200)

// render to multiple destinations using a background thread
async function render(){
  // save a ‘retina’ image...
  await canvas.saveAs("rainbox.png", {density:2})
  // ...or use a shorthand for canvas.toBuffer("png")
  let pngData = await canvas.png
  // ...or embed it in a string
  let pngEmbed = `<img src="${await canvas.toDataURL("png")}">`
}
render()

// ...or save the file synchronously from the main thread
canvas.saveAsSync("rainbox.pdf")

Multi-page sequences

import {Canvas} from 'skia-canvas'

let canvas = new Canvas(400, 400),
    ctx = canvas.getContext("2d"),
    {width, height} = canvas

for (const color of ['orange', 'yellow', 'green', 'skyblue', 'purple']){
  ctx = canvas.newPage()
  ctx.fillStyle = color
  ctx.fillRect(0,0, width, height)
  ctx.fillStyle = 'white'
  ctx.arc(width/2, height/2, 40, 0, 2 * Math.PI)
  ctx.fill()
}

async function render(){
  // save to a multi-page PDF file
  await canvas.saveAs("all-pages.pdf")

  // save to files named `page-01.png`, `page-02.png`, etc.
  await canvas.saveAs("page-{2}.png")
}
render()

Rendering to a window

import {Window} from 'skia-canvas'

let win = new Window(300, 300)
win.title = "Canvas Window"
win.on("draw", e => {
  let ctx = e.target.canvas.getContext("2d")
  ctx.lineWidth = 25 + 25 * Math.cos(e.frame / 10)
  ctx.beginPath()
  ctx.arc(150, 150, 50, 0, 2 * Math.PI)
  ctx.stroke()

  ctx.beginPath()
  ctx.arc(150, 150, 10, 0, 2 * Math.PI)
  ctx.stroke()
  ctx.fill()
})

Acknowledgements

This project is deeply indebted to the work of the Rust Skia project whose Skia bindings provide a safe and idiomatic interface to the mess of C++ that lies underneath. Many thanks to the developers of node-canvas for their terrific set of unit tests. In the absence of an Acid Test for canvas, these routines were invaluable.

Notable contributors

@mpaparno contributed support for SVG rendering, raw image-buffer handling, WEBP import/export and numerous bugfixes
@Salmondx developed the initial Raw image loading & rendering routines
@lucasmerlin helped get GPU rendering working on Vulkan
@cprecioso & @saantonandre corrected and expanded upon the TypeScript type definitions
@meihuanyu contributed filter & path rendering fixes

Copyright

vlcn-io/cr-sqlite

2024-12-08T01:35:02Z

Convergent, Replicated SQLite. Multi-writer and CRDT support for SQLite

cr-sqlite - Convergent, Replicated, SQLite

A component of the vulcan project.

Examples

Example applications using cr-sqlite to sync state.

Vite starter - Example | Repository
TodoMVC - Example | Repository
Svelte Store
Tutorials
WIP Local-First Presentation Editor
Basic setup & sync via an Observable Notebook

"It's like Git, for your data."

CR-SQLite is a run-time loadable extension for SQLite and libSQL. It allows merging different SQLite databases together that have taken independent writes.

In other words, you can write to your SQLite database while offline. I can write to mine while offline. We can then both come online and merge our databases together, without conflict.

In technical terms: cr-sqlite adds multi-master replication and partition tolerance to SQLite via conflict free replicated data types (CRDTs) and/or causally ordered event logs.

When is this useful?

Syncing data between devices
Implementing realtime collaboration
Offline editing
Being resilient to network conditions
Enabling instantaneous interactions

All of the above involve a merging of independent edits problem. If your database can handle this for you, you don't need custom code in your application to handle those 5 cases.

Discussions of these problems in the application space:

Sponsors

Companies:

Individuals: robinvasan | iansinnott | davefowler | barbalex | MohannadNaj

Perf

Perf data: https://github.com/vlcn-io/cr-sqlite/blob/main/py/perf/perf.ipynb

Currently inserts into CRRs are 2.5x slower than inserts into regular SQLite tables.
Reads are the same speed

Usage

The full documentation site is available here.

crsqlite exposes three main APIs:

A function extension (crsql_as_crr) to upgrade existing tables to "crrs" or "conflict free replicated relations"
- SELECT crsql_as_crr('table_name')
A virtual table (crsql_changes) to ask the database for changesets or to apply changesets from another database
- SELECT "table", "pk", "cid", "val", "col_version", "db_version", "site_id", cl, seq FROM crsql_changes WHERE db_version > x AND site_id = crsql_site_id() -- to get local changes
- SELECT "table", "pk", "cid", "val", "col_version", "db_version", "site_id", cl, seq FROM crsql_changes WHERE db_version > x AND site_id != some_site_id -- to get all changes excluding those synced from some actor
- INSERT INTO crsql_changes VALUES ([patches received from select on another peer])
And crsql_begin_alter('table_name') & crsql_alter_commit('table_name') primitives to allow altering table definitions that have been upgraded to crrs.
- Until we move forward with extending the syntax of SQLite to be CRR aware, altering CRRs looks like:
```
SELECT crsql_begin_alter('table_name');
-- 1 or more alterations to `table_name`
ALTER TABLE table_name ...;
SELECT crsql_commit_alter('table_name');
```
  A future version of cr-sqlite may extend the SQL syntax to make this more natural.

Application code uses the function extension to enable crr support on tables.

Networking code uses the crsql_changes virtual table to fetch and apply changes.

Usage looks like:

-- load the extension if it is not statically linked
.load crsqlite
.mode qbox
-- create tables as normal
create table foo (a primary key not null, b);
create table baz (a primary key not null, b, c, d);

-- update those tables to be crrs / crdts
select crsql_as_crr('foo');
select crsql_as_crr('baz');

-- insert some data / interact with tables as normal
insert into foo (a,b) values (1,2);
insert into baz (a,b,c,d) values ('a', 'woo', 'doo', 'daa');

-- ask for a record of what has changed
select "table", "pk", "cid", "val", "col_version", "db_version", "site_id", "cl", "seq" from crsql_changes;

┌───────┬─────────────┬─────┬───────┬─────────────┬────────────┬──────────────────────────────────────┬────┬─────┐
│ table │     pk      │ cid │  val  │ col_version │ db_version │ "site_id" │ cl │ seq │
├───────┼─────────────┼─────┼───────┼─────────────┼────────────┼──────────────────────────────────────┼────┼─────┤
│ 'foo' │ x'010901'   │ 'b' │ 2     │ 1           │ 1          │ x'049c48eadf4440d7944ed9ec88b13ea5'  │ 1  │ 0   │
│ 'baz' │ x'010b0161' │ 'b' │ 'woo' │ 1           │ 2          │ x'049c48eadf4440d7944ed9ec88b13ea5'  │ 1  │ 0   │
│ 'baz' │ x'010b0161' │ 'c' │ 'doo' │ 1           │ 2          │ x'049c48eadf4440d7944ed9ec88b13ea5'  │ 1  │ 1   │
│ 'baz' │ x'010b0161' │ 'd' │ 'daa' │ 1           │ 2          │ x'049c48eadf4440d7944ed9ec88b13ea5'  │ 1  │ 2   │
└───────┴─────────────┴─────┴───────┴─────────────┴────────────┴──────────────────────────────────────┴────┴─────┘

-- merge changes from a peer
insert into crsql_changes
  ("table", "pk", "cid", "val", "col_version", "db_version", "site_id", "cl", "seq")
  values
  ('foo', x'010905', 'b', 'thing', 5, 5, X'7096E2D505314699A59C95FABA14ABB5', 1, 0);
insert into crsql_changes ("table", "pk", "cid", "val", "col_version", "db_version", "site_id", "cl", "seq")
  values
  ('baz', x'010b0161', 'b', 123, 101, 233, X'7096E2D505314699A59C95FABA14ABB5', 1, 0);

-- check that peer's changes were applied
sqlite> select * from foo;
┌───┬─────────┐
│ a │    b    │
├───┼─────────┤
│ 1 │ 2       │
│ 5 │ 'thing' │
└───┴─────────┘

select * from baz;
┌─────┬─────┬───────┬───────┐
│  a  │  b  │   c   │   d   │
├─────┼─────┼───────┼───────┤
│ 'a' │ 123 │ 'doo' │ 'daa' │
└─────┴─────┴───────┴───────┘

-- tear down the extension before closing the connection
-- https://sqlite.org/forum/forumpost/c94f943821
select crsql_finalize();

Packages

Pre-built binaries of the extension are available in the releases section.

These can be loaded into sqlite via the load_extension command from any language (Python, NodeJS, C++, Rust, etc.) that has SQLite bindings.

The entrypoint to the loadable extension is sqlite3_crsqlite_init so you'll either need to provide that to load_extension or rename your binary to crsqlite.[dylib/dll/so]. See the linked sqlite load_extension docs.

load_extension(extension_path, 'sqlite3_crsqlite_init')

Note: if you're using cr-sqlite as a run time loadable extension, loading the extension should be the first operation you do after opening a connection to the database. The extension needs to be loaded on every connection you create.

For a WASM build that works in the browser, see the js directory.

For UI integrations (e.g., React) see the js directory.

How does it work?

There are two approaches with very different tradeoffs. Both will eventually be supported by cr-sqlite. v1 (and current releases) support the first approach. v2 will support both approaches.

Approach 1: History-free CRDTs

Approach 1 is characterized by the following properties:

Keeps no history / only keeps the current state
Automatically handles merge conflicts. No options for manual merging.
Tables are Grow Only Sets or variants of Observe-Remove Sets
Rows are maps of CRDTs. The column names being the keys, column values being a specific CRDT type
Columns can be counter, fractional index or last write wins CRDTs.
1. multi-value registers, RGA and others to come in future iterations

Tables which should be synced are defined as a composition of other types of CRDTs.

Example table definition:

CREATE CLSet post (
 id INTEGER PRIMARY KEY NOT NULL,
 views COUNTER,
 content PERITEXT,
 owner_id LWW INTEGER
);

note: given that extensions can't extend the SQLite syntax this is notional. We are, however, extending the libSQL syntax so this will be available in that fork. In base SQLite you'd run the select crsql_as_crr function as seen earlier.

Under approach 1, merging two tables works roughly like so:

Rows are identified by primary key
Tables are unioned (and a delete log is consulted) such that both tables will have the same rows.

If a row was modified in multiple places, then we merge the row. Merging a row involves merging each column of that row according to the semantics of the CRDT for the column.

Last-write wins just picks the lastest write
Counter CRDT sums the values
Multi-value registers keep all conflicting values
Fractional indices are taken as last write

For more background see this post.

Notes:

LWW, Fractional Index, Observe-Remove sets are available now.
Counter and rich-text CRDTs are still being implemented.
Custom SQL syntax will be available in our libSQL integration. The SQLite extension requires a slightly different syntax than what is depicted above.

Approach 2: Causal Event Log

To be implemented in v2 of cr-sqlite

Approach 2 has the following properties:

A history of every modification that happens to the database is kept
1. This history can be garbage collected in certain network topologies
Merge conflicts can be automatically handled (via CRDT style rules) or the developer can define their own conflict resolution plan.
The developer can choose to fork the data on merge conflict rather than merging
Forks can live indefinitely or a specific fork can be chosen and other forks dropped

This is much more akin to git and event sourcing but with the drawback being that it is much more write heavy and much more space intensive.

Building

For a stable version, build against a release tag as main may not be 100% stable.

You'll need to install Rust.

Installing Rust: https://www.rust-lang.org/tools/install

Run Time Loadable Extension

Instructions on building a native library that can be loaded into SQLite in non-wasm environments.

rustup toolchain install nightly # make sure you have the rust nightly toolchain
git clone --recurse-submodules git@github.com:vlcn-io/cr-sqlite.git
cd cr-sqlite/core
make loadable

This will create a shared library at dist/crsqlite.[lib extension]

[lib extension]:

Linux: .so
Darwin / OS X: .dylib
Windows: .dll

WASM

For a WASM build that works in the browser, see the js repository.

CLI

Instructions on building a sqlite3 CLI that has cr-sqlite statically linked and pre-loaded.

In the core directory of the project, run:

make sqlite3

This will create a sqlite3 binary at dist/sqlite3

Tests

core:

cd core
make test

py integration tests:

cd core
make loadable
cd ../py/correctness
./install-and-test.sh

JS APIs

JS APIs for using cr-sqlite in the browser are not yet documented but exist in the js repo. You can also see examples of them in use here:

Research & Prior Art

cr-sqlite was inspired by and built on ideas from these papers:

GraphiteEditor/Graphite

2024-12-08T01:45:24Z

2D vector & raster editor that melds traditional layers & tools with a modern node-based, non-destructive, procedural workflow.

Redefining state-of-the-art graphics editing

Graphite is an in-development raster and vector graphics package that's free and open source. It is powered by a node graph compositing engine that fuses layers with nodes, providing a fully nondestructive editing experience.

Presently, Graphite is a lightweight vector graphics editor that runs in your browser. Its node-based compositor lets you apply image effects and co-create art with generative AI.

Photo editing, digital painting, desktop publishing, VFX compositing, and motion graphics are additional competencies planned on the roadmap to make Graphite a highly versatile content creation tool.

Launch the latest alpha release of the Graphite editor and learn more at the project website.

⭐ Please remember to star this project here on GitHub! ⭐

Red Dress (click to open this artwork and explore it yourself)

Valley of Spires (click to open this artwork and explore it yourself)

Discord community

Join our Discord community to chat with the community and development team. Feel free to lurk, ask questions, give suggestions, or get involved in the project. We are always seeking new collaborators to help test, design, and develop the software and this is where we communicate.

Building/contributing

We need Rust and web developers! See instructions here for setting up the project and getting started.

We are also in search of artists to create beautiful sample work in Graphite and illustrations for the website and social media. Please get in touch if you are able to help out.

By submitting code for inclusion in the project, you are agreeing to license your changes under the Apache 2.0 license, and that you have the authority to do so. Some directories may have other licenses, like dual-licensed MIT/Apache 2.0, and code submissions to those directories mean you agree to the applicable license(s).

Features and roadmap

See the web page for this information.

Support the project

Please consider pledging a monthly donation. Your contribution helps sustain this volunteer-run open source project and brings powerful, free creative tools to the masses.

vercel/turborepo

2024-12-08T01:45:24Z

Build system optimized for JavaScript and TypeScript, written in Rust

Turborepo

Turborepo is a high-performance build system for JavaScript and TypeScript codebases, written in Rust.

Getting Started

Visit https://turbo.build/repo to get started with Turborepo.

Contributing

See CONTRIBUTING.md for more information.

Community

The Turborepo community can be found on GitHub Discussions, where you can ask questions, voice ideas, and share your projects.

To chat with other community members, you can join the Turborepo Discord.

Our Code of Conduct applies to all Turborepo community channels.

Who is using Turborepo?

Turborepo is used by the world's leading companies. Check out the Turborepo Showcase to learn more.

Updates

Follow @turborepo on X for project updates.

Authors

Turborepo

Jared Palmer (@jaredpalmer)

Security

If you believe you have found a security vulnerability in Turborepo, we encourage you to responsibly disclose this and not open a public issue. We will investigate all legitimate reports. Email security@vercel.com to disclose any security vulnerabilities.

https://vercel.com/security

Myriad-Dreamin/tinymist

2024-12-08T01:45:24Z

Tinymist [ˈtaɪni mɪst] is an integrated language service for Typst [taɪpst].

Tinymist

Tinymist [ˈtaɪni mɪst] is an integrated language service for Typst [taɪpst]. You can also call it "微霭" [wēi ǎi] in Chinese.

It contains:

an analyzing library for Typst, see tinymist-query.
a CLI for Typst, see tinymist.
- which provides a language server for Typst, see Language Features.
- which provides a preview server for Typst, see Preview Feature.
a VSCode extension for Typst, see Tinymist VSCode Extension.

Features

Language service (LSP) features:

Semantic highlighting
- The "semantic highlighting" is supplementary to "syntax highlighting".
Code actions
- Also known as "quick fixes" or "refactorings".
Formatting (Reformatting)
- Provide the user with support for formatting whole documents, using typstfmt or typstyle.
Document highlight
- Highlight all break points in a loop context.
- (Todo) Highlight all exit points in a function context.
- (Todo) Highlight all captures in a closure context.
- (Todo) Highlight all occurrences of a symbol in a document.
Document links
- Renders path or link references in the document, such as image("path.png") or bibliography(style: "path.csl").
Document symbols
- Also known as "document outline" or "table of contents" in Typst.
Folding ranges
- You can collapse code/content blocks and headings.
Goto definitions
- Right-click on a symbol and select "Go to Definition".
- Or ctrl+click on a symbol.
References
- Right-click on a symbol and select "Go to References" or "Find References".
- Or ctrl+click on a symbol.
Hover tips
- Also known as "hovering tooltip".
- Render docs according to tidy style.
Inlay hints
- Inlay hints are special markers that appear in the editor and provide you with additional information about your code, like the names of the parameters that a called method expects.
Color Provider
- View all inlay colorful label for color literals in your document.
- Change the color literal's value by a color picker or its code presentation.
Code Lens
- Should give contextual buttons along with code. For example, a button for exporting your document to various formats at the start of the document.
Rename symbols and embedded paths
Help with function and method signatures
Workspace Symbols
Code Action
- Increasing/Decreasing heading levels.
- Turn equation into "inline", "block" or "multiple-line block" styles.
experimental/onEnter
- Enter inside triple-slash comments automatically inserts ///
- Enter in the middle or after a trailing space in // inserts //
- Enter inside //! doc comments automatically inserts //!
- Enter inside equation markups automatically inserts indents.

Extra features:

Compiles to PDF on save (configurable to as-you-type, or other options).
Compiles to SVG, PNG, HTML, Markdown, Text, and other formats by commands, vscode tasks, or code lenses.
Provides code lenses for exporting to PDF/SVG/PNG/etc.
Provides a status bar item to show the current document's compilation status and words count.
Editor tools:
- View a list of templates in template gallery. (tinymist.showTemplateGallery)
- Click a button in template gallery to initialize a new project with a template. (tinymist.initTemplate and tinymist.initTemplateInPlace)
- Trace execution in current document (tinymist.profileCurrentFile).

Versioning and Release Cycle

Tinymist's versions follow the Semantic Versioning scheme, in format of MAJOR.MINOR.PATCH. Besides, tinymist follows special rules for the version number:

If a version is suffixed with -rcN ( ), e.g. 0.11.0-rc1 and 0.12.1-rc1, it means this version is a release candidate. It is used to test publish script and E2E functionalities. These versions will not be published to the marketplace.
If the PATCH number is odd, e.g. 0.11.1 and 0.12.3, it means this version is a nightly release. The nightly release will use both tinymist and typst at main branch. They will be published as prerelease version to the marketplace.
Otherwise, if the PATCH number is even, e.g. 0.11.0 and 0.12.2, it means this version is a regular release. The regular release will always use the recent stable version of tinymist and typst.

The release cycle is as follows:

If there is a typst version update, a new major or minor version will be released intermediately. This means tinymist will always align the minor version with typst.
If there is at least a bug or feature added this week, a new patch version will be released.

Installation

Follow the instructions to enable tinymist in your favorite editor.

Installing Regular/Nightly Prebuilds from GitHub

Note: if you are not knowing what is a regular/nightly release, please don't follow this section.

Besides published releases specific for each editors, you can also download the latest regular/nightly prebuilts from GitHub and install them manually.

Regular prebuilts can be found in GitHub Releases.
Nightly prebuilts can be found in GitHub Actions. For example, if you are seeking a nightly release for the featured PR: build: bump version to 0.11.17-rc1, you could click and go to the action page run for the related commits and download the artifacts.

To install extension file (the file with .vsix extension) manually, please Ctrl+Shift+X in the editor window and drop the downloaded vsix file into the opened extensions view.

Documentation

See Online Documentation.

Packaging

Stable Channel:

Nightly Channel:

Roadmap

The development in typst v0.12.0 has been finished. We'll slow down for a while to catch regressions and bugs by changes. We are also planning to implement the following features in typst v0.13.0 or spare time in weekend:

Spell checking: There is already a branch but no suitable (default) spell checking library is found.
Type checking: complete the type checker.
Static Linter: linting code statically according to feedback of the type checker and succeeding code analysis.
Periscope renderer: It is disabled since vscode reject to render SVGs containing foreignObjects.
Inlay hint: It is disabled by default because of performance issues.
Find references of dictionary fields and named function arguments.
A reliable way of configuring projects's entry files and files to export across editors. See GitHub Issue 530.
Improve symbol view's appearance.
Improve package view.
- Navigate to symbols by clicking on the symbol name in the view.
- Automatically locate the symbol item in the view when viewing local documentation.
- Remember the recently invoked package commands, e.g. "Open Docs of @preview/cetz:0.3.1", "Open directory of @preview/touying:0.5.3".
Improve label view.
- Group labels.
- Search labels.
- Keep (persist) group preferences.
Improve Typst Preview.
- Browsing mode: if no main file is specified, the preview will be in browsing mode and use the recently focused file as the main.
- Pin drop-down: Set the file to preview in the drop-down for clients that doesn't support passing arguments to the preview command.
- Render in web worker (another thread) to reduce overhead on the electron's main thread.

If you are interested by any above features, please feel free to send Issues to discuss or PRs to implement to GitHub.

Contributing

Please read the CONTRIBUTING.md file for contribution guidelines.

Maintainers

Get list of maintainers from MAINTAINERS.typ. Or programmatically by yarn maintainers

[!NOTE]

You can add extra arguments for specific information. For example, yarn maintainers --input="action=maintainers".

Acknowledgements

Partially code is inherited from typst-lsp
The integrating offline handwritten-stroke recognizer is powered by Detypify.
The integrating preview service is powered by typst-preview.
The integrating local package management functions are adopted from vscode-typst-sync.

aimacode/aima-scala

2024-12-08T01:45:35Z

aima-scala

Scala implementation of algorithms from Russell And Norvig's Artificial Intelligence - A Modern Approach 3rd Edition. You can use this in conjunction with a course on AI, or for study on your own.

Formating the code

sbt scalafmt

Test Coverage

sbt ";clean;test"
sbt coverageReport

reports are generated in the target/scoverage-report folder of the sub-projects

Fausto-Korpsvart/Catppuccin-GTK-Theme

2024-12-08T01:35:15Z

GTK Theme with Catppuccin colour scheme

overleaf/toolkit

2024-12-08T01:45:41Z

Overleaf Toolkit

This repository contains the Overleaf Toolkit, the standard tools for running a local instance of Overleaf. This toolkit will help you to set up and administer both Overleaf Community Edition (free to use, and community supported), and Overleaf Server Pro (commercial, with professional support).

The Developer wiki contains further documentation on releases, features and other configuration elements.

Getting Started

Clone this repository locally:

git clone https://github.com/overleaf/toolkit.git ./overleaf-toolkit

Then follow the Quick Start Guide.

Documentation

See Documentation Index

Contributing

See the CONTRIBUTING file.

Getting Help

Users of the free Community Edition should open an issue on github.

Users of Server Pro should contact support@overleaf.com for assistance.

In both cases, it is a good idea to include the output of the bin/doctor script in your message.

ava-labs/icm-contracts

2024-12-07T01:35:30Z

Smart contracts built on top of Avalanche Interchain Messaging (ICM) to facilitate EVM cross-chain application development.

ourzora/zora-protocol

2024-12-08T01:45:52Z

Monorepo for Zora Protocol (contracts & sdks)

hyperledger-labs/yui-ibc-solidity

2024-12-08T01:45:52Z

IBC in Solidity

ethpandaops/optimism-package

2024-12-08T01:45:58Z

A kurtosis package for deploying an optimistic rollup

Muhammed-Rahif/Notpad

2024-12-08T01:46:03Z

The ordinary notepad that everyone know, but it's rich!

alienator88/Pearcleaner

2024-12-08T01:46:06Z

A free, source-available and fair-code licensed mac app cleaner

Pearcleaner

Status: Maintained (Bi-weekly release cadence)
Version: 4.0.2
Download · Commits

A free, source-available and fair-code licensed Mac app cleaner inspired by Freemacsoft's AppCleaner and Sun Knudsen's Privacy Guides post on his app-cleaner script. This project was born out of wanting to learn more on how macOS deals with app installation/uninstallation and getting more Swift experience. If you have suggestions I'm open to hearing them, submit a feature request!

Translations

If you are able to contribute to translations for the app, please see this discussion: https://github.com/alienator88/Pearcleaner/discussions/137

License

[!IMPORTANT] Pearcleaner is licensed under Apache 2.0 with Commons Clause. This means that you can do anything you'd like with the source, modify it, contribute to it, etc., but the license explicitly prohibits any form of monetization for Pearcleaner or any modified versions of it. See full license HERE

Features

Orphaned file search for finding remaining files from previously uninstalled applications
Development environments file/cache cleaning
Sentinel monitor helper that can be enabled to watch Trash folder for deleted apps to cleanup after the fact(Extremely small (210KB) and uses ~2mb of ram to run in the background and file watch)
Mini mode which can be enabled from Settings
Menubar icon option
CLI support
Drag/drop applications support
Optional Finder Extension which allows you to uninstall an app directly from Finder by right click > Pearcleaner Uninstall
Theme System available with custom color selector
Differentiate between regular, Safari web-apps and mobile apps with badges like web and iOS
Has clean uninstall menu option for the Pearcleaner app itself if you want to stop using it and get rid of all files and launch items
Export app bundles for migrating apps and their cache to a new system
Export app file list search results
Optional Homebrew cleanup
Include extra directories to search for apps in
Exclude files/folders from the orphaned file search
Custom auto-updater that pulls latest release notes and binaries from GitHub Releases (Pearcleaner should run from /Applications folder to avoid permission issues)

Screenshots

Issues

[!WARNING]

When submitting issues, please use the appropriate issue template corresponding with your problem HERE

Beta versions of macOS will not be supported until general release

Requirements

[!NOTE]

MacOS 13.0+ [Non-beta releases]

Full Disk permission to search for files

Accessibility permission to delete/restore files

Automation permission to control Finder and Terminal

Getting Pearcleaner

Releases

Pre-compiled, always up-to-date versions are available from my releases page.

Homebrew

You can add the app via Homebrew:

brew install pearcleaner

Thanks

Much appreciation to Freemacsoft's AppCleaner and Sun Knudsen's app-cleaner script
DharsanB for sponsoring my Apple Developer account

Other Apps

Pearcleaner - An opensource app cleaner with privacy in mind

Sentinel - A GUI for controlling gatekeeper status on your Mac

Viz - Utility for extracting text from images, videos, qr/barcodes

RoboCupAtHome/RuleBook

2024-12-07T01:35:51Z

Rulebook for RoboCup @Home 2024

RuleBook for RoboCup @Home 2025

PDF Downloads

Rulebook
Score sheets

The current version for 2025 is draft

Changelog

RoboCup@Home teams and team members are welcome to post GitHub issues for clarifications, questions etc.

Improvements are also welcome in the form of pull requests (see guidelines).

FAQ

See the Frequently Asked Questions here.

Or ask us via Telegram: https://t.me/RoboCupAtHome

Question, issues, and contribution Guidelines

When asking questions please follow these Guidelines.
Guidelines for setting up an issue are here. Open an issue for:
- Typos, misspellings, and broken references.
- The rules are unclear or poorly specified.
- One rule contradicts another.
- The RuleBook contains outdated information
- Scoresheets are cryptic of hard to read.
- Compilation and build errors.
To contribute directly with the rulebook (Pull Requests), please read the conventions in the Contribution Guidelines and in the Workflow Guidelines

RoboCup

RoboCup is an international joint project to promote AI, robotics, and related fields. It is an attempt to foster AI and intelligent robotics research by providing standard problems where a wide range of technologies can be integrated and examined. More information can be found at http://www.robocup.org/.

RoboCup@Home

The RoboCup@Home league aims to develop service and assistive robot technology with high relevance for future personal domestic applications. It is the largest international annual compe- tition for autonomous service robots and is part of the RoboCup initiative. A set of benchmark tests is used to evaluate the robots abilities and performance in a realistic non-standardized home environment setting. Focus lies on the following domains but is not limited to: Human-Robot- Interaction and Cooperation, Navigation and Mapping in dynamic environments, Computer Vision and Object Recognition under natural light conditions, Object Manipulation, Adaptive Behaviors, Behavior Integration, Ambient Intelligence, Standardization and System Integration. It is collocated with the RoboCup symposium.

Contributing

The RoboCup @Home Organizing Committee is always looking for volunteers to assist in the events and willing to help to maintain the sites. Volunteers can help in tasks such as organizing and categorizing publications, team videos, memories, scores, press, and writing examples and extended rule explanations. Other projects such as developing utilities for the competition and tools for referees are also welcome. If you would like to collaborate, write to any of the members of the Organizing Committee.

Contributions and improvements to the rulebook are always welcome in the form of pull requests.

Building

The easiest way is to utilize the docker.io/texlive/texlive:TL2022-historic image to build everything.

For convenience, you should execute build.sh to make the PDFs with a container runtime. (This requires either docker or podman to be installed)

Recorded data

Data is recorded from the robots during the competition. It is available through the following links:

European Open 2016

All challenges

Leipzig 2016

More information

@Home Website: http://athome.robocup.org/
- @Home Website (backup): http://robocupathome.org/
Telegram group for Q&A and rules discussion: https://t.me/RoboCupAtHome
Whatsapp group: Please join the Telegram group
RoboCup 2019 Website https://2019.robocup.org
General mailing list
Youtube channel
Facebook Page
RoboCup @Home Wiki
Forum
Gitter.im chat

Hanxven/LeagueAkari

2024-12-07T01:36:05Z

✨兴趣使然、功能全面的英雄联盟工具集。

基于 LCU API 的英雄联盟客户端工具集

1. League Akari

简体中文 | English

Gitee 镜像站点：https://gitee.com/Hanxven/LeagueAkari

1.1 已支持功能

战绩查询
- 同大区玩家战绩查询以及有限的跨区查询
自动操作
- 接受对局
- 点赞玩家
- 返回房间
- 英雄选择（普通模式、大乱斗模式等）
- 英雄禁用
- 重新匹配对局
- 自动回复
对局分析
- 战绩分析
- 开黑分析
其他功能
- 重生倒计时
- 英雄选择或游戏内发送 KDA 简报
- 虚假段位、修改生涯背景、卸下头像框和勋章
- 英雄选择时秒退，最后一秒秒退
- 大乱斗无 CD 换英雄
- 创建 5x5 训练房间
- 同大区以及有限的跨大区观战
- 自定义在线状态
- 游戏内发送自定义文本
- 修复客户端窗口大小
外部数据源
- 英雄平衡性信息 (Fandom)
- OP.GG
其他功能
- 敬请探索

1.2 使用方法

在右侧 Release 中寻找最新的构建版本压缩包，解压后即可运行。

运行时不需要管理员权限，但高权限会提供更多的功能。

支持腾讯服以及非腾讯服客户端。

[!NOTE] 一旦检测到游戏客户端，则会自动连接，无需考虑启动顺序。

League Akari 会很好地处理中途启动的情景，并始终维持合适的连接状态。

1.3 企鹅群组

一个适合吹水开黑聊天的好地方，甚至还能反馈 BUG 和提出建议。

企鹅群组：301157623 (已满，随缘加入)

1.4 测试版本

在群聊中，定期会发布测试版本，测试版本会包含若干将在下个正式版本中发布的功能。

2. 加入到开发

百密一疏，各种问题总是难以避免，作为使用者，您可以：

2.1 GitHub Issues

GitHub Issues 是最重要的反馈渠道，请精准描述您的需求、遇到的问题或任何可行的想法。

2.2 加入开发

如果您对此项目感兴趣，欢迎加入到开发之中，提交 PR，为其添加更多功能。

3. 编译 & 构建 & 运行

本章节指示如何通过源码构建 League Akari。

3.1 Electron 主程序

安装依赖：yarn install

dev：yarn dev

build（for Windows only）: yarn build:win

3.2 Node 原生模块 (可选)

需要 MSVC 编译环境和 node-gyp。

Node Addons 使 League Akari 具备更强大的功能，因为它为其提供了操作系统级 API 的调用入口。这些模块位于 addons/ 下。

如果你修改了 addons/* 下的模块内容，请重新编译。

yarn install

node-gyp configure
node-gyp build

之后将 la-input-win64.node 以及 la-tools-win64.node 复制到应用源码 src/main/native 下，并更新类型定义 src/main/node-addon.d.ts。

4. 参考

League Akari 的实现参考了许多现有的优秀开源项目，这些项目为软件的部分模块开发提供了清晰的思路指导，特此表示感谢。❤️

项目名称	描述
⭐⭐⭐ Pengu Loader	用于 UX 客户端调试和逆向工程工具
⭐⭐⭐ League of Legends LCU and Riot Client API Docs	LCU API 文档参考
⭐⭐ Community Dragon	资源管理和参考文档
⭐⭐ Seraphine	缝合重灾区，提供了集成思路
⭐ fix-lcu-window	修复客户端窗口大小问题的思路借鉴
⭐ Joi	OP.GG 相关实现的参考
⭐ lol-helper	(曾经的) 卡炫彩功能和工具设计的参考
⭐ vscode-league-respawn-timer	重生倒计时功能的参考
⭐ LeaguePrank	趣味功能的实现参考
⭐ LCU API	LCU API 早期参考文档

5. FAQ - 常见问题及回答

常见问题及解答

6. 免责声明

本软件作为基于 Riot 提供的 League Client Update (LCU) API 开发的辅助工具，由于其设计和实施均未采用侵入性技术手段，理论上不会直接干预或修改游戏数据。然而，需明确指出的是，虽然本软件在原理上并未直接修改游戏内部数据，但在游戏环境的持续更新和演变中 (如未来腾讯可能的反作弊系统或其他保护服务的更新)，无法完全排除由于版本更新导致的兼容性问题或其他意外后果。

特此强调，对于使用本软件可能带来的任何后果，包括但不限于游戏账户的封禁、数据损坏或其他任何形式的游戏体验负面影响，本软件的开发者将不承担任何责任。用户在决定使用本软件时，应充分考虑并自行承担由此产生的所有风险和后果。

本声明旨在全面而详尽地通知用户关于本软件使用的可能风险，以便用户在使用过程中做出充分的风险评估和明智的决策。感谢您的关注，同时敬请遵守相关游戏规则和使用指南，确保一种健康和公平的游戏环境。

coaidev/coai

2024-12-07T01:36:05Z

🚀 Next Generation AI One-Stop Internationalization Solution. 🚀 下一代 AI 一站式 B/C 端解决方案，支持 OpenAI，Midjourney，Claude，讯飞星火，Stable Diffusion，DALL·E，ChatGLM，通义千问，腾讯混元，360 智脑，百川 AI，火山方舟，新必应，Gemini，Moonshot 等模型，支持对话分享，自定义预设，云端同步，模型市场，支持弹性计费和订阅计划模式，支持图片解析，支持联网搜索，支持模型缓存，丰富美观的后台管理与仪表盘数据统计。

🥳 Chat Nio

🚀 Next Generation AIGC One-Stop Business Solution

"Chat Nio > Next Web + One API"

English · 简体中文 · Docs · Discord · Deployment Guide

📝 Features

🤖️ Rich Model Support: Multi-model service provider support (OpenAI / Anthropic / Gemini / Midjourney and more than ten compatible formats & private LLM support)
🤯 Beautiful UI Design: UI compatible with PC / Pad / Mobile, following Shadcn UI & Tremor Charts design standards, rich and beautiful interface design and backend dashboard
🎃 Complete Markdown Support: Support for LaTeX formulas / Mermaid mind maps / table rendering / code highlighting / chart drawing / progress bars and other advanced Markdown syntax support
👀 Multi-theme Support: Support for multiple theme switching, including Light Mode for light themes and Dark Mode for dark themes. 👉 Custom Color Scheme
📚 Internationalization Support: Support for internationalization, multi-language switching 🇨🇳 🇺🇸 🇯🇵 🇷🇺 👉 Welcome to contribute translations Pull Request
🎨 Text-to-Image Support: Support for multiple text-to-image models: OpenAI DALL-E✅ & Midjourney (support for U/V/R operations)✅ & Stable Diffusion✅ etc.
📡 Powerful Conversation Sync: Zero-cost cross-device conversation sync support for users, support for conversation sharing (link sharing & save as image & share management), no need for WebDav / WebRTC and other dependencies and complex learning costs
🎈 Model Market & Preset System: Support for customizable model market in the backend, providing model introductions, tags, and other parameters. Site owners can customize model introductions according to the situation. Also supports a preset system, including custom presets and cloud synchronization functions.
📖 Rich File Parsing: Out-of-the-box, supports file parsing for all models (PDF / Docx / Pptx / Excel / image formats parsing), supports more cloud image storage solutions (S3 / R2 / MinIO etc.), supports OCR image recognition 👉 See project Chat Nio Blob Service for details (supports Vercel / Docker one-click deployment)
🌏 Full Model Internet Search: Based on the SearXNG open-source engine, supports rich search engines such as Google / Bing / DuckDuckGo / Yahoo / Wikipedia / Arxiv / Qwant, supports safe search mode, content truncation, image proxy, test search availability, and other functions.
💕 Progressive Web App (PWA): Supports PWA applications & desktop support (desktop based on Tauri)
🤩 Comprehensive Backend Management: Supports beautiful and rich dashboard, announcement & notification management, user management, subscription management, gift code & redemption code management, price setting, subscription setting, custom model market, custom site name & logo, SMTP email settings, and other functions
🤑 Multiple Billing Methods: Supports 💴 Subscription and 💴 Elastic Billing two billing methods. Elastic billing supports per-request billing / token billing / no billing / anonymous calls and minimum request points detection and other powerful features
🎉 Innovative Model Caching: Supports enabling model caching: i.e., under the same request parameter hash, if it has been requested before, it will directly return the cached result (hitting the cache will not be billed), reducing the number of requests. You can customize whether to cache models, cache time, multiple cache result numbers, and other advanced cache settings
🥪 Additional Features (Support Discontinued): 🍎 AI Project Generator Function / 📂 Batch Article Generation Function / 🥪 AI Card Function (Deprecated)
😎 Excellent Channel Management: Self-written excellent channel algorithm, supports ⚡ multi-channel management, supports 🥳priority setting for channel call order, supports 🥳weight setting for load balancing probability distribution of channels at the same priority, supports 🥳user grouping, 🥳automatic retry on failure, 🥳model redirection, 🥳built-in upstream hiding, 🥳channel status management and other powerful enterprise-level functions
⭐ OpenAI API Distribution & Proxy System: Supports calling various large models in OpenAI API standard format, integrates powerful channel management functions, only needs to deploy one site to achieve simultaneous development of B/C-end business💖
👌 Quick Upstream Synchronization: Channel settings, model market, price settings, and other settings can quickly synchronize with upstream sites, modify your site configuration based on this, quickly build your site, save time and effort, one-click synchronization, quick launch
👋 SEO Optimization: Supports SEO optimization, supports custom site name, site logo, and other SEO optimization settings to make search engines crawl faster, making your site stand out👋
🎫 Multiple Redemption Code Systems: Supports multiple redemption code systems, supports gift codes and redemption codes, supports batch generation, gift codes are suitable for promotional distribution, redemption codes are suitable for card sales, for gift codes of one type, a user can only redeem one code, which to some extent reduces the situation of one user redeeming multiple times in promotions😀
🥰 Business-Friendly License: Adopts the Apache-2.0 open-source license, friendly for commercial secondary development & distribution (please also comply with the provisions of the Apache-2.0 license, do not use for illegal purposes)

✨ Chat Nio Business

✅ Beautiful commercial-grade UI, elegant frontend interface and backend management

✅ Supports TTS & STT, plugin marketplace, RAG knowledge base and other rich features and modules

✅ More payment providers, more billing models and advanced order management

✅ Supports more authentication methods, including SMS login, OAuth login, etc.

✅ Supports model monitoring, channel health detection, fault alarm automatic channel switching

✅ Supports multi-tenant API Key distribution system, enterprise-level token permission management and visitor restrictions

✅ Supports security auditing, logging, model rate limiting, API Gateway and other advanced features

✅ Supports promotion rewards, professional data statistics, user profile analysis and other business analysis capabilities

✅ Supports Discord/Telegram/Feishu and other bot integration capabilities (extension modules)

...

👉 Learn More

🔨 Supported Models

OpenAI & Azure OpenAI (✅ Vision ✅ Function Calling)
Anthropic Claude (✅ Vision ✅ Function Calling)
Google Gemini & PaLM2 (✅ Vision)
Midjourney (✅ Mode Toggling ✅ U/V/R Actions)
iFlytek SparkDesk (✅ Vision ✅ Function Calling)
Zhipu AI ChatGLM (✅ Vision)
Alibaba Tongyi Qwen
Tencent Hunyuan
Baichuan AI
Moonshot AI (👉 OpenAI)
DeepSeek AI (👉 OpenAI)
ByteDance Skylark (✅ Function Calling)
Groq Cloud AI
OpenRouter (👉 OpenAI)
360 GPT
LocalAI / Ollama (👉 OpenAI)

👻 OpenAI Compatible API Proxy

Chat Completions (/v1/chat/completions)
Image Generation (/v1/images)
Model List (/v1/models)
Dashboard Billing (/v1/billing)

📦 Deployment

[!TIP] After successful deployment, the admin account is root, with the default password chatnio123456

✨ Zeabur (One-Click)

Zeabur provides a certain free quota, you can use non-paid regions for one-click deployment, and also supports plan subscriptions and elastic billing for flexible expansion.

Click Deploy to deploy, and enter the domain name you wish to bind, wait for the deployment to complete.

After deployment is complete, please visit your domain name and log in to the backend management using the username root and password chatnio123456. Please follow the prompts to change the password in the chatnio backend in a timely manner.

🐳 BTPanel (One-Click)

Install Baota Panel by visiting BTPanel and install using the stable version script.
Log in to the panel and click Docker on the left to enter Docker management.
If prompted that Docker / Docker Compose is not installed, you can install according to the guide above.
After installation is complete, enter App Store, search for CoAI and click Install.
Configure basic application information such as your domain name, port, etc., and click Confirm (default configuration can be used).
First-time installation may take 1-2 minutes to complete database initialization. If you encounter issues, please check the panel running logs for troubleshooting.
Visit your configured domain or server http://[ip]:[port], log in to the backend management using username root and password chatnio123456.

AlibabaCloud ComputeNest (One-Click)

Access the CoAI service on ComputeNest International Edition. If you are in China, please visit ComputeNest, and fill in the deployment parameters as prompted.
Select the payment type, fill in the instance parameters and network parameters, and click Next: Confirm Order.
After confirming the deployment parameters and checking the estimated price, click Create Now and wait for the service instance to be deployed.
Click Service Instance on the left. After the service instance is deployed, click the instance ID to enter the details interface.
Click the address in Use Now on the details interface to enter the CoAI interface. The default username is root and the password is chatnio123456 to log in to the backend management.
For more operation details and payment information, see：Service Details.

⚡ Docker Compose Installation (Recommended)

[!NOTE] After successful execution, the host machine mapping address is http://localhost:8000

git clone --depth=1 --branch=main --single-branch https://github.com/Deeptrain-Community/chatnio.git
cd chatnio
docker-compose up -d # Run the service
# To use the stable version, use docker-compose -f docker-compose.stable.yaml up -d instead
# To use Watchtower for automatic updates, use docker-compose -f docker-compose.watch.yaml up -d instead

Version update (If Watchtower automatic updates are enabled, manual updates are not necessary):

docker-compose down 
docker-compose pull
docker-compose up -d

MySQL database mount directory: ~/db

Redis database mount directory: ~/redis

Configuration file mount directory: ~/config

⚡ Docker Installation (Lightweight runtime, commonly used for external MYSQL/RDS services)

[!NOTE] After successful execution, the host machine address is http://localhost:8094.

To use the stable version, use programzmh/chatnio:stable instead of programzmh/chatnio:latest

docker run -d --name chatnio \
   --network host \
   -v ~/config:/config \
   -v ~/logs:/logs \
   -v ~/storage:/storage \
   -e MYSQL_HOST=localhost \
   -e MYSQL_PORT=3306 \
   -e MYSQL_DB=chatnio \
   -e MYSQL_USER=root \
   -e MYSQL_PASSWORD=chatnio123456 \
   -e REDIS_HOST=localhost \
   -e REDIS_PORT=6379 \
   -e SECRET=secret \
   -e SERVE_STATIC=true \
   programzmh/chatnio:latest

--network host means using the host machine's network, allowing the Docker container to use the host's network. You can modify this as needed.

SECRET: JWT secret key, generate a random string and modify accordingly

SERVE_STATIC: Whether to enable static file serving (normally this doesn't need to be changed, see FAQ below for details)

-v ~/config:/config mounts the configuration file, -v ~/logs:/logs mounts the host machine directory for log files, -v ~/storage:/storage mounts the directory for additional feature generated files

MySQL and Redis services need to be configured. Please refer to the information above to modify the environment variables accordingly

Version update (After enabling Watchtower, manual updates are not necessary. After execution, follow the steps above to run again):

docker stop chatnio
docker rm chatnio
docker pull programzmh/chatnio:latest

⚒ Compile and Install

[!NOTE] After successful deployment, the default port is 8094, and the access address is http://localhost:8094

Config settings (~/config/config.yaml) can be overridden using environment variables. For example, the MYSQL_HOST environment variable can override the mysql.host configuration item

git clone https://github.com/Deeptrain-Community/chatnio.git
cd chatnio

cd app
npm install -g pnpm
pnpm install
pnpm build

cd ..
go build -o chatnio

# e.g. using nohup (you can also use systemd or other service manager)
nohup ./chatnio > output.log & # using nohup to run in background

📦 Tech Stack

🥗 Frontend: React + Redux + Radix UI + Tailwind CSS
🍎 Backend: Golang + Gin + Redis + MySQL
🍒 Application Technology: PWA + WebSocket

🤯 Why Create This Project & Project Advantages

We found that most AIGC commercial sites on the market are frontend-oriented lightweight deployment projects with beautiful UI interface designs, such as the commercial version of Next Chat. Due to its personal privatization-oriented design, there are some limitations in secondary commercial development, presenting some issues, such as:
1. Difficult conversation synchronization, for example, requiring services like WebDav, high user learning costs, and difficulties in real-time cross-device synchronization.
2. Insufficient billing, for example, only supporting elastic billing or only subscription-based, unable to meet the needs of different users.
3. Inconvenient file parsing, for example, only supporting uploading images to an image hosting service first, then returning to the site to input the URL direct link in the input box, without built-in file parsing functionality.
4. No support for conversation URL sharing, for example, only supporting conversation screenshot sharing, unable to support conversation URL sharing (or only supporting tools like ShareGPT, which cannot promote the site).
5. Insufficient channel management, for example, the backend only supports OpenAI format channels, making it difficult to be compatible with other format channels. And only one channel can be filled in, unable to support multi-channel management.
6. No API call support, for example, only supporting user interface calls, unable to support API proxying and management.
Another type is API distribution-oriented sites with powerful distribution systems, such as projects based on One API. Although these projects support powerful API proxying and management, they lack interface design and some C-end features, such as:
1. Insufficient user interface, for example, only supporting API calls, without built-in user interface chat. User interface chat requires manually copying the key and going to other sites to use, which has a high learning cost for ordinary users.
2. No subscription system, for example, only supporting elastic billing, lacking billing design for C-end users, unable to meet different user needs, and not user-friendly in terms of cost perception for users without a foundation.
3. Insufficient C-end features, for example, only supporting API calls, not supporting conversation synchronization, conversation sharing, file parsing, and other functions.
4. Insufficient load balancing, the open-source version does not support the weight parameter, unable to achieve balanced load distribution probability for channels at the same priority (New API also solves this pain point, with a more beautiful UI).

Therefore, we hope to combine the advantages of these two types of projects to create a project that has both a powerful API distribution system and a rich user interface design, thus meeting the needs of C-end users while developing B-end business, improving user experience, reducing user learning costs, and increasing user stickiness.

Thus, Chat Nio was born. We hope to create a project that has both a powerful API distribution system and a rich user interface design, becoming the next-generation open-source AIGC project's one-stop commercial solution.

❤ Donations

If you find this project helpful, you can give it a Star to show your support!

outerbase/studio

2024-12-07T01:36:05Z

An invoice generator app built using Next.js, Typescript, and Shadcn

Invoify

Invoify is a web-based invoice generator application built with Next.js 13, TypeScript, React, and the Shadcn UI library. It provides an easy way to create and manage professional invoices.

Invoify

Technologies

Core Technologies

Next.js: React framework for SSR and client-side navigation.
TypeScript: JavaScript superset with static typing.
Shadcn-UI: UI library for enhanced visuals.
Tailwind: Utility-first CSS framework.
React Hook Form: Form management for React.
Zod: TypeScript-first schema validation.
Puppeteer: PDF generation with headless browsers.

Additional Dependencies

Nodemailer: Node.js module for sending emails.
Lucide Icons: Collection of customizable SVG icons.

Roadmap

Easily Create Invoices: Utilize a simple form to quickly generate invoices.
Save for Future Access: Store your invoices directly in your browser for easy retrieval.
Retrieve Invoices Effortlessly: Load and access invoices seamlessly from your saved list.
Flexible Download Options: Download invoices directly or send them via email in PDF format.
Template Variety: Choose from multiple (currently 2) invoice templates.
Live Preview: Edit the form and see changes in real-time with the live preview feature.
Export in Various Formats: Export invoices in different formats, including JSON, XLSX, CSV, and XML.
I18N Support: i18n support with multiple languages for UI and templates.
Themeable Templates: Select a theme color for the invoice
Custom Inputs: Define your own inputs that are missing from the default invoice builder. (Ex: VAT number)
Individual Tax for Line Items: Add tax details for a specific line item other than the general tax

Demo

[!NOTE] Please be advised that there are currently issues when using this application in the Mozilla Firefox browser. For more information, refer to Issue #11.

Visit the live demo to see Invoify in action.

Getting Started

Follow these instructions to get Invoify up and running on your local machine.

Prerequisites

Node.js and npm installed on your system.

Installation

Clone the repository:

git clone https://github.com/al1abb/invoify.git
cd invoify

Install dependencies
```
npm install
```
Create an .env.local file with this content (This step is for sending pdf to email feature):
```
NODEMAILER_EMAIL=your_email@example.com
NODEMAILER_PW=your_email_password
```
Start development server
```
npm run dev
```
Open your web browser and access the application at http://localhost:3000

License

Distributed under the MIT License. See LICENSE.txt for more information.

github/docs

2024-12-08T01:46:30Z

The open-source repo for docs.github.com

GitHub Docs

This repository contains the documentation website code and Markdown source files for docs.github.com.

GitHub's Docs team works on pre-production content in a private repo that regularly syncs with this public repo.

Use the table of contents icon on the top right corner of this document to navigate to a specific section quickly.

Contributing

We accept different types of contributions, including some that don't require you to write a single line of code. For detailed instructions on how to get started with our project, see "About contributing to GitHub Docs."

Ways to contribute

On the GitHub Docs site, you can contribute by clicking the Make a contribution button at the bottom of the page to open a pull request for quick fixes like typos, updates, or link fixes.

You can also contribute by creating a local environment or opening a Codespace. For more information, see "Setting up your environment to work on GitHub Docs."

For more complex contributions, please open an issue using the most appropriate issue template to describe the changes you'd like to see.

If you're looking for a way to contribute, you can scan through our help wanted board to find open issues already approved for work.

And that's it!

If you're having trouble with your GitHub account, contact Support.

That's how you can easily become a member of the GitHub Docs community. ✨

READMEs

In addition to the README you're reading right now, this repo includes other READMEs that describe the purpose of each subdirectory in more detail:

License

The GitHub product documentation in the assets, content, and data folders are licensed under a CC-BY license.

All other code in this repository is licensed under the MIT license.

When using the GitHub logos, be sure to follow the GitHub logo guidelines.

Thanks 💜

Thanks for all your contributions and efforts towards improving the GitHub documentation. We thank you for being part of our ✨ community ✨!

epicweb-dev/epic-stack

2024-12-08T01:46:30Z

This is a Full Stack app starter with the foundational things setup and configured for you to hit the ground running on your next EPIC idea.

The Epic Stack 🚀

Ditch analysis paralysis and start shipping Epic Web apps.

This is an opinionated project starter and reference that allows teams to ship their ideas to production faster and on a more stable foundation based on the experience of Kent C. Dodds and contributors.

npx create-epic-app@latest

The Epic Stack

Watch Kent's Introduction to The Epic Stack

"The Epic Stack" by Kent C. Dodds

Docs

Read the docs (please 🙏).

Support

🆘 Join the discussion on GitHub and the KCD Community on Discord.
💡 Create an idea discussion for suggestions.
🐛 Open a GitHub issue to report a bug.

Branding

Want to talk about the Epic Stack in a blog post or talk? Great! Here are some assets you can use in your material: EpicWeb.dev/brand

Thanks

You rock 🪨

4gray/iptvnator

2024-12-08T01:46:30Z

📺 Cross-platform IPTV player application with multiple features, such as support of m3u and m3u8 playlists, favorites, TV guide, TV archive/catchup and more.

IPTVnator - IPTV Player Application

Telegram channel for discussions

IPTVnator is a video player application that provides support for IPTV playlist playback (m3u, m3u8). The application allows users to import playlists using remote URLs or by uploading files from the local file system. Additionally, it supports EPG information in XMLTV format which can be provided via URL.

The application is a cross-platform, open-source project built with Electron (currently migrating to Tauri) and Angular.

⚠️ Note: IPTVnator does not provide any playlists or other digital content. The channels and pictures in the screenshots are for demonstration purposes only.

Features

M3u and M3u8 playlist support 📺
Xtream Code (XC) and Stalker portal (STB) support
External player support - MPV, VLC
Add playlists from the file system or remote URLs 📂
Automatic playlist updates on application startup
Channel search functionality 🔍
EPG support (TV Guide) with detailed information
TV archive/catchup/timeshift functionality
Group-based channel list
Favorite channels management
Global favorites aggregated from all playlists
HTML video player with HLS.js support or Video.js-based player
Internationalization with support for 8 languages (English, Russian, German, Korean, Spanish, Chinese, French, Italian)
Custom "User Agent" header configuration for playlists
Light and Dark themes
Docker version available for self-hosting

Screenshots:

Welcome screen: Playlists overview	Main player interface with channels sidebar and video player

Welcome screen: Add playlist via file upload	Welcome screen: Add playlist via URL

EPG Sidebar: TV guide on the right side	General application settings

Playlist settings

Note: First version of the application which was developed as a PWA is available in an extra git branch.

Download

Download the latest version of the application for macOS, Windows, and Linux from the release page.

Alternatively, you can install the application using one of the following package managers:

Snap

$ sudo snap install iptvnator

Arch

Also available as an Arch PKG, iptvnator-bin, in the AUR (using your favourite AUR-helper, .e.g. yay)

$ yay -S iptvnator-bin

Gentoo

You can install IPTVnator from the gentoo-zh overlay

sudo eselect repository enable gentoo-zh
sudo emerge --sync gentoo-zh
sudo emerge iptvnator-bin

How to Build and Develop

Requirements:

Node.js with npm
Rust (required for tauri)

Clone this repository and install project dependencies:
```
$ npm install
```
Start the application:
```
$ npm run tauri dev
```

This will open the Tauri version in a separate window, while the PWA version will be available at http://localhost:4200.

To run only the Angular app without Tauri, use:

$ npm run serve

Disclaimer

IPTVnator doesn't provide any playlists or other digital content.

skills/github-pages

2024-12-08T01:36:42Z

Create a site or blog from your GitHub repositories with GitHub Pages.

LOG1997/log-lottery

2024-12-08T01:36:18Z

🎈🎈🎈🎈年会抽奖程序，threejs+vue3 3D球体动态抽奖应用。

ZUGFeRD/quba-viewer

2024-12-07T01:36:36Z

Quba is a viewer for electronic invoices

Quba-Viewer

Quba-Viewer (homepage) is a cross platform open source application to display structured (i.e., XML) and hybrid (Factur-X/ZUGFeRD PDF) electronic invoices.

Documentation-wise there is a doc RE Architecture, Development, Debugging and testing in general and some Electron and E-Invoice peculiarities like interprocess (IPC) communication, XSLT and codelists in particular.

History

1.4.2 01.11.2024

- #54 print only prints first page - #83 NaN for VAT amount and rounding amount - #82 Drag&Drop does not work

1.4.1 30.10.2024

- #78 default language to OS language, if de|en|fr - #68 structured invoice data won't get parsed from zugferd-file - #23 Does not work on intel macs - #63 Mention license in about window - #58 electron 20 to 33 - #53 MSI version

1.4.0 15.01.2024

- Online validation works again - Corrected validation now possible with personalized account - Be able to visualize Order-X, i.e. CIO #30 - be able to display IDs in whatever language - Factur-X / ZUGFeRD is now visualized by default and requires no click - first attempts intel mac - be able to load same filename multiple times #35 - menu item context for print (/xml and /pdf) and validate - get rid off temp files - changed translation from 3 files to one, translations provided by the app

1.3.0 14.04.2023

- support ubl credit notes

1.2.1 13.10.2022

- upgrade to Electron 20 enabling e.g. newer Mac versions to run it

1.2.0 26.09.2022

- Allow to validate files via Mustangserver

1.1.5 15.08.2022

- XML invoice with large PDF (approx. > 1.4 MB) attached can't be opened #16

1.1.4 23.06.2022

- No close button for attachments #15

1.1.3 28.04.2022

- Accept invoices as command line arguments #12

1.1.2 11.11.2021

- Allow drag&drop of files #9

1.1.1 24.09.2021

- The window for the example files can now be closed - plain pdf without embedded xml not to show button for visualization

1.1.0 20.09.2021

- #6 New document shows still old data / invoice - Switch to vue.js - i18n: Support for EN and FR - support FX Referenzprofil XRechnung

1.0 23.08.2021

- #5 Codelists are now resolved - Support for first FX Extended Element, i.e., Cash Discount in XML

0.5 27.07.2021

- Support for PDF - Support for Factur-X/ZUGFeRD - Display errors as dialog instead of hiding them

0.2 10.06.2021

- Support for XRechnung (UBL) - "dark" theming - possibility to open multiple files at the same time in the same viewer (tabs) - print

0.1 2021-03-31

- Initial release - Support for XRechnung (UN/CEFACT CII) - release for Windows on 2021-03-31, for Linux on 2101-04-16 and for Mac on 2021-04-27

Pedigree

The FeRD had published visualization XSLTs for ZF1 (~=UN/CEFACT C13B) as open source, unfortunately the ones for the UN/CEFACT C16B-based version 2 remains proprietary. The Kosit released XSLT for both CII and UBL of the XRechnung (XR) which has been used for various online viewers but also for offline viewers like Ultramarinviewer and Open XRechnung Toolbox. Quba uses this work added translations and at least experimental support for Factur-X/ZUGFeRD profiles higher than EN16931.

Known issues

While XRechnung, EN16931 and below should work, not all FX attributes/elements of the Extended Profile have yet been mapped, feel free to report missing ones
conversion of XSLT to sef.json xslt3 -xsl:ubl-creditnote-xr.xsl -export:ubl-creditnote-xr.sef.json -t on windows works only in cmd.exe, not powershell

ZUGFeRD/quba-viewer

2024-12-08T01:46:59Z

Quba is a viewer for electronic invoices

Quba-Viewer

Quba-Viewer (homepage) is a cross platform open source application to display structured (i.e., XML) and hybrid (Factur-X/ZUGFeRD PDF) electronic invoices.

History

1.4.2 01.11.2024

- #54 print only prints first page - #83 NaN for VAT amount and rounding amount - #82 Drag&Drop does not work

1.4.1 30.10.2024

1.4.0 15.01.2024

1.3.0 14.04.2023

- support ubl credit notes

1.2.1 13.10.2022

- upgrade to Electron 20 enabling e.g. newer Mac versions to run it

1.2.0 26.09.2022

- Allow to validate files via Mustangserver

1.1.5 15.08.2022

- XML invoice with large PDF (approx. > 1.4 MB) attached can't be opened #16

1.1.4 23.06.2022

- No close button for attachments #15

1.1.3 28.04.2022

- Accept invoices as command line arguments #12

1.1.2 11.11.2021

- Allow drag&drop of files #9

1.1.1 24.09.2021

- The window for the example files can now be closed - plain pdf without embedded xml not to show button for visualization

1.1.0 20.09.2021

- #6 New document shows still old data / invoice - Switch to vue.js - i18n: Support for EN and FR - support FX Referenzprofil XRechnung

1.0 23.08.2021

- #5 Codelists are now resolved - Support for first FX Extended Element, i.e., Cash Discount in XML

0.5 27.07.2021

- Support for PDF - Support for Factur-X/ZUGFeRD - Display errors as dialog instead of hiding them

0.2 10.06.2021

- Support for XRechnung (UBL) - "dark" theming - possibility to open multiple files at the same time in the same viewer (tabs) - print

0.1 2021-03-31

- Initial release - Support for XRechnung (UN/CEFACT CII) - release for Windows on 2021-03-31, for Linux on 2101-04-16 and for Mac on 2021-04-27

Pedigree

Known issues

While XRechnung, EN16931 and below should work, not all FX attributes/elements of the Extended Profile have yet been mapped, feel free to report missing ones
conversion of XSLT to sef.json xslt3 -xsl:ubl-creditnote-xr.xsl -export:ubl-creditnote-xr.sef.json -t on windows works only in cmd.exe, not powershell

itplr-kosit/validator-configuration-xrechnung

2024-12-08T01:46:59Z

Configuration for validating documents against the German XRechnung standard using the KoSIT validation tool

Validator Configuration for XRechnung

Validator configuration for Universal Business Language (UBL) and UN/CEFACT Invoice and CreditNote documents against EN16931:2017 and the German Core Invoice Usage Specification (CIUS) XRechnung and XRechnung Extension. It needs the Validator to execute the rules.

For general questions to the standard "XRechnung" and related components refer to the official operations and support website.

For further technical information see the specific pages on usage and on development.

Releases

Each release contains all necessary XML Schema and Schematron rules to validate UBL and CII invoice XML documents for conformity with CIUS and Extension XRechnung.

In particular this Validator Configuration integrates the following third-party artifacts:

Versioning

The git master branch always has the latest code. Since March 2018 new releases have the following naming and versioning scheme:

The source code gets tagged with release-${release-date} and distributions get the name validator-configuration-xrechnung_${xrechnung-version}_${release-date} e.g. validator-configuration-xrechnung_3.0.2_2024-06-20.zip

GitHub 1C Enterprise Daily Trending

SAP/abap-file-formats

sqlfluff/sqlfluff

The SQL Linter for Humans

Table of Contents

Dialects Supported

Templates Supported

VS Code Extension

Getting Started

Documentation

Releases

SQLFluff on Slack

SQLFluff on Twitter

Contributing

Sponsors

imputnet/cobalt

cobalt monorepo

thank you

ethics

contributing

licenses

QuivrHQ/MegaParse

MegaParse - Your Parser for every type of documents

Key Features 🎯

Support

Example

Installation

Usage

Use MegaParse Vision

(Optional) Use LlamaParse for Improved Results

Use as an API

BenchMark

In Construction 🚧

Star History

MrOtherGuy/firefox-csshacks

Collection of random CSS hacks for Firefox

Setup

Set the pref to load stylesheets

Setting up files

Find the profile folder

Creating the stylesheet files

Set up files manually

Set up files using git

Style categories

Usage

Important!

Further miscallaneous notes

Theme

sdmg15/Best-websites-a-programmer-should-visit

Best-websites-a-programmer-should-visit

📖 Index

❓ When you get stuck

📰 News

📚 Magazines

👶 Coding practice for beginners

💰 CryptoCurrency

💡 For those who want to start a small project but can't find the ideas

🗣️ General Coding advice

🎨 Coding Style

🛠️ General Tools

🐚 Bash and Shell scripting

🎤 Interview Preparation

🎬 Documentaries

🎓 MOOCs for learning something new

🧑‍💻 Sites related to your preferred programming language (For me C++)

🤖 Learn AI

📢 Seminar, research writing, talks, etc

📦 Everything in one place

📺 YouTube Channels

✍️ Good Articles

🎧 Podcasts

🔄 Building a Simple Compiler/Interpreter

🧑‍🏫 Tutorials

👀 Watch others code

🧠 What should a programmer know

⚔️ Competitive programming

📖 Computer Books

🔴 Video Tutorials

💻 Online Compiler and Sharing Code snippets

📝 Blogs of Developers