GitHub 1C Enterprise Daily Trending

unitreerobotics/unitree_rl_gym

2024-12-19T01:28:39Z

Unitree RL GYM

This is a simple example of using Unitree Robots for reinforcement learning, including Unitree Go2, H1, H1_2, G1

Isaac Gym	Mujoco	Physical

1. Installation

Create a new python virtual env with python 3.8

Install pytorch 2.3.1 with cuda-12.1:

pip install torch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121

Install Isaac Gym
- Download and install Isaac Gym Preview 4 from https://developer.nvidia.com/isaac-gym
- cd isaacgym/python && pip install -e .
- Try running an example cd examples && python 1080_balls_of_solitude.py
- For troubleshooting check docs isaacgym/docs/index.html
Install rsl_rl (PPO implementation)
- Clone https://github.com/leggedrobotics/rsl_rl
- cd rsl_rl && git checkout v1.0.2 && pip install -e .
Install unitree_rl_gym
- Navigate to the folder unitree_rl_gym
- pip install -e .
Install unitree_sdk2py (Optional for depoly on real robot)
- Clone https://github.com/unitreerobotics/unitree_sdk2_python
- cd unitree_sdk2_python & pip install -e .

2. Train in Isaac Gym

Train: python legged_gym/scripts/train.py --task=go2
- To run on CPU add following arguments: --sim_device=cpu, --rl_device=cpu (sim on CPU and rl on GPU is possible).
- To run headless (no rendering) add --headless.
- Important : To improve performance, once the training starts press v to stop the rendering. You can then enable it later to check the progress.
- The trained policy is saved in logs/<experiment_name>/<date_time>_<run_name>/model_<iteration>.pt. Where <experiment_name> and <run_name> are defined in the train config.
- The following command line arguments override the values set in the config files:
- --task TASK: Task name.
- --resume: Resume training from a checkpoint
- --experiment_name EXPERIMENT_NAME: Name of the experiment to run or load.
- --run_name RUN_NAME: Name of the run.
- --load_run LOAD_RUN: Name of the run to load when resume=True. If -1: will load the last run.
- --checkpoint CHECKPOINT: Saved model checkpoint number. If -1: will load the last checkpoint.
- --num_envs NUM_ENVS: Number of environments to create.
- --seed SEED: Random seed.
- --max_iterations MAX_ITERATIONS: Maximum number of training iterations.
Play:python legged_gym/scripts/play.py --task=go2
- By default, the loaded policy is the last model of the last run of the experiment folder.
- Other runs/model iteration can be selected by setting load_run and checkpoint in the train config.

2.1 Play Demo

Go2	G1	H1	H1_2

3. Sim in Mujoco

3.1 Mujoco Usage

To execute sim2sim in mujoco, execute the following command:

python deploy/deploy_mujoco/deploy_mujoco.py {config_name}

config_name: The file name of the configuration file. The configuration file will be found under deploy/deploy_mujoco/configs/, for example g1.yaml, h1.yaml, h1_2.yaml.

example:

python deploy/deploy_mujoco/deploy_mujoco.py g1.yaml

3.2 Mujoco Demo

G1	H1	H1_2

4. Depoly on Physical Robot

reference to Deploy on Physical Robot(English) | 实物部署（简体中文）

seleniumbase/SeleniumBase

2024-12-19T01:28:39Z

Python APIs for web automation, testing, and bypassing bot-detection.

SeleniumBase

All-in-one Browser Automation Framework:
Web Crawling / Testing / Scraping / Stealth

SeleniumBase is the professional toolkit for web automation activities. Built for testing websites, bypassing CAPTCHAs, enhancing productivity, completing tasks, and scaling your business.

📚 Learn from over 200 examples in the SeleniumBase/examples/ folder.

🐙 Note that UC Mode / CDP Mode (Stealth Mode) have their own ReadMe files.

ℹ️ Most scripts run with raw python, although some scripts use Syntax Formats that expect pytest (a Python unit-testing framework included with SeleniumBase that can discover, collect, and run tests automatically).

📗 Here's raw_google.py, which performs a Google search:

from seleniumbase import SB

with SB(test=True) as sb:
    sb.open("https://google.com/ncr")
    sb.type('[title="Search"]', "SeleniumBase GitHub page\n")
    sb.click('[href*="github.com/seleniumbase/"]')
    sb.save_screenshot_to_logs()  # ./latest_logs/
    print(sb.get_page_title())

python raw_google.py

📗 Here's an example of bypassing Cloudflare's challenge page: SeleniumBase/examples/cdp_mode/raw_gitlab.py

from seleniumbase import SB

with SB(uc=True, test=True, locale_code="en") as sb:
    url = "https://gitlab.com/users/sign_in"
    sb.activate_cdp_mode(url)
    sb.uc_gui_click_captcha()
    sb.sleep(2)

📗 Here's test_get_swag.py, which tests an e-commerce site:

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)  # Call pytest

class MyTestClass(BaseCase):
    def test_swag_labs(self):
        self.open("https://www.saucedemo.com")
        self.type("#user-name", "standard_user")
        self.type("#password", "secret_sauce\n")
        self.assert_element("div.inventory_list")
        self.click('button[name*="backpack"]')
        self.click("#shopping_cart_container a")
        self.assert_text("Backpack", "div.cart_item")
        self.click("button#checkout")
        self.type("input#first-name", "SeleniumBase")
        self.type("input#last-name", "Automation")
        self.type("input#postal-code", "77123")
        self.click("input#continue")
        self.click("button#finish")
        self.assert_text("Thank you for your order!")

pytest test_get_swag.py

(The default browser is --chrome if not set.)

📗 Here's test_coffee_cart.py, which verifies an e-commerce site:

pytest test_coffee_cart.py --demo

(--demo mode slows down tests and highlights actions)

📗 Here's test_demo_site.py, which covers several actions:

pytest test_demo_site.py

Easy to type, click, select, toggle, drag & drop, and more.

(For more examples, see the SeleniumBase/examples/ folder.)

Explore the README:

Get Started / Installation

Basic Example / Usage

Common Test Methods

Fun Facts / Learn More

Demo Mode / Debugging

Command-line Options

Directory Configuration

SeleniumBase Dashboard

Generating Test Reports

▶️ How is SeleniumBase different from raw Selenium? (click to expand)

💡 SeleniumBase is a Python framework for browser automation and testing. SeleniumBase uses Selenium/WebDriver APIs and incorporates test-runners such as pytest, pynose, and behave to provide organized structure, test discovery, test execution, test state (eg. passed, failed, or skipped), and command-line options for changing default settings (eg. browser selection). With raw Selenium, you would need to set up your own options-parser for configuring tests from the command-line.

💡 SeleniumBase's driver manager gives you more control over automatic driver downloads. (Use --driver-version=VER with your pytest run command to specify the version.) By default, SeleniumBase will download a driver version that matches your major browser version if not set.

💡 SeleniumBase automatically detects between CSS Selectors and XPath, which means you don't need to specify the type of selector in your commands (but optionally you could).

💡 SeleniumBase methods often perform multiple actions in a single method call. For example, self.type(selector, text) does the following:
1. Waits for the element to be visible.
2. Waits for the element to be interactive.
3. Clears the text field.
4. Types in the new text.
5. Presses Enter/Submit if the text ends in "\n".
With raw Selenium, those actions require multiple method calls.

💡 SeleniumBase uses default timeout values when not set:
✅ self.click("button")
With raw Selenium, methods would fail instantly (by default) if an element needed more time to load:
❌ self.driver.find_element(by="css selector", value="button").click()
(Reliable code is better than unreliable code.)

💡 SeleniumBase lets you change the explicit timeout values of methods:
✅ self.click("button", timeout=10)
With raw Selenium, that requires more code:
❌ WebDriverWait(driver, 10).until(EC.element_to_be_clickable("css selector", "button")).click()
(Simple code is better than complex code.)

💡 SeleniumBase gives you clean error output when a test fails. With raw Selenium, error messages can get very messy.

💡 SeleniumBase gives you the option to generate a dashboard and reports for tests. It also saves screenshots from failing tests to the ./latest_logs/ folder. Raw Selenium does not have these options out-of-the-box.

💡 SeleniumBase includes desktop GUI apps for running tests, such as SeleniumBase Commander for pytest and SeleniumBase Behave GUI for behave.

💡 SeleniumBase has its own Recorder / Test Generator for creating tests from manual browser actions.

💡 SeleniumBase comes with test case management software, ("CasePlans"), for organizing tests and step descriptions.

💡 SeleniumBase includes tools for building data apps, ("ChartMaker"), which can generate JavaScript from Python.

📚 Learn about different ways of writing tests:

📗📝 Here's test_simple_login.py, which uses BaseCase class inheritance, and runs with pytest or pynose. (Use self.driver to access Selenium's raw driver.)

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)

class TestSimpleLogin(BaseCase):
    def test_simple_login(self):
        self.open("seleniumbase.io/simple/login")
        self.type("#username", "demo_user")
        self.type("#password", "secret_pass")
        self.click('a:contains("Sign in")')
        self.assert_exact_text("Welcome!", "h1")
        self.assert_element("img#image1")
        self.highlight("#image1")
        self.click_link("Sign out")
        self.assert_text("signed out", "#top_message")

📘📝 Here's raw_login_sb.py, which uses the SB Context Manager. Runs with pure python. (Use sb.driver to access Selenium's raw driver.)

from seleniumbase import SB

with SB() as sb:
    sb.open("seleniumbase.io/simple/login")
    sb.type("#username", "demo_user")
    sb.type("#password", "secret_pass")
    sb.click('a:contains("Sign in")')
    sb.assert_exact_text("Welcome!", "h1")
    sb.assert_element("img#image1")
    sb.highlight("#image1")
    sb.click_link("Sign out")
    sb.assert_text("signed out", "#top_message")

📙📝 Here's raw_login_driver.py, which uses the Driver Manager. Runs with pure python. (The driver is an improved version of Selenium's raw driver, with more methods.)

from seleniumbase import Driver

driver = Driver()
try:
    driver.open("seleniumbase.io/simple/login")
    driver.type("#username", "demo_user")
    driver.type("#password", "secret_pass")
    driver.click('a:contains("Sign in")')
    driver.assert_exact_text("Welcome!", "h1")
    driver.assert_element("img#image1")
    driver.highlight("#image1")
    driver.click_link("Sign out")
    driver.assert_text("signed out", "#top_message")
finally:
    driver.quit()

Set up Python & Git:

🔵 Add Python and Git to your System PATH.

🔵 Using a Python virtual env is recommended.

Install SeleniumBase:

You can install seleniumbase from PyPI or GitHub:

🔵 How to install seleniumbase from PyPI:

pip install seleniumbase

(Add --upgrade OR -U to upgrade SeleniumBase.)
(Add --force-reinstall to upgrade indirect packages.)
(Use pip3 if multiple versions of Python are present.)

🔵 How to install seleniumbase from a GitHub clone:

git clone https://github.com/seleniumbase/SeleniumBase.git
cd SeleniumBase/
pip install -e .

🔵 How to upgrade an existing install from a GitHub clone:

git pull
pip install -e .

🔵 Type seleniumbase or sbase to verify that SeleniumBase was installed successfully:

 ___      _          _             ___              
/ __| ___| |___ _ _ (_)_  _ _ __  | _ ) __ _ ______ 
\__ \/ -_) / -_) ' \| | \| | '  \ | _ \/ _` (_-< -_)
|___/\___|_\___|_||_|_|\_,_|_|_|_\|___/\__,_/__|___|
----------------------------------------------------

╭──────────────────────────────────────────────────╮
│  * USAGE: "seleniumbase [COMMAND] [PARAMETERS]"  │
│  *    OR:        "sbase [COMMAND] [PARAMETERS]"  │
│                                                  │
│ COMMANDS:        PARAMETERS / DESCRIPTIONS:      │
│    get / install    [DRIVER_NAME] [OPTIONS]      │
│    methods          (List common Python methods) │
│    options          (List common pytest options) │
│    behave-options   (List common behave options) │
│    gui / commander  [OPTIONAL PATH or TEST FILE] │
│    behave-gui       (SBase Commander for Behave) │
│    caseplans        [OPTIONAL PATH or TEST FILE] │
│    mkdir            [DIRECTORY] [OPTIONS]        │
│    mkfile           [FILE.py] [OPTIONS]          │
│    mkrec / codegen  [FILE.py] [OPTIONS]          │
│    recorder         (Open Recorder Desktop App.) │
│    record           (If args: mkrec. Else: App.) │
│    mkpres           [FILE.py] [LANG]             │
│    mkchart          [FILE.py] [LANG]             │
│    print            [FILE] [OPTIONS]             │
│    translate        [SB_FILE.py] [LANG] [ACTION] │
│    convert          [WEBDRIVER_UNITTEST_FILE.py] │
│    extract-objects  [SB_FILE.py]                 │
│    inject-objects   [SB_FILE.py] [OPTIONS]       │
│    objectify        [SB_FILE.py] [OPTIONS]       │
│    revert-objects   [SB_FILE.py] [OPTIONS]       │
│    encrypt / obfuscate                           │
│    decrypt / unobfuscate                         │
│    proxy            (Start a basic proxy server) │
│    download server  (Get Selenium Grid JAR file) │
│    grid-hub         [start|stop] [OPTIONS]       │
│    grid-node        [start|stop] --hub=[HOST/IP] │
│                                                  │
│ *  EXAMPLE => "sbase get chromedriver stable"    │
│ *  For command info => "sbase help [COMMAND]"    │
│ *  For info on all commands => "sbase --help"    │
╰──────────────────────────────────────────────────╯

🔵 Downloading webdrivers:

✅ SeleniumBase automatically downloads webdrivers as needed, such as chromedriver.

▶️ Here's sample output from a chromedriver download. (click to expand)

*** chromedriver to download = 131.0.6778.108 (Latest Stable) 

Downloading chromedriver-mac-arm64.zip from:
https://storage.googleapis.com/chrome-for-testing-public/131.0.6778.108/mac-arm64/chromedriver-mac-arm64.zip ...
Download Complete!

Extracting ['chromedriver'] from chromedriver-mac-arm64.zip ...
Unzip Complete!

The file [chromedriver] was saved to:
~/github/SeleniumBase/seleniumbase/drivers/
chromedriver

Making [chromedriver 131.0.6778.108] executable ...
[chromedriver 131.0.6778.108] is now ready for use!

Basic Example / Usage:

🔵 If you've cloned SeleniumBase, you can run tests from the examples/ folder.

Here's my_first_test.py:

cd examples/
pytest my_first_test.py

Here's the full code for my_first_test.py:

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)

class MyTestClass(BaseCase):
    def test_swag_labs(self):
        self.open("https://www.saucedemo.com")
        self.type("#user-name", "standard_user")
        self.type("#password", "secret_sauce\n")
        self.assert_element("div.inventory_list")
        self.assert_exact_text("Products", "span.title")
        self.click('button[name*="backpack"]')
        self.click("#shopping_cart_container a")
        self.assert_exact_text("Your Cart", "span.title")
        self.assert_text("Backpack", "div.cart_item")
        self.click("button#checkout")
        self.type("#first-name", "SeleniumBase")
        self.type("#last-name", "Automation")
        self.type("#postal-code", "77123")
        self.click("input#continue")
        self.assert_text("Checkout: Overview")
        self.assert_text("Backpack", "div.cart_item")
        self.assert_text("29.99", "div.inventory_item_price")
        self.click("button#finish")
        self.assert_exact_text("Thank you for your order!", "h2")
        self.assert_element('img[alt="Pony Express"]')
        self.js_click("a#logout_sidebar_link")
        self.assert_element("div#login_button_container")

By default, CSS Selectors are used for finding page elements.
If you're new to CSS Selectors, games like CSS Diner can help you learn.
For more reading, here's an advanced guide on CSS attribute selectors.

Here are some common SeleniumBase methods:

self.open(url)  # Navigate the browser window to the URL.
self.type(selector, text)  # Update the field with the text.
self.click(selector)  # Click the element with the selector.
self.click_link(link_text)  # Click the link containing text.
self.go_back()  # Navigate back to the previous URL.
self.select_option_by_text(dropdown_selector, option)
self.hover_and_click(hover_selector, click_selector)
self.drag_and_drop(drag_selector, drop_selector)
self.get_text(selector)  # Get the text from the element.
self.get_current_url()  # Get the URL of the current page.
self.get_page_source()  # Get the HTML of the current page.
self.get_attribute(selector, attribute)  # Get element attribute.
self.get_title()  # Get the title of the current page.
self.switch_to_frame(frame)  # Switch into the iframe container.
self.switch_to_default_content()  # Leave the iframe container.
self.open_new_window()  # Open a new window in the same browser.
self.switch_to_window(window)  # Switch to the browser window.
self.switch_to_default_window()  # Switch to the original window.
self.get_new_driver(OPTIONS)  # Open a new driver with OPTIONS.
self.switch_to_driver(driver)  # Switch to the browser driver.
self.switch_to_default_driver()  # Switch to the original driver.
self.wait_for_element(selector)  # Wait until element is visible.
self.is_element_visible(selector)  # Return element visibility.
self.is_text_visible(text, selector)  # Return text visibility.
self.sleep(seconds)  # Do nothing for the given amount of time.
self.save_screenshot(name)  # Save a screenshot in .png format.
self.assert_element(selector)  # Verify the element is visible.
self.assert_text(text, selector)  # Verify text in the element.
self.assert_exact_text(text, selector)  # Verify text is exact.
self.assert_title(title)  # Verify the title of the web page.
self.assert_downloaded_file(file)  # Verify file was downloaded.
self.assert_no_404_errors()  # Verify there are no broken links.
self.assert_no_js_errors()  # Verify there are no JS errors.

🔵 For the complete list of SeleniumBase methods, see: Method Summary

Fun Facts / Learn More:

✅ SeleniumBase automatically handles common WebDriver actions such as launching web browsers before tests, saving screenshots during failures, and closing web browsers after tests.

✅ SeleniumBase lets you customize tests via command-line options.

✅ SeleniumBase uses simple syntax for commands. Example:

self.type("input", "dogs\n")  # (The "\n" presses ENTER)

Most SeleniumBase scripts can be run with pytest, pynose, or pure python. Not all test runners can run all test formats. For example, tests that use the sb pytest fixture can only be run with pytest. (See Syntax Formats) There's also a Gherkin test format that runs with behave.

pytest coffee_cart_tests.py --rs
pytest test_sb_fixture.py --demo
pytest test_suite.py --rs --html=report.html --dashboard

pynose basic_test.py --mobile
pynose test_suite.py --headless --report --show-report

python raw_sb.py
python raw_test_scripts.py

behave realworld.feature
behave calculator.feature -D rs -D dashboard

✅ pytest includes automatic test discovery. If you don't specify a specific file or folder to run, pytest will automatically search through all subdirectories for tests to run based on the following criteria:

Python files that start with test_ or end with _test.py.
Python methods that start with test_.

With a SeleniumBase pytest.ini file present, you can modify default discovery settings. The Python class name can be anything because seleniumbase.BaseCase inherits unittest.TestCase to trigger autodiscovery.

✅ You can do a pre-flight check to see which tests would get discovered by pytest before the actual run:

pytest --co -q

✅ You can be more specific when calling pytest or pynose on a file:

pytest [FILE_NAME.py]::[CLASS_NAME]::[METHOD_NAME]

pynose [FILE_NAME.py]:[CLASS_NAME].[METHOD_NAME]

✅ No More Flaky Tests! SeleniumBase methods automatically wait for page elements to finish loading before interacting with them (up to a timeout limit). This means you no longer need random time.sleep() statements in your scripts.

✅ SeleniumBase supports all major browsers and operating systems:

Browsers: Chrome, Edge, Firefox, and Safari.

Systems: Linux/Ubuntu, macOS, and Windows.

✅ SeleniumBase works on all popular CI/CD platforms:

✅ SeleniumBase includes an automated/manual hybrid solution called MasterQA to speed up manual testing with automation while manual testers handle validation.

✅ SeleniumBase supports running tests while offline (assuming webdrivers have previously been downloaded when online).

✅ For a full list of SeleniumBase features, Click Here.

Demo Mode / Debugging:

🔵 Demo Mode helps you see what a test is doing. If a test is moving too fast for your eyes, run it in Demo Mode to pause the browser briefly between actions, highlight page elements being acted on, and display assertions:

pytest my_first_test.py --demo

🔵 time.sleep(seconds) can be used to make a test wait at a specific spot:

import time; time.sleep(3)  # Do nothing for 3 seconds.

🔵 Debug Mode with Python's built-in pdb library helps you debug tests:

import pdb; pdb.set_trace()
import pytest; pytest.set_trace()
breakpoint()  # Shortcut for "import pdb; pdb.set_trace()"

(pdb commands: n, c, s, u, d => next, continue, step, up, down)

🔵 To pause an active test that throws an exception or error, (and keep the browser window open while Debug Mode begins in the console), add --pdb as a pytest option:

pytest test_fail.py --pdb

🔵 To start tests in Debug Mode, add --trace as a pytest option:

pytest test_coffee_cart.py --trace

🔵 Command-line Options:

✅ Here are some useful command-line options that come with pytest:

-v  # Verbose mode. Prints the full name of each test and shows more details.
-q  # Quiet mode. Print fewer details in the console output when running tests.
-x  # Stop running the tests after the first failure is reached.
--html=report.html  # Creates a detailed pytest-html report after tests finish.
--co | --collect-only  # Show what tests would get run. (Without running them)
--co -q  # (Both options together!) - Do a dry run with full test names shown.
-n=NUM  # Multithread the tests using that many threads. (Speed up test runs!)
-s  # See print statements. (Should be on by default with pytest.ini present.)
--junit-xml=report.xml  # Creates a junit-xml report after tests finish.
--pdb  # If a test fails, enter Post Mortem Debug Mode. (Don't use with CI!)
--trace  # Enter Debug Mode at the beginning of each test. (Don't use with CI!)
-m=MARKER  # Run tests with the specified pytest marker.

✅ SeleniumBase provides additional pytest command-line options for tests:

--browser=BROWSER  # (The web browser to use. Default: "chrome".)
--chrome  # (Shortcut for "--browser=chrome". On by default.)
--edge  # (Shortcut for "--browser=edge".)
--firefox  # (Shortcut for "--browser=firefox".)
--safari  # (Shortcut for "--browser=safari".)
--settings-file=FILE  # (Override default SeleniumBase settings.)
--env=ENV  # (Set the test env. Access with "self.env" in tests.)
--account=STR  # (Set account. Access with "self.account" in tests.)
--data=STRING  # (Extra test data. Access with "self.data" in tests.)
--var1=STRING  # (Extra test data. Access with "self.var1" in tests.)
--var2=STRING  # (Extra test data. Access with "self.var2" in tests.)
--var3=STRING  # (Extra test data. Access with "self.var3" in tests.)
--variables=DICT  # (Extra test data. Access with "self.variables".)
--user-data-dir=DIR  # (Set the Chrome user data directory to use.)
--protocol=PROTOCOL  # (The Selenium Grid protocol: http|https.)
--server=SERVER  # (The Selenium Grid server/IP used for tests.)
--port=PORT  # (The Selenium Grid port used by the test server.)
--cap-file=FILE  # (The web browser's desired capabilities to use.)
--cap-string=STRING  # (The web browser's desired capabilities to use.)
--proxy=SERVER:PORT  # (Connect to a proxy server:port as tests are running)
--proxy=USERNAME:PASSWORD@SERVER:PORT  # (Use an authenticated proxy server)
--proxy-bypass-list=STRING # (";"-separated hosts to bypass, Eg "*.foo.com")
--proxy-pac-url=URL  # (Connect to a proxy server using a PAC_URL.pac file.)
--proxy-pac-url=USERNAME:PASSWORD@URL  # (Authenticated proxy with PAC URL.)
--proxy-driver  # (If a driver download is needed, will use: --proxy=PROXY.)
--multi-proxy  # (Allow multiple authenticated proxies when multi-threaded.)
--agent=STRING  # (Modify the web browser's User-Agent string.)
--mobile  # (Use the mobile device emulator while running tests.)
--metrics=STRING  # (Set mobile metrics: "CSSWidth,CSSHeight,PixelRatio".)
--chromium-arg="ARG=N,ARG2"  # (Set Chromium args, ","-separated, no spaces.)
--firefox-arg="ARG=N,ARG2"  # (Set Firefox args, comma-separated, no spaces.)
--firefox-pref=SET  # (Set a Firefox preference:value set, comma-separated.)
--extension-zip=ZIP  # (Load a Chrome Extension .zip|.crx, comma-separated.)
--extension-dir=DIR  # (Load a Chrome Extension directory, comma-separated.)
--disable-features="F1,F2"  # (Disable features, comma-separated, no spaces.)
--binary-location=PATH  # (Set path of the Chromium browser binary to use.)
--driver-version=VER  # (Set the chromedriver or uc_driver version to use.)
--sjw  # (Skip JS Waits for readyState to be "complete" or Angular to load.)
--wfa  # (Wait for AngularJS to be done loading after specific web actions.)
--pls=PLS  # (Set pageLoadStrategy on Chrome: "normal", "eager", or "none".)
--headless  # (The default headless mode. Linux uses this mode by default.)
--headless1  # (Use Chrome's old headless mode. Fast, but has limitations.)
--headless2  # (Use Chrome's new headless mode, which supports extensions.)
--headed  # (Run tests in headed/GUI mode on Linux OS, where not default.)
--xvfb  # (Run tests using the Xvfb virtual display server on Linux OS.)
--xvfb-metrics=STRING  # (Set Xvfb display size on Linux: "Width,Height".)
--locale=LOCALE_CODE  # (Set the Language Locale Code for the web browser.)
--interval=SECONDS  # (The autoplay interval for presentations & tour steps)
--start-page=URL  # (The starting URL for the web browser when tests begin.)
--archive-logs  # (Archive existing log files instead of deleting them.)
--archive-downloads  # (Archive old downloads instead of deleting them.)
--time-limit=SECONDS  # (Safely fail any test that exceeds the time limit.)
--slow  # (Slow down the automation. Faster than using Demo Mode.)
--demo  # (Slow down and visually see test actions as they occur.)
--demo-sleep=SECONDS  # (Set the wait time after Slow & Demo Mode actions.)
--highlights=NUM  # (Number of highlight animations for Demo Mode actions.)
--message-duration=SECONDS  # (The time length for Messenger alerts.)
--check-js  # (Check for JavaScript errors after page loads.)
--ad-block  # (Block some types of display ads from loading.)
--host-resolver-rules=RULES  # (Set host-resolver-rules, comma-separated.)
--block-images  # (Block images from loading during tests.)
--do-not-track  # (Indicate to websites that you don't want to be tracked.)
--verify-delay=SECONDS  # (The delay before MasterQA verification checks.)
--ee | --esc-end  # (Lets the user end the current test via the ESC key.)
--recorder  # (Enables the Recorder for turning browser actions into code.)
--rec-behave  # (Same as Recorder Mode, but also generates behave-gherkin.)
--rec-sleep  # (If the Recorder is enabled, also records self.sleep calls.)
--rec-print  # (If the Recorder is enabled, prints output after tests end.)
--disable-cookies  # (Disable Cookies on websites. Pages might break!)
--disable-js  # (Disable JavaScript on websites. Pages might break!)
--disable-csp  # (Disable the Content Security Policy of websites.)
--disable-ws  # (Disable Web Security on Chromium-based browsers.)
--enable-ws  # (Enable Web Security on Chromium-based browsers.)
--enable-sync  # (Enable "Chrome Sync" on websites.)
--uc | --undetected  # (Use undetected-chromedriver to evade bot-detection.)
--uc-cdp-events  # (Capture CDP events when running in "--undetected" mode.)
--log-cdp  # ("goog:loggingPrefs", {"performance": "ALL", "browser": "ALL"})
--remote-debug  # (Sync to Chrome Remote Debugger chrome://inspect/#devices)
--ftrace | --final-trace  # (Debug Mode after each test. Don't use with CI!)
--dashboard  # (Enable the SeleniumBase Dashboard. Saved at: dashboard.html)
--dash-title=STRING  # (Set the title shown for the generated dashboard.)
--enable-3d-apis  # (Enables WebGL and 3D APIs.)
--swiftshader  # (Chrome "--use-gl=angle" / "--use-angle=swiftshader-webgl")
--incognito  # (Enable Chrome's Incognito mode.)
--guest  # (Enable Chrome's Guest mode.)
--dark  # (Enable Chrome's Dark mode.)
--devtools  # (Open Chrome's DevTools when the browser opens.)
--rs | --reuse-session  # (Reuse browser session for all tests.)
--rcs | --reuse-class-session  # (Reuse session for tests in class.)
--crumbs  # (Delete all cookies between tests reusing a session.)
--disable-beforeunload  # (Disable the "beforeunload" event on Chrome.)
--window-position=X,Y  # (Set the browser's starting window position.)
--window-size=WIDTH,HEIGHT  # (Set the browser's starting window size.)
--maximize  # (Start tests with the browser window maximized.)
--screenshot  # (Save a screenshot at the end of each test.)
--no-screenshot  # (No screenshots saved unless tests directly ask it.)
--visual-baseline  # (Set the visual baseline for Visual/Layout tests.)
--wire  # (Use selenium-wire's webdriver for replacing selenium webdriver.)
--external-pdf  # (Set Chromium "plugins.always_open_pdf_externally":True.)
--timeout-multiplier=MULTIPLIER  # (Multiplies the default timeout values.)
--list-fail-page  # (After each failing test, list the URL of the failure.)

(See the full list of command-line option definitions here. For detailed examples of command-line options, see customizing_test_runs.md)

🔵 During test failures, logs and screenshots from the most recent test run will get saved to the latest_logs/ folder. Those logs will get moved to archived_logs/ if you add --archive_logs to command-line options, or have ARCHIVE_EXISTING_LOGS set to True in settings.py, otherwise log files with be cleaned up at the start of the next test run. The test_suite.py collection contains tests that fail on purpose so that you can see how logging works.

cd examples/

pytest test_suite.py --chrome

pytest test_suite.py --firefox

An easy way to override seleniumbase/config/settings.py is by using a custom settings file. Here's the command-line option to add to tests: (See examples/custom_settings.py) --settings_file=custom_settings.py (Settings include default timeout values, a two-factor auth key, DB credentials, S3 credentials, and other important settings used by tests.)

🔵 To pass additional data from the command-line to tests, add --data="ANY STRING". Inside your tests, you can use self.data to access that.

Directory Configuration:

🔵 When running tests with pytest, you'll want a copy of pytest.ini in your root folders. When running tests with pynose, you'll want a copy of setup.cfg in your root folders. These files specify default configuration details for tests. Test folders should also include a blank init.py file to allow your test files to import other files from that folder.

🔵 sbase mkdir DIR creates a folder with config files and sample tests:

sbase mkdir ui_tests

That new folder will have these files:

ui_tests/
├── __init__.py
├── my_first_test.py
├── parameterized_test.py
├── pytest.ini
├── requirements.txt
├── setup.cfg
├── test_demo_site.py
└── boilerplates/
    ├── __init__.py
    ├── base_test_case.py
    ├── boilerplate_test.py
    ├── classic_obj_test.py
    ├── page_objects.py
    ├── sb_fixture_test.py
    └── samples/
        ├── __init__.py
        ├── google_objects.py
        ├── google_test.py
        ├── sb_swag_test.py
        └── swag_labs_test.py

ProTip™: You can also create a boilerplate folder without any sample tests in it by adding -b or --basic to the sbase mkdir command:

sbase mkdir ui_tests --basic

That new folder will have these files:

ui_tests/
├── __init__.py
├── pytest.ini
├── requirements.txt
└── setup.cfg

Of those files, the pytest.ini config file is the most important, followed by a blank __init__.py file. There's also a setup.cfg file (for pynose). Finally, the requirements.txt file can be used to help you install seleniumbase into your environments (if it's not already installed).

Log files from failed tests:

Let's try an example of a test that fails:

""" test_fail.py """
from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)

class MyTestClass(BaseCase):

    def test_find_army_of_robots_on_xkcd_desert_island(self):
        self.open("https://xkcd.com/731/")
        self.assert_element("div#ARMY_OF_ROBOTS", timeout=1)  # This should fail

You can run it from the examples/ folder like this:

pytest test_fail.py

🔵 You'll notice that a logs folder, ./latest_logs/, was created to hold information (and screenshots) about the failing test. During test runs, past results get moved to the archived_logs folder if you have ARCHIVE_EXISTING_LOGS set to True in settings.py, or if your run tests with --archive-logs. If you choose not to archive existing logs, they will be deleted and replaced by the logs of the latest test run.

SeleniumBase Dashboard:

🔵 The --dashboard option for pytest generates a SeleniumBase Dashboard located at dashboard.html, which updates automatically as tests run and produce results. Example:

pytest --dashboard --rs --headless

🔵 Additionally, you can host your own SeleniumBase Dashboard Server on a port of your choice. Here's an example of that using Python's http.server:

python -m http.server 1948

🔵 Now you can navigate to http://localhost:1948/dashboard.html in order to view the dashboard as a web app. This requires two different terminal windows: one for running the server, and another for running the tests, which should be run from the same directory. (Use Ctrl+C to stop the http server.)

🔵 Here's a full example of what the SeleniumBase Dashboard may look like:

pytest test_suite.py test_image_saving.py --dashboard --rs --headless

Generating Test Reports:

🔵 `pytest` HTML Reports:

✅ Using --html=report.html gives you a fancy report of the name specified after your test suite completes.

pytest test_suite.py --html=report.html

✅ When combining pytest html reports with SeleniumBase Dashboard usage, the pie chart from the Dashboard will get added to the html report. Additionally, if you set the html report URL to be the same as the Dashboard URL when also using the dashboard, (example: --dashboard --html=dashboard.html), then the Dashboard will become an advanced html report when all the tests complete.

✅ Here's an example of an upgraded html report:

pytest test_suite.py --dashboard --html=report.html

If viewing pytest html reports in Jenkins, you may need to configure Jenkins settings for the html to render correctly. This is due to Jenkins CSP changes.

You can also use --junit-xml=report.xml to get an xml report instead. Jenkins can use this file to display better reporting for your tests.

pytest test_suite.py --junit-xml=report.xml

🔵 `pynose` Reports:

The --report option gives you a fancy report after your test suite completes.

pynose test_suite.py --report

(NOTE: You can add --show-report to immediately display pynose reports after the test suite completes. Only use --show-report when running tests locally because it pauses the test run.)

🔵 `behave` Dashboard & Reports:

(The behave_bdd/ folder can be found in the examples/ folder.)

behave behave_bdd/features/ -D dashboard -D headless

You can also use --junit to get .xml reports for each behave feature. Jenkins can use these files to display better reporting for your tests.

behave behave_bdd/features/ --junit -D rs -D headless

🔵 Allure Reports:

See: https://allurereport.org/docs/pytest/

SeleniumBase no longer includes allure-pytest as part of installed dependencies. If you want to use it, install it first:

pip install allure-pytest

Now your tests can create Allure results files, which can be processed by Allure Reports.

pytest test_suite.py --alluredir=allure_results

Using a Proxy Server:

If you wish to use a proxy server for your browser tests (Chromium or Firefox), you can add --proxy=IP_ADDRESS:PORT as an argument on the command line.

pytest proxy_test.py --proxy=IP_ADDRESS:PORT

If the proxy server that you wish to use requires authentication, you can do the following (Chromium only):

pytest proxy_test.py --proxy=USERNAME:PASSWORD@IP_ADDRESS:PORT

SeleniumBase also supports SOCKS4 and SOCKS5 proxies:

pytest proxy_test.py --proxy="socks4://IP_ADDRESS:PORT"

pytest proxy_test.py --proxy="socks5://IP_ADDRESS:PORT"

To make things easier, you can add your frequently-used proxies to PROXY_LIST in proxy_list.py, and then use --proxy=KEY_FROM_PROXY_LIST to use the IP_ADDRESS:PORT of that key.

pytest proxy_test.py --proxy=proxy1

Changing the User-Agent:

🔵 If you wish to change the User-Agent for your browser tests (Chromium and Firefox only), you can add --agent="USER AGENT STRING" as an argument on the command-line.

pytest user_agent_test.py --agent="Mozilla/5.0 (Nintendo 3DS; U; ; en) Version/1.7412.EU"

Handling Pop-Up Alerts:

🔵 self.accept_alert() automatically waits for and accepts alert pop-ups. self.dismiss_alert() automatically waits for and dismisses alert pop-ups. On occasion, some methods like self.click(SELECTOR) might dismiss a pop-up on its own because they call JavaScript to make sure that the readyState of the page is complete before advancing. If you're trying to accept a pop-up that got dismissed this way, use this workaround: Call self.find_element(SELECTOR).click() instead, (which will let the pop-up remain on the screen), and then use self.accept_alert() to accept the pop-up (more on that here). If pop-ups are intermittent, wrap code in a try/except block.

Building Guided Tours for Websites:

🔵 Learn about SeleniumBase Interactive Walkthroughs (in the examples/tour_examples/ folder). It's great for prototyping a website onboarding experience.

Production Environments & Integrations:

▶️ Here are some things you can do to set up a production environment for your testing. (click to expand)

You can set up a Jenkins build server for running tests at regular intervals. For a real-world Jenkins example of headless browser automation in action, check out the SeleniumBase Jenkins example on Azure or the SeleniumBase Jenkins example on Google Cloud.
You can use the Selenium Grid to scale your testing by distributing tests on several machines with parallel execution. To do this, check out the SeleniumBase selenium_grid folder, which should have everything you need, including the Selenium Grid ReadMe to help you get started.
If you're using the SeleniumBase MySQL feature to save results from tests running on a server machine, you can install MySQL Workbench to help you read & write from your DB more easily.
If you're using AWS, you can set up an Amazon S3 account for saving log files and screenshots from your tests. To activate this feature, modify settings.py with connection details in the S3 section, and add --with-s3-logging on the command-line when running your tests.

Here's an example of running tests with some additional features enabled:

pytest [YOUR_TEST_FILE.py] --with-db-reporting --with-s3-logging

Detailed Method Specifications and Examples:

🔵 Navigating to a web page: (and related commands)

self.open("https://xkcd.com/378/")  # This method opens the specified page.

self.go_back()  # This method navigates the browser to the previous page.

self.go_forward()  # This method navigates the browser forward in history.

self.refresh_page()  # This method reloads the current page.

self.get_current_url()  # This method returns the current page URL.

self.get_page_source()  # This method returns the current page source.

ProTip™: You can use the self.get_page_source() method with Python's find() command to parse through HTML to find something specific. (For more advanced parsing, see the BeautifulSoup example.)

source = self.get_page_source()
head_open_tag = source.find('<head>')
head_close_tag = source.find('</head>', head_open_tag)
everything_inside_head = source[head_open_tag+len('<head>'):head_close_tag]

🔵 Clicking:

To click an element on the page:

self.click("div#my_id")

ProTip™: In most web browsers, you can right-click on a page and select Inspect Element to see the CSS selector details that you'll need to create your own scripts.

🔵 Typing Text:

self.type(selector, text) # updates the text from the specified element with the specified value. An exception is raised if the element is missing or if the text field is not editable. Example:

self.type("input#id_value", "2012")

You can also use self.add_text() or the WebDriver .send_keys() command, but those won't clear the text box first if there's already text inside.

🔵 Getting the text from an element on a page:

text = self.get_text("header h2")

🔵 Getting the attribute value from an element on a page:

attribute = self.get_attribute("#comic img", "title")

🔵 Asserting existence of an element on a page within some number of seconds:

self.wait_for_element_present("div.my_class", timeout=10)

(NOTE: You can also use: self.assert_element_present(ELEMENT))

🔵 Asserting visibility of an element on a page within some number of seconds:

self.wait_for_element_visible("a.my_class", timeout=5)

(NOTE: The short versions of that are self.find_element(ELEMENT) and self.assert_element(ELEMENT). The find_element() version returns the element.)

Since the line above returns the element, you can combine that with .click() as shown below:

self.find_element("a.my_class", timeout=5).click()

# But you're better off using the following statement, which does the same thing:

self.click("a.my_class")  # DO IT THIS WAY!

ProTip™: You can use dots to signify class names (Ex: div.class_name) as a simplified version of div[class="class_name"] within a CSS selector.

You can also use *= to search for any partial value in a CSS selector as shown below:

self.click('a[name*="partial_name"]')

🔵 Asserting visibility of text inside an element on a page within some number of seconds:

self.assert_text("Make it so!", "div#trek div.picard div.quotes")
self.assert_text("Tea. Earl Grey. Hot.", "div#trek div.picard div.quotes", timeout=3)

(NOTE: self.find_text(TEXT, ELEMENT) and self.wait_for_text(TEXT, ELEMENT) also do this. For backwards compatibility, older method names were kept, but the default timeout may be different.)

🔵 Asserting Anything:

self.assert_true(var1 == var2)

self.assert_false(var1 == var2)

self.assert_equal(var1, var2)

🔵 Useful Conditional Statements: (with creative examples)

❓ is_element_visible(selector): (visible on the page)

if self.is_element_visible('div#warning'):
    print("Red Alert: Something bad might be happening!")

❓ is_element_present(selector): (present in the HTML)

if self.is_element_present('div#top_secret img.tracking_cookie'):
    self.contact_cookie_monster()  # Not a real SeleniumBase method
else:
    current_url = self.get_current_url()
    self.contact_the_nsa(url=current_url, message="Dark Zone Found")  # Not a real SeleniumBase method

def is_there_a_cloaked_klingon_ship_on_this_page():
    if self.is_element_present("div.ships div.klingon"):
        return not self.is_element_visible("div.ships div.klingon")
    return False

❓ is_text_visible(text, selector): (text visible on element)

if self.is_text_visible("You Shall Not Pass!", "h1"):
    self.open("https://www.youtube.com/watch?v=3xYXUeSmb-Y")

▶️ Click for a longer example of is_text_visible():

def get_mirror_universe_captain_picard_superbowl_ad(superbowl_year):
    selector = "div.superbowl_%s div.commercials div.transcript div.picard" % superbowl_year
    if self.is_text_visible("Yes, it was I who summoned you all here.", selector):
        return "Picard Paramount+ Superbowl Ad 2020"
    elif self.is_text_visible("Commander, signal the following: Our Network is Secure!"):
        return "Picard Mirror Universe iboss Superbowl Ad 2018"
    elif self.is_text_visible("For the Love of Marketing and Earl Grey Tea!", selector):
        return "Picard Mirror Universe HubSpot Superbowl Ad 2015"
    elif self.is_text_visible("Delivery Drones... Engage", selector):
        return "Picard Mirror Universe Amazon Superbowl Ad 2015"
    elif self.is_text_visible("Bing it on Screen!", selector):
        return "Picard Mirror Universe Microsoft Superbowl Ad 2015"
    elif self.is_text_visible("OK Glass, Make it So!", selector):
        return "Picard Mirror Universe Google Superbowl Ad 2015"
    elif self.is_text_visible("Number One, I've Never Seen Anything Like It.", selector):
        return "Picard Mirror Universe Tesla Superbowl Ad 2015"
    elif self.is_text_visible("Let us make sure history never forgets the name ... Facebook", selector):
        return "Picard Mirror Universe Facebook Superbowl Ad 2015"
    elif self.is_text_visible("""With the first link, the chain is forged.
                              The first speech censored, the first thought forbidden,
                              the first freedom denied, chains us all irrevocably.""", selector):
        return "Picard Mirror Universe Wikimedia Superbowl Ad 2015"
    else:
        raise Exception("Reports of my assimilation are greatly exaggerated.")

❓ is_link_text_visible(link_text):

if self.is_link_text_visible("Stop! Hammer time!"):
    self.click_link("Stop! Hammer time!")

🔵 Switching Tabs:

If your test opens up a new tab/window, you can switch to it. (SeleniumBase automatically switches to new tabs that don't open to about:blank URLs.)

self.switch_to_window(1)  # This switches to the new tab (0 is the first one)

🔵 How to handle iframes:

🔵 iframes follow the same principle as new windows: You must first switch to the iframe if you want to perform actions in there:

self.switch_to_frame("iframe")
# ... Now perform actions inside the iframe
self.switch_to_parent_frame()  # Exit the current iframe

To exit from multiple iframes, use self.switch_to_default_content(). (If inside a single iframe, this has the same effect as self.switch_to_parent_frame().)

self.switch_to_frame('iframe[name="frame1"]')
self.switch_to_frame('iframe[name="frame2"]')
# ... Now perform actions inside the inner iframe
self.switch_to_default_content()  # Back to the main page

🔵 You can also use a context manager to act inside iframes:

with self.frame_switch("iframe"):
    # ... Now perform actions while inside the code block
# You have left the iframe

This also works with nested iframes:

with self.frame_switch('iframe[name="frame1"]'):
    with self.frame_switch('iframe[name="frame2"]'):
        # ... Now perform actions while inside the code block
    # You are now back inside the first iframe
# You have left all the iframes

🔵 How to execute custom jQuery scripts:

jQuery is a powerful JavaScript library that allows you to perform advanced actions in a web browser. If the web page you're on already has jQuery loaded, you can start executing jQuery scripts immediately. You'd know this because the web page would contain something like the following in the HTML:

<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.3/jquery.min.js"></script>

🔵 It's OK if you want to use jQuery on a page that doesn't have it loaded yet. To do so, run the following command first:

self.activate_jquery()

▶️ Here are some examples of using jQuery in your scripts. (click to expand)

self.execute_script("jQuery, window.scrollTo(0, 600)")  # Scrolling the page

self.execute_script("jQuery('#annoying-widget').hide()")  # Hiding elements on a page

self.execute_script("jQuery('#hidden-widget').show(0)")  # Showing hidden elements on a page

self.execute_script("jQuery('#annoying-button a').remove()")  # Removing elements on a page

self.execute_script("jQuery('%s').mouseover()" % (mouse_over_item))  # Mouse-over elements on a page

self.execute_script("jQuery('input#the_id').val('my_text')")  # Fast text input on a page

self.execute_script("jQuery('div#dropdown a.link').click()")  # Click elements on a page

self.execute_script("return jQuery('div#amazing')[0].text")  # Returns the css "text" of the element given

self.execute_script("return jQuery('textarea')[2].value")  # Returns the css "value" of the 3rd textarea element on the page

(Most of the above commands can be done directly with built-in SeleniumBase methods.)

🔵 How to handle a restrictive CSP:

❗ Some websites have a restrictive Content Security Policy to prevent users from loading jQuery and other external libraries onto their websites. If you need to use jQuery or another JS library on those websites, add --disable-csp as a pytest command-line option to load a Chromium extension that bypasses the CSP.

🔵 More JavaScript fun:

▶️ In this example, JavaScript creates a referral button on a page, which is then clicked. (click to expand)

start_page = "https://xkcd.com/465/"
destination_page = "https://github.com/seleniumbase/SeleniumBase"
self.open(start_page)
referral_link = '''<a class='analytics test' href='%s'>Free-Referral Button!</a>''' % destination_page
self.execute_script('''document.body.innerHTML = \"%s\"''' % referral_link)
self.click("a.analytics")  # Clicks the generated button

(Due to popular demand, this traffic generation example has been included in SeleniumBase with the self.generate_referral(start_page, end_page) and the self.generate_traffic(start_page, end_page, loops) methods.)

🔵 How to use deferred asserts:

Let's say you want to verify multiple different elements on a web page in a single test, but you don't want the test to fail until you verified several elements at once so that you don't have to rerun the test to find more missing elements on the same page. That's where deferred asserts come in. Here's an example:

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)

class DeferredAssertTests(BaseCase):
    def test_deferred_asserts(self):
        self.open("https://xkcd.com/993/")
        self.wait_for_element("#comic")
        self.deferred_assert_element('img[alt="Brand Identity"]')
        self.deferred_assert_element('img[alt="Rocket Ship"]')  # Will Fail
        self.deferred_assert_element("#comicmap")
        self.deferred_assert_text("Fake Item", "ul.comicNav")  # Will Fail
        self.deferred_assert_text("Random", "ul.comicNav")
        self.deferred_assert_element('a[name="Super Fake !!!"]')  # Will Fail
        self.deferred_assert_exact_text("Brand Identity", "#ctitle")
        self.deferred_assert_exact_text("Fake Food", "#comic")  # Will Fail
        self.process_deferred_asserts()

deferred_assert_element() and deferred_assert_text() will save any exceptions that would be raised. To flush out all the failed deferred asserts into a single exception, make sure to call self.process_deferred_asserts() at the end of your test method. If your test hits multiple pages, you can call self.process_deferred_asserts() before navigating to a new page so that the screenshot from your log files matches the URL where the deferred asserts were made.

🔵 How to access raw WebDriver:

If you need access to any commands that come with standard WebDriver, you can call them directly like this:

self.driver.delete_all_cookies()
capabilities = self.driver.capabilities
self.driver.find_elements("partial link text", "GitHub")

(In general, you'll want to use the SeleniumBase versions of methods when available.)

🔵 How to retry failing tests automatically:

You can use pytest --reruns=NUM to retry failing tests that many times. Add --reruns-delay=SECONDS to wait that many seconds between retries. Example:

pytest --reruns=1 --reruns-delay=1

You can use the @retry_on_exception() decorator to retry failing methods. (First import: from seleniumbase import decorators). To learn more about SeleniumBase decorators, click here.

"Catch bugs in QA before deploying code to Production!"

Wrap-Up

If you see something, say something!

https://github.com/mdmintz

luckjiawei/frpc-desktop

2024-12-19T01:28:39Z

Xiaomi Home Integration for Home Assistant

English | 简体中文

Xiaomi Home Integration is an integrated component of Home Assistant supported by Xiaomi official. It allows you to use Xiaomi IoT smart devices in Home Assistant.

Installation

Home Assistant version requirement:

Core $\geq$ 2024.11.0

Operating System $\geq$ 13.0

Method 1: Git clone from GitHub

cd config
git clone https://github.com/XiaoMi/ha_xiaomi_home.git
cd ha_xiaomi_home
./install.sh /config

We recommend this installation method, for it is convenient to switch to a tag when updating xiaomi_home to a certain version.

For example, update to version v1.0.0

cd config/ha_xiaomi_home
git checkout v1.0.0
./install.sh /config

Method 2: HACS

HACS > Overflow Menu > Custom repositories > Repository: https://github.com/XiaoMi/ha_xiaomi_home.git & Category: Integration > ADD

Xiaomi Home has not been added to the HACS store as a default yet. It's coming soon.

Method 3: Manually installation via Samba / FTPS

Download and copy custom_components/xiaomi_home folder to config/custom_components folder in your Home Assistant.

Configuration

Login

Settings > Devices & services > ADD INTEGRATION > Search Xiaomi Home > NEXT > Click here to login > Sign in with Xiaomi account

Add MIoT Devices

After logging in successfully, a dialog box named "Select Home and Devices" pops up. You can select the home containing the device that you want to import in Home Assistant.

Multiple User Login

After a Xiaomi account login and its user configuration are completed, you can continue to add other Xiaomi accounts in the configured Xiaomi Home Integration page.

Method: Settings > Devices & services > Configured > Xiaomi Home > ADD HUB > NEXT > Click here to login > Sign in with Xiaomi account

Update Configurations

You can change the configurations in the "Configuration Options" dialog box, in which you can update your user nickname and the list of the devices importing from Xiaomi Home APP, etc.

Method: Settings > Devices & services > Configured > Xiaomi Home > CONFIGURE > Select the option to update

Debug Mode for Action

You can manually send Action command message with parameters to the device when the debug mode for action is activated. The user interface for sending the Action command with parameters is shown as a Text entity.

Method: Settings > Devices & services > Configured > Xiaomi Home > CONFIGURE > Debug mode for action

Security

Xiaomi Home Integration and the affiliated cloud interface is provided by Xiaomi officially. You need to use your Xiaomi account to login to get your device list. Xiaomi Home Integration implements OAuth 2.0 login process, which does not keep your account password in the Home Assistant application. However, due to the limitation of the Home Assistant platform, the user information (including device information, certificates, tokens, etc.) of your Xiaomi account will be saved in the Home Assistant configuration file in clear text after successful login. You need to ensure that your Home Assistant configuration file is properly stored. The exposure of your configuration file may result in others logging in with your identity.

FAQ

Does Xiaomi Home Integration support all Xiaomi Home devices?

Xiaomi Home Integration currently supports most categories of Home device. Only a few categories are not supported. They are Bluetooth device, infrared device and virtual device.
Does Xiaomi Home Integration support multiple Xiaomi accounts?

Yes, it supports multiple Xiaomi accounts. Furthermore, Xiaomi Home Integration allows that devices belonging to different accounts can be added to a same area.
Does Xiaomi Home Integration support local control?

Local control is implemented by Xiaomi Central Hub Gateway (firmware version 3.4.0_0000 above) or Xiaomi home devices with built-in central hub gateway (software version 0.8.0 above) inside. If you do not have a Xiaomi central hub gateway or other devices having central hub gateway function, all control commands are sent through Xiaomi Cloud. The firmware for Xiaomi central hub gateway including the built-in central hub gateway supporting Home Assistant local control feature has not been released yet. Please refer to MIoT team's notification for upgrade plans.

Xiaomi central hub gateway is only available in mainland China. In other regions, it is not available.

Xiaomi Home Integration can also implement partial local control by enabling Xiaomi LAN control function. Xiaomi LAN control function can only control IP devices (devices connected to the router via WiFi or ethernet cable) in the same local area network as Home Assistant. It cannot control BLE Mesh, ZigBee, etc. devices. This function may cause some abnormalities. We recommend not to use this function. Xiaomi LAN control function is enabled by Settings > Devices & services > Configured > Xiaomi Home > CONFIGURE > Update LAN control configuration

Xiaomi LAN control function is not restricted by region. It is available in all regions. However, if there is a central gateway in the local area network where Home Assistant is located, even Xiaomi LAN control function is enabled in the integration, it will not take effect.
In which regions is Xiaomi Home Integration available?

Xiaomi Home Integration can be used in the mainland of China, Europe, India, Russia, Singapore, and USA. As user data in Xiaomi Cloud of different regions is isolated, you need to choose your region when importing MIoT devices in the configuration process. Xiaomi Home Integration allows you to import devices of different regions to a same area.

Principle of Messaging

Control through the Cloud

Image 1: Cloud control architecture

Xiaomi Home Integration subscribes to the interested device messages on the MQTT Broker in MIoT Cloud. When a device property changes or a device event occurs, the device sends an upstream message to MIoT Cloud, and the MQTT Broker pushes the subscribed device message to Xiaomi Home Integration. Because Xiaomi Home Integration does not need to poll to obtain the current device property value in the cloud, it can immediately receive the notification message when the properties change or the events occur. Thanks to the message subscription mechanism, Xiaomi Home Integration only queries the properties of all devices from the cloud once when the integration configuration is completed, which puts little access pressure on the cloud.

Xiaomi Home Integration sends command messages to the devices via the HTTP interface of MIoT Cloud to control devices. The device reacts and responds after receiving the downstream message sent forward by MIoT Cloud.

Control locally

Image 2: Local control architecture

Xiaomi central hub gateway contains a standard MQTT Broker, which implements a complete subscribe-publish mechanism. Xiaomi Home Integration subscribes to the interested device messages through Xiaomi central hub gateway. When a device property changes or a device event occurs, the device sends an upstream message to Xiaomi central hub gateway, and the MQTT Broker pushes the subscribed device message to Xiaomi Home Integration.

When Xiaomi Home Integration needs to control a device, it publishes a device command message to the MQTT Broker, which is then forwarded to the device by Xiaomi central hub gateway. The device reacts and responds after receiving the downstream message from the gateway.

Mapping Relationship between MIoT-Spec-V2 and Home Assistant Entity

MIoT-Spec-V2 is the abbreviation for MIoT Specification Version 2, which is an IoT protocol formulated by Xiaomi IoT platform to give a standard functional description of IoT devices. It includes function definition (referred to as data model by other IoT platforms), interaction model, message format, and encoding.

In MIoT-Spec-V2 protocol, a product is defined as a device. A device contains several services. A service may have some properties, events and actions. Xiaomi Home Integration creates Home Assistant entities according to MIoT-Spec-V2. The conversion relationship is as follows.

General Conversion

Property

format	access	value-list	value-range	Entity in Home Assistant
writable	string	-	-	Text
writable	bool	-	-	Switch
writable	not string & not bool	existent	-	Select
writable	not string & not bool	non-existent	existent	Number
not writable	-	-	-	Sensor

Event

MIoT-Spec-V2 event is transformed to Event entity in Home Assistant. The event's parameters are also passed to entity's _trigger_event.

Action

in	Entity in Home Assistant
empty	Button
not empty	Notify

If the debug mode for action is activated, the Text entity will be created when the "in" field in the action spec is not empty.

The "Attribute" item in the entity details page displays the format of the input parameter which is an ordered list, enclosed in square brackets []. The string elements in the list are enclosed in double quotation marks "".

For example, the "Attributes" item in the details page of the Notify entity converted by the "Intelligent Speaker Execute Text Directive" action of xiaomi.wifispeaker.s12 siid=5, aiid=5 instance shows the action params as [Text Content(str), Silent Execution(bool)]. A properly formatted input is ["Hello", true].

Specific Conversion

MIoT-Spec-V2 uses URN for defining types. The format is urn:<namespace>:<type>:<name>:<value>[:<vendor-product>:<version>], in which name is a human-readable word or phrase describing the instance of device, service, property, event and action. Xiaomi Home Integration first determines whether to convert the MIoT-Spec-V2 instance into a specific Home Assistant entity based on the instance's name. For the instance that does not meet the specific conversion rules, general conversion rules are used for conversion.

namespace is the namespace of MIoT-Spec-V2 instance. When its value is miot-spec-v2, it means that the specification is defined by Xiaomi. When its value is bluetooth-spec, it means that the specification is defined by Bluetooth Special Interest Group (SIG). When its value is not miot-spec-v2 or bluetooth-spec, it means that the specification is defined by other vendors. If MIoT-Spec-V2 namespace is not miot-spec-v2, a star mark * is added in front of the entity's name .

Device

The conversion follows SPEC_DEVICE_TRANS_MAP.

{
    '<device instance name>':{
        'required':{
            '<service instance name>':{
                'required':{
                    'properties': {
                        '<property instance name>': set<property access: str>
                    },
                    'events': set<event instance name: str>,
                    'actions': set<action instance name: str>
                },
                'optional':{
                    'properties': set<property instance name: str>,
                    'events': set<event instance name: str>,
                    'actions': set<action instance name: str>
                }
            }
        },
        'optional':{
            '<service instance name>':{
                'required':{
                    'properties': {
                        '<property instance name>': set<property access: str>
                    },
                    'events': set<event instance name: str>,
                    'actions': set<action instance name: str>
                },
                'optional':{
                    'properties': set<property instance name: str>,
                    'events': set<event instance name: str>,
                    'actions': set<action instance name: str>
                }
            }
        },
        'entity': str
    }
}

The "required" field under "device instance name" indicates the required services of the device. The "optional" field under "device instance name" indicates the optional services of the device. The "entity" field indicates the Home Assistant entity to be created. The "required" and the "optional" field under "service instance name" are required and optional properties, events and actions of the service respectively. The value of "property instance name" under "required" "properties" field is the access mode of the property. The condition for a successful match is that the value of "property instance name" is a subset of the access mode of the corresponding MIoT-Spec-V2 property instance.

Home Assistant entity will not be created if MIoT-Spec-V2 device instance does not contain all required services, properties, events or actions.

Service

The conversion follows SPEC_SERVICE_TRANS_MAP.

{
    '<service instance name>':{
        'required':{
            'properties': {
                '<property instance name>': set<property access: str>
            },
            'events': set<event instance name: str>,
            'actions': set<action instance name: str>
        },
        'optional':{
            'properties': set<property instance name: str>,
            'events': set<event instance name: str>,
            'actions': set<action instance name: str>
        },
        'entity': str
    }
}

The "required" field under "service instance name" indicates the required properties, events and actions of the service. The "optional" field indicates the optional properties, events and actions of the service. The "entity" field indicates the Home Assistant entity to be created. The value of "property instance name" under "required" "properties" field is the access mode of the property. The condition for a successful match is that the value of "property instance name" is a subset of the access mode of the corresponding MIoT-Spec-V2 property instance.

Home Assistant entity will not be created if MIoT-Spec-V2 service instance does not contain all required properties, events or actions.

Property

The conversion follows SPEC_PROP_TRANS_MAP.

{
    'entities':{
        '<entity name>':{
            'format': set<str>,
            'access': set<str>
        }
    },
    'properties': {
        '<property instance name>':{
            'device_class': str,
            'entity': str
        }
    }
}

The "format" field under "entity name" represents the data format of the property, and matching with one value indicates a successful match. The "access" field under "entity name" represents the access mode of the property, and matching with all values is considered a successful match.

The "entity" field under "property instance name", of which value is one of entity name under "entities" field, indicates the Home Assistant entity to be created. The "device_class" field under "property instance name" indicates the Home Assistant entity's _attr_device_class.

Event

The conversion follows SPEC_EVENT_TRANS_MAP.

{
    '<event instance name>': str
}

The value of the event instance name indicates _attr_device_class of the Home Assistant entity to be created.

MIoT-Spec-V2 Filter

spec_filter.json is used to filter out the MIoT-Spec-V2 instance that will not be converted to Home Assistant entity.

The format of spec_filter.json is as follows.

{
    "<MIoT-Spec-V2 device instance>":{
        "services": list<service_iid: str>,
        "properties": list<service_iid.property_iid: str>,
        "events": list<service_iid.event_iid: str>,
        "actions": list<service_iid.action_iid: str>,
    }
}

The key of spec_filter.json dictionary is the urn excluding the "version" field of the MIoT-Spec-V2 device instance. The firmware of different versions of the same product may be associated with the MIoT-Spec-V2 device instances of different versions. It is required that the MIoT-Spec-V2 instance of a higher version must contain all MIoT-Spec-V2 instances of the lower versions when a vendor defines the MIoT-Spec-V2 of its product on MIoT platform. Thus, the key of spec_filter.json does not need to specify the version number of MIoT-Spec-V2 device instance.

The value of "services", "properties", "events" or "actions" fields under "device instance" is the instance id (iid) of the service, property, event or action that will be ignored in the conversion process. Wildcard matching is supported.

Example:

{
    "urn:miot-spec-v2:device:television:0000A010:xiaomi-rmi1":{
        "services": ["*"]   # Filter out all services. It is equivalent to completely ignoring the device with such MIoT-Spec-V2 device instance.
    },
    "urn:miot-spec-v2:device:gateway:0000A019:xiaomi-hub1": {
        "services": ["3"],  # Filter out the service whose iid=3.
        "properties": ["4.*"]   # Filter out all properties in the service whose iid=4.
        "events": ["4.1"],  # Filter out the iid=1 event in the iid=4 service.
        "actions": ["4.1"]  # Filter out the iid=1 action in the iid=4 service.
    }
}

Device information service (urn:miot-spec-v2:service:device-information:00007801) of all devices will never be converted to Home Assistant entity.

Multiple Language Support

There are 8 languages available for selection in the config flow language option of Xiaomi Home, including Simplified Chinese, Traditional Chinese, English, Spanish, Russian, French, German, and Japanese. The config flow page in Simplified Chinese and English has been manually reviewed by the developer. Other languages are translated by machine translation. If you want to modify the words and sentences in the config flow page, you need to modify the json file of the certain language in custom_components/xiaomi_home/translations/ and custom_components/xiaomi_home/miot/i18n/ directory.

When displaying Home Assistant entity name, Xiaomi Home downloads the multiple language file configured by the device vendor from MIoT Cloud, which contains translations for MIoT-Spec-V2 instances of the device. multi_lang.json is a locally maintained multiple language dictionary, which has a higher priority than the multiple language file obtained from the cloud and can be used to supplement or modify the multiple language translation of devices.

The format of multi_lang.json is as follows.

{
    "<MIoT-Spec-V2 device instance>": {
        "<language code>": {
            "<instance code>": <translation: str>
        }
    }
}

The key of multi_lang.json dictionary is the urn excluding the "version" field of the MIoT-Spec-V2 device instance.

The language code is zh-Hans, zh-Hant, en, es, ru, fr, de, or ja, corresponding to the 8 selectable languages mentioned above.

The instance code is the code of the MIoT-Spec-V2 instance, which is in the format of:

service:<siid>                  # service
service:<siid>:property:<piid>  # property
service:<siid>:property:<piid>:valuelist:<value> # the value in value-list of a property
service:<siid>:event:<eiid>     # event
service:<siid>:action:<aiid>    # action

siid, piid, eiid, aiid and value are all decimal three-digit integers.

Example:

{
    "urn:miot-spec-v2:device:health-pot:0000A051:chunmi-a1": {
        "zh-Hant": {
            "service:002": "養生壺",
            "service:002:property:001": "工作狀態",
            "service:002:property:001:valuelist:000": "待機中",
            "service:002:action:002": "停止烹飪",
            "service:005:event:001": "烹飪完成"
        }
    }
}

If you edit specv2entity.py, spec_filter.json or multi_lang.json in the custom_components/xiaomi_home/miot/specs directory in your Home Assistant, you need to update the entity conversion rule in the integration's CONFIGURE page to take effect. Method: Settings > Devices & services > Configured > Xiaomi Home > CONFIGURE > Update Entity Conversion Rule

Documents

License
Contribution Guidelines: English | 简体中文
ChangeLog
Development Documents: https://developers.home-assistant.io/docs/creating_component_index

Directory Structure

miot: core code.
miot/miot_client: Adding a login user in the integration needs adding a miot_client instance.
miot/miot_cloud: Contains functions related to the cloud service, including OAuth login process, HTTP interface functions (to get the user information, to send the device control command, etc.)
miot/miot_device: Device entity, including device information, processing logic of property, event and action.
miot/miot_mips: Message bus for subscribing and publishing method.
miot/miot_spec: Parse MIoT-Spec-V2.
miot/miot_lan: Device LAN control, including device discovery, device control, etc.
miot/miot_mdns: Central hub gateway service LAN discovery.
miot/miot_network: Obtain network status and network information.
miot/miot_storage: File storage for the integration.
miot/test: Test scripts.
config_flow: Config flow.

dillongoostudios/goo-engine

2024-12-19T01:29:28Z

Run iOS app without actually installing it!

LiveContainer

Run iOS app without actually installing it!

Allows you to install unlimited apps (10 apps limit of free developer account do not apply here!), have multiple versions of an app installed and multiple data containers.
When JIT is available, codesign is entirely bypassed, no need to sign your apps before installing. Otherwise, app will be signed with the same certificate used by LiveContainer.

Compatibility

Unfortunately, not all apps work in LiveContainer, so we have a compatibility list to tell if there is apps that have issues. If they aren't on this list, then it's likely going run. However, if it doesn't work, please make an issue about it.

Usage

Requires AltStore or SideStore

Build from source or get prebuilt ipa in the Actions tab
Open LiveContainer, tap the plus icon in the upper right hand corner and select IPA files to install.
Choose the app you want to open in the next launch.
You can long-press the app to manage it.

Without JIT

Without JIT, guest apps need to be codesigned, which requires retrieving the certificate and password from SideStore or AltStore. This process involves applying a tweak to SideStore/AltStore, allowing it to expose the certificate to LiveContainer.

Open Settings in LiveContainer, tap "Patch SideStore/AltStore", and the app will switch to SideStore/AltStore to reinstall it with the tweak applied. If you use AltWidget, select "Keep Extension."
Wait for the installation to finish, then reopen SideStore/AltStore.
Return to LiveContainer and press "Test JIT-Less Mode." If it says "Test Passed," JIT-less mode is ready.
Install your app via the "Apps" tab.
Tap the run icon, it will attempt to restart LiveContainer with guest app loaded.

Note: If you update or reinstall SideStore/AltStore, you'll need to reapply the patch. Re-patch is not needed when you refresh your store.

With JIT (requires SideStore)

Tap the play icon, it will jump to SideStore and exit.
In SideStore, hold down LiveContainer and tap Enable JIT. If you have SideStore build supporting JIT URL scheme, it jumps back to LiveContainer with JIT enabled and the guest app is ready to use.

Add to Home Screen

Long press the app and you will see 2 ways to add your app to home screen:

Launch URL: Copy the provided URL, create a shortcut that opens the app, and add it to your home screen. You’ll need to create a separate shortcut for each app.
Create App Clip: Install an App Clip MDM profile, which adds the app to your home screen. Note that launching the app includes an extra jump, and the App Clip will remain in the app switcher.

Multiple LiveContainers

Using two LiveContainers allows you to run two different apps simultaneously, with almost seamless data transfer between the LiveContainers.

To install a second LiveContainer, go to Settings and tap "Install Another LiveContainer."

The first LiveContainer (blue icon) always launches by default. If an app is already running in the first container, you'll be prompted to either open it in the second LiveContainer (gray icon) or terminate the current app and relaunch it in the first. If the app is already running in the second container, it will switch automatically. To use an app in the second container, you must convert this app to a shared app. You can do that by opening the first LiveContainer (blue), long press on your app, open the settings of your app and then "Convert to Shared App". After that, you can launch your app using LiveContainer2 (grey).

Fix File Picker & Local Notification

Some apps may experience issues with their file pickers or not be able to apply for notification permission in LiveContainer. To resolve this, enable "Fix File Picker & Local Notification" in the app-specific settings.

"Open In App" Support

Tap the link icon in the top-right corner of the "Apps" tab and input the URL. LiveContainer will detect the appropriate app and ask if you want to launch it.
What's more, you can share a web page to LiveContainer using this shortcut. Be sure to add this shortcut to "Favorites" in share sheet actions.

JIT Support

To enable JIT for a guest app:

Enter your SideJITServer/JITStreamer address and device UDID in LiveContainer settings.
Enable "Launch with JIT" in its app-specific settings.
LiveContainer will communicate with the server before launching the app to enable JIT.

Installing external tweaks

LiveContainer comes with its own TweakLoader, which automatically load CydiaSubstrate and tweaks. TweakLoader is injected to every app you install. You can override TweakLoader.dylib symlink with your own implementation if you wish.

.dylib files in Tweaks folder are global, they are loaded to all apps. You can create app-specific tweaks folder and switch between them instantly.

To install tweaks, you can use the built-in tweak manager in LiveContainer, which will automatically sign tweaks as you import. Otherwise, you can manually add them and then use the tweak manager to sign them.

Hiding Apps

LiveContainer offers three levels of app hiding:

Lock Apps: In the app-specific settings, you can lock an app, requiring Face ID or a password to launch it.
Hide Locked Apps: Once locked, you can hide the app. It will appear as a placeholder in the "Hidden Apps" section, and authentication is needed to view or launch it.
Strict Hiding Mode: For complete invisibility, enable "Strict Hiding Mode" in LiveContainer's settings after authentication. Hidden apps will no longer appear in the interface. To access them, triple-tap the "x Apps in total" text on the main screen and authenticate to unlock the apps when you relaunch LiveContainer.

Building

export THEOS=/path/to/theos
git submodule update --init --recursive
make package

Project structure

Main executable

Core of LiveContainer
Contains the logic of setting up guest environment and loading guest app.
If no app is selected, it loads LiveContainerSwiftUI.

LiveContainerSwiftUI

SwiftUI rewrite of LiveContainerUI (by @hugeBlack)
Lanaguage file Localizable.xcstrings is in here for multilingual support

TweakLoader

A simple tweak injector, which loads CydiaSubstrate and load tweaks.
Injected to every app you install in LiveContainer.

AltStoreTweak

The tweak that got injected into SideStore/AltStore to retrieve certificate from it everytime it launches.

ZSign

The app signer shipped with LiveContainer.
Originally made by zhlynn.
LiveContainer uses Feather's version of ZSign modified by hcrysalis.
Changes are made to meet LiveContainer's need.

How does it work?

Patching guest executable

Patch __PAGEZERO segment:
- Change vmaddr to 0xFFFFC000 (0x100000000 - 0x4000)
- Change vmsize to 0x4000
Change MH_EXECUTE to MH_DYLIB.
Inject a load command to load TweakLoader.dylib

Patching `@executable_path`

Call _NSGetExecutablePath with an invalid buffer pointer input -> SIGSEGV
Do some magic stuff to overwrite the contents of executable_path.

Patching `NSBundle.mainBundle`

This property is overwritten with the guest app's bundle.

Bypassing Library Validation

JIT is optional to bypass codesigning. In JIT-less mode, all executables are signed so this does not apply.
Derived from Restoring Dyld Memory Loading

dlopening the executable

Call dlopen with the guest app's executable
TweakLoader loads all tweaks in the selected folder
Find the entry point
Jump to the entry point
The guest app's entry point calls UIApplicationMain and start up like any other iOS apps.

Multi-Account support & Keychain Semi-Separation

3 keychain access groups are created and LiveContainer allocates them to each container of same app. So you can create 3 container with different keychain access groups.

Why only 3?

The original thought was 256, but due to a SideStore bug (latest AltStore don't have it), we can only declare 3 keychain access groups before SideStore fails to sign. So the limit is 3.

Limitations

Entitlements from the guest app are not applied to the host app. This isn't a big deal since sideloaded apps requires only basic entitlements.
App Permissions are globally applied.
Guest app containers are not sandboxed. This means one guest app can access other guest apps' data.
App extensions aren't supported. they cannot be registered because: LiveContainer is sandboxed, SpringBoard doesn't know what apps are installed in LiveContainer, and they take up App ID.
Only one guest app can run at a time. This is much more like 3 apps limit where you have to disable an app to run another (switching between app in LiveContainer is instant).
Remote push notification might not work. ~~If you have a paid developer account then you don't even have to use LiveContainer~~
Querying custom URL schemes might not work(?)

TODO

Use ChOma instead of custom MachO parser

License

Apache License 2.0

Credits

xpn's blogpost: Restoring Dyld Memory Loading
LinusHenze's CFastFind: MIT license
fishhook: BSD 3-Clause license
MBRoundProgressView
@haxi0 for icon
@Vishram1123 for the initial shortcut implementation.
@hugeBlack for SwiftUI contribution
@Staubgeborener for automatic AltStore/SideStore source updater
@fkunn1326 for improved app hiding
@slds1 for dynamic color feature

KreitinnSoftware/MiceWine-Application

2024-12-20T01:29:07Z

MiceWine is a project that aims to run Windows applications and games on Android smartphones.

MiceWine Emulator

MiceWine is a project that aims to run Windows applications and games on Android smartphones.

It uses a customized build of Wine compiled for Android and Box64 to run in the best possible way.

Actually in constant development.

It's use a XServer based on Termux-X11

Compatible Android Versions

Support Android 10+

Compatible GPUs

GPU	Support	Driver
Adreno 7xx	Supported	Turnip/Zink
Adreno 6xx	Supported	Turnip/Zink
Others	Parcially Supported	Native/Zink (Experimental)

About Native Vulkan on Non-Adreno GPUs is Experimental, Require Some Testing

Third Party Open Source Applications:

ClassiCube/ClassiCube

2024-12-20T01:29:07Z

Custom Minecraft Classic / ClassiCube client written in C from scratch (formerly ClassicalSharp in C#)

ClassiCube is a custom Minecraft Classic compatible client written in C from scratch.

[!IMPORTANT] ClassiCube is not affiliated with (or supported by) Mojang AB, Minecraft, or Microsoft in any way.

ClassiCube is not trying to replicate modern Minecraft.
It will never support survival, Minecraft accounts, or modern Minecraft servers.

What ClassiCube is

ClassiCube aims to replicate the 2009 Minecraft Classic client while offering optional enhancements to improve gameplay. ClassiCube can run on many systems, including desktop, web, mobile, and even some consoles.

Features (click to expand)

Much better performance and minimal memory usage compared to original Minecraft Classic
Optional enhancements including custom blocks, models, and environment colors
Partially supports some features of Minecraft Classic versions before 0.30
Works with effectively all graphics cards that support OpenGL or Direct3D 9
Runs on Windows, macOS, Linux, Android, iOS, and in a web browser
Also runs on OpenBSD, FreeBSD, NetBSD, Solaris, Haiku, IRIX, SerenityOS
Although in various stages of early development, also runs on various consoles

You can download ClassiCube here and the very latest builds here.

We need your help

ClassiCube strives to replicate the original Minecraft Classic experience by strictly adhering to clean room reverse engineering approach.

If you're interested in documenting or verifying the behaviour of the original Minecraft Classic, please get in contact on the ClassiCube Discord

How to play

Initially, you will need to run ClassiCube.exe to download the required assets from minecraft.net and classicube.net.
Just click 'OK' to the dialog menu that appears when you start the launcher.

Note: When running from within VirtualBox, disable Mouse Integration, otherwise the in-game camera won't work properly.

Singleplayer mode Run ClassiCube.exe, then click Singleplayer at the main menu.

Multiplayer mode Run ClassiCube.exe. You can connect to LAN/locally hosted servers, and classicube.net servers if you have a ClassiCube account.

Stuck on OpenGL 1.1?

The most common reason for being stuck on OpenGL 1.1 is non-working GPU drivers - so if possible, you should try either installing or updating the drivers for your GPU.

Otherwise:

On Windows, you can still run the OpenGL build of ClassiCube anyways.
(You can try downloading and using the MESA software renderer from here for slightly better performance though)
On other operating systems, you will have to compile the game yourself.
Don't forget to add -DCC_BUILD_GL11 to the compilation command line so that the compiled game supports OpenGL 1.1.

Supported systems

ClassiCube runs on:

Windows - 95 and later
macOS - 10.5 or later (can be compiled for 10.3/10.4 though)
Linux - needs libcurl and libopenal
Android - 2.3 or later
iOS - 8.0 or later
Most web browsers (even runs on IE11)

And also runs on:

Raspberry Pi - needs libcurl and libopenal
FreeBSD - needs libexecinfo, curl and openal-soft packages (can download from here)
NetBSD - needs libexecinfo, curl and openal-soft packages (can download from here)
OpenBSD - needs libexecinfo, curl and openal packages
Solaris - needs curl and openal packages
Haiku - needs openal package (if you have a GitHub account, can download from here)
BeOS - untested on actual hardware
IRIX - needs curl and openal packages
SerenityOS - needs SDL2
Classic Mac OS (System 7 and later)
Dreamcast - unfinished, but usable (can download from here)
Saturn - unfinished, major rendering and stability issues (can download from here)
Switch - unfinished, but usable (can download from here)
Wii U - unfinished, major issues, broken on real hardware (can download from here)
Wii - unfinished, but usable (can download from here)
GameCube - unfinished, but usable (can download from here)
Nintendo 64 - unfinished, moderate rendering issues (can download from here)
3DS - unfinished, but usable (can download from here)
DS/DSi - unfinished, rendering issues (can download from here)
PS Vita - unfinished, rendering issues (can download from here)
PSP - unfinished, rendering issues (can download from here)
PS3 - unfinished, rendering issues (can download from here)
PS2 - unfinished, major rendering and stability issues (can download from here)
PS1 - unfinished, major rendering and stability issues (can download from here)
Xbox 360 - completely unfinished, broken on real hardware (can download from here)
Xbox - unfinished, major rendering issues (can download from here)

Compiling

Note: The instructions below automatically compile ClassiCube with the recommended defaults for the platform.
If you (not recommended) want to override the defaults (e.g. to compile OpenGL build on Windows), see here for details.

Compiling - Windows

Using Visual Studio

Open ClassiCube.sln (File -> Open -> Project/Solution)
Compile/Build it (Build -> Build Solution).

If you get a The Windows SDK version 5.1 was not found compilation error, see here for how to fix

Using Visual Studio (command line)

Use 'Developer Tools for Visual Studio' from Start Menu
Navigate to the directory with ClassiCube's source code
Run cl.exe *.c /link user32.lib gdi32.lib winmm.lib dbghelp.lib shell32.lib comdlg32.lib /out:ClassiCube.exe

Using MinGW-w64

Assuming that you used the installer from https://sourceforge.net/projects/mingw-w64/ :

Install MinGW-W64
Use either Run Terminal from Start Menu or run mingw-w64.bat in the installation folder
Navigate to the directory with ClassiCube's source code
Run gcc -fno-math-errno *.c -o ClassiCube.exe -mwindows -lwinmm

Using MinGW

Assuming that you used the installer from https://osdn.net/projects/mingw/ :

Install MinGW. You need mingw32-base-bin and msys-base-bin packages.
Run msys.bat in the C:\MinGW\msys\1.0 folder.
Navigate to the directory with ClassiCube's source code
Run gcc -fno-math-errno *.c -o ClassiCube.exe -mwindows -lwinmm

Using TCC (Tiny C Compiler)

Setting up TCC:

Download and extract tcc-0.9.27-win64-bin.zip from https://bellard.org/tcc/
Download winapi-full-for-0.9.27.zip from https://bellard.org/tcc/
Copy winapi folder and _mingw_dxhelper.h from winapi-full-for-0.9.27.zip into TCC's include folder

Compiling with TCC:

Navigate to the directory with ClassiCube's source code
In ExtMath.c, change fabsf to fabs and sqrtf to sqrt
Run tcc.exe -o ClassiCube.exe *.c -lwinmm -lgdi32 -luser32 -lcomdlg32 -lshell32
(Note: You may need to specify the full path to tcc.exe instead of just tcc.exe)

Compiling - Linux

Using gcc/clang

Install X11, XInput2, and OpenGL development libraries if necessary.
For Ubuntu, these are the libx11-dev, libxi-dev and libgl1-mesa-dev packages
Run either:
- make linux or
- cc -fno-math-errno src/*.c -o ClassiCube -rdynamic -lpthread -lX11 -lXi -lGL -ldl

Cross compiling for Windows (32 bit):

Install MinGW-w64 if necessary. (Ubuntu: gcc-mingw-w64 package)
Run i686-w64-mingw32-gcc -fno-math-errno src/*.c -o ClassiCube.exe -mwindows -lwinmm

Cross compiling for Windows (64 bit):

Install MinGW-w64 if necessary. (Ubuntu: gcc-mingw-w64 package)
Run x86_64-w64-mingw32-gcc -fno-math-errno src/*.c -o ClassiCube.exe -mwindows -lwinmm

Raspberry Pi

Although the regular linux compiliation flags will work fine, to take full advantage of the hardware:

gcc -fno-math-errno src/*.c -o ClassiCube -DCC_BUILD_RPI -rdynamic -lpthread -lX11 -lXi -lEGL -lGLESv2 -ldl

Compiling - macOS

Install a C compiler if necessary. The easiest way of obtaining one is by installing Xcode.
Run either:
- make darwin or
- cc -fno-math-errno src/*.c src/*.m -o ClassiCube -framework Cocoa -framework OpenGL -framework IOKit -lobjc

Using Xcode GUI

Open the misc/macOS/CCMAC.xcodeproj project in Xcode
Compile the project

Compiling - for Android

NOTE: If you are distributing a modified version, please change the package ID from com.classicube.android.client to something else - otherwise Android users won't be able to have both ClassiCube and your modified version installed at the same time on their Android device

Using Android Studio GUI

Open android folder in Android Studio (TODO explain more detailed)

Using command line (gradle)

Run gradlew in android folder (TODO explain more detailed)

Compiling - for iOS

iOS version will have issues as it's incomplete and only tested in iOS Simulator

NOTE: If you are distributing a modified version, please change the bundle ID from com.classicube.ios.client to something else - otherwise iOS users won't be able to have both ClassiCube and your modified version installed at the same time on their iOS device

Using Xcode GUI

Open the misc/ios/CCIOS.xcodeproj project in Xcode
Compile the project

Using command line (Xcode)

xcodebuild -sdk iphoneos -configuration Debug (TODO explain more detailed)

Compiling - webclient

Install emscripten if necessary.
Run either:
- make web or
- emcc src/*.c -s ALLOW_MEMORY_GROWTH=1 -s TOTAL_STACK=1Mb --js-library interop_web.js

The generated javascript file has some issues. See here for how to fix

For details on how to integrate the webclient into a website, see here

Compiling - consoles

All console ports need assistance from someone experienced with homebrew development - if you're interested, please get in contact on the ClassiCube Discord.

Nintendo consoles (click to expand)

Switch

Run make switch. You'll need libnx and mesa

NOTE: It is highly recommended that you install the precompiled devkitpro packages from here - you need the switch-dev group and the switch-mesa switch-glm packages)

Wii U

Run make wiiu. You'll need wut

NOTE: It is highly recommended that you install the precompiled devkitpro packages from here - you need the wiiu-dev group)

3DS

Run make 3ds. You'll need libctru

NOTE: It is highly recommended that you install the precompiled devkitpro packages from here - you need the 3ds-dev group)

Wii

Run make wii. You'll need libogc

NOTE: It is highly recommended that you install the precompiled devkitpro packages from here - you need the wii-dev group)

GameCube

Run make gamecube. You'll need libogc

NOTE: It is highly recommended that you install the precompiled devkitpro packages from here - you need the gamecube-dev group)

Nintendo DS/DSi

Run make ds. You'll need BlocksDS

Nintendo 64

Run make n64. You'll need the opengl branch of libdragon

Sony consoles (click to expand)

PlayStation Vita

Run make vita. You'll need vitasdk

PlayStation Portable

Run make psp. You'll need pspsdk

NOTE: It is recommended that you install the precompiled pspsdk version from here

PlayStation 3

Run make ps3. You'll need PSL1GHT

PlayStation 2

Run make ps2. You'll need ps2sdk

PlayStation 1

Run make ps1. You'll need PSn00bSDK

Microsoft consoles (click to expand)

Xbox 360

Run make 360. You'll need libxenon

Xbox (original)

Run make xbox. You'll need nxdk

SEGA consoles (click to expand)

SEGA consoles

Dreamcast

Run make dreamcast. You'll need KallistiOS

Saturn

Run make saturn. You'll need libyaul

Compiling - other platforms (click to expand)

FreeBSD

Install libxi, libexecinfo, curl and openal-soft packages if needed
Run either:
- make freebsd or
- cc src/*.c -o ClassiCube -I /usr/local/include -L /usr/local/lib -lm -lpthread -lX11 -lXi -lGL -lexecinfo

OpenBSD

Install libexecinfo, curl and openal packages if needed
Run either:
- make opensd or
- cc src/*.c -o ClassiCube -I /usr/X11R6/include -I /usr/local/include -L /usr/X11R6/lib -L /usr/local/lib -lm -lpthread -lX11 -lXi -lGL -lexecinfo

NetBSD

Install libexecinfo, curl and openal-soft packages if needed
Run either:
- make netbsd or
- cc src/*.c -o ClassiCube -I /usr/X11R7/include -I /usr/pkg/include -L /usr/X11R7/lib -L /usr/pkg/lib -lpthread -lX11 -lXi -lGL -lexecinfo

DragonflyBSD

Install libxi, libexecinfo, curl and openal-soft packages if needed
Run either:
- make dragonfly or
- cc src/*.c -o ClassiCube -I /usr/local/include -L /usr/local/lib -lm -lpthread -lX11 -lXi -lGL -lexecinfo

Solaris

Install required packages if needed
Run either:
- make sunos or
- gcc -fno-math-errno src/*.c -o ClassiCube -lsocket -lX11 -lXi -lGL

Haiku

Install gcc, haiku_devel, openal_devel packages if needed
Run either:
- make haiku or
- cc -fno-math-errno src/*.c src/*.cpp -o ClassiCube -lGL -lnetwork -lstdc++ -lbe -lgame -ltracker

BeOS

Install a C compiler
Run either:
- make beos or
- cc -fno-math-errno src/*.c src/*.cpp -o ClassiCube -lGL -lbe -lgame -ltracker

IRIX

Install required packages if needed
Run either:
- make irix or
- gcc -fno-math-errno src/*.c -o ClassiCube -lGL -lX11 -lXi -lpthread -ldl`

SerenityOS

Install SDL2 port if needed
Run either:
- make serenityos or
- cc src/*.c -o ClassiCube -lgl -lSDL2

Classic Mac OS

Install Retro68
Run either
- make macclassic_68k (For a M68k build)
- make macclassic_ppc (For a PPC build)

The PowerPC build will usually perform much better

Other systems

You'll have to write the necessary code. You should read portability.md in doc folder.

Documentation

Functions and variables in .h files are mostly documented.

Further information (e.g. style) for ClassiCube's source code can be found in the doc and misc folders.

Known compilation errors

Fixes for compilation errors when using musl or old glibc for C standard library

Tips

Press escape (after joining a world) or pause to switch to the pause menu.
Pause menu -> Options -> Controls lists all of the key combinations used by the client.
Note that toggling 'vsync' to on will minimise CPU usage, while off will maximimise chunk loading speed.
Press F to cycle view distance. Lower view distances can improve performance.
If the server has disabled hacks, key combinations such as fly and speed will not do anything.
To see the list of built in commands, type /client.
To see help for a given built in command, type /client help <command name>.

Open source technologies (click to expand)

curl - HTTP/HTTPS for linux and macOS
FreeType - Font handling for all platforms
GCC - Compiles client for linux
MinGW-w64 - Compiles client for windows
Clang - Compiles client for macOS
Emscripten - Compiles client for web
RenderDoc - Graphics debugging
BearSSL - SSL/TLS support on consoles
libnx - Backend for Switch
Ryujinx - Emulator used to test Switch port
wut - Backend for Wii U
Cemu - Emulator used to test Wii U port
libctru - Backend for 3DS
citro3D - Rendering backend for 3DS
Citra - Emulator used to test 3DS port
libogc - Backend for Wii and GameCube
libfat - Filesystem backend for Wii/GC
Dolphin - Emulator used to test Wii/GC port
libdragon - Backend for Nintendo 64
ares - Emulator used to test Nintendo 64 port
BlocksDS - Backend for Nintendo DS
melonDS - Emulator used to test Nintendo DS port
vitasdk - Backend for PS Vita
Vita3K - Emulator used to test Vita port
pspsdk - Backend for PSP
PPSSPP - Emulator used to test PSP port
PSL1GHT - Backend for PS3
RPCS3 - Emulator used to test PS3 port
ps2sdk - Backend for PS2
PCSX2 - Emulator used to test PS2 port
PSn00bSDK - Backend for PS1
duckstation - Emulator used to test PS1 port
libxenon - Backend for Xbox 360
nxdk - Backend for Xbox
xemu - Emulator used to test Xbox port
cxbx-reloaded - Emulator used to test Xbox port
KallistiOS - Backend for Dreamcast
GLdc - Basis of rendering backend for Dreamcast
flycast - Emulator used to test Dreamcast port
libyaul - Backend for Saturn
mednafen - Emulator used to test Saturn port

Sound Credits

ClassiCube uses sounds from Freesound.org
Full credits are listed in doc/sound-credits.md

mailru/easyjson

2024-12-20T01:31:02Z

Fast JSON serializer for golang.

easyjson

Package easyjson provides a fast and easy way to marshal/unmarshal Go structs to/from JSON without the use of reflection. In performance tests, easyjson outperforms the standard encoding/json package by a factor of 4-5x, and other JSON encoding packages by a factor of 2-3x.

easyjson aims to keep generated Go code simple enough so that it can be easily optimized or fixed. Another goal is to provide users with the ability to customize the generated code by providing options not available with the standard encoding/json package, such as generating "snake_case" names or enabling omitempty behavior by default.

Usage

Install:

# for Go < 1.17
go get -u github.com/mailru/easyjson/...

or

# for Go >= 1.17
go get github.com/mailru/easyjson && go install github.com/mailru/easyjson/...@latest

Run:

easyjson -all <file>.go

The above will generate <file>_easyjson.go containing the appropriate marshaler and unmarshaler funcs for all structs contained in <file>.go.

Please note that easyjson requires a full Go build environment and the GOPATH environment variable to be set. This is because easyjson code generation invokes go run on a temporary file (an approach to code generation borrowed from ffjson).

Serialize

someStruct := &SomeStruct{Field1: "val1", Field2: "val2"}
rawBytes, err := easyjson.Marshal(someStruct)

Deserialize

someStruct := &SomeStruct{}
err := easyjson.Unmarshal(rawBytes, someStruct)

Please see the GoDoc for more information and features.

Options

Usage of easyjson:
  -all
    	generate marshaler/unmarshalers for all structs in a file
  -build_tags string
        build tags to add to generated file
  -gen_build_flags string
        build flags when running the generator while bootstrapping
  -byte
        use simple bytes instead of Base64Bytes for slice of bytes
  -leave_temps
    	do not delete temporary files
  -no_std_marshalers
    	don't generate MarshalJSON/UnmarshalJSON funcs
  -noformat
    	do not run 'gofmt -w' on output file
  -omit_empty
    	omit empty fields by default
  -output_filename string
    	specify the filename of the output
  -pkg
    	process the whole package instead of just the given file
  -snake_case
    	use snake_case names instead of CamelCase by default
  -lower_camel_case
        use lowerCamelCase instead of CamelCase by default
  -stubs
    	only generate stubs for marshaler/unmarshaler funcs
  -disallow_unknown_fields
        return error if some unknown field in json appeared
  -disable_members_unescape
        disable unescaping of \uXXXX string sequences in member names

Using -all will generate marshalers/unmarshalers for all Go structs in the file excluding those structs whose preceding comment starts with easyjson:skip. For example:

//easyjson:skip
type A struct {}

If -all is not provided, then only those structs whose preceding comment starts with easyjson:json will have marshalers/unmarshalers generated. For example:

//easyjson:json
type A struct {}

Additional option notes:

-snake_case tells easyjson to generate snake_case field names by default (unless overridden by a field tag). The CamelCase to snake_case conversion algorithm should work in most cases (ie, HTTPVersion will be converted to "http_version").
-build_tags will add the specified build tags to generated Go sources.
-gen_build_flags will execute the easyjson bootstapping code to launch the actual generator command with provided flags. Multiple arguments should be separated by space e.g. -gen_build_flags="-mod=mod -x".

Structure json tag options

Besides standard json tag options like 'omitempty' the following are supported:

'nocopy' - disables allocation and copying of string values, making them refer to original json buffer memory. This works great for short lived objects which are not hold in memory after decoding and immediate usage. Note if string requires unescaping it will be processed as normally.
'intern' - string "interning" (deduplication) to save memory when the very same string dictionary values are often met all over the structure. See below for more details.

Generated Marshaler/Unmarshaler Funcs

For Go struct types, easyjson generates the funcs MarshalEasyJSON / UnmarshalEasyJSON for marshaling/unmarshaling JSON. In turn, these satisfy the easyjson.Marshaler and easyjson.Unmarshaler interfaces and when used in conjunction with easyjson.Marshal / easyjson.Unmarshal avoid unnecessary reflection / type assertions during marshaling/unmarshaling to/from JSON for Go structs.

easyjson also generates MarshalJSON and UnmarshalJSON funcs for Go struct types compatible with the standard json.Marshaler and json.Unmarshaler interfaces. Please be aware that using the standard json.Marshal / json.Unmarshal for marshaling/unmarshaling will incur a significant performance penalty when compared to using easyjson.Marshal / easyjson.Unmarshal.

Additionally, easyjson exposes utility funcs that use the MarshalEasyJSON and UnmarshalEasyJSON for marshaling/unmarshaling to and from standard readers and writers. For example, easyjson provides easyjson.MarshalToHTTPResponseWriter which marshals to the standard http.ResponseWriter. Please see the GoDoc listing for the full listing of utility funcs that are available.

Controlling easyjson Marshaling and Unmarshaling Behavior

Go types can provide their own MarshalEasyJSON and UnmarshalEasyJSON funcs that satisfy the easyjson.Marshaler / easyjson.Unmarshaler interfaces. These will be used by easyjson.Marshal and easyjson.Unmarshal when defined for a Go type.

Go types can also satisfy the easyjson.Optional interface, which allows the type to define its own omitempty logic.

Type Wrappers

easyjson provides additional type wrappers defined in the easyjson/opt package. These wrap the standard Go primitives and in turn satisfy the easyjson interfaces.

The easyjson/opt type wrappers are useful when needing to distinguish between a missing value and/or when needing to specifying a default value. Type wrappers allow easyjson to avoid additional pointers and heap allocations and can significantly increase performance when used properly.

Memory Pooling

easyjson uses a buffer pool that allocates data in increasing chunks from 128 to 32768 bytes. Chunks of 512 bytes and larger will be reused with the help of sync.Pool. The maximum size of a chunk is bounded to reduce redundant memory allocation and to allow larger reusable buffers.

easyjson's custom allocation buffer pool is defined in the easyjson/buffer package, and the default behavior pool behavior can be modified (if necessary) through a call to buffer.Init() prior to any marshaling or unmarshaling. Please see the GoDoc listing for more information.

String interning

During unmarshaling, string field values can be optionally interned to reduce memory allocations and usage by deduplicating strings in memory, at the expense of slightly increased CPU usage.

This will work effectively only for string fields being decoded that have frequently the same value (e.g. if you have a string field that can only assume a small number of possible values).

To enable string interning, add the intern keyword tag to your json tag on string fields, e.g.:

type Foo struct {
  UUID  string `json:"uuid"`         // will not be interned during unmarshaling
  State string `json:"state,intern"` // will be interned during unmarshaling
}

Issues, Notes, and Limitations

easyjson is still early in its development. As such, there are likely to be bugs and missing features when compared to encoding/json. In the case of a missing feature or bug, please create a GitHub issue. Pull requests are welcome!
Unlike encoding/json, object keys are case-sensitive. Case-insensitive matching is not currently provided due to the significant performance hit when doing case-insensitive key matching. In the future, case-insensitive object key matching may be provided via an option to the generator.
easyjson makes use of unsafe, which simplifies the code and provides significant performance benefits by allowing no-copy conversion from []byte to string. That said, unsafe is used only when unmarshaling and parsing JSON, and any unsafe operations / memory allocations done will be safely deallocated by easyjson. Set the build tag easyjson_nounsafe to compile it without unsafe.
easyjson is compatible with Google App Engine. The appengine build tag (set by App Engine's environment) will automatically disable the use of unsafe, which is not allowed in App Engine's Standard Environment. Note that the use with App Engine is still experimental.
Floats are formatted using the default precision from Go's strconv package. As such, easyjson will not correctly handle high precision floats when marshaling/unmarshaling JSON. Note, however, that there are very few/limited uses where this behavior is not sufficient for general use. That said, a different package may be needed if precise marshaling/unmarshaling of high precision floats to/from JSON is required.
While unmarshaling, the JSON parser does the minimal amount of work needed to skip over unmatching parens, and as such full validation is not done for the entire JSON value being unmarshaled/parsed.
Currently there is no true streaming support for encoding/decoding as typically for many uses/protocols the final, marshaled length of the JSON needs to be known prior to sending the data. Currently this is not possible with easyjson's architecture.
easyjson parser and codegen based on reflection, so it won't work on package main files, because they cant be imported by parser.

Benchmarks

Most benchmarks were done using the example 13kB example JSON (9k after eliminating whitespace). This example is similar to real-world data, is well-structured, and contains a healthy variety of different types, making it ideal for JSON serialization benchmarks.

Note:

For small request benchmarks, an 80 byte portion of the above example was used.
For large request marshaling benchmarks, a struct containing 50 regular samples was used, making a ~500kB output JSON.
Benchmarks are showing the results of easyjson's default behaviour, which makes use of unsafe.

Benchmarks are available in the repository and can be run by invoking make.

easyjson vs. encoding/json

easyjson is roughly 5-6 times faster than the standard encoding/json for unmarshaling, and 3-4 times faster for non-concurrent marshaling. Concurrent marshaling is 6-7x faster if marshaling to a writer.

easyjson vs. ffjson

easyjson uses the same approach for JSON marshaling as ffjson, but takes a significantly different approach to lexing and parsing JSON during unmarshaling. This means easyjson is roughly 2-3x faster for unmarshaling and 1.5-2x faster for non-concurrent unmarshaling.

As of this writing, ffjson seems to have issues when used concurrently: specifically, large request pooling hurts ffjson's performance and causes scalability issues. These issues with ffjson can likely be fixed, but as of writing remain outstanding/known issues with ffjson.

easyjson and ffjson have similar performance for small requests, however easyjson outperforms ffjson by roughly 2-5x times for large requests when used with a writer.

easyjson vs. go/codec

go/codec provides compile-time helpers for JSON generation. In this case, helpers do not work like marshalers as they are encoding-independent.

easyjson is generally 2x faster than go/codec for non-concurrent benchmarks and about 3x faster for concurrent encoding (without marshaling to a writer).

In an attempt to measure marshaling performance of go/codec (as opposed to allocations/memcpy/writer interface invocations), a benchmark was done with resetting length of a byte slice rather than resetting the whole slice to nil. However, the optimization in this exact form may not be applicable in practice, since the memory is not freed between marshaling operations.

easyjson vs 'ujson' python module

ujson is using C code for parsing, so it is interesting to see how plain golang compares to that. It is important to note that the resulting object for python is slower to access, since the library parses JSON object into dictionaries.

easyjson is slightly faster for unmarshaling and 2-3x faster than ujson for marshaling.

Benchmark Results

ffjson results are from February 4th, 2016, using the latest ffjson and go1.6. go/codec results are from March 4th, 2016, using the latest go/codec and go1.6.

Unmarshaling

lib	json size	MB/s	allocs/op	B/op
standard	regular	22	218	10229
standard	small	9.7	14	720

easyjson	regular	125	128	9794
easyjson	small	67	3	128

ffjson	regular	66	141	9985
ffjson	small	17.6	10	488

codec	regular	55	434	19299
codec	small	29	7	336

ujson	regular	103	N/A	N/A

Marshaling, one goroutine.

lib	json size	MB/s	allocs/op	B/op
standard	regular	75	9	23256
standard	small	32	3	328
standard	large	80	17	1.2M

easyjson	regular	213	9	10260
easyjson*	regular	263	8	742
easyjson	small	125	1	128
easyjson	large	212	33	490k
easyjson*	large	262	25	2879

ffjson	regular	122	153	21340
ffjson**	regular	146	152	4897
ffjson	small	36	5	384
ffjson**	small	64	4	128
ffjson	large	134	7317	818k
ffjson**	large	125	7320	827k

codec	regular	80	17	33601
codec***	regular	108	9	1153
codec	small	42	3	304
codec***	small	56	1	48
codec	large	73	483	2.5M
codec***	large	103	451	66007

ujson	regular	92	N/A	N/A

* marshaling to a writer, ** using ffjson.Pool(), *** reusing output slice instead of resetting it to nil

Marshaling, concurrent.

lib	json size	MB/s	allocs/op	B/op
standard	regular	252	9	23257
standard	small	124	3	328
standard	large	289	17	1.2M

easyjson	regular	792	9	10597
easyjson*	regular	1748	8	779
easyjson	small	333	1	128
easyjson	large	718	36	548k
easyjson*	large	2134	25	4957

ffjson	regular	301	153	21629
ffjson**	regular	707	152	5148
ffjson	small	62	5	384
ffjson**	small	282	4	128
ffjson	large	438	7330	1.0M
ffjson**	large	131	7319	820k

codec	regular	183	17	33603
codec***	regular	671	9	1157
codec	small	147	3	304
codec***	small	299	1	48
codec	large	190	483	2.5M
codec***	large	752	451	77574

* marshaling to a writer, ** using ffjson.Pool(), *** reusing output slice instead of resetting it to nil

web-infra-dev/midscene

2024-12-20T01:31:23Z

An AI-powered automation SDK can control the page, perform assertions, and extract data in JSON format using natural language.

Midscene.js

English | 简体中文 | 日本語

Joyful UI Automation

Midscene.js is an AI-powered automation SDK can control the page, perform assertions, and extract data in JSON format using natural language.

Features ✨

Natural Language Interaction 👆: Describe the steps, and let Midscene plan and control the user interface for you
Understand UI, Answer in JSON 🔍: Provide prompts regarding the desired data format, and then receive the expected response in JSON format.
Intuitive Assertion 🤔: Make assertions in natural language; it’s all based on AI understanding.
Out-of-box LLM 🪓: It is fine to use public multimodal LLMs like GPT-4o. There is no need for any custom training.
Visualized Report 🎞️: With our visualized report file, you can easily understand and debug the whole process.
Brand New Experience! 🔥: Experience a whole new world of automation development. Enjoy!

Resources 📄

License

Midscene.js is MIT licensed.

amazon-ion/ion-tests

2024-12-19T01:31:32Z

Test vectors for testing compliant Ion implementations.

dbt-labs/docs.getdbt.com

2024-12-19T01:31:43Z

The code behind docs.getdbt.com

We use docusaurus to power docs.getdbt.com.

Code of conduct

Please review the dbt docs contributors code of conduct. Creating an inclusive and equitable environment for our documents is more important than any other aspect. Syntax errors can be corrected, but trust, once lost, is difficult to gain back.

Contributing

We welcome contributions from community members to this repo:

Fixes: When you notice an error, you can use the Edit this page button at the bottom of each page to suggest a change.
New documentation: If you contributed code in dbt-core, we encourage you to also write the docs here! Please reach out in the dbt community if you need help finding a place for these docs.
Major rewrites: You can file an issue to propose ideas for a content area that requires attention.

You can use components documented in the docusaurus library.

Writing content

The dbt Labs docs are written in Markdown and sometimes HTML. When writing content, refer to the style guide and content types to help you understand our writing standards and how we break down information in the product documentation.

SME and editorial reviews

All PRs that are submitted will be reviewed by the dbt Labs Docs team for editorial review.

Content that is submitted by our users and the open-source community are also reviewed by our dbt Labs subject matter experts (SMEs) to help ensure technical accuracy.

Versioning and single-sourcing content

We now enable you to reuse content between different docs pages, version pages, and establish product variables in the dbt Labs product documentation. To learn more about how to single source content between versions, product variables, and other content, see Single-sourcing content.

Adding tabbed components to a page

You can add code snippets and other content in a tabbed view. To learn more about adding tabbed components, see Adding page components.

Running the Docs site locally

You can click a link available in a Vercel bot PR comment to see and review your changes rendered on a staging server. You are also able to see and review your proposed modifications locally on your computer. Our setup instructions use homebrew:

Prerequisites

(Mac Terminal) Install Xcode Command Line Tools
- Open a terminal window, run xcode-select --install, and follow the on-screen prompts in the pop-up window.
(Mac and Linux) Install homebrew
- Copy and paste /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" in to a terminal window and follow the prompts. Once the installation has completed, follow the Next Steps instructions listed in terminal.
(Windows) Install Node.js

(Mac and Linux only) Install node: brew install node
Clone this repo: git clone https://github.com/dbt-labs/docs.getdbt.com.git
cd into the repo: cd docs.getdbt.com
cd into the website subdirectory: cd website
Install the required node packages: make install or npm install (optional — install any updates)
Build the website: make run or npm start
Before pushing your changes to a branch, run make build or npm run build and check that all links work

Advisory:

If you run into an fatal error: 'vips/vips8' file not found error when you run npm install, you may need to run brew install vips. Warning: this one will take a while -- go ahead and grab some coffee!

JuliaParallel/ClusterManagers.jl

2024-12-19T01:31:55Z

NirDiamant/GenAI_Agents

2024-12-20T01:32:05Z

This repository provides tutorials and implementations for various Generative AI Agent techniques, from basic to advanced. It serves as a comprehensive guide for building intelligent, interactive AI systems.

🌟 Support This Project: Your sponsorship fuels innovation in GenAI agent development. Become a sponsor to help maintain and expand this valuable resource!

GenAI Agents: Comprehensive Repository for Development and Implementation 🚀

Welcome to one of the most extensive and dynamic collections of Generative AI (GenAI) agent tutorials and implementations available today. This repository serves as a comprehensive resource for learning, building, and sharing GenAI agents, ranging from simple conversational bots to complex, multi-agent systems.

📫 Stay Updated!

🚀
Cutting-edge
Updates

💡
Expert
Insights

🎯
Top 1%
Content

Join thousands of AI enthusiasts getting unique cutting edge insights and free tutorials!

Introduction

Generative AI agents are at the forefront of artificial intelligence, revolutionizing the way we interact with and leverage AI technologies. This repository is designed to guide you through the development journey, from basic agent implementations to advanced, cutting-edge systems.

Our goal is to provide a valuable resource for everyone - from beginners taking their first steps in AI to seasoned practitioners pushing the boundaries of what's possible. By offering a range of examples from foundational to complex, we aim to facilitate learning, experimentation, and innovation in the rapidly evolving field of GenAI agents.

Furthermore, this repository serves as a platform for showcasing innovative agent creations. Whether you've developed a novel agent architecture or found an innovative application for existing techniques, we encourage you to share your work with the community.

Related Projects

📚 Dive into my comprehensive guide on RAG techniques to learn about integrating external knowledge into AI systems, enhancing their capabilities with up-to-date and relevant information retrieval.

🖋️ Explore my Prompt Engineering Techniques guide for an extensive collection of prompting strategies, from fundamental concepts to advanced methods, improving your ability to communicate effectively with AI language models.

A Community-Driven Knowledge Hub

This repository grows stronger with your contributions! Join our vibrant Discord community — the central hub for shaping and advancing this project together 🤝

GenAI Agents Discord Community

Whether you're a novice eager to learn or an expert ready to share your knowledge, your insights can shape the future of GenAI agents. Join us to propose ideas, get feedback, and collaborate on innovative implementations. For contribution guidelines, please refer to our CONTRIBUTING.md file. Let's advance GenAI agent technology together!

🔗 For discussions on GenAI, agents, or to explore knowledge-sharing opportunities, feel free to connect on LinkedIn.

Key Features

🎓 Learn to build GenAI agents from beginner to advanced levels
🧠 Explore a wide range of agent architectures and applications
📚 Step-by-step tutorials and comprehensive documentation
🛠️ Practical, ready-to-use agent implementations
🌟 Regular updates with the latest advancements in GenAI
🤝 Share your own agent creations with the community

GenAI Agent Implementations

Explore our extensive list of GenAI agent implementations, sorted by categories:

🌱 Beginner-Friendly Agents

Simple Conversational Agent
- LangChain
- PydanticAI
Overview 🔎

A context-aware conversational AI maintains information across interactions, enabling more natural dialogues.

Implementation 🛠️

Integrates a language model, prompt template, and history manager to generate contextual responses and track conversation sessions.
Simple Question Answering Agent

Overview 🔎

Answering (QA) agent using LangChain and OpenAI's language model understands user queries and provides relevant, concise answers.

Implementation 🛠️

Combines OpenAI's GPT model, a prompt template, and an LLMChain to process user questions and generate AI-driven responses in a streamlined manner.
Simple Data Analysis Agent

Overview 🔎

An AI-powered data analysis agent interprets and answers questions about datasets using natural language, combining language models with data manipulation tools for intuitive data exploration.

Implementation 🛠️

Integrates a language model, data manipulation framework, and agent framework to process natural language queries and perform data analysis on a synthetic dataset, enabling accessible insights for non-technical users.

🔧 Framework Tutorial: LangGraph

Introduction to LangGraph: Building Modular AI Workflows

Overview 🔎

This tutorial introduces LangGraph, a powerful framework for creating modular, graph-based AI workflows. Learn how to leverage LangGraph to build more complex and flexible AI agents that can handle multi-step processes efficiently.

Implementation 🛠️

Step-by-step guide on using LangGraph to create a StateGraph workflow. The tutorial covers key concepts such as state management, node creation, and graph compilation. It demonstrates these principles by constructing a simple text analysis pipeline, serving as a foundation for more advanced agent architectures.

🎓 Educational and Research Agents

ATLAS: Academic Task and Learning Agent System

Overview 🔎

ATLAS demonstrates how to build an intelligent multi-agent system that transforms academic support through AI-powered assistance. The system leverages LangGraph's workflow framework to coordinate multiple specialized agents that provide personalized academic planning, note-taking, and advisory support.

Implementation 🛠️

Implements a state-managed multi-agent architecture using four specialized agents (Coordinator, Planner, Notewriter, and Advisor) working in concert through LangGraph's workflow framework. The system features sophisticated workflows for profile analysis and academic support, with continuous adaptation based on student performance and feedback.

Additional Resources 📚
- YouTube Explanation
- Blog Post
Scientific Paper Agent - Literature Review

Overview 🔎

An intelligent research assistant that helps users navigate, understand, and analyze scientific literature through an orchestrated workflow. The system combines academic APIs with sophisticated paper processing techniques to automate literature review tasks, enabling researchers to efficiently extract insights from academic papers while maintaining research rigor and quality control.

Implementation 🛠️

Leverages LangGraph to create a five-node workflow system including decision making, planning, tool execution, and quality validation nodes. The system integrates the CORE API for paper access, PDFplumber for document processing, and advanced language models for analysis. Key features include a retry mechanism for robust paper downloads, structured data handling through Pydantic models, and quality-focused improvement cycles with human-in-the-loop validation options.

Additional Resources 📚
- YouTube Explanation
- Blog Post
Chiron - A Feynman-Enhanced Learning Agent

Overview 🔎

An adaptive learning agent that guides users through educational content using a structured checkpoint system and Feynman-style teaching. The system processes learning materials (either user-provided or web-retrieved), verifies understanding through interactive checkpoints, and provides simplified explanations when needed, creating a personalized learning experience that mimics one-on-one tutoring.

Implementation 🛠️

Uses LangGraph to orchestrate a learning workflow that includes checkpoint definition, context building, understanding verification, and Feynman teaching nodes. The system integrates web search for dynamic content retrieval, employs semantic chunking for context processing, and manages embeddings for relevant information retrieval. Key features include a 70% understanding threshold for progression, interactive human-in-the-loop validation, and structured output through Pydantic models for consistent data handling.

Additional Resources 📚
- YouTube Explanation

💼 Business and Professional Agents

Customer Support Agent (LangGraph)

Overview 🔎

An intelligent customer support agent using LangGraph categorizes queries, analyzes sentiment, and provides appropriate responses or escalates issues.

Implementation 🛠️

Utilizes LangGraph to create a workflow combining state management, query categorization, sentiment analysis, and response generation.
Essay Grading Agent (LangGraph)

Overview 🔎

An automated essay grading system using LangGraph and an LLM model evaluates essays based on relevance, grammar, structure, and depth of analysis.

Implementation 🛠️

Utilizes a state graph to define the grading workflow, incorporating separate grading functions for each criterion.
Travel Planning Agent (LangGraph)

Overview 🔎

A Travel Planner using LangGraph demonstrates how to build a stateful, multi-step conversational AI application that collects user input and generates personalized travel itineraries.

Implementation 🛠️

Utilizes StateGraph to define the application flow, incorporates custom PlannerState for process management.
GenAI Career Assistant Agent

Overview 🔎

The GenAI Career Assistant demonstrates how to create a multi-agent system that provides personalized guidance for careers in Generative AI. Using LangGraph and Gemini LLM, the system delivers customized learning paths, resume assistance, interview preparation, and job search support.

Implementation 🛠️

Leverages a multi-agent architecture using LangGraph to coordinate specialized agents (Learning, Resume, Interview, Job Search) through TypedDict-based state management. The system employs sophisticated query categorization and routing while integrating with external tools like DuckDuckGo for job searches and dynamic content generation.

Additional Resources 📚
- YouTube Explanation
Project Manager Assistant Agent

Overview 🔎

An AI agent designed to assist in project management tasks by automating the process of creating actionable tasks from project descriptions, identifying dependencies, scheduling work, and assigning tasks to team members based on expertise. The system includes risk assessment and self-reflection capabilities to optimize project plans through multiple iterations, aiming to minimize overall project risk.

Implementation 🛠️

Leverages LangGraph to orchestrate a workflow of specialized nodes including task generation, dependency mapping, scheduling, allocation, and risk assessment. Each node uses GPT-4o-mini for structured outputs following Pydantic models. The system implements a feedback loop for self-improvement, where risk scores trigger reflection cycles that generate insights to optimize the project plan. Visualization tools display Gantt charts of the generated schedules across iterations.

Additional Resources 📚
- YouTube Explanation
Contract Analysis Assistant (ClauseAI)

Overview 🔎

ClauseAI demonstrates how to build an AI-powered contract analysis system using a multi-agent approach. The system employs specialized AI agents for different aspects of contract review, from clause analysis to compliance checking, and leverages LangGraph for workflow orchestration and Pinecone for efficient clause retrieval and comparison.

Implementation 🛠️

Implements a sophisticated state-based workflow using LangGraph to coordinate multiple AI agents through contract analysis stages. The system features Pydantic models for data validation, vector storage with Pinecone for clause comparison, and LLM-based analysis for generating comprehensive contract reports. The implementation includes parallel processing capabilities and customizable report generation based on user requirements.

Additional Resources 📚
- YouTube Explanation
E2E Testing Agent

Overview 🔎

The E2E Testing Agent demonstrates how to build an AI-powered system that converts natural language test instructions into executable end-to-end web tests. Using LangGraph for workflow orchestration and Playwright for browser automation, the system enables users to specify test cases in plain English while handling the complexity of test generation and execution.

Implementation 🛠️

Implements a structured workflow using LangGraph to coordinate test generation, validation, and execution. The system features TypedDict state management, integration with Playwright for browser automation, and LLM-based code generation for converting natural language instructions into executable test scripts. The implementation includes DOM state analysis, error handling, and comprehensive test reporting.

Additional Resources 📚
- YouTube Explanation

🎨 Creative and Content Generation Agents

GIF Animation Generator Agent (LangGraph)

Overview 🔎

A GIF animation generator that integrates LangGraph for workflow management, GPT-4 for text generation, and DALL-E for image creation, producing custom animations from user prompts.

Implementation 🛠️

Utilizes LangGraph to orchestrate a workflow that generates character descriptions, plots, and image prompts using GPT-4, creates images with DALL-E 3, and assembles them into GIFs using PIL. Employs asynchronous programming for efficient parallel processing.
TTS Poem Generator Agent (LangGraph)

Overview 🔎

An advanced text-to-speech (TTS) agent using LangGraph and OpenAI's APIs classifies input text, processes it based on content type, and generates corresponding speech output.

Implementation 🛠️

Utilizes LangGraph to orchestrate a workflow that classifies input text using GPT models, applies content-specific processing, and converts the processed text to speech using OpenAI's TTS API. The system adapts its output based on the identified content type (general, poem, news, or joke).
Music Compositor Agent (LangGraph)

Overview 🔎

An AI Music Compositor using LangGraph and OpenAI's language models generates custom musical compositions based on user input. The system processes the input through specialized components, each contributing to the final musical piece, which is then converted to a playable MIDI file.

Implementation 🛠️

LangGraph orchestrates a workflow that transforms user input into a musical composition, using ChatOpenAI (GPT-4) to generate melody, harmony, and rhythm, which are then style-adapted. The final AI-generated composition is converted to a MIDI file using music21 and can be played back using pygame.
Content Intelligence: Multi-Platform Content Generation Agent

Overview 🔎

Content Intelligence demonstrates how to build an advanced content generation system that transforms input text into platform-optimized content across multiple social media channels. The system employs LangGraph for workflow orchestration to analyze content, conduct research, and generate tailored content while maintaining brand consistency across different platforms.

Implementation 🛠️

Implements a sophisticated workflow using LangGraph to coordinate multiple specialized nodes (Summary, Research, Platform-Specific) through the content generation process. The system features TypedDict and Pydantic models for state management, integration with Tavily Search for research enhancement, and platform-specific content generation using GPT-4. The implementation includes parallel processing for multiple platforms and customizable content templates.

Additional Resources 📚
- YouTube Explanation
Business Meme Generator Using LangGraph and Memegen.link

Overview 🔎

The Business Meme Generator demonstrates how to create an AI-powered system that generates contextually relevant memes based on company website analysis. Using LangGraph for workflow orchestration, the system combines Groq's Llama model for text analysis and the Memegen.link API to automatically produce brand-aligned memes for digital marketing.

Implementation 🛠️

Implements a state-managed workflow using LangGraph to coordinate website content analysis, meme concept generation, and image creation. The system features Pydantic models for data validation, asynchronous processing with aiohttp, and integration with external APIs (Groq, Memegen.link) to create a complete meme generation pipeline with customizable templates.

Additional Resources 📚
- YouTube Explanation
Murder Mystery Game with LLM Agents

Overview 🔎

A text-based detective game that utilizes autonomous LLM agents as interactive characters in a procedurally generated murder mystery. Drawing inspiration from the UNBOUNDED paper, the system creates unique scenarios each time, with players taking on the role of Sherlock Holmes to solve the case through character interviews and deductive reasoning.

Implementation 🛠️

Leverages two LangGraph workflows - a main game loop for story/character generation and game progression, and a conversation sub-graph for character interactions. The system uses a combination of LLM-powered narrative generation, character AI, and structured game mechanics to create an immersive investigative experience with replayable storylines.

Additional Resources 📚
- YouTube Explanation

📊 Analysis and Information Processing Agents

Memory-Enhanced Conversational Agent

Overview 🔎

A memory-enhanced conversational AI agent incorporates short-term and long-term memory systems to maintain context within conversations and across multiple sessions, improving interaction quality and personalization.

Implementation 🛠️

Integrates a language model with separate short-term and long-term memory stores, utilizes a prompt template incorporating both memory types, and employs a memory manager for storage and retrieval. The system includes an interaction loop that updates and utilizes memories for each response.
Multi-Agent Collaboration System

Overview 🔎

A multi-agent collaboration system combining historical research with data analysis, leveraging large language models to simulate specialized agents working together to answer complex historical questions.

Implementation 🛠️

Utilizes a base Agent class to create specialized HistoryResearchAgent and DataAnalysisAgent, orchestrated by a HistoryDataCollaborationSystem. The system follows a five-step process: historical context provision, data needs identification, historical data provision, data analysis, and final synthesis.
Self-Improving Agent

Overview 🔎

A Self-Improving Agent using LangChain engages in conversations, learns from interactions, and continuously improves its performance over time through reflection and adaptation.

Implementation 🛠️

Integrates a language model with chat history management, response generation, and a reflection mechanism. The system employs a learning system that incorporates insights from reflection to enhance future performance, creating a continuous improvement loop.
Task-Oriented Agent

Overview 🔎

A language model application using LangChain that summarizes text and translates the summary to Spanish, combining custom functions, structured tools, and an agent for efficient text processing.

Implementation 🛠️

Utilizes custom functions for summarization and translation, wrapped as structured tools. Employs a prompt template to guide the agent, which orchestrates the use of tools. An agent executor manages the process, taking input text and producing both an English summary and its Spanish translation.
Internet Search and Summarize Agent

Overview 🔎

An intelligent web research assistant that combines web search capabilities with AI-powered summarization, automating the process of gathering information from the internet and distilling it into concise, relevant summaries.

Implementation 🛠️

Integrates a web search module using DuckDuckGo's API, a result parser, and a text summarization engine leveraging OpenAI's language models. The system performs site-specific or general searches, extracts relevant content, generates concise summaries, and compiles attributed results for efficient information retrieval and synthesis.
Multi agent research team - Autogen

Overview 🔎

This technique explores a multi-agent system for collaborative research using the AutoGen library. It employs agents to solve tasks collaboratively, focusing on efficient execution and quality assurance. The system enhances research by distributing tasks among specialized agents.

Implementation 🛠️

Agents are configured with specific roles using the GPT-4 model, including admin, developer, planner, executor, and quality assurance. Interaction management ensures orderly communication with defined transitions. Task execution involves collaborative planning, coding, execution, and quality checking, demonstrating a scalable framework for various domains.

Additional Resources 📚
- comprehensive solution with UI
- Blogpost
Sales Call Analyzer

Overview 🔎

An intelligent system that automates the analysis of sales call recordings by combining audio transcription with advanced natural language processing. The analyzer transcribes audio using OpenAI's Whisper, processes the text using NLP techniques, and generates comprehensive reports including sentiment analysis, key phrases, pain points, and actionable recommendations to improve sales performance.

Implementation 🛠️

Utilizes multiple components in a structured workflow: OpenAI Whisper for audio transcription, CrewAI for task automation and agent management, and LangChain for orchestrating the analysis pipeline. The system processes audio through a series of steps from transcription to detailed analysis, leveraging custom agents and tasks to generate structured JSON reports containing insights about customer sentiment, sales opportunities, and recommended improvements.

Additional Resources 📚
- YouTube Explanation
Weather Emergency & Response System

Overview 🔎

A comprehensive system demonstrating two agent graph implementations for weather emergency response: a real-time graph processing live weather data, and a hybrid graph combining real and simulated data for testing high-severity scenarios. The system handles complete workflow from data gathering through emergency plan generation, with automated notifications and human verification steps.

Implementation 🛠️

Utilizes LangGraph for orchestrating complex workflows with state management, integrating OpenWeatherMap API for real-time data, and Gemini for analysis and response generation. The system incorporates email notifications, social media monitoring simulation, and severity-based routing with configurable human verification for low/medium severity events.

Additional Resources 📚
- YouTube Explanation
Self-Healing Codebase System

Overview 🔎

An intelligent system that automatically detects, diagnoses, and fixes runtime code errors using LangGraph workflow orchestration and ChromaDB vector storage. The system maintains a memory of encountered bugs and their fixes through vector embeddings, enabling pattern recognition for similar errors across the codebase.

Implementation 🛠️

Utilizes a state-based graph workflow that processes function definitions and runtime arguments through specialized nodes for error detection, code analysis, and fix generation. Incorporates ChromaDB for vector-based storage of bug patterns and fixes, with automated search and retrieval capabilities for similar error patterns, while maintaining code execution safety through structured validation steps.

Additional Resources 📚
- YouTube Explanation
DataScribe: AI-Powered Schema Explorer

Overview 🔎

An intelligent agent system that enables intuitive exploration and querying of relational databases through natural language interactions. The system utilizes a fleet of specialized agents, coordinated by a stateful Supervisor, to handle schema discovery, query planning, and data analysis tasks while maintaining contextual understanding through vector-based relationship graphs.

Implementation 🛠️

Leverages LangGraph for orchestrating a multi-agent workflow including discovery, inference, and planning agents, with NetworkX for relationship graph visualization and management. The system incorporates dynamic state management through TypedDict classes, maintains database context between sessions using a db_graph attribute, and includes safety measures to prevent unauthorized database modifications.

📰 News and Information Agents

News TL;DR using LangGraph

Overview 🔎

A news summarization system that generates concise TL;DR summaries of current events based on user queries. The system leverages large language models for decision making and summarization while integrating with news APIs to access up-to-date content, allowing users to quickly catch up on topics of interest through generated bullet-point summaries.

Implementation 🛠️

Utilizes LangGraph to orchestrate a workflow combining multiple components: GPT-4o-mini for generating search terms and article summaries, NewsAPI for retrieving article metadata, BeautifulSoup for web scraping article content, and Asyncio for concurrent processing. The system follows a structured pipeline from query processing through article selection and summarization, managing the flow between components to produce relevant TL;DRs of current news articles.

Additional Resources 📚
- YouTube Explanation
AInsight: AI/ML Weekly News Reporter

Overview 🔎

AInsight demonstrates how to build an intelligent news aggregation and summarization system using a multi-agent architecture. The system employs three specialized agents (NewsSearcher, Summarizer, Publisher) to automatically collect, process and summarize AI/ML news for general audiences through LangGraph-based workflow orchestration.

Implementation 🛠️

Implements a state-managed multi-agent system using LangGraph to coordinate the news collection (Tavily API), technical content summarization (GPT-4), and report generation processes. The system features modular architecture with TypedDict-based state management, external API integration, and markdown report generation with customizable templates.

Additional Resources 📚
- YouTube Explanation
Journalism-Focused AI Assistant

Overview 🔎

A specialized AI assistant that helps journalists tackle modern journalistic challenges like misinformation, bias, and information overload. The system integrates fact-checking, tone analysis, summarization, and grammar review tools to enhance the accuracy and efficiency of journalistic work while maintaining ethical reporting standards.

Implementation 🛠️

Leverages LangGraph to orchestrate a workflow of specialized components including language models for analysis and generation, web search integration via DuckDuckGo's API, document parsing tools like PyMuPDFLoader and WebBaseLoader, text splitting with RecursiveCharacterTextSplitter, and structured JSON outputs. Each component works together through a unified workflow to analyze content, verify facts, detect bias, extract quotes, and generate comprehensive reports.
Blog Writer (Open AI Swarm)

Overview 🔎

A multi-agent system for collaborative blog post creation using OpenAI's Swarm package. It leverages specialized agents to perform research, planning, writing, and editing tasks efficiently.

Implementation 🛠️

Utilizes OpenAI's Swarm Package to manage agent interactions. Includes an admin, researcher, planner, writer, and editor, each with specific roles. The system follows a structured workflow: topic setting, outlining, research, drafting, and editing. This approach enhances content creation through task distribution, specialization, and collaborative problem-solving.

Additional Resources 📚
- Swarm Repo
Podcast Internet Search and Generate Agent 🎙️

Overview 🔎

A two step agent that first searches the internet for a given topic and then generates a podcast on the topic found. The search step uses a search agent and search function to find the most relevant information. The second step uses a podcast generation agent and generation function to create a podcast on the topic found.

Implementation 🛠️

Utilizes LangGraph to orchestrate a two-step workflow. The first step involves a search agent and function to gather information from the internet. The second step uses a podcast generation agent and function to create a podcast based on the gathered information.

🛍️ Shopping and Product Analysis Agents

ShopGenie - Redefining Online Shopping Customer Experience

Overview 🔎

An AI-powered shopping assistant that helps customers make informed purchasing decisions even without domain expertise. The system analyzes product information from multiple sources, compares specifications and reviews, identifies the best option based on user needs, and delivers recommendations through email with supporting video reviews, creating a comprehensive shopping experience.

Implementation 🛠️

Uses LangGraph to orchestrate a workflow combining Tavily for web search, Llama-3.1-70B for structured data analysis and product comparison, and YouTube API for review video retrieval. The system processes search results through multiple nodes including schema mapping, product comparison, review identification, and email generation. Key features include structured Pydantic models for consistent data handling, retry mechanisms for robust API interactions, and email delivery through SMTP for sharing recommendations.

Additional Resources 📚
- YouTube Explanation
Car Buyer AI Agent

Overview 🔎

The Smart Product Buyer AI Agent demonstrates how to build an intelligent system that assists users in making informed purchasing decisions. Using LangGraph and LLM-based intelligence, the system processes user requirements, scrapes product listings from websites like AutoTrader, and provides detailed analysis and recommendations for car purchases.

Implementation 🛠️

Implements a state-based workflow using LangGraph to coordinate user interaction, web scraping, and decision support. The system features TypedDict state management, async web scraping with Playwright, and integrates with external APIs for comprehensive product analysis. The implementation includes a Gradio interface for real-time chat interaction and modular scraper architecture for easy extension to additional product categories.

Additional Resources 📚
- YouTube Explanation

🎯 Task Management and Productivity Agents

Taskifier - Intelligent Task Allocation & Management

Overview 🔎

An intelligent task management system that analyzes user work styles and creates personalized task breakdown strategies, born from the observation that procrastination often stems from task ambiguity among students and early-career professionals. The system evaluates historical work patterns, gathers relevant task information through web search, and generates customized step-by-step approaches to optimize productivity and reduce workflow paralysis.

Implementation 🛠️

Leverages LangGraph for orchestrating a multi-step workflow including work style analysis, information gathering via Tavily API, and customized plan generation. The system maintains state through the process, integrating historical work pattern data with fresh task research to output detailed, personalized task execution plans aligned with the user's natural working style.

Additional Resources 📚
- YouTube Explanation
Grocery Management Agents System

Overview 🔎

A multi-agent system built with CrewAI that automates grocery management tasks including receipt interpretation, expiration date tracking, inventory management, and recipe recommendations. The system uses specialized agents to extract data from receipts, estimate product shelf life, track consumption, and suggest recipes to minimize food waste.

Implementation 🛠️

Implements four specialized agents using CrewAI - a Receipt Interpreter that extracts item details from receipts, an Expiration Date Estimator that determines shelf life using online sources, a Grocery Tracker that maintains inventory based on consumption, and a Recipe Recommender that suggests meals using available ingredients. Each agent has specific tools and tasks orchestrated through a crew workflow.

Additional Resources 📚
- YouTube Explanation

🔍 Quality Assurance and Testing Agents

LangGraph-Based Systems Inspector

Overview 🔎

A comprehensive testing and validation tool for LangGraph-based applications that automatically analyzes system architecture, generates test cases, and identifies potential vulnerabilities through multi-agent inspection. The inspector employs specialized AI testers to evaluate different aspects of the system, from basic functionality to security concerns and edge cases.

Implementation 🛠️

Integrates LangGraph for workflow orchestration, multiple LLM-powered testing agents, and a structured evaluation pipeline that includes static analysis, test case generation, and results verification. The system uses Pydantic for data validation, NetworkX for graph representation, and implements a modular architecture that allows for parallel test execution and comprehensive result analysis.

Additional Resources 📚
- YouTube Explanation
- Blog Post
EU Green Deal FAQ Bot

Overview 🔎

The EU Green Deal FAQ Bot demonstrates how to build a RAG-based AI agent that helps businesses understand EU green deal policies. The system processes complex regulatory documents into manageable chunks and provides instant, accurate answers to common questions about environmental compliance, emissions reporting, and waste management requirements.

Implementation 🛠️

Implements a sophisticated RAG pipeline using FAISS vectorstore for document storage, semantic chunking for preprocessing, and multiple specialized agents (Retriever, Summarizer, Evaluator) for query processing. The system features query rephrasing for improved accuracy, cross-reference with gold Q&A datasets for answer validation, and comprehensive evaluation metrics to ensure response quality and relevance.

Additional Resources 📚
- YouTube Explanation
Systematic Review Automation System + Paper Draft Creation

Overview 🔎

A comprehensive system for automating academic systematic reviews using a directed graph architecture and LangChain components. The system generates complete, publication-ready systematic review papers, automatically processing everything from literature search through final draft generation with multiple revision cycles.

Implementation 🛠️

Utilizes a state-based graph workflow that handles paper search and selection (up to 3 papers), PDF processing, and generates a complete academic paper with all standard sections (abstract, introduction, methods, results, conclusions, references). The system incorporates multiple revision cycles with automated critique and improvement phases, all orchestrated through LangGraph state management.

Additional Resources 📚
- YouTube Explanation

🌟 Special Advanced Technique 🌟

Sophisticated Controllable Agent for Complex RAG Tasks 🤖

Overview 🔎

An advanced RAG solution designed to tackle complex questions that simple semantic similarity-based retrieval cannot solve. This approach uses a sophisticated deterministic graph as the "brain" 🧠 of a highly controllable autonomous agent, capable of answering non-trivial questions from your own data.

Implementation 🛠️

• Implement a multi-step process involving question anonymization, high-level planning, task breakdown, adaptive information retrieval and question answering, continuous re-planning, and rigorous answer verification to ensure grounded and accurate responses.

Getting Started

To begin exploring and building GenAI agents:

Clone this repository:

git clone https://github.com/NirDiamant/GenAI_Agents.git

Navigate to the technique you're interested in:
```
cd all_agents_tutorials/technique-name
```
Follow the detailed implementation guide in each technique's notebook.

Contributing

We welcome contributions from the community! If you have a new technique or improvement to suggest:

Fork the repository
Create your feature branch: git checkout -b feature/AmazingFeature
Commit your changes: git commit -m 'Add some AmazingFeature'
Push to the branch: git push origin feature/AmazingFeature
Open a pull request

Contributors

License

This project is licensed under a custom non-commercial license - see the LICENSE file for details.

⭐️ If you find this repository helpful, please consider giving it a star!

Keywords: GenAI, Generative AI, Agents, NLP, AI, Machine Learning, Natural Language Processing, LLM, Conversational AI, Task-Oriented AI

mostafaalagamy/Metrolist

2024-12-19T01:32:05Z

YouTube Music client for Android

Metrolist

YouTube Music client for Android

Screenshots

Features

Play any song or video from YT Music
Background playback
Personalized quick picks
Library management
Download and cache songs for offline playback
Search for songs, albums, artists, videos and playlists
Live lyrics
Login support
Skip silence
Import playlists
Audio normalization
Adjust tempo/pitch
Local playlist management
Reorder songs in playlist or queue
Light - Dark - black - Dynamic theme
Sleep timer
Material 3
etc.

Download Now

Warning

If you're in a region where YouTube Music is not supported, you won't be able to use this app unless you have a proxy or VPN to connect to a YTM supported region.

Q: Why Metrolist isn't showing in Android Auto?

Go to Android Auto's settings and tap multiple times on the version in the bottom to enable developer settings
In the three dots menu at the top-right of the screen, click "Developer settings"
Enable "Unknown sources"

Disclaimer

This project and its contents are not affiliated with, funded, authorized, endorsed by, or in any way associated with YouTube, Google LLC, Metrolist Group LLC or any of its affiliates and subsidiaries.

Any trademark, service mark, trade name, or other intellectual property rights used in this project are owned by the respective owners.

reportportal/reportportal

2024-12-19T01:32:27Z

Main Repository. ReportPortal starts here - see readme below.

DS4SD/docling-serve

2024-12-19T01:32:27Z

Running Docling as an API service

SpecterOps/BloodHound-Legacy

2024-12-20T01:36:29Z

Six Degrees of Domain Admin

Latest Version of BloodHound Community Edition is Released

For the latest version of BloodHound you may follow this link to the BloodHound Community Edition repository.

Deprecation Notice

This repository will be archived in the near future.

Getting Started with BloodHound

To get started with BloodHound, check out the BloodHound docs.

About BloodHound

BloodHound is a single page Javascript web application, built on top of Linkurious, compiled with Electron, with a Neo4j database fed by a C# data collector.

BloodHound uses graph theory to reveal the hidden and often unintended relationships within an Active Directory or Azure environment. Attackers can use BloodHound to easily identify highly complex attack paths that would otherwise be impossible to quickly identify. Defenders can use BloodHound to identify and eliminate those same attack paths. Both blue and red teams can use BloodHound to easily gain a deeper understanding of privilege relationships in an Active Directory or Azure environment.

BloodHound was created by @_wald0, @CptJesus, and @harmj0y.

BloodHound is maintained by the BloodHound Enterprise team.

About BloodHound Enterprise

BloodHound Enterprise is an Attack Path Management solution that continuously maps and quantifies Active Directory Attack Paths. You can remove millions, even billions of Attack Paths within your existing architecture and eliminate the attacker’s easiest, most reliable, and most attractive techniques.

Downloading BloodHound Binaries

Pre-Compiled BloodHound binaries can be found here.

The rolling release will always be updated to the most recent source. Tagged releases are considered "stable" but will likely not have new features or fixes.

Creating example data

A sample database generator can be found here

You can create your own example Active Directory environment using BadBlood.

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

XiaoMi/ha_xiaomi_home

2024-12-19T01:36:31Z

Xiaomi Home Integration for Home Assistant

English | 简体中文

Xiaomi Home Integration is an integrated component of Home Assistant supported by Xiaomi official. It allows you to use Xiaomi IoT smart devices in Home Assistant.

Installation

Home Assistant version requirement:

Core $\geq$ 2024.11.0

Operating System $\geq$ 13.0

Method 1: Git clone from GitHub

cd config
git clone https://github.com/XiaoMi/ha_xiaomi_home.git
cd ha_xiaomi_home
./install.sh /config

We recommend this installation method, for it is convenient to switch to a tag when updating xiaomi_home to a certain version.

For example, update to version v1.0.0

cd config/ha_xiaomi_home
git checkout v1.0.0
./install.sh /config

Method 2: HACS

HACS > Overflow Menu > Custom repositories > Repository: https://github.com/XiaoMi/ha_xiaomi_home.git & Category: Integration > ADD

Xiaomi Home has not been added to the HACS store as a default yet. It's coming soon.

Method 3: Manually installation via Samba / FTPS

Download and copy custom_components/xiaomi_home folder to config/custom_components folder in your Home Assistant.

Configuration

Login

Settings > Devices & services > ADD INTEGRATION > Search Xiaomi Home > NEXT > Click here to login > Sign in with Xiaomi account

Add MIoT Devices

After logging in successfully, a dialog box named "Select Home and Devices" pops up. You can select the home containing the device that you want to import in Home Assistant.

Multiple User Login

After a Xiaomi account login and its user configuration are completed, you can continue to add other Xiaomi accounts in the configured Xiaomi Home Integration page.

Method: Settings > Devices & services > Configured > Xiaomi Home > ADD HUB > NEXT > Click here to login > Sign in with Xiaomi account

Update Configurations

You can change the configurations in the "Configuration Options" dialog box, in which you can update your user nickname and the list of the devices importing from Xiaomi Home APP, etc.

Method: Settings > Devices & services > Configured > Xiaomi Home > CONFIGURE > Select the option to update

Debug Mode for Action

Method: Settings > Devices & services > Configured > Xiaomi Home > CONFIGURE > Debug mode for action

Security

FAQ

Does Xiaomi Home Integration support all Xiaomi Home devices?

Xiaomi Home Integration currently supports most categories of Home device. Only a few categories are not supported. They are Bluetooth device, infrared device and virtual device.
Does Xiaomi Home Integration support multiple Xiaomi accounts?

Yes, it supports multiple Xiaomi accounts. Furthermore, Xiaomi Home Integration allows that devices belonging to different accounts can be added to a same area.
Does Xiaomi Home Integration support local control?

Local control is implemented by Xiaomi Central Hub Gateway (firmware version 3.4.0_0000 above) or Xiaomi home devices with built-in central hub gateway (software version 0.8.0 above) inside. If you do not have a Xiaomi central hub gateway or other devices having central hub gateway function, all control commands are sent through Xiaomi Cloud. The firmware for Xiaomi central hub gateway including the built-in central hub gateway supporting Home Assistant local control feature has not been released yet. Please refer to MIoT team's notification for upgrade plans.

Xiaomi central hub gateway is only available in mainland China. In other regions, it is not available.

Xiaomi Home Integration can also implement partial local control by enabling Xiaomi LAN control function. Xiaomi LAN control function can only control IP devices (devices connected to the router via WiFi or ethernet cable) in the same local area network as Home Assistant. It cannot control BLE Mesh, ZigBee, etc. devices. This function may cause some abnormalities. We recommend not to use this function. Xiaomi LAN control function is enabled by Settings > Devices & services > Configured > Xiaomi Home > CONFIGURE > Update LAN control configuration

Xiaomi LAN control function is not restricted by region. It is available in all regions. However, if there is a central gateway in the local area network where Home Assistant is located, even Xiaomi LAN control function is enabled in the integration, it will not take effect.
In which regions is Xiaomi Home Integration available?

Xiaomi Home Integration can be used in the mainland of China, Europe, India, Russia, Singapore, and USA. As user data in Xiaomi Cloud of different regions is isolated, you need to choose your region when importing MIoT devices in the configuration process. Xiaomi Home Integration allows you to import devices of different regions to a same area.

Principle of Messaging

Control through the Cloud

Image 1: Cloud control architecture

Control locally

Image 2: Local control architecture

Mapping Relationship between MIoT-Spec-V2 and Home Assistant Entity

General Conversion

Property

format	access	value-list	value-range	Entity in Home Assistant
writable	string	-	-	Text
writable	bool	-	-	Switch
writable	not string & not bool	existent	-	Select
writable	not string & not bool	non-existent	existent	Number
not writable	-	-	-	Sensor

Event

MIoT-Spec-V2 event is transformed to Event entity in Home Assistant. The event's parameters are also passed to entity's _trigger_event.

Action

in	Entity in Home Assistant
empty	Button
not empty	Notify

If the debug mode for action is activated, the Text entity will be created when the "in" field in the action spec is not empty.

Specific Conversion

Device

The conversion follows SPEC_DEVICE_TRANS_MAP.

{
    '<device instance name>':{
        'required':{
            '<service instance name>':{
                'required':{
                    'properties': {
                        '<property instance name>': set<property access: str>
                    },
                    'events': set<event instance name: str>,
                    'actions': set<action instance name: str>
                },
                'optional':{
                    'properties': set<property instance name: str>,
                    'events': set<event instance name: str>,
                    'actions': set<action instance name: str>
                }
            }
        },
        'optional':{
            '<service instance name>':{
                'required':{
                    'properties': {
                        '<property instance name>': set<property access: str>
                    },
                    'events': set<event instance name: str>,
                    'actions': set<action instance name: str>
                },
                'optional':{
                    'properties': set<property instance name: str>,
                    'events': set<event instance name: str>,
                    'actions': set<action instance name: str>
                }
            }
        },
        'entity': str
    }
}

Home Assistant entity will not be created if MIoT-Spec-V2 device instance does not contain all required services, properties, events or actions.

Service

The conversion follows SPEC_SERVICE_TRANS_MAP.

{
    '<service instance name>':{
        'required':{
            'properties': {
                '<property instance name>': set<property access: str>
            },
            'events': set<event instance name: str>,
            'actions': set<action instance name: str>
        },
        'optional':{
            'properties': set<property instance name: str>,
            'events': set<event instance name: str>,
            'actions': set<action instance name: str>
        },
        'entity': str
    }
}

Home Assistant entity will not be created if MIoT-Spec-V2 service instance does not contain all required properties, events or actions.

Property

The conversion follows SPEC_PROP_TRANS_MAP.

{
    'entities':{
        '<entity name>':{
            'format': set<str>,
            'access': set<str>
        }
    },
    'properties': {
        '<property instance name>':{
            'device_class': str,
            'entity': str
        }
    }
}

Event

The conversion follows SPEC_EVENT_TRANS_MAP.

{
    '<event instance name>': str
}

The value of the event instance name indicates _attr_device_class of the Home Assistant entity to be created.

MIoT-Spec-V2 Filter

spec_filter.json is used to filter out the MIoT-Spec-V2 instance that will not be converted to Home Assistant entity.

The format of spec_filter.json is as follows.

{
    "<MIoT-Spec-V2 device instance>":{
        "services": list<service_iid: str>,
        "properties": list<service_iid.property_iid: str>,
        "events": list<service_iid.event_iid: str>,
        "actions": list<service_iid.action_iid: str>,
    }
}

Example:

{
    "urn:miot-spec-v2:device:television:0000A010:xiaomi-rmi1":{
        "services": ["*"]   # Filter out all services. It is equivalent to completely ignoring the device with such MIoT-Spec-V2 device instance.
    },
    "urn:miot-spec-v2:device:gateway:0000A019:xiaomi-hub1": {
        "services": ["3"],  # Filter out the service whose iid=3.
        "properties": ["4.*"]   # Filter out all properties in the service whose iid=4.
        "events": ["4.1"],  # Filter out the iid=1 event in the iid=4 service.
        "actions": ["4.1"]  # Filter out the iid=1 action in the iid=4 service.
    }
}

Device information service (urn:miot-spec-v2:service:device-information:00007801) of all devices will never be converted to Home Assistant entity.

Multiple Language Support

The format of multi_lang.json is as follows.

{
    "<MIoT-Spec-V2 device instance>": {
        "<language code>": {
            "<instance code>": <translation: str>
        }
    }
}

The key of multi_lang.json dictionary is the urn excluding the "version" field of the MIoT-Spec-V2 device instance.

The language code is zh-Hans, zh-Hant, en, es, ru, fr, de, or ja, corresponding to the 8 selectable languages mentioned above.

The instance code is the code of the MIoT-Spec-V2 instance, which is in the format of:

service:<siid>                  # service
service:<siid>:property:<piid>  # property
service:<siid>:property:<piid>:valuelist:<value> # the value in value-list of a property
service:<siid>:event:<eiid>     # event
service:<siid>:action:<aiid>    # action

siid, piid, eiid, aiid and value are all decimal three-digit integers.

Example:

{
    "urn:miot-spec-v2:device:health-pot:0000A051:chunmi-a1": {
        "zh-Hant": {
            "service:002": "養生壺",
            "service:002:property:001": "工作狀態",
            "service:002:property:001:valuelist:000": "待機中",
            "service:002:action:002": "停止烹飪",
            "service:005:event:001": "烹飪完成"
        }
    }
}

If you edit specv2entity.py, spec_filter.json or multi_lang.json in the custom_components/xiaomi_home/miot/specs directory in your Home Assistant, you need to update the entity conversion rule in the integration's CONFIGURE page to take effect. Method: Settings > Devices & services > Configured > Xiaomi Home > CONFIGURE > Update Entity Conversion Rule

Documents

License
Contribution Guidelines: English | 简体中文
ChangeLog
Development Documents: https://developers.home-assistant.io/docs/creating_component_index

Directory Structure

miot: core code.
miot/miot_client: Adding a login user in the integration needs adding a miot_client instance.
miot/miot_cloud: Contains functions related to the cloud service, including OAuth login process, HTTP interface functions (to get the user information, to send the device control command, etc.)
miot/miot_device: Device entity, including device information, processing logic of property, event and action.
miot/miot_mips: Message bus for subscribing and publishing method.
miot/miot_spec: Parse MIoT-Spec-V2.
miot/miot_lan: Device LAN control, including device discovery, device control, etc.
miot/miot_mdns: Central hub gateway service LAN discovery.
miot/miot_network: Obtain network status and network information.
miot/miot_storage: File storage for the integration.
miot/test: Test scripts.
config_flow: Config flow.

seleniumbase/SeleniumBase

2024-12-19T01:36:31Z

Python APIs for web automation, testing, and bypassing bot-detection.

SeleniumBase

All-in-one Browser Automation Framework:
Web Crawling / Testing / Scraping / Stealth

SeleniumBase is the professional toolkit for web automation activities. Built for testing websites, bypassing CAPTCHAs, enhancing productivity, completing tasks, and scaling your business.

📚 Learn from over 200 examples in the SeleniumBase/examples/ folder.

🐙 Note that UC Mode / CDP Mode (Stealth Mode) have their own ReadMe files.

📗 Here's raw_google.py, which performs a Google search:

from seleniumbase import SB

with SB(test=True) as sb:
    sb.open("https://google.com/ncr")
    sb.type('[title="Search"]', "SeleniumBase GitHub page\n")
    sb.click('[href*="github.com/seleniumbase/"]')
    sb.save_screenshot_to_logs()  # ./latest_logs/
    print(sb.get_page_title())

python raw_google.py

📗 Here's an example of bypassing Cloudflare's challenge page: SeleniumBase/examples/cdp_mode/raw_gitlab.py

from seleniumbase import SB

with SB(uc=True, test=True, locale_code="en") as sb:
    url = "https://gitlab.com/users/sign_in"
    sb.activate_cdp_mode(url)
    sb.uc_gui_click_captcha()
    sb.sleep(2)

📗 Here's test_get_swag.py, which tests an e-commerce site:

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)  # Call pytest

class MyTestClass(BaseCase):
    def test_swag_labs(self):
        self.open("https://www.saucedemo.com")
        self.type("#user-name", "standard_user")
        self.type("#password", "secret_sauce\n")
        self.assert_element("div.inventory_list")
        self.click('button[name*="backpack"]')
        self.click("#shopping_cart_container a")
        self.assert_text("Backpack", "div.cart_item")
        self.click("button#checkout")
        self.type("input#first-name", "SeleniumBase")
        self.type("input#last-name", "Automation")
        self.type("input#postal-code", "77123")
        self.click("input#continue")
        self.click("button#finish")
        self.assert_text("Thank you for your order!")

pytest test_get_swag.py

(The default browser is --chrome if not set.)

📗 Here's test_coffee_cart.py, which verifies an e-commerce site:

pytest test_coffee_cart.py --demo

(--demo mode slows down tests and highlights actions)

📗 Here's test_demo_site.py, which covers several actions:

pytest test_demo_site.py

Easy to type, click, select, toggle, drag & drop, and more.

(For more examples, see the SeleniumBase/examples/ folder.)

Explore the README:

Get Started / Installation

Basic Example / Usage

Common Test Methods

Fun Facts / Learn More

Demo Mode / Debugging

Command-line Options

Directory Configuration

SeleniumBase Dashboard

Generating Test Reports

▶️ How is SeleniumBase different from raw Selenium? (click to expand)

💡 SeleniumBase automatically detects between CSS Selectors and XPath, which means you don't need to specify the type of selector in your commands (but optionally you could).

💡 SeleniumBase gives you clean error output when a test fails. With raw Selenium, error messages can get very messy.

💡 SeleniumBase includes desktop GUI apps for running tests, such as SeleniumBase Commander for pytest and SeleniumBase Behave GUI for behave.

💡 SeleniumBase has its own Recorder / Test Generator for creating tests from manual browser actions.

💡 SeleniumBase comes with test case management software, ("CasePlans"), for organizing tests and step descriptions.

💡 SeleniumBase includes tools for building data apps, ("ChartMaker"), which can generate JavaScript from Python.

📚 Learn about different ways of writing tests:

📗📝 Here's test_simple_login.py, which uses BaseCase class inheritance, and runs with pytest or pynose. (Use self.driver to access Selenium's raw driver.)

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)

class TestSimpleLogin(BaseCase):
    def test_simple_login(self):
        self.open("seleniumbase.io/simple/login")
        self.type("#username", "demo_user")
        self.type("#password", "secret_pass")
        self.click('a:contains("Sign in")')
        self.assert_exact_text("Welcome!", "h1")
        self.assert_element("img#image1")
        self.highlight("#image1")
        self.click_link("Sign out")
        self.assert_text("signed out", "#top_message")

📘📝 Here's raw_login_sb.py, which uses the SB Context Manager. Runs with pure python. (Use sb.driver to access Selenium's raw driver.)

from seleniumbase import SB

with SB() as sb:
    sb.open("seleniumbase.io/simple/login")
    sb.type("#username", "demo_user")
    sb.type("#password", "secret_pass")
    sb.click('a:contains("Sign in")')
    sb.assert_exact_text("Welcome!", "h1")
    sb.assert_element("img#image1")
    sb.highlight("#image1")
    sb.click_link("Sign out")
    sb.assert_text("signed out", "#top_message")

📙📝 Here's raw_login_driver.py, which uses the Driver Manager. Runs with pure python. (The driver is an improved version of Selenium's raw driver, with more methods.)

from seleniumbase import Driver

driver = Driver()
try:
    driver.open("seleniumbase.io/simple/login")
    driver.type("#username", "demo_user")
    driver.type("#password", "secret_pass")
    driver.click('a:contains("Sign in")')
    driver.assert_exact_text("Welcome!", "h1")
    driver.assert_element("img#image1")
    driver.highlight("#image1")
    driver.click_link("Sign out")
    driver.assert_text("signed out", "#top_message")
finally:
    driver.quit()

Set up Python & Git:

🔵 Add Python and Git to your System PATH.

🔵 Using a Python virtual env is recommended.

Install SeleniumBase:

You can install seleniumbase from PyPI or GitHub:

🔵 How to install seleniumbase from PyPI:

pip install seleniumbase

(Add --upgrade OR -U to upgrade SeleniumBase.)
(Add --force-reinstall to upgrade indirect packages.)
(Use pip3 if multiple versions of Python are present.)

🔵 How to install seleniumbase from a GitHub clone:

git clone https://github.com/seleniumbase/SeleniumBase.git
cd SeleniumBase/
pip install -e .

🔵 How to upgrade an existing install from a GitHub clone:

git pull
pip install -e .

🔵 Type seleniumbase or sbase to verify that SeleniumBase was installed successfully:

 ___      _          _             ___              
/ __| ___| |___ _ _ (_)_  _ _ __  | _ ) __ _ ______ 
\__ \/ -_) / -_) ' \| | \| | '  \ | _ \/ _` (_-< -_)
|___/\___|_\___|_||_|_|\_,_|_|_|_\|___/\__,_/__|___|
----------------------------------------------------

╭──────────────────────────────────────────────────╮
│  * USAGE: "seleniumbase [COMMAND] [PARAMETERS]"  │
│  *    OR:        "sbase [COMMAND] [PARAMETERS]"  │
│                                                  │
│ COMMANDS:        PARAMETERS / DESCRIPTIONS:      │
│    get / install    [DRIVER_NAME] [OPTIONS]      │
│    methods          (List common Python methods) │
│    options          (List common pytest options) │
│    behave-options   (List common behave options) │
│    gui / commander  [OPTIONAL PATH or TEST FILE] │
│    behave-gui       (SBase Commander for Behave) │
│    caseplans        [OPTIONAL PATH or TEST FILE] │
│    mkdir            [DIRECTORY] [OPTIONS]        │
│    mkfile           [FILE.py] [OPTIONS]          │
│    mkrec / codegen  [FILE.py] [OPTIONS]          │
│    recorder         (Open Recorder Desktop App.) │
│    record           (If args: mkrec. Else: App.) │
│    mkpres           [FILE.py] [LANG]             │
│    mkchart          [FILE.py] [LANG]             │
│    print            [FILE] [OPTIONS]             │
│    translate        [SB_FILE.py] [LANG] [ACTION] │
│    convert          [WEBDRIVER_UNITTEST_FILE.py] │
│    extract-objects  [SB_FILE.py]                 │
│    inject-objects   [SB_FILE.py] [OPTIONS]       │
│    objectify        [SB_FILE.py] [OPTIONS]       │
│    revert-objects   [SB_FILE.py] [OPTIONS]       │
│    encrypt / obfuscate                           │
│    decrypt / unobfuscate                         │
│    proxy            (Start a basic proxy server) │
│    download server  (Get Selenium Grid JAR file) │
│    grid-hub         [start|stop] [OPTIONS]       │
│    grid-node        [start|stop] --hub=[HOST/IP] │
│                                                  │
│ *  EXAMPLE => "sbase get chromedriver stable"    │
│ *  For command info => "sbase help [COMMAND]"    │
│ *  For info on all commands => "sbase --help"    │
╰──────────────────────────────────────────────────╯

🔵 Downloading webdrivers:

✅ SeleniumBase automatically downloads webdrivers as needed, such as chromedriver.

▶️ Here's sample output from a chromedriver download. (click to expand)

*** chromedriver to download = 131.0.6778.108 (Latest Stable) 

Downloading chromedriver-mac-arm64.zip from:
https://storage.googleapis.com/chrome-for-testing-public/131.0.6778.108/mac-arm64/chromedriver-mac-arm64.zip ...
Download Complete!

Extracting ['chromedriver'] from chromedriver-mac-arm64.zip ...
Unzip Complete!

The file [chromedriver] was saved to:
~/github/SeleniumBase/seleniumbase/drivers/
chromedriver

Making [chromedriver 131.0.6778.108] executable ...
[chromedriver 131.0.6778.108] is now ready for use!

Basic Example / Usage:

🔵 If you've cloned SeleniumBase, you can run tests from the examples/ folder.

Here's my_first_test.py:

cd examples/
pytest my_first_test.py

Here's the full code for my_first_test.py:

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)

class MyTestClass(BaseCase):
    def test_swag_labs(self):
        self.open("https://www.saucedemo.com")
        self.type("#user-name", "standard_user")
        self.type("#password", "secret_sauce\n")
        self.assert_element("div.inventory_list")
        self.assert_exact_text("Products", "span.title")
        self.click('button[name*="backpack"]')
        self.click("#shopping_cart_container a")
        self.assert_exact_text("Your Cart", "span.title")
        self.assert_text("Backpack", "div.cart_item")
        self.click("button#checkout")
        self.type("#first-name", "SeleniumBase")
        self.type("#last-name", "Automation")
        self.type("#postal-code", "77123")
        self.click("input#continue")
        self.assert_text("Checkout: Overview")
        self.assert_text("Backpack", "div.cart_item")
        self.assert_text("29.99", "div.inventory_item_price")
        self.click("button#finish")
        self.assert_exact_text("Thank you for your order!", "h2")
        self.assert_element('img[alt="Pony Express"]')
        self.js_click("a#logout_sidebar_link")
        self.assert_element("div#login_button_container")

By default, CSS Selectors are used for finding page elements.
If you're new to CSS Selectors, games like CSS Diner can help you learn.
For more reading, here's an advanced guide on CSS attribute selectors.

Here are some common SeleniumBase methods:

self.open(url)  # Navigate the browser window to the URL.
self.type(selector, text)  # Update the field with the text.
self.click(selector)  # Click the element with the selector.
self.click_link(link_text)  # Click the link containing text.
self.go_back()  # Navigate back to the previous URL.
self.select_option_by_text(dropdown_selector, option)
self.hover_and_click(hover_selector, click_selector)
self.drag_and_drop(drag_selector, drop_selector)
self.get_text(selector)  # Get the text from the element.
self.get_current_url()  # Get the URL of the current page.
self.get_page_source()  # Get the HTML of the current page.
self.get_attribute(selector, attribute)  # Get element attribute.
self.get_title()  # Get the title of the current page.
self.switch_to_frame(frame)  # Switch into the iframe container.
self.switch_to_default_content()  # Leave the iframe container.
self.open_new_window()  # Open a new window in the same browser.
self.switch_to_window(window)  # Switch to the browser window.
self.switch_to_default_window()  # Switch to the original window.
self.get_new_driver(OPTIONS)  # Open a new driver with OPTIONS.
self.switch_to_driver(driver)  # Switch to the browser driver.
self.switch_to_default_driver()  # Switch to the original driver.
self.wait_for_element(selector)  # Wait until element is visible.
self.is_element_visible(selector)  # Return element visibility.
self.is_text_visible(text, selector)  # Return text visibility.
self.sleep(seconds)  # Do nothing for the given amount of time.
self.save_screenshot(name)  # Save a screenshot in .png format.
self.assert_element(selector)  # Verify the element is visible.
self.assert_text(text, selector)  # Verify text in the element.
self.assert_exact_text(text, selector)  # Verify text is exact.
self.assert_title(title)  # Verify the title of the web page.
self.assert_downloaded_file(file)  # Verify file was downloaded.
self.assert_no_404_errors()  # Verify there are no broken links.
self.assert_no_js_errors()  # Verify there are no JS errors.

🔵 For the complete list of SeleniumBase methods, see: Method Summary

Fun Facts / Learn More:

✅ SeleniumBase automatically handles common WebDriver actions such as launching web browsers before tests, saving screenshots during failures, and closing web browsers after tests.

✅ SeleniumBase lets you customize tests via command-line options.

✅ SeleniumBase uses simple syntax for commands. Example:

self.type("input", "dogs\n")  # (The "\n" presses ENTER)

pytest coffee_cart_tests.py --rs
pytest test_sb_fixture.py --demo
pytest test_suite.py --rs --html=report.html --dashboard

pynose basic_test.py --mobile
pynose test_suite.py --headless --report --show-report

python raw_sb.py
python raw_test_scripts.py

behave realworld.feature
behave calculator.feature -D rs -D dashboard

Python files that start with test_ or end with _test.py.
Python methods that start with test_.

✅ You can do a pre-flight check to see which tests would get discovered by pytest before the actual run:

pytest --co -q

✅ You can be more specific when calling pytest or pynose on a file:

pytest [FILE_NAME.py]::[CLASS_NAME]::[METHOD_NAME]

pynose [FILE_NAME.py]:[CLASS_NAME].[METHOD_NAME]

✅ SeleniumBase supports all major browsers and operating systems:

Browsers: Chrome, Edge, Firefox, and Safari.

Systems: Linux/Ubuntu, macOS, and Windows.

✅ SeleniumBase works on all popular CI/CD platforms:

✅ SeleniumBase includes an automated/manual hybrid solution called MasterQA to speed up manual testing with automation while manual testers handle validation.

✅ SeleniumBase supports running tests while offline (assuming webdrivers have previously been downloaded when online).

✅ For a full list of SeleniumBase features, Click Here.

Demo Mode / Debugging:

pytest my_first_test.py --demo

🔵 time.sleep(seconds) can be used to make a test wait at a specific spot:

import time; time.sleep(3)  # Do nothing for 3 seconds.

🔵 Debug Mode with Python's built-in pdb library helps you debug tests:

import pdb; pdb.set_trace()
import pytest; pytest.set_trace()
breakpoint()  # Shortcut for "import pdb; pdb.set_trace()"

(pdb commands: n, c, s, u, d => next, continue, step, up, down)

🔵 To pause an active test that throws an exception or error, (and keep the browser window open while Debug Mode begins in the console), add --pdb as a pytest option:

pytest test_fail.py --pdb

🔵 To start tests in Debug Mode, add --trace as a pytest option:

pytest test_coffee_cart.py --trace

🔵 Command-line Options:

✅ Here are some useful command-line options that come with pytest:

-v  # Verbose mode. Prints the full name of each test and shows more details.
-q  # Quiet mode. Print fewer details in the console output when running tests.
-x  # Stop running the tests after the first failure is reached.
--html=report.html  # Creates a detailed pytest-html report after tests finish.
--co | --collect-only  # Show what tests would get run. (Without running them)
--co -q  # (Both options together!) - Do a dry run with full test names shown.
-n=NUM  # Multithread the tests using that many threads. (Speed up test runs!)
-s  # See print statements. (Should be on by default with pytest.ini present.)
--junit-xml=report.xml  # Creates a junit-xml report after tests finish.
--pdb  # If a test fails, enter Post Mortem Debug Mode. (Don't use with CI!)
--trace  # Enter Debug Mode at the beginning of each test. (Don't use with CI!)
-m=MARKER  # Run tests with the specified pytest marker.

✅ SeleniumBase provides additional pytest command-line options for tests:

--browser=BROWSER  # (The web browser to use. Default: "chrome".)
--chrome  # (Shortcut for "--browser=chrome". On by default.)
--edge  # (Shortcut for "--browser=edge".)
--firefox  # (Shortcut for "--browser=firefox".)
--safari  # (Shortcut for "--browser=safari".)
--settings-file=FILE  # (Override default SeleniumBase settings.)
--env=ENV  # (Set the test env. Access with "self.env" in tests.)
--account=STR  # (Set account. Access with "self.account" in tests.)
--data=STRING  # (Extra test data. Access with "self.data" in tests.)
--var1=STRING  # (Extra test data. Access with "self.var1" in tests.)
--var2=STRING  # (Extra test data. Access with "self.var2" in tests.)
--var3=STRING  # (Extra test data. Access with "self.var3" in tests.)
--variables=DICT  # (Extra test data. Access with "self.variables".)
--user-data-dir=DIR  # (Set the Chrome user data directory to use.)
--protocol=PROTOCOL  # (The Selenium Grid protocol: http|https.)
--server=SERVER  # (The Selenium Grid server/IP used for tests.)
--port=PORT  # (The Selenium Grid port used by the test server.)
--cap-file=FILE  # (The web browser's desired capabilities to use.)
--cap-string=STRING  # (The web browser's desired capabilities to use.)
--proxy=SERVER:PORT  # (Connect to a proxy server:port as tests are running)
--proxy=USERNAME:PASSWORD@SERVER:PORT  # (Use an authenticated proxy server)
--proxy-bypass-list=STRING # (";"-separated hosts to bypass, Eg "*.foo.com")
--proxy-pac-url=URL  # (Connect to a proxy server using a PAC_URL.pac file.)
--proxy-pac-url=USERNAME:PASSWORD@URL  # (Authenticated proxy with PAC URL.)
--proxy-driver  # (If a driver download is needed, will use: --proxy=PROXY.)
--multi-proxy  # (Allow multiple authenticated proxies when multi-threaded.)
--agent=STRING  # (Modify the web browser's User-Agent string.)
--mobile  # (Use the mobile device emulator while running tests.)
--metrics=STRING  # (Set mobile metrics: "CSSWidth,CSSHeight,PixelRatio".)
--chromium-arg="ARG=N,ARG2"  # (Set Chromium args, ","-separated, no spaces.)
--firefox-arg="ARG=N,ARG2"  # (Set Firefox args, comma-separated, no spaces.)
--firefox-pref=SET  # (Set a Firefox preference:value set, comma-separated.)
--extension-zip=ZIP  # (Load a Chrome Extension .zip|.crx, comma-separated.)
--extension-dir=DIR  # (Load a Chrome Extension directory, comma-separated.)
--disable-features="F1,F2"  # (Disable features, comma-separated, no spaces.)
--binary-location=PATH  # (Set path of the Chromium browser binary to use.)
--driver-version=VER  # (Set the chromedriver or uc_driver version to use.)
--sjw  # (Skip JS Waits for readyState to be "complete" or Angular to load.)
--wfa  # (Wait for AngularJS to be done loading after specific web actions.)
--pls=PLS  # (Set pageLoadStrategy on Chrome: "normal", "eager", or "none".)
--headless  # (The default headless mode. Linux uses this mode by default.)
--headless1  # (Use Chrome's old headless mode. Fast, but has limitations.)
--headless2  # (Use Chrome's new headless mode, which supports extensions.)
--headed  # (Run tests in headed/GUI mode on Linux OS, where not default.)
--xvfb  # (Run tests using the Xvfb virtual display server on Linux OS.)
--xvfb-metrics=STRING  # (Set Xvfb display size on Linux: "Width,Height".)
--locale=LOCALE_CODE  # (Set the Language Locale Code for the web browser.)
--interval=SECONDS  # (The autoplay interval for presentations & tour steps)
--start-page=URL  # (The starting URL for the web browser when tests begin.)
--archive-logs  # (Archive existing log files instead of deleting them.)
--archive-downloads  # (Archive old downloads instead of deleting them.)
--time-limit=SECONDS  # (Safely fail any test that exceeds the time limit.)
--slow  # (Slow down the automation. Faster than using Demo Mode.)
--demo  # (Slow down and visually see test actions as they occur.)
--demo-sleep=SECONDS  # (Set the wait time after Slow & Demo Mode actions.)
--highlights=NUM  # (Number of highlight animations for Demo Mode actions.)
--message-duration=SECONDS  # (The time length for Messenger alerts.)
--check-js  # (Check for JavaScript errors after page loads.)
--ad-block  # (Block some types of display ads from loading.)
--host-resolver-rules=RULES  # (Set host-resolver-rules, comma-separated.)
--block-images  # (Block images from loading during tests.)
--do-not-track  # (Indicate to websites that you don't want to be tracked.)
--verify-delay=SECONDS  # (The delay before MasterQA verification checks.)
--ee | --esc-end  # (Lets the user end the current test via the ESC key.)
--recorder  # (Enables the Recorder for turning browser actions into code.)
--rec-behave  # (Same as Recorder Mode, but also generates behave-gherkin.)
--rec-sleep  # (If the Recorder is enabled, also records self.sleep calls.)
--rec-print  # (If the Recorder is enabled, prints output after tests end.)
--disable-cookies  # (Disable Cookies on websites. Pages might break!)
--disable-js  # (Disable JavaScript on websites. Pages might break!)
--disable-csp  # (Disable the Content Security Policy of websites.)
--disable-ws  # (Disable Web Security on Chromium-based browsers.)
--enable-ws  # (Enable Web Security on Chromium-based browsers.)
--enable-sync  # (Enable "Chrome Sync" on websites.)
--uc | --undetected  # (Use undetected-chromedriver to evade bot-detection.)
--uc-cdp-events  # (Capture CDP events when running in "--undetected" mode.)
--log-cdp  # ("goog:loggingPrefs", {"performance": "ALL", "browser": "ALL"})
--remote-debug  # (Sync to Chrome Remote Debugger chrome://inspect/#devices)
--ftrace | --final-trace  # (Debug Mode after each test. Don't use with CI!)
--dashboard  # (Enable the SeleniumBase Dashboard. Saved at: dashboard.html)
--dash-title=STRING  # (Set the title shown for the generated dashboard.)
--enable-3d-apis  # (Enables WebGL and 3D APIs.)
--swiftshader  # (Chrome "--use-gl=angle" / "--use-angle=swiftshader-webgl")
--incognito  # (Enable Chrome's Incognito mode.)
--guest  # (Enable Chrome's Guest mode.)
--dark  # (Enable Chrome's Dark mode.)
--devtools  # (Open Chrome's DevTools when the browser opens.)
--rs | --reuse-session  # (Reuse browser session for all tests.)
--rcs | --reuse-class-session  # (Reuse session for tests in class.)
--crumbs  # (Delete all cookies between tests reusing a session.)
--disable-beforeunload  # (Disable the "beforeunload" event on Chrome.)
--window-position=X,Y  # (Set the browser's starting window position.)
--window-size=WIDTH,HEIGHT  # (Set the browser's starting window size.)
--maximize  # (Start tests with the browser window maximized.)
--screenshot  # (Save a screenshot at the end of each test.)
--no-screenshot  # (No screenshots saved unless tests directly ask it.)
--visual-baseline  # (Set the visual baseline for Visual/Layout tests.)
--wire  # (Use selenium-wire's webdriver for replacing selenium webdriver.)
--external-pdf  # (Set Chromium "plugins.always_open_pdf_externally":True.)
--timeout-multiplier=MULTIPLIER  # (Multiplies the default timeout values.)
--list-fail-page  # (After each failing test, list the URL of the failure.)

(See the full list of command-line option definitions here. For detailed examples of command-line options, see customizing_test_runs.md)

cd examples/

pytest test_suite.py --chrome

pytest test_suite.py --firefox

🔵 To pass additional data from the command-line to tests, add --data="ANY STRING". Inside your tests, you can use self.data to access that.

Directory Configuration:

🔵 sbase mkdir DIR creates a folder with config files and sample tests:

sbase mkdir ui_tests

That new folder will have these files:

ui_tests/
├── __init__.py
├── my_first_test.py
├── parameterized_test.py
├── pytest.ini
├── requirements.txt
├── setup.cfg
├── test_demo_site.py
└── boilerplates/
    ├── __init__.py
    ├── base_test_case.py
    ├── boilerplate_test.py
    ├── classic_obj_test.py
    ├── page_objects.py
    ├── sb_fixture_test.py
    └── samples/
        ├── __init__.py
        ├── google_objects.py
        ├── google_test.py
        ├── sb_swag_test.py
        └── swag_labs_test.py

ProTip™: You can also create a boilerplate folder without any sample tests in it by adding -b or --basic to the sbase mkdir command:

sbase mkdir ui_tests --basic

That new folder will have these files:

ui_tests/
├── __init__.py
├── pytest.ini
├── requirements.txt
└── setup.cfg

Log files from failed tests:

Let's try an example of a test that fails:

""" test_fail.py """
from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)

class MyTestClass(BaseCase):

    def test_find_army_of_robots_on_xkcd_desert_island(self):
        self.open("https://xkcd.com/731/")
        self.assert_element("div#ARMY_OF_ROBOTS", timeout=1)  # This should fail

You can run it from the examples/ folder like this:

pytest test_fail.py

SeleniumBase Dashboard:

🔵 The --dashboard option for pytest generates a SeleniumBase Dashboard located at dashboard.html, which updates automatically as tests run and produce results. Example:

pytest --dashboard --rs --headless

🔵 Additionally, you can host your own SeleniumBase Dashboard Server on a port of your choice. Here's an example of that using Python's http.server:

python -m http.server 1948

🔵 Here's a full example of what the SeleniumBase Dashboard may look like:

pytest test_suite.py test_image_saving.py --dashboard --rs --headless

Generating Test Reports:

🔵 `pytest` HTML Reports:

✅ Using --html=report.html gives you a fancy report of the name specified after your test suite completes.

pytest test_suite.py --html=report.html

✅ Here's an example of an upgraded html report:

pytest test_suite.py --dashboard --html=report.html

If viewing pytest html reports in Jenkins, you may need to configure Jenkins settings for the html to render correctly. This is due to Jenkins CSP changes.

You can also use --junit-xml=report.xml to get an xml report instead. Jenkins can use this file to display better reporting for your tests.

pytest test_suite.py --junit-xml=report.xml

🔵 `pynose` Reports:

The --report option gives you a fancy report after your test suite completes.

pynose test_suite.py --report

(NOTE: You can add --show-report to immediately display pynose reports after the test suite completes. Only use --show-report when running tests locally because it pauses the test run.)

🔵 `behave` Dashboard & Reports:

(The behave_bdd/ folder can be found in the examples/ folder.)

behave behave_bdd/features/ -D dashboard -D headless

You can also use --junit to get .xml reports for each behave feature. Jenkins can use these files to display better reporting for your tests.

behave behave_bdd/features/ --junit -D rs -D headless

🔵 Allure Reports:

See: https://allurereport.org/docs/pytest/

SeleniumBase no longer includes allure-pytest as part of installed dependencies. If you want to use it, install it first:

pip install allure-pytest

Now your tests can create Allure results files, which can be processed by Allure Reports.

pytest test_suite.py --alluredir=allure_results

Using a Proxy Server:

If you wish to use a proxy server for your browser tests (Chromium or Firefox), you can add --proxy=IP_ADDRESS:PORT as an argument on the command line.

pytest proxy_test.py --proxy=IP_ADDRESS:PORT

If the proxy server that you wish to use requires authentication, you can do the following (Chromium only):

pytest proxy_test.py --proxy=USERNAME:PASSWORD@IP_ADDRESS:PORT

SeleniumBase also supports SOCKS4 and SOCKS5 proxies:

pytest proxy_test.py --proxy="socks4://IP_ADDRESS:PORT"

pytest proxy_test.py --proxy="socks5://IP_ADDRESS:PORT"

To make things easier, you can add your frequently-used proxies to PROXY_LIST in proxy_list.py, and then use --proxy=KEY_FROM_PROXY_LIST to use the IP_ADDRESS:PORT of that key.

pytest proxy_test.py --proxy=proxy1

Changing the User-Agent:

🔵 If you wish to change the User-Agent for your browser tests (Chromium and Firefox only), you can add --agent="USER AGENT STRING" as an argument on the command-line.

pytest user_agent_test.py --agent="Mozilla/5.0 (Nintendo 3DS; U; ; en) Version/1.7412.EU"

Handling Pop-Up Alerts:

Building Guided Tours for Websites:

🔵 Learn about SeleniumBase Interactive Walkthroughs (in the examples/tour_examples/ folder). It's great for prototyping a website onboarding experience.

Production Environments & Integrations:

▶️ Here are some things you can do to set up a production environment for your testing. (click to expand)

You can set up a Jenkins build server for running tests at regular intervals. For a real-world Jenkins example of headless browser automation in action, check out the SeleniumBase Jenkins example on Azure or the SeleniumBase Jenkins example on Google Cloud.
You can use the Selenium Grid to scale your testing by distributing tests on several machines with parallel execution. To do this, check out the SeleniumBase selenium_grid folder, which should have everything you need, including the Selenium Grid ReadMe to help you get started.
If you're using the SeleniumBase MySQL feature to save results from tests running on a server machine, you can install MySQL Workbench to help you read & write from your DB more easily.
If you're using AWS, you can set up an Amazon S3 account for saving log files and screenshots from your tests. To activate this feature, modify settings.py with connection details in the S3 section, and add --with-s3-logging on the command-line when running your tests.

Here's an example of running tests with some additional features enabled:

pytest [YOUR_TEST_FILE.py] --with-db-reporting --with-s3-logging

Detailed Method Specifications and Examples:

🔵 Navigating to a web page: (and related commands)

self.open("https://xkcd.com/378/")  # This method opens the specified page.

self.go_back()  # This method navigates the browser to the previous page.

self.go_forward()  # This method navigates the browser forward in history.

self.refresh_page()  # This method reloads the current page.

self.get_current_url()  # This method returns the current page URL.

self.get_page_source()  # This method returns the current page source.

source = self.get_page_source()
head_open_tag = source.find('<head>')
head_close_tag = source.find('</head>', head_open_tag)
everything_inside_head = source[head_open_tag+len('<head>'):head_close_tag]

🔵 Clicking:

To click an element on the page:

self.click("div#my_id")

ProTip™: In most web browsers, you can right-click on a page and select Inspect Element to see the CSS selector details that you'll need to create your own scripts.

🔵 Typing Text:

self.type(selector, text) # updates the text from the specified element with the specified value. An exception is raised if the element is missing or if the text field is not editable. Example:

self.type("input#id_value", "2012")

You can also use self.add_text() or the WebDriver .send_keys() command, but those won't clear the text box first if there's already text inside.

🔵 Getting the text from an element on a page:

text = self.get_text("header h2")

🔵 Getting the attribute value from an element on a page:

attribute = self.get_attribute("#comic img", "title")

🔵 Asserting existence of an element on a page within some number of seconds:

self.wait_for_element_present("div.my_class", timeout=10)

(NOTE: You can also use: self.assert_element_present(ELEMENT))

🔵 Asserting visibility of an element on a page within some number of seconds:

self.wait_for_element_visible("a.my_class", timeout=5)

(NOTE: The short versions of that are self.find_element(ELEMENT) and self.assert_element(ELEMENT). The find_element() version returns the element.)

Since the line above returns the element, you can combine that with .click() as shown below:

self.find_element("a.my_class", timeout=5).click()

# But you're better off using the following statement, which does the same thing:

self.click("a.my_class")  # DO IT THIS WAY!

ProTip™: You can use dots to signify class names (Ex: div.class_name) as a simplified version of div[class="class_name"] within a CSS selector.

You can also use *= to search for any partial value in a CSS selector as shown below:

self.click('a[name*="partial_name"]')

🔵 Asserting visibility of text inside an element on a page within some number of seconds:

self.assert_text("Make it so!", "div#trek div.picard div.quotes")
self.assert_text("Tea. Earl Grey. Hot.", "div#trek div.picard div.quotes", timeout=3)

(NOTE: self.find_text(TEXT, ELEMENT) and self.wait_for_text(TEXT, ELEMENT) also do this. For backwards compatibility, older method names were kept, but the default timeout may be different.)

🔵 Asserting Anything:

self.assert_true(var1 == var2)

self.assert_false(var1 == var2)

self.assert_equal(var1, var2)

🔵 Useful Conditional Statements: (with creative examples)

❓ is_element_visible(selector): (visible on the page)

if self.is_element_visible('div#warning'):
    print("Red Alert: Something bad might be happening!")

❓ is_element_present(selector): (present in the HTML)

if self.is_element_present('div#top_secret img.tracking_cookie'):
    self.contact_cookie_monster()  # Not a real SeleniumBase method
else:
    current_url = self.get_current_url()
    self.contact_the_nsa(url=current_url, message="Dark Zone Found")  # Not a real SeleniumBase method

def is_there_a_cloaked_klingon_ship_on_this_page():
    if self.is_element_present("div.ships div.klingon"):
        return not self.is_element_visible("div.ships div.klingon")
    return False

❓ is_text_visible(text, selector): (text visible on element)

if self.is_text_visible("You Shall Not Pass!", "h1"):
    self.open("https://www.youtube.com/watch?v=3xYXUeSmb-Y")

▶️ Click for a longer example of is_text_visible():

def get_mirror_universe_captain_picard_superbowl_ad(superbowl_year):
    selector = "div.superbowl_%s div.commercials div.transcript div.picard" % superbowl_year
    if self.is_text_visible("Yes, it was I who summoned you all here.", selector):
        return "Picard Paramount+ Superbowl Ad 2020"
    elif self.is_text_visible("Commander, signal the following: Our Network is Secure!"):
        return "Picard Mirror Universe iboss Superbowl Ad 2018"
    elif self.is_text_visible("For the Love of Marketing and Earl Grey Tea!", selector):
        return "Picard Mirror Universe HubSpot Superbowl Ad 2015"
    elif self.is_text_visible("Delivery Drones... Engage", selector):
        return "Picard Mirror Universe Amazon Superbowl Ad 2015"
    elif self.is_text_visible("Bing it on Screen!", selector):
        return "Picard Mirror Universe Microsoft Superbowl Ad 2015"
    elif self.is_text_visible("OK Glass, Make it So!", selector):
        return "Picard Mirror Universe Google Superbowl Ad 2015"
    elif self.is_text_visible("Number One, I've Never Seen Anything Like It.", selector):
        return "Picard Mirror Universe Tesla Superbowl Ad 2015"
    elif self.is_text_visible("Let us make sure history never forgets the name ... Facebook", selector):
        return "Picard Mirror Universe Facebook Superbowl Ad 2015"
    elif self.is_text_visible("""With the first link, the chain is forged.
                              The first speech censored, the first thought forbidden,
                              the first freedom denied, chains us all irrevocably.""", selector):
        return "Picard Mirror Universe Wikimedia Superbowl Ad 2015"
    else:
        raise Exception("Reports of my assimilation are greatly exaggerated.")

❓ is_link_text_visible(link_text):

if self.is_link_text_visible("Stop! Hammer time!"):
    self.click_link("Stop! Hammer time!")

🔵 Switching Tabs:

If your test opens up a new tab/window, you can switch to it. (SeleniumBase automatically switches to new tabs that don't open to about:blank URLs.)

self.switch_to_window(1)  # This switches to the new tab (0 is the first one)

🔵 How to handle iframes:

🔵 iframes follow the same principle as new windows: You must first switch to the iframe if you want to perform actions in there:

self.switch_to_frame("iframe")
# ... Now perform actions inside the iframe
self.switch_to_parent_frame()  # Exit the current iframe

To exit from multiple iframes, use self.switch_to_default_content(). (If inside a single iframe, this has the same effect as self.switch_to_parent_frame().)

self.switch_to_frame('iframe[name="frame1"]')
self.switch_to_frame('iframe[name="frame2"]')
# ... Now perform actions inside the inner iframe
self.switch_to_default_content()  # Back to the main page

🔵 You can also use a context manager to act inside iframes:

with self.frame_switch("iframe"):
    # ... Now perform actions while inside the code block
# You have left the iframe

This also works with nested iframes:

with self.frame_switch('iframe[name="frame1"]'):
    with self.frame_switch('iframe[name="frame2"]'):
        # ... Now perform actions while inside the code block
    # You are now back inside the first iframe
# You have left all the iframes

🔵 How to execute custom jQuery scripts:

<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.3/jquery.min.js"></script>

🔵 It's OK if you want to use jQuery on a page that doesn't have it loaded yet. To do so, run the following command first:

self.activate_jquery()

▶️ Here are some examples of using jQuery in your scripts. (click to expand)

self.execute_script("jQuery, window.scrollTo(0, 600)")  # Scrolling the page

self.execute_script("jQuery('#annoying-widget').hide()")  # Hiding elements on a page

self.execute_script("jQuery('#hidden-widget').show(0)")  # Showing hidden elements on a page

self.execute_script("jQuery('#annoying-button a').remove()")  # Removing elements on a page

self.execute_script("jQuery('%s').mouseover()" % (mouse_over_item))  # Mouse-over elements on a page

self.execute_script("jQuery('input#the_id').val('my_text')")  # Fast text input on a page

self.execute_script("jQuery('div#dropdown a.link').click()")  # Click elements on a page

self.execute_script("return jQuery('div#amazing')[0].text")  # Returns the css "text" of the element given

self.execute_script("return jQuery('textarea')[2].value")  # Returns the css "value" of the 3rd textarea element on the page

(Most of the above commands can be done directly with built-in SeleniumBase methods.)

🔵 How to handle a restrictive CSP:

🔵 More JavaScript fun:

▶️ In this example, JavaScript creates a referral button on a page, which is then clicked. (click to expand)

start_page = "https://xkcd.com/465/"
destination_page = "https://github.com/seleniumbase/SeleniumBase"
self.open(start_page)
referral_link = '''<a class='analytics test' href='%s'>Free-Referral Button!</a>''' % destination_page
self.execute_script('''document.body.innerHTML = \"%s\"''' % referral_link)
self.click("a.analytics")  # Clicks the generated button

🔵 How to use deferred asserts:

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)

class DeferredAssertTests(BaseCase):
    def test_deferred_asserts(self):
        self.open("https://xkcd.com/993/")
        self.wait_for_element("#comic")
        self.deferred_assert_element('img[alt="Brand Identity"]')
        self.deferred_assert_element('img[alt="Rocket Ship"]')  # Will Fail
        self.deferred_assert_element("#comicmap")
        self.deferred_assert_text("Fake Item", "ul.comicNav")  # Will Fail
        self.deferred_assert_text("Random", "ul.comicNav")
        self.deferred_assert_element('a[name="Super Fake !!!"]')  # Will Fail
        self.deferred_assert_exact_text("Brand Identity", "#ctitle")
        self.deferred_assert_exact_text("Fake Food", "#comic")  # Will Fail
        self.process_deferred_asserts()

🔵 How to access raw WebDriver:

If you need access to any commands that come with standard WebDriver, you can call them directly like this:

self.driver.delete_all_cookies()
capabilities = self.driver.capabilities
self.driver.find_elements("partial link text", "GitHub")

(In general, you'll want to use the SeleniumBase versions of methods when available.)

🔵 How to retry failing tests automatically:

You can use pytest --reruns=NUM to retry failing tests that many times. Add --reruns-delay=SECONDS to wait that many seconds between retries. Example:

pytest --reruns=1 --reruns-delay=1

You can use the @retry_on_exception() decorator to retry failing methods. (First import: from seleniumbase import decorators). To learn more about SeleniumBase decorators, click here.

"Catch bugs in QA before deploying code to Production!"

Wrap-Up

If you see something, say something!

https://github.com/mdmintz

jbengler/tidyplots

2024-12-20T01:36:46Z

Tidy Plots for Scientific Papers

tidyplots

The goal of tidyplots is to streamline the creation of publication-ready plots for scientific papers. It allows to gradually add, remove and adjust plot components using a consistent and intuitive syntax.

Installation

You can install the released version of tidyplots from CRAN with:

install.packages("tidyplots")

And the development version from GitHub with:

# install.packages("devtools")
devtools::install_github("jbengler/tidyplots")

Usage

Here are some examples. Also have a look at the getting started guide and the full documentation.

library(tidyplots)

study %>% 
  tidyplot(x = treatment, y = score, color = treatment) %>% 
  add_mean_bar(alpha = 0.4) %>% 
  add_sem_errorbar() %>% 
  add_data_points_beeswarm()

energy %>% 
  tidyplot(x = year, y = power, color = energy_source) %>% 
  add_barstack_absolute()

energy %>% 
  dplyr::filter(year %in% c(2005, 2010, 2015, 2020)) %>% 
  tidyplot(y = power, color = energy_source) %>% 
  add_donut() %>% 
  split_plot(by = year)

energy_week %>% 
  tidyplot(x = date, y = power, color = energy_source) %>% 
  add_areastack_absolute()

energy_week %>% 
  tidyplot(x = date, y = power, color = energy_source) %>% 
  add_areastack_relative()

study %>% 
  tidyplot(x = group, y = score, color = dose) %>% 
  add_mean_bar(alpha = 0.4) %>% 
  add_mean_dash() %>% 
  add_mean_value()

time_course %>%
  tidyplot(x = day, y = score, color = treatment) %>%
  add_mean_line() %>%
  add_mean_dot() %>%
  add_sem_ribbon()

climate %>%
  tidyplot(x = month, y = year, color = max_temperature) %>%
  add_heatmap()

study %>% 
  tidyplot(x = treatment, y = score, color = treatment) %>% 
  add_boxplot() %>% 
  add_test_pvalue(ref.group = 1)

gene_expression %>% 
  dplyr::filter(external_gene_name %in% c("Apol6", "Col5a3", "Vgf", "Bsn")) %>% 
  tidyplot(x = condition, y = expression, color = sample_type) %>% 
  add_mean_dash() %>% 
  add_sem_errorbar() %>% 
  add_data_points_beeswarm() %>% 
  add_test_asterisks(hide_info = TRUE) %>% 
  remove_x_axis_title() %>% 
  split_plot(by = external_gene_name)

study %>% 
  tidyplot(x = treatment, y = score, color = treatment) %>% 
  add_mean_bar(alpha = 0.4) %>% 
  add_sem_errorbar() %>% 
  add_data_points_beeswarm() %>% 
  view_plot(title = "Default color scheme: 'friendly'") %>% 
  adjust_colors(colors_discrete_apple) %>% 
  view_plot(title = "Alternative color scheme: 'apple'")

Documentation

Package index
Overview of all tidyplots functions
Get started
Getting started guide
Visualizing data
Article with examples for common data visualizations
Advanced plotting
Article about advanced plotting techniques and workflows
Color schemes
Article about the use of color schemes

Acknowledgements

I would like to thank Lars Binkle-Ladisch for our insightful discussions and for consistently challenging my decisions regarding the naming of functions and their arguments.

Many thanks to the R and tidyverse communities. tidyplots is built upon their software and coding paradigms, and it would not have been possible without their contributions.

tidyplots relies on several fantastic packages that handle all the heavy lifting behind the scenes. These include cli, dplyr, forcats, ggbeeswarm, ggplot2, ggpubr, ggrastr, ggrepel, glue, Hmisc, htmltools, lifecycle, patchwork, purrr, rlang, scales, stringr, tidyr, and tidyselect.

rgwood/systemctl-tui

2024-12-19T01:37:04Z

A fast, simple TUI for interacting with systemd services and their logs

systemctl-tui

A fast, simple TUI for interacting with systemd services and their logs.

systemctl-tui can quickly browse service status and logs, and start/stop/restart services. It aims to do a small number of things well.

Install

Note: this project only works on Linux (WSL works if you have systemd enabled). Binaries are published for x64 and ARM64 in the GitHub releases, and distro packages are available.

If you'd rather build from scratch you will need Rust installed. Then either:

Run cargo install systemctl-tui --locked
Clone the repo and run cargo build --release to get a release binary at target/release/systemctl-tui

Distro Packages

Packaging status

Arch Linux

systemctl-tui can be installed from the official repositories:

pacman -S systemctl-tui

Nix

A Nix package is available and can be installed as follows:

nix-shell -p systemctl-tui

Optional:

Alias systemctl-tui to st for quick access
Create a symlink so systemctl-tui can be used with sudo:

sudo ln -s ~/.cargo/bin/systemctl-tui /usr/bin/systemctl-tui

Help

Credits

Inspired by the truly wonderful Lazygit
sysz is so cool
Used ratatui-template to get started
systemd code partially taken from servicer

thomas-mauran/chess-tui

2024-12-19T01:37:04Z

A Git-compatible VCS that is both simple and powerful

Jujutsu—a version control system

Homepage • Installation • Getting Started • Development Roadmap • Contributing

Introduction

Jujutsu is a powerful version control system for software projects. You use it to get a copy of your code, track changes to the code, and finally publish those changes for others to see and use. It is designed from the ground up to be easy to use—whether you're new or experienced, working on brand new projects alone, or large scale software projects with large histories and teams.

Jujutsu is unlike most other systems, because internally it abstracts the user interface and version control algorithms from the storage systems used to serve your content. This allows it to serve as a VCS with many possible physical backends, that may have their own data or networking models—like Mercurial or Breezy, or hybrid systems like Google's cloud-based design, Piper/CitC.

Today, we use Git repositories as a storage layer to serve and track content, making it compatible with many of your favorite Git-based tools, right now! All core developers use Jujutsu to develop Jujutsu, right here on GitHub. But it should hopefully work with your favorite Git forges, too.

We combine many distinct design choices and concepts from other version control systems into a single tool. Some of those sources of inspiration include:

Git: We make an effort to be fast—with a snappy UX, efficient algorithms, correct data structures, and good-old-fashioned attention to detail. The default storage backend uses Git repositories for "physical storage", for wide interoperability and ease of onboarding.
Mercurial & Sapling: There are many Mercurial-inspired features, such as the revset language to select commits. There is no explicit index or staging area. Branches are "anonymous" like Mercurial, so you don't need to make up a name for each small change. Primitives for rewriting history are powerful and simple. Formatting output is done with a robust template language that can be configured by the user.
Darcs: Jujutsu keeps track of conflicts as first-class objects in its model; they are first-class in the same way commits are, while alternatives like Git simply think of conflicts as textual diffs. While not as rigorous as systems like Darcs (which is based on a formalized theory of patches, as opposed to snapshots), the effect is that many forms of conflict resolution can be performed and propagated automatically.

And it adds several innovative, useful features of its own:

Working-copy-as-a-commit: Changes to files are recorded automatically as normal commits, and amended on every subsequent change. This "snapshot" design simplifies the user-facing data model (commits are the only visible object), simplifies internal algorithms, and completely subsumes features like Git's stashes or the index/staging-area.
Operation log & undo: Jujutsu records every operation that is performed on the repository, from commits, to pulls, to pushes. This makes debugging problems like "what just happened?" or "how did I end up here?" easier, especially when you're helping your coworker answer those questions about their repository! And because everything is recorded, you can undo that mistake you just made with ease. Version control has finally entered the 1960s!
Automatic rebase and conflict resolution: When you modify a commit, every descendent is automatically rebased on top of the freshly-modified one. This makes "patch-based" workflows a breeze. If you resolve a conflict in a commit, the resolution of that conflict is also propagated through descendants as well. In effect, this is a completely transparent version of git rebase --update-refs combined with git rerere, supported by design.

[!WARNING] The following features are available for use, but experimental; they may have bugs, backwards incompatible storage changes, and user-interface changes!

Safe, concurrent replication: Have you ever wanted to store your version controlled repositories inside a Dropbox folder? Or continuously backup repositories to S3? No? Well, now you can!

The fundamental problem with using filesystems like Dropbox and backup tools like rsync on your typical Git/Mercurial repositories is that they rely on local filesystem operations being atomic, serialized, and non-concurrent with respect to other reads and writes—which is not true when operating on distributed file systems, or when operations like concurrent file copies (for backup) happen while lock files are being held.

Jujutsu is instead designed to be safe under concurrent scenarios; simply using rsync or Dropbox and then using that resulting repository should never result in a repository in a corrupt state. The worst that should happen is that it will expose conflicts between the local and remote state, leaving you to resolve them.

The command-line tool is called jj for now because it's easy to type and easy to replace (rare in English). The project is called "Jujutsu" because it matches "jj".

Jujutsu is relatively young, with lots of work to still be done. If you have any questions, or want to talk about future plans, please join us on Discord , start a GitHub Discussion, or send an IRC message to #jujutsu on Libera Chat. The developers monitor all of these channels[^bridge].

[^bridge]: To be more precise, the #jujutsu Libera IRC channel is bridged to one of the channels on jj's Discord. Some of the developers stay on Discord and use the bridge to follow IRC.

News and Updates 📣

Feb 2024: Version 0.14 is released, which deprecates "jj checkout" and "jj merge", as well as jj init --git, which is now just called jj git init.
Oct 2023: Version 0.10.0 is released! Now includes a bundled merge and diff editor for all platforms, "immutable revsets" to avoid accidentally edit-ing the wrong revisions, and lots of polish.
Jan 2023: Martin gave a presentation about Google's plans for Jujutsu at Git Merge 2022! See the slides or the recording.

Related Media

Mar 2024: Chris Krycho started a YouTube series about Jujutsu.
Feb 2024: Chris Krycho published an article about Jujutsu called jj init and Steve Klabnik followed up with the Jujutsu Tutorial.
Jan 2024: Jujutsu was featured in an LWN.net article called Jujutsu: a new, Git-compatible version control system.
Jan 2023: Martin's Talk about Jujutsu at Git Merge 2022, video and the associated slides.

The wiki also contains a more extensive list of media references.

Getting started

[!IMPORTANT] Jujutsu is an experimental version control system. While Git compatibility is stable, and most developers use it daily for all their needs, there may still be work-in-progress features, suboptimal UX, and workflow gaps that make it unusable for your particular use.

Follow the installation instructions to obtain and configure jj.

The best way to get started is probably to go through the tutorial. Also see the Git comparison, which includes a table of jj vs. git commands.

As you become more familiar with Jujutsu, the following resources may be helpful:

The FAQ.
The Glossary.
The jj help command (e.g. jj help rebase).

If you are using a prerelease version of jj, you would want to consult the docs for the prerelease (main branch) version. You can also get there from the docs for the latest release by using the website's version switcher. The version switcher is visible in the header of the website when you scroll to the top of any page.

Features

Compatible with Git

Jujutsu is designed so that the underlying data and storage model is abstract. Today, it features two backends—one of them uses a Git repository for storage, while the other is a native storage backend[^native-backend]. The Git backend uses the libgit2 C library and the gitoxide Rust library.

[^native-backend]: At this time, there's practically no reason to use the native backend. The backend exists mainly to make sure that it's possible to eventually add functionality that cannot easily be added to the Git backend.

The Git backend is fully featured and maintained, and allows you to use Jujutsu with any Git remote. The commits you create will look like regular Git commits. You can fetch branches from a regular Git remote and push branches to the remote. You can always switch back to Git.

Here is how you can explore a GitHub repository with jj.

You can even have a "co-located" local repository where you can use both jj and git commands interchangeably.

The working copy is automatically committed

Jujutsu uses a real commit to represent the working copy. Checking out a commit results a new working-copy commit on top of the target commit. Almost all commands automatically amend the working-copy commit.

The working-copy being a commit means that commands never fail because the working copy is dirty (no "error: Your local changes to the following files..."), and there is no need for git stash. Also, because the working copy is a commit, commands work the same way on the working-copy commit as on any other commit, so you can set the commit message before you're done with the changes.

The repo is the source of truth

With Jujutsu, the working copy plays a smaller role than with Git. Commands snapshot the working copy before they start, then they update the repo, and then the working copy is updated (if the working-copy commit was modified). Almost all commands (even checkout!) operate on the commits in the repo, leaving the common functionality of snapshotting and updating of the working copy to centralized code. For example, jj restore (similar to git restore) can restore from any commit and into any commit, and jj describe can set the commit message of any commit (defaults to the working-copy commit).

Entire repo is under version control

All operations you perform in the repo are recorded, along with a snapshot of the repo state after the operation. This means that you can easily revert to an earlier repo state, or to simply undo a particular operation (which does not necessarily have to be the most recent operation).

Conflicts can be recorded in commits

If an operation results in conflicts, information about those conflicts will be recorded in the commit(s). The operation will succeed. You can then resolve the conflicts later. One consequence of this design is that there's no need to continue interrupted operations. Instead, you get a single workflow for resolving conflicts, regardless of which command caused them. This design also lets Jujutsu rebase merge commits correctly (unlike both Git and Mercurial).

Basic conflict resolution:

Juggling conflicts:

Automatic rebase

Whenever you modify a commit, any descendants of the old commit will be rebased onto the new commit. Thanks to the conflict design described above, that can be done even if there are conflicts. Bookmarks pointing to rebased commits will be updated. So will the working copy if it points to a rebased commit.

Comprehensive support for rewriting history

Besides the usual rebase command, there's jj describe for editing the description (commit message) of an arbitrary commit. There's also jj diffedit, which lets you edit the changes in a commit without checking it out. To split a commit into two, use jj split. You can even move part of the changes in a commit to any other commit using jj squash -i --from X --into Y.

Status

The tool is fairly feature-complete, but some important features like (the equivalent of) git blame are not yet supported. There are also several performance bugs. It's likely that workflows and setups different from what the core developers use are not well supported, e.g. there is no native support for email-based workflows.

Today, all core developers use jj to work on jj. I (Martin von Zweigbergk) have almost exclusively used jj to develop the project itself since early January 2021. I haven't had to re-clone from source (I don't think I've even had to restore from backup).

There will be changes to workflows and backward-incompatible changes to the on-disk formats before version 1.0.0. Even the binary's name may change (i.e. away from jj). For any format changes, we'll try to implement transparent upgrades (as we've done with recent changes), or provide upgrade commands or scripts if requested.

Related work

There are several tools trying to solve similar problems as Jujutsu. See related work for details.

Contributing

We welcome outside contributions, and there's plenty of things to do, so don't be shy. Please ask if you want a pointer on something you can help with, and hopefully we can all figure something out.

We do have a few policies and suggestions for contributors. The broad TL;DR:

Bug reports are very welcome!
Every commit that lands in the main branch is code reviewed.
Please behave yourself, and obey the Community Guidelines.
There is a mandatory CLA you must agree to. Importantly, it does not transfer copyright ownership to Google or anyone else; it simply gives us the right to safely redistribute and use your changes.

Mandatory Google Disclaimer

I (Martin von Zweigbergk, martinvonz@google.com) started Jujutsu as a hobby project in late 2019, and it has evolved into my full-time project at Google, with several other Googlers (now) assisting development in various capacities. That said, this is not a Google product.

License

Jujutsu is available as Open Source Software, under the Apache 2.0 license. See LICENSE for details about copyright and redistribution.

lynxworld/lynx

2024-12-19T01:37:10Z

Cypher query execution framework

Lynx: A Graph Query Framework for Multiple Heterogeneous Data Sources

Lynx is a general graph query framework to simplify querying graph data by converting complex statements into basic graph operations. Instead of connecting directly to the data sources, Lynx retrieves data through user-implemented interfaces for those graph operations.

View Demo · Report Bug · Request Feature

Table of Contents

About The Project

Introduction

(back to top)

Architecture and Workflow

Getting Started

Import

Import Lynx by Maven.

  <dependency>
      <groupId>org.grapheco</groupId>
      <artifactId>lynx</artifactId>
      <version>0.6</version>
  </dependency>

How to use

Implementation
- Implement the required interfaces for your data source
- Configure the basic settings
Runner
- Set up the query environment
- Execute your graph queries

(back to top)

Demo Video

We have a demo video at VLDB2023:

Citation

Our paper has been accepted as a demo paper at VLDB2023:

@article{shen2023lynx,
  title={Lynx: A Graph Query Framework for Multiple Heterogeneous Data Sources},
  author={Shen, Zhihong and Hu, Chuan and Zhao, Zihao},
  journal={Proceedings of the VLDB Endowment},
  volume={16},
  number={12},
  pages={3926--3929},
  year={2023},
  publisher={VLDB Endowment}
}

YanG-1989/m3u

2024-12-20T01:37:44Z

直播源

仓库简介

个人爱好，收集整合！佛系更新…

Telegram 群「传送门」

软件下载,资源集合,没其他东西 (后期可能会分享喜欢的东西)...

疑难杂症「传送门」

有问题请留言,基本上置顶已经解答了所有的问题...

订阅地址「懒人必备」

Gather.m3u「精简版」---(修改 hosts 方法)

订阅地址「GitHub版」,ipv6·源为主...

Gather.m3u「完整版」

订阅地址「VPS版」,请勿直接打开...

直播集合「一切随缘」

Live.m3u「多平台」

直播平台订阅地址「收集网友喜欢的直播间...」

个性化 M3U 定制

定制自己的直播 M3U 列表「分享自己喜欢的主播..」

垃圾话语录「历史记录」

2024.12.13

维护 Gather「GitHub」「增加 IPV6·其他源」
维护 Gather「完整版」「修复 IPV6·失效源」

2024.12.2

维护 Gather「修复 •台湾失效源」

2024.11.30

维护 Live「增加 Twitch 直播代理」

2024.11.25

维护 Gather「完整版」「增加公益AKTV·源」

2024.11.23

维护 Live「增加 YouTube 限制 / 修复虎牙断流」

2024.11.21

维护 Live「修复 YY 直播代理」

2024.11.18

维护 Gather「完整版」「替换 IPV6·源-山东」

声明「佛曰需要」

本资料中的所有资源均来自网络，仅作为爱好者的收集和分享之用，版权归原作者或原版权方所有。如有侵权，请联系本人删除。本资料仅供学习与交流使用，请勿用于商业用途。

classroomio/classroomio

2024-12-19T01:37:44Z

The Open Source Education Platform. A Simple and Beautiful Alternative to Moodle LMS, EdX, Thinkific and Teachable

github/CopilotForXcode

2024-12-20T01:38:11Z

Xcode extension for GitHub Copilot

GitHub Copilot for Xcode

GitHub Copilot is an AI pair programmer tool that helps you write code faster and smarter. Copilot for Xcode is an Xcode extension that provides inline coding suggestions as you type.

Beta Preview Policy

Use of the GitHub Copilot Xcode Extension is subject to GitHub's Pre-Release Terms. We want to remind you that:

Beta Previews may not be supported or may change at any time. You may receive confidential information through those programs that must remain confidential while the program is private. We'd love your feedback to make our Beta Previews better.

Requirements

macOS 12+
Xcode 8+
A GitHub Copilot subscription. To learn more, visit https://github.com/features/copilot.

Getting Started

Install via Homebrew:
```
brew install --cask github-copilot-for-xcode
```
Or download the dmg from the latest release. Drag GitHub Copilot for Xcode into the Applications folder:

Updates can be downloaded and installed by the app.
A background item will be added to enable Copilot to start when Xcode is opened.
Two permissions are required: Accessibility and Xcode Source Editor Extension. For more on why these permissions are required see TROUBLESHOOTING.md.

The first time the application is run the Accessibility permission should be requested:

The Xcode Source Editor Extension permission needs to be enabled manually. Click Extension Permission from the Copilot for Xcode settings to open the System Preferences to the Extensions panel. Select Xcode Source Editor and enable GitHub Copilot:
After granting the extension permission, please restart Xcode to ensure the Github Copilot menu is available and not disabled under the Xcode Editor menu.

Keyboard shortcuts can be set for all menu items in the Key Bindings section of Xcode preferences.
To sign into GitHub Copilot, click the Sign in button in the settings application. This will open a browser window and copy a code to the clipboard. Paste the code into the GitHub login page and authorize the application.
To install updates, click Check for Updates from the menu item or in the settings application.

After installing a new version, Xcode must be restarted to use the new version correctly.

New versions can also be installed from dmg files downloaded from the releases page. When installing a new version via dmg, the application must be run manually the first time to accept the downloaded from the internet warning.
To avoid confusion, we recommend disabling Predictive code completion under Xcode > Preferences > Text Editing > Editing.
Press tab to accept the first line of a suggestion, hold option to view the full suggestion, and press option + tab to accept the full suggestion.

License

This project is licensed under the terms of the MIT open source license. Please refer to LICENSE.txt for the full terms.

Privacy

We follow responsible practices in accordance with our Privacy Statement.

To get the latest security fixes, please use the latest version of the GitHub Copilot for Xcode.

Support

We’d love to get your help in making GitHub Copilot better! If you have feedback or encounter any problems, please reach out on our Feedback forum.

Acknowledgements

Thank you to @intitni for creating the original project that this is based on.

Attributions can be found under About when running the app or in Credits.rtf.

circuitikz/circuitikz

2024-12-19T01:37:59Z

CircuiTikZ TeX/LaTeX package for drawing circuits

circuitikz

This package provides a set of macros on top of TikZ for naturally typesetting electrical and electronic networks.

It was born mainly for writing Massimo Redaelli's exercise book and exams sheets for the Elettrotecnica courses at Politecnico di Milano, Italy. He wanted a tool that was easy to use, with a lean syntax, native to LaTeX, and supporting directly PDF output format.

Requirements

circuitikz is included with the most common LaTeX systems, so it should work out of the box. Anyway, the main dependency is on TikZ/PGF, xstring and siunitx (if used):

Tikz/PGF graphics library, version 3.1.5b
xstring newer than 2009/03/13
siunitx, better if v2 or newer.

Installation

The stable version is available on CTAN and is included within the major latex distributions(Texlive, Miktex). If you want to test the latest version, have a look at http://circuitikz.github.io/circuitikz/. There you can find the latest git-version as a single file, just copy it to your project or to your local tex tree.

Usage

Stable version

Just place

  \usepackage[RPvoltages, siunitx]{circuitikz}% or your preferred options

Be sure to check the options you like — you should at least state one of the "voltage direction" ones, like RPvoltages shown above, or the usage of siunitx special syntax. Be sure to check the relevant sections in the manual.

For ConTeXt, you should be able to use

  \usemodule[circuitikz]

in the preamble and compile away, both with PS and PDF target output.

It is higly recommended to go through the tutorials in the manual and to learn a bit of TikZ before for example, go through at least the basic TikZ one, but there are more suggestions in the documentation.

Notice that for the time being, due to this bug, automatic (CI) checks for ConTeXt compatibility is disabled. I (Romano) am still checking it at home, but I am not able to have it integrated in the CI.

Development or newer version

If you want to use the git version, go to the GitHub page for the project, download the last version (or the one you want to use) — it's the links called "download the latest version in a single file". You will have a file called circuitikzgit.sty that you should put somewhere in your LaTeX input path; the best place is in the same directory where your main file live. Then, just use it instead of the normal one:

  \usepackage[siunitx, RPvoltages]{circuitikzgit}

or, for ConTeXt,

  \usemodule[circuitikzgit]

Be aware that options and style files can (and do) change between versions, so you have to check them manually, and that the recommended version is always the one that come with your TeX/LaTeX distribution; use this trick only if you really need a newer version!

Older version

There is a roll-back mechanism in the package, that let you use older versions if you need them; the details are in the manual, look at the section named "Incompatibilities between versions" for details.

You have all the code and the corresponding manual available in the in the github page of the project, since version 0.6 onward.

Contact the authors

For bug reports and enhacement suggestions, the preferred way is to use the issue page on the project. Please be ready to provide an example code showing the bug, if any.

Please do not use the issue page for generic help on how to use the package. The manual has a (growing) set of examples; also the questions tagged circuitikz on https://tex.stackexchange.com/ are often answered promptly. You can also find help on matrix and on TopTeX.

Since more or less version 0.9.0 (May 2019) Romano is the main maintainer of the project. You can contact the authors via GitHub (preferred) or via their email addresses:

GitHub main project page
Massimo Redaelli (m.redaelli@gmail.com)
Stefan Lindner (stefan.lindner@fau.de)
Stefan Erhardt (stefan.erhardt@fau.de)
Romano Giannetti (romano.giannetti@gmail.com)

License and copyright

This file may be distributed and/or modified

under the LaTeX Project Public License and/or
under the GNU Public License.

zaidmukaddam/miniperplx

2024-12-19T01:38:12Z

A minimalistic AI-powered search engine that helps you find information on the internet. Powered by Vercel AI SDK! Search with models like GPT-4o mini, GPT-4o and Claude 3.5 Sonnet(New)!

MiniPerplx

A minimalistic AI-powered search engine that helps you find information on the internet.

ProductHunt Launch

Upvote MiniPerplx on ProductHunt to show your support!

Features

AI-powered search: Get answers to your questions using Anthropic's Models.
Web search: Search the web using Tavily's API.
URL Specific search: Get information from a specific URL.
Weather: Get the current weather for any location using OpenWeather's API.
Programming: Run code snippets in multiple languages using E2B's API.
Maps: Get the location of any place using Google Maps API.
Results Overview: Get a quick overview of the results from different providers.
Translation: Translate text to different languages using Microsoft's Translator API.

Built with

LLM used

Deploy your own

Local development

To run the example locally you need to:

Sign up for accounts with the AI providers you want to use. OpenAI and Anthropic are required, Tavily is required for the web search feature.
Obtain API keys for each provider.
Set the required environment variables as shown in the .env.example file, but in a new file called .env.local.
pnpm install to install the required dependencies.
pnpm dev to launch the development server.

License

This project is licensed under the MIT License - see the LICENSE file for details.

konfig-dev/konfig

2024-12-20T01:38:36Z

SDK & API Docs Generator. Sunset as of December 2024

Konfig

The monorepo that holds everything...

Getting started

Get the repository on your local machine. Takes a minute.

git clone https://github.com/konfig-dev/konfig --recursive
cd konfig

This repository has submodules so pull all of them. Also take a few minutes.

git submodule update --init --recursive --remote --merge

Environment Setup

Run Postgres as a background process

# in /konfig
brew install postgresql
mkdir -p postgres/data
initdb -D ./postgres/data
pg_ctl -D ./postgres/data start

Setup .env file in generator/konfig-dash to something like:

DATABASE_URL="postgresql://dylanhuang@localhost:5432/konfig_dev?connection_limit=1"
TEST_DATABASE_URL="postgresql://dylanhuang@localhost:5432/konfig_test?connection_limit=1"

AWS_ACCESS_KEY_ID=XXXXXX
AWS_SECRET_ACCESS_KEY=XXXXXX

# Used to encrypt/decrypt session cookies. Change this value and re-deploy to log out all users of your app at once.
SESSION_SECRET=ZUWpQ9pB4fB5FFpjHLi8Z2qadzXkdTKhHBsXmGmjNdxtrZbevaCYWSpw7G7cHBhh

Paste this into your ~/.zshrc or ~/.bashrc

if [ -f $HOME/.envvars ]; then
    . $HOME/.envvars
else
    print "404: ~/.envvars not found."
fi

Then create ~/.envvars with values from Dylan.

How to run Konfig

Make sure node_modules is initiated in konfig-dash

cd generator/konfig-dash
yarn # takes some time
yarn rw prisma migrate dev # setup the DB

Start the server with yarn dev

# inside generator/konfig-dash
yarn dev

Start generator/konfig-generator-api w/ IntelliJ
cd into an SDK repo and run konfig generate -d

Making Changes

See Changesets

Helicone/helicone

2024-12-20T01:38:36Z

🧊 Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, Anthropic SDK, LiteLLM, LLamaIndex, LangChain, and more. 🍓 YC W23

🔍 Observability	🕸️ Agent Tracing	💬 Prompt Management
📊 Evaluations	📚 Datasets	🎛️ Fine-tuning

Docs • Discord • Roadmap • Changelog • Bug reports

See Helicone in Action! (Free)

Helicone is the all-in-one, open-source LLM developer platform

🔌 Integrate: One-line of code to log all your requests to OpenAI, Anthropic, LangChain, Gemini, TogetherAI, LlamaIndex, LiteLLM, OpenRouter, and more
📊 Observe: Inspect and debug traces & sessions for agents, chatbots, document processing pipelines, and more
📈 Analyze: Track metrics like cost, latency, quality, and more. Export to PostHog in one-line for custom dashboards
🎮 Playground: Rapidly test and iterate on prompts, sessions and traces in our UI
🧠 Prompt Management: Version and experiment with prompts using production data. Your prompts remain under your control, always accessible.
🔍 Evaluate: Automatically run evals on traces or sessions using the latest platforms: LastMile or Ragas (more coming soon)
🎛️ Fine-tune: Fine-tune with one of our fine-tuning partners: OpenPipe or Autonomi (more coming soon)
🛜 Gateway: Caching, custom rate limits, LLM security, and more with our gateway
🛡️ Enterprise Ready: SOC 2 and GDPR compliant

🎁 Generous monthly free tier (100k requests/month) - No credit card required!

Quick Start ⚡️ One line of code

Get your write-only API key by signing up here.

Update only the baseURL in your code:

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: `https://oai.helicone.ai/v1/${process.env.HELICONE_API_KEY}`,
});

or - use headers for more secure environments

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: `https://oai.helicone.ai/v1`,
  defaultHeaders: {
   "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
  },
});

🎉 You're all set! View your logs at Helicone.

This quick start uses Helicone Cloud with OpenAI. For other providers or self-hosted options, see below.

Get Started For Free

Helicone Cloud (Recommended)

The fastest and most reliable way to get started with Helicone. Get started for free at Helicone US or Helicone EU. Your first 100k requests are free every month, after which you'll pay based on usage. Try our demo to see Helicone in action!

Integrations: View our supported integrations.

Latency Concerns: Helicone's Cloud offering is deployed on Cloudflare workers and ensures the lowest latency (~10ms) add-on to your API requests. View our latency benchmarks.

Self-Hosting Open Source LLM Observability with Helicone

Docker

Helicone is simple to self-host and update. To get started locally, just use our docker-compose file.

Pre-Request:

Copy the shared directory to the valhalla directory
Create a valhalla folder in the valhalla directory and put /valhalla/jawn in it

# Clone the repository
git clone https://github.com/Helicone/helicone.git
cd docker
cp .env.example .env

# Start the services
docker compose up

Helm

For Enterprise workloads, we also have a production-ready Helm chart available. To access, contact us at enterprise@helicone.ai.

Manual (Not Recommended)

Manual deployment is not recommended. Please use Docker or Helm. If you must, follow the instructions here.

Architecture

Helicone is comprised of five services:

Web: Frontend Platform (NextJS)
Worker: Proxy Logging (Cloudflare Workers)
Jawn: Dedicated Server for serving collecting logs (Express + Tsoa)
Supabase: Application Database and Auth
ClickHouse: Analytics Database
Minio: Object Storage for logs.

LLM Observability Integrations

Main Integrations

Integration	Supports	Description
Generic Gateway	Python, Node.js, Python w/package, LangChain JS, LangChain, cURL	Flexible integration method for various LLM providers
Async Logging (OpenLLMetry)	JS/TS, Python	Asynchronous logging for multiple LLM platforms
OpenAI	JS/TS, Python	-
Azure OpenAI	JS/TS, Python	-
Anthropic	JS/TS, Python	-
Ollama	JS/TS	Run and use large language models locally
AWS Bedrock	JS/TS	-
Gemini API	JS/TS	-
Gemini Vertex AI	JS/TS	Gemini models on Google Cloud's Vertex AI
Vercel AI	JS/TS	AI SDK for building AI-powered applications
Anyscale	JS/TS, Python	-
TogetherAI	JS/TS, Python	-
Hyperbolic	JS/TS, Python	High-performance AI inference platform
Groq	JS/TS, Python	High-performance models
DeepInfra	JS/TS, Python	Serverless AI inference for various models
OpenRouter	JS/TS, Python	Unified API for multiple AI models
LiteLLM	JS/TS, Python	Proxy server supporting multiple LLM providers
Fireworks AI	JS/TS, Python	Fast inference API for open-source LLMs

Supported Frameworks

Framework	Supports	Description
LangChain	JS/TS, Python	-
LlamaIndex	Python	Framework for building LLM-powered data applications
CrewAI	-	Framework for orchestrating role-playing AI agents
Big-AGI	JS/TS	Generative AI suite
ModelFusion	JS/TS	Abstraction layer for integrating AI models into JavaScript and TypeScript applications

Other Integrations

Integration	Description
PostHog	Product analytics platform. Build custom dashboards.
RAGAS	Evaluation framework for retrieval-augmented generation
Open WebUI	Web interface for interacting with local LLMs
MetaGPT	Multi-agent framework
Open Devin	AI software engineer
Mem0 EmbedChain	Framework for building RAG applications
Dify	LLMOps platform for AI-native application development

This list may be out of date. Don't see your provider or framework? Check out the latest integrations in our docs. If not found there, request a new integration by contacting help@helicone.ai.

Community 🌍

Learn this repo with Greptile

learnthisrepo.com/helicone |

Contributing

We ❤️ our contributors! We warmly welcome contributions for documentation, integrations, costs, and feature requests.

If you have an idea for how Helicone can be better, create a GitHub issue or vote on the roadmap
Update costs instructions in costs/README.md
Join discord to ask questions

License

Helicone is licensed under the Apache v2.0 License.

Additional Resources

Data Management: Manage and export your Helicone data with our API.
- Guides: ETL, Request Exporting
Data Ownership: Learn about Data Ownership and Autonomy

For more information, visit our documentation.

Contributors

jujumilk3/leaked-system-prompts

2024-12-19T01:28:39Z

Periodic cyber security newsletters that capture the latest news, summaries of conference talks, research, best practices, tools, events, vulnerabilities, and analysis of trending threats and attacks

wwb521/live

2024-12-20T01:28:35Z

更新高质量电视直播源，欢迎大家使用，永久免费

jd-opensource/nutui

2024-12-19T01:38:26Z

京东风格的移动端 Vue 组件库，支持多端小程序(A Vue.js UI Toolkit for Mobile Web)

GitHub 1C Enterprise Daily Trending

unitreerobotics/unitree_rl_gym

Unitree RL GYM

1. Installation

2. Train in Isaac Gym

2.1 Play Demo

3. Sim in Mujoco

3.1 Mujoco Usage

3.2 Mujoco Demo

4. Depoly on Physical Robot

seleniumbase/SeleniumBase

SeleniumBase

Set up Python & Git:

Install SeleniumBase:

🔵 Downloading webdrivers:

Basic Example / Usage:

Here are some common SeleniumBase methods:

Fun Facts / Learn More:

Demo Mode / Debugging:

🔵 Command-line Options:

Directory Configuration:

Log files from failed tests:

SeleniumBase Dashboard:

Generating Test Reports:

🔵 pytest HTML Reports:

🔵 pynose Reports:

🔵 behave Dashboard & Reports:

🔵 Allure Reports:

Using a Proxy Server:

Changing the User-Agent:

Handling Pop-Up Alerts:

Building Guided Tours for Websites:

Production Environments & Integrations:

Detailed Method Specifications and Examples:

🔵 Switching Tabs:

🔵 How to handle iframes:

🔵 How to execute custom jQuery scripts:

🔵 How to handle a restrictive CSP:

🔵 More JavaScript fun:

🔵 How to use deferred asserts:

🔵 How to access raw WebDriver:

🔵 How to retry failing tests automatically:

Wrap-Up

luckjiawei/frpc-desktop

Xiaomi Home Integration for Home Assistant

Installation

Method 1: Git clone from GitHub

Method 2: HACS

Method 3: Manually installation via Samba / FTPS

Configuration

Login

Add MIoT Devices

Multiple User Login

Update Configurations

Debug Mode for Action

Security

FAQ

Principle of Messaging

Control through the Cloud

Control locally

Mapping Relationship between MIoT-Spec-V2 and Home Assistant Entity

General Conversion

Specific Conversion

MIoT-Spec-V2 Filter

Multiple Language Support

Documents

Directory Structure

dillongoostudios/goo-engine

LiveContainer

Compatibility

Usage

Without JIT

With JIT (requires SideStore)

Add to Home Screen

Multiple LiveContainers

Fix File Picker & Local Notification

"Open In App" Support

JIT Support

Installing external tweaks

Hiding Apps

🔵 `pytest` HTML Reports:

🔵 `pynose` Reports:

🔵 `behave` Dashboard & Reports:

Patching `@executable_path`

Patching `NSBundle.mainBundle`