Merge branch 'main' into feature/controllers

docs/book/examples/cookbook_llm_regression_testing.md

@@ -86,13 +86,13 @@ ws = CloudWorkspace(token="YOUR_API_TOKEN", url="https://app.evidently.cloud")
 Create a Project:
 
 ```python 
-project = ws.create_project("Regression testing example", team_id="YOUR_TEAM_ID")
+project = ws.create_project("Regression testing example", org_id="YOUR_ORG_ID")
 project.description = "My project description"
 project.save()
 ```
 
 {% hint style="info" %}
-**Need help?** Check how to find API key and [create a Team](../installation/cloud_account.md).
+**Need help?** Check how to find [API key](../installation/cloud_account.md).
 {% endhint %}
 
 # 3. Prepare the Dataset

docs/book/examples/tutorial-llm.md

@@ -137,18 +137,18 @@ assistant_logs.head(3)
 To be able to save and share results and get a live monitoring dashboard, create a Project in Evidently Cloud. Here's how to set it up:
 
 * **Sign up**. If you do not have one yet, create a free [Evidently Cloud account](https://app.evidently.cloud/signup) and name your Organization.
-* **Add a Team**. Click **Teams** in the left menu. Create a Team, copy and save the Team ID. ([Team page](https://app.evidently.cloud/teams)).
+* **Create an Organization** when you log in for the first time. Get an ID of your organization. [Organizations page](https://app.evidently.cloud/organizations).
 * **Get your API token**. Click the **Key** icon in the left menu to go. Generate and save the token. ([Token page](https://app.evidently.cloud/token)).
 * **Connect to Evidently Cloud**. Pass your API key to connect. 
 
 ```python
 ws = CloudWorkspace(token="YOUR_TOKEN", 
                     url="https://app.evidently.cloud")
 ```
-* **Create a Project**. Create a new Project inside your Team, adding your title and description:
+* **Create a Project**. Create a new Project inside your Organization, adding your title and description:
 
 ```python
-project = ws.create_project("My project title", team_id="YOUR_TEAM_ID")
+project = ws.create_project("My project title", org_id="YOUR_ORG_ID")
 project.description = "My project description"
 project.save()
 ```

docs/book/get-started/cloud_quickstart_llm.md

@@ -12,8 +12,7 @@ Need help? Ask on [Discord](https://discord.com/invite/xZjKRaNp8b).
 
 Set up your Evidently Cloud workspace:
 * **Sign up** for a free [Evidently Cloud account](https://app.evidently.cloud/signup).
-* **Create an Organization** when you log in for the first time.
-* **Create a Team**. Click Teams in the left menu, create a Team, and save the Team ID ([Team page](https://app.evidently.cloud/teams)).
+* **Create an Organization** when you log in for the first time. Get an ID of your organization. [Organizations page](https://app.evidently.cloud/organizations). 
 * **Get your API token**. Click the **Key** icon in the left menu. Generate and save the token. ([Token page](https://app.evidently.cloud/token)).
 
 Now, switch to your Python environment.
@@ -49,10 +48,10 @@ Connect to Evidently Cloud using your API token:
 ws = CloudWorkspace(token="YOUR_API_TOKEN", url="https://app.evidently.cloud")
 ```
 
-Create a Project within your Team:
+Create a Project within your Organization:
 
 ```python
-project = ws.create_project("My test project", team_id="YOUR_TEAM_ID")
+project = ws.create_project("My test project", org_id="YOUR_ORG_ID")
 project.description = "My project description"
 project.save()
 ```

docs/book/get-started/cloud_quickstart_tabular.md

@@ -6,8 +6,7 @@ description: ML Monitoring “Hello world.” From data to dashboard in a couple
 
 Set up your Evidently Cloud workspace:
 * **Sign up**. If you do not have one yet, sign up for a free [Evidently Cloud account](https://app.evidently.cloud/signup).
-* **Create an Organization**. When you log in the first time, create and name your Organization.
-* **Create a Team**. Click **Teams** in the left menu. Create a Team, copy and save the Team ID. ([Team page](https://app.evidently.cloud/teams)).
+* **Create an Organization** when you log in for the first time. Get an ID of your organization. [Organizations page](https://app.evidently.cloud/organizations).
 * **Get your API token**. Click the **Key** icon in the left menu. Generate and save the token. ([Token page](https://app.evidently.cloud/token)).
 
 You can now go to your Python environment.
@@ -39,10 +38,10 @@ Connect to Evidently Cloud using your access token.
 ws = CloudWorkspace(token="YOUR_TOKEN_HERE", url="https://app.evidently.cloud")
 ```
 
-Create a new Project inside your Team. Pass the `team_id`.
+Create a new Project inside your Organization. Pass the `org_id`.
 
 ```python
-project = ws.create_project("My test project", team_id="YOUR_TEAM_ID")
+project = ws.create_project("My test project", org_id="YOUR_ORG_ID")
 project.description = "My project description"
 project.save()
 ```

docs/book/get-started/cloud_quickstart_tracing.md

@@ -56,7 +56,7 @@ Initialize the OpenAI client. Pass the token as an environment variable:
 client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
 ```
 
-Set up tracing parameters. Copy the Team ID from the [Teams page](https://app.evidently.cloud/teams), and give a name to identify your tracing dataset.
+Set up tracing parameters. Give it a name to identify your tracing dataset.
 
 ```python
 init_tracing(

docs/book/installation/cloud_account.md

@@ -10,15 +10,7 @@ If not yet, [sign up for a free Evidently Cloud account](https://app.evidently.c
 
 After logging in, create an **Organization** and name it.
 
-# 3. Create a Team 
-
-Go to the **Teams** icon in the left menu, create a Team, and name it. ([Team page](https://app.evidently.cloud/teams)).
-
-{% hint style="info" %}
-**Do I always need a Team?** Yes. Every Project must be within a Team. Teams act as "folders" to organize your work, and you can create multiple Teams. If you work alone, simply create a Team without external users. 
-{% endhint %}
-
-# 4. Connect from Python
+# 3. Connect from Python
 
 You will need an access token to interact with Evidently Cloud from your Python environment.
 

docs/book/projects/add_project.md

@@ -13,16 +13,10 @@ You can create a Project using the Python API or directly in the user interface.
 
 ## Add a new Project - API
 
-{% hint style="success" %}
-Team management is a Pro feature available in the `Evidently Cloud` and `Evidently Enterprise`.
-{% endhint %}
-
-In Evidently Cloud and Enterprise, you must create a Team before adding a Project. To get your Team ID, go to the [Teams page](https://app.evidently.cloud/teams), select your Team, and copy the ID from there.
-
-To create a Project inside a workspace `ws` and Team with a `team_id`, assign a name and description, and save the changes:
+To create a Project inside a workspace `ws` and Organization ([see organizations](https://app.evidently.cloud/organizations)) with an `org_id`, assign a name and description, and save the changes:
 
 ```
-project = ws.create_project("My test project", team_id="YOUR_TEAM_ID")
+project = ws.create_project("My test project", org_id="YOUR_ORG_ID")
 project.description = "My project description"
 project.save()
 ```
@@ -37,7 +31,7 @@ project.save()
 
 ## Add a new Project - UI
 
-Click on the “plus” sign on the home page, create a Team if you do not have one yet and type your Project name and description.
+Click on the “plus” sign on the home page, type your Project name and description.
 
 ![](../.gitbook/assets/cloud/add_project_wide-min.png)
 

docs/book/reference/all-metrics.md

@@ -263,6 +263,7 @@ Check for regular expression matches.
 | **JSONMatch()** <ul><li>Compares two columns of a dataframe and checks whether the two objects in each row of the dataframe are matching JSONs or not. </li><li>Returns True/False for every input. </li></ul> Example use:<br> `JSONMatch(with_column="column_2")`| **Required:** <br> `with_column : str` <br><br>**Optional:**<ul><li>`display_name`</li> |
 | **ContainsLink()** <ul><li>Checks if the text contains at least one valid URL. </li><li>Returns True/False for each row. </li></ul> | **Required:** n/a <br>**Optional:**<ul><li>`display_name`</li></ul> |
 | **IsValidPython()** <ul><li>Checks if the text is valid Python code without syntax errors.</li><li>Returns True/False for every input. </li></ul>| **Required:** <br>n/a<br><br>**Optional:**<ul><li>`display_name`</li></ul> |
+| **IsValidSQL()** <ul><li>Checks if the text in a specified column is a valid SQL query without executing the query.</li><li>Returns True/False for every input. </li></ul>| **Required:** <br>n/a<br><br>**Optional:**<ul><li>`display_name`</li></ul> |
 
 ## Descriptors: Text stats
 

requirements.min.txt

@@ -31,4 +31,5 @@ openai==1.16.2
 evaluate==0.4.1
 transformers[torch]==4.39.3
 sentence-transformers==2.7.0
+sqlvalidator==0.0.20
 chromadb==0.4.0

setup.cfg

@@ -102,6 +102,9 @@ ignore_missing_imports = True
 [mypy-litellm.*]
 ignore_missing_imports = True
 
+[mypy-sqlvalidator.*]
+ignore_missing_imports = True
+
 [mypy-chromadb.*]
 ignore_missing_imports = True
 

setup.py

@@ -103,6 +103,7 @@
             "evaluate>=0.4.1",
             "transformers[torch]>=4.39.3",
             "sentence-transformers>=2.7.0",
+            "sqlvalidator>=0.0.20",
             "chromadb>=0.4.0",
         ],
         "spark": ["pyspark>=3.4.0"],

src/evidently/_version.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # coding: utf-8
 
-version_info = (0, 4, 41)
+version_info = (0, 5, 0)
 __version__ = ".".join(map(str, version_info))
 
 

src/evidently/descriptors/__init__.py

@@ -8,6 +8,7 @@
 from .hf_descriptor import HuggingFaceToxicityModel
 from .is_valid_json_descriptor import IsValidJSON
 from .is_valid_python_descriptor import IsValidPython
+from .is_valid_sql_descriptor import IsValidSQL
 from .json_match_descriptor import JSONMatch
 from .json_schema_match_descriptor import JSONSchemaMatch
 from .llm_judges import BiasLLMEval
@@ -74,6 +75,7 @@
     "WordMatch",
     "WordNoMatch",
     "IsValidJSON",
+    "IsValidSQL",
     "JSONSchemaMatch",
     "IsValidPython",
     "_registry",

src/evidently/descriptors/_registry.py

@@ -127,6 +127,9 @@
     "evidently.descriptors.custom_descriptor.CustomPairColumnEval",
     "evidently:descriptor:CustomPairColumnEval",
 )
+register_type_alias(
+    FeatureDescriptor, "evidently.descriptors.is_valid_sql_descriptor.IsValidSQL", "evidently:descriptor:IsValidSQL"
+)
 register_type_alias(
     FeatureDescriptor,
     "evidently.descriptors.json_match_descriptor.JSONMatch",

src/evidently/descriptors/is_valid_sql_descriptor.py

@@ -0,0 +1,11 @@
+from evidently.features import is_valid_sql_feature
+from evidently.features.generated_features import FeatureDescriptor
+from evidently.features.generated_features import GeneratedFeature
+
+
+class IsValidSQL(FeatureDescriptor):
+    class Config:
+        type_alias = "evidently:descriptor:IsValidSQL"
+
+    def feature(self, column_name: str) -> GeneratedFeature:
+        return is_valid_sql_feature.IsValidSQL(column_name, self.display_name)

src/evidently/features/_registry.py

@@ -104,6 +104,9 @@
 register_type_alias(
     GeneratedFeatures, "evidently.features.contains_link_feature.ContainsLink", "evidently:feature:ContainsLink"
 )
+register_type_alias(
+    GeneratedFeatures, "evidently.features.is_valid_sql_feature.IsValidSQL", "evidently:feature:IsValidSQL"
+)
 register_type_alias(
     GeneratedFeatures, "evidently.features.exact_match_feature.ExactMatchFeature", "evidently:feature:ExactMatchFeature"
 )

src/evidently/features/is_valid_sql_feature.py

@@ -0,0 +1,43 @@
+from typing import Any
+from typing import ClassVar
+from typing import Optional
+
+from evidently import ColumnType
+from evidently.features.generated_features import ApplyColumnGeneratedFeature
+
+
+class IsValidSQL(ApplyColumnGeneratedFeature):
+    class Config:
+        type_alias = "evidently:feature:IsValidSQL"
+
+    __feature_type__: ClassVar = ColumnType.Categorical
+    display_name_template: ClassVar = "SQL Validity Check for {column_name}"
+    column_name: str
+
+    def __init__(self, column_name: str, display_name: Optional[str] = None):
+        self.column_name = column_name
+        self.display_name = display_name
+        super().__init__()
+
+    def apply(self, value: Any):
+        if value is None or not isinstance(value, str):
+            return False
+
+        return self.is_valid_sql(value)
+
+    def is_valid_sql(self, query: str) -> bool:
+        import sqlvalidator
+
+        queries = query.strip().split(";")  # Split by semicolon
+
+        for q in queries:
+            q = q.strip()  # Remove extra whitespace
+            if not q:  # Skip empty queries
+                continue
+
+            try:
+                sqlvalidator.format_sql(q)  # Validate SQL syntax
+            except Exception:
+                return False  # Invalid SQL
+
+        return True  # All queries are valid