Skip to content

Commit

Permalink
documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
akamor committed Dec 2, 2024
1 parent 6748c8b commit 29f8d31
Show file tree
Hide file tree
Showing 6 changed files with 83 additions and 3 deletions.
3 changes: 3 additions & 0 deletions docs/source/redact/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ TextualNer class
:members:
:no-undoc-members:

.. autoclass:: tonic_textual.classes.record_api_request_options.RecordApiRequestOptions
:members:

Dataset class
-----------------------------------------

Expand Down
24 changes: 24 additions & 0 deletions docs/source/redact/redacting_text.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ This produces the following output:
"new_text": "[ORGANIZATION_P5XLAH]"
}
`redact` calls can also be recorded so you can view and analyze results in our UI. To learn more, read :ref:`record-api-call-section`

Bulk redact raw text
---------------------
In the same way that you use the `redact` method to redact strings, you can use the `redact_bulk` method to redact many strings at the same time.
Expand Down Expand Up @@ -300,3 +302,25 @@ This produces the following output:
}
Note that LLM Synthesis is non-deterministic — you will likely get different results each time you run it.


.. _record-api-call-section:

Recording API requests
----------------------
When you redact text with the :meth:`redact<tonic_textual.redact_api.TextualNer.redact>` method you can optionally record these requests for later viewing and analysis in the UI. The `redact` method takes an optional argument called `record_options` (:class:`RecordApiRequestOptions<tonic_textual.classes.record_api_request_options.RecordApiRequestOptions>`). To record an API request you can do the following:

.. code-block:: python
from tonic_textual.redact_api import TextualNer
from tonic_textual.classes.record_api_request_options import RecordApiRequestOptions
ner = TextualNer()
ner.redact("My name is John Doe", record_options=RecordApiRequestOptions(
record=True,
retention_time_in_hours=1,
tags=["my_first_request"])
)
The above code will run the redaction as normal, and then record the API Request and its results. The request itself will be automatically purged after 1 hour. Results are viewable in the UI from the API Explorer page.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "tonic-textual"
version = "3.0.1"
version = "3.0.2"
description = "Wrappers around the Tonic Textual API"
authors = ["Adam Kamor <[email protected]>", "Joe Ferrara <[email protected]>", "Ander Steele <[email protected]>", "Ethan Philpott <[email protected]>", "Lyon Van Voorhis <[email protected]>", "Kirill Medvedev <[email protected]>", "Travis Matthews <[email protected]>"]
license = "MIT"
Expand Down
2 changes: 1 addition & 1 deletion tonic_textual/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "3.0.1"
__version__ = "3.0.2"
36 changes: 36 additions & 0 deletions tonic_textual/classes/record_api_request_options.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
from typing import List, Optional


class RecordApiRequestOptions(dict):
"""
Class to denote if an API request should be recorded.
Parameters
----------
record : bool
Indicates whether the request should be recorded.
retention_time_in_hours: int
The number of hours to store the request. Afterwards, the request is automatically purged..
tags : Optional[List[str]]
An optional list of tags to use for the request. Makes searching for requests in the UI easier.
"""
def __init__(self, record: bool, retention_time_in_hours: int, tags: Optional[List[str]] = []):
self.record = record
self.retention_time_in_hours = retention_time_in_hours
self.tags = tags

dict.__init__(
self,
record=record,
retention_time_in_hours=retention_time_in_hours,
tags = tags
)

def to_dict(self):
return {
"record": self.record,
"retention_time_in_hours": self.retention_time_in_hours,
"tags": self.tags,
}
19 changes: 18 additions & 1 deletion tonic_textual/redact_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
SingleDetectionResult,
)
from tonic_textual.classes.httpclient import HttpClient
from tonic_textual.classes.record_api_request_options import RecordApiRequestOptions
from tonic_textual.classes.redact_api_responses.bulk_redaction_response import BulkRedactionResponse
from tonic_textual.classes.redact_api_responses.redaction_response import (
RedactionResponse,
Expand Down Expand Up @@ -50,6 +51,8 @@ class TextualNer:
>>> textual = TonicTextual("https://textual.tonic.ai")
"""

default_record_options = RecordApiRequestOptions(False, 0, [])

def __init__(
self, base_url: str = "https://textual.tonic.ai", api_key: Optional[str] = None, verify: bool = True
):
Expand Down Expand Up @@ -221,6 +224,7 @@ def redact(
random_seed: Optional[int] = None,
label_block_lists: Optional[Dict[str, List[str]]] = None,
label_allow_lists: Optional[Dict[str, List[str]]] = None,
record_options: Optional[RecordApiRequestOptions] = default_record_options
) -> RedactionResponse:
"""Redacts a string. Depending on the configured handling for each sensitive
data type, values are either redacted, synthesized, or ignored.
Expand Down Expand Up @@ -252,7 +256,9 @@ def redact(
A dictionary of (entity type, additional values). When a piece of text matches a listed regular expression,
the text is marked as the entity type and is included in the redaction or synthesis.
record_options: Optional[RecordApiRequestOptions]
An optional value to record API request and result for analysis in UI. The default value is False.
Returns
-------
RedactionResponse
Expand Down Expand Up @@ -292,6 +298,12 @@ def redact(
for k, v in label_allow_lists.items()
}

if record_options.record:
record_payload = { "retentionTimeInHours": record_options.retention_time_in_hours, "tags": record_options.tags, "record": True}
payload["recordApiRequestOptions"] = record_payload
else:
payload["recordApiRequestOptions"] = None

return self.send_redact_request("/api/redact", payload, random_seed)

def redact_bulk(
Expand All @@ -302,6 +314,7 @@ def redact_bulk(
random_seed: Optional[int] = None,
label_block_lists: Optional[Dict[str, List[str]]] = None,
label_allow_lists: Optional[Dict[str, List[str]]] = None,
record: Optional[RecordApiRequestOptions] = False
) -> BulkRedactionResponse:
"""Redacts a string. Depending on the configured handling for each sensitive
data type, values are either redacted, synthesized, or ignored.
Expand Down Expand Up @@ -333,6 +346,8 @@ def redact_bulk(
A dictionary of (entity type, additional values). When a piece of text matches a listed regular expression,
the text is marked as the entity type and is included in the redaction or synthesis.
record: Optional[bool]
An optional value to record API request and result for analysis in UI. The default value is False.
Returns
-------
Expand Down Expand Up @@ -373,6 +388,8 @@ def redact_bulk(
for k, v in label_allow_lists.items()
}

payload["record"] = True if record else False

return self.send_redact_bulk_request("/api/redact/bulk", payload, random_seed)

def llm_synthesis(
Expand Down

0 comments on commit 29f8d31

Please sign in to comment.