Skip to content

Commit

Permalink
feat(docs): enhance Python bindings documentation and CLI
Browse files Browse the repository at this point in the history
Improve documentation and clarity of the Python bindings and CLI interface,
with particular focus on streaming operations and API completeness.

Documentation Improvements:
- Add comprehensive docstrings for all Python functions
- Enhance streaming_decrypt_from_storage documentation with parallel processing details
- Add detailed examples using ThreadPoolExecutor for chunk retrieval
- Include minimum size requirements in encryption documentation
- Add CLI usage examples and command descriptions

Technical Details:
- Clarify get_chunks callback interface for streaming operations
- Add key features section for streaming functionality
- Improve error descriptions and type hints
- Add memory efficiency notes for large file operations

Version bump to 0.32.3 to reflect documentation improvements.
  • Loading branch information
dirvine committed Nov 19, 2024
1 parent 87f5593 commit b0ca5e5
Show file tree
Hide file tree
Showing 3 changed files with 256 additions and 70 deletions.
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ license = "GPL-3.0"
name = "self_encryption"
readme = "README.md"
repository = "https://github.com/maidsafe/self_encryption"
version = "0.32.2"
version = "0.32.3"

[features]
default = []
Expand Down
64 changes: 57 additions & 7 deletions self_encryption/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
- Streaming operations for large files
- Parallel chunk processing
- Both in-memory and file-based operations
- Command-line interface for all operations
Basic Usage:
>>> from self_encryption import encrypt, decrypt
Expand All @@ -27,27 +28,74 @@
... return (Path("chunks") / hash_hex).read_bytes()
>>> decrypt_from_storage(data_map, "output.dat", get_chunk)
Advanced Features:
- Hierarchical data maps for large files
- Streaming decryption with parallel chunk retrieval
- Chunk verification and validation
- XorName operations for content addressing
Command Line Usage:
The library includes a command-line interface for all operations:
# Encrypt a file
$ self-encryption encrypt-file input.dat chunks/
# Decrypt a file
$ self-encryption decrypt-file data_map.json chunks/ output.dat
# Verify a chunk
$ self-encryption verify chunks/abc123.dat
# Shrink a data map
$ self-encryption shrink data_map.json chunks/ optimized_map.json
For more information about CLI commands:
$ self-encryption --help
Classes:
DataMap - Contains metadata about encrypted chunks
Methods:
new(chunk_infos) -> DataMap
with_child(chunk_infos, child) -> DataMap
child() -> Optional[int]
is_child() -> bool
len() -> int
infos() -> List[Tuple[int, bytes, bytes, int]]
EncryptedChunk - Represents an encrypted chunk of data
Methods:
new(content: bytes) -> EncryptedChunk
content() -> bytes
from_bytes(content: bytes) -> EncryptedChunk
XorName - Content-addressed names for chunks
Methods:
new(bytes) -> XorName
from_content(content) -> XorName
as_bytes() -> bytes
Functions:
encrypt(data: bytes) -> Tuple[DataMap, List[EncryptedChunk]]
Encrypt data in memory, returning a data map and encrypted chunks.
The input data must be at least 3072 bytes.
encrypt_from_file(input_path: str, output_dir: str) -> Tuple[DataMap, List[str]]
Encrypt a file and store chunks to disk. Returns a data map and chunk names.
The input file must be at least 3072 bytes.
decrypt(data_map: DataMap, chunks: List[EncryptedChunk]) -> bytes
Decrypt data using provided chunks in memory.
decrypt_from_storage(data_map: DataMap, output_path: str, get_chunk: Callable) -> None
shrink_data_map(data_map: DataMap, store_chunk: Callable) -> Tuple[DataMap, List[EncryptedChunk]]
Decrypt data using chunks from storage, writing directly to a file.
Suitable for files that can fit in memory.
streaming_decrypt_from_storage(data_map: DataMap, output_path: str, get_chunks: Callable) -> None
Decrypt data using parallel chunk retrieval for improved performance.
Optimized for large files and remote storage backends.
Retrieves multiple chunks in parallel for better throughput.
shrink_data_map(data_map: DataMap, store_chunk: Callable) -> Tuple[DataMap, List[EncryptedChunk]]
Shrink a data map by recursively encrypting it. Useful for large files.
verify_chunk(name: XorName, content: bytes) -> EncryptedChunk
Verify the integrity of an encrypted chunk.
For more information about specific functions or classes, use help() on the individual items:
For more detailed documentation about specific functions or classes:
>>> help(self_encryption.DataMap)
>>> help(self_encryption.encrypt)
"""
Expand All @@ -67,6 +115,8 @@

from .cli import cli

__version__ = "0.32.2"

__all__ = [
"DataMap",
"EncryptedChunk",
Expand Down
Loading

0 comments on commit b0ca5e5

Please sign in to comment.