Skip to content

Commit

Permalink
[Docs] add src code ref for offloading size limitation
Browse files Browse the repository at this point in the history
  • Loading branch information
DenChenn committed Oct 12, 2024
1 parent 719db18 commit f0f0d69
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions docs/user_guide/concepts/main_concepts/data_management.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,13 @@ In Flyte, data is categorized into metadata and raw data to optimize data handli
Flyte stores raw data in an object store (e.g., S3), uploading it on first use and passing only a reference thereafter.
Tasks can then access this data via Flyte’s automated download or streaming, enabling efficient access to large datasets without needing to transfer full copies.

*Source code reference for auto-offloading value sizes limitation:*

.. literalinclude:: https://github.com/flyteorg/static-resources/blob/ddec386ab74897e3c09d89fdaf305e77a2ecc52c/flyte/flytepropeller/pkg/controller/config/config.go
:caption: flytepropeller/pkg/controller/config/config.go
:language: go
:lines: 184-192

Data Flow and Security
~~~~~~~~~~~~~~~~~~~~~~

Expand Down

0 comments on commit f0f0d69

Please sign in to comment.