Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[flyte deck] Streaming Decks #2779

Open
wants to merge 51 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
01182b4
[Flyte Decl] Streaming Decks
Future-Outlier Oct 1, 2024
9616fc3
print
Future-Outlier Oct 1, 2024
98ae7c7
sleep more
Future-Outlier Oct 1, 2024
4e92bb0
add dummy deck
Future-Outlier Oct 2, 2024
4df18b5
nit
Future-Outlier Oct 2, 2024
ebb4d4e
dummy deck
Future-Outlier Oct 2, 2024
99522d9
update
Future-Outlier Oct 2, 2024
c19d67d
nit
Future-Outlier Oct 2, 2024
67cd829
test
Future-Outlier Oct 2, 2024
06da3df
return html
Future-Outlier Oct 2, 2024
6d99d69
Change Deck
Future-Outlier Oct 2, 2024
b805cd7
fix
Future-Outlier Oct 2, 2024
4c97758
fix recursion error
Future-Outlier Oct 2, 2024
7b3574a
remove redundant code
Future-Outlier Oct 2, 2024
9b60564
add dummy deck to deck init
Future-Outlier Oct 2, 2024
18c994f
Better Dummy Deck Logic
Future-Outlier Oct 2, 2024
39f39d1
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Oct 7, 2024
aabcbbb
Deck Publish
Future-Outlier Oct 7, 2024
9ca43f3
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Nov 25, 2024
ed56352
litn
Future-Outlier Nov 27, 2024
b559fc9
remove dummy deck
Future-Outlier Nov 27, 2024
fc5578f
nit
Future-Outlier Nov 27, 2024
7139468
use auto refresh tab, 5 seconds as interval
Future-Outlier Dec 2, 2024
e0aee9e
revert
Future-Outlier Dec 2, 2024
3727588
test setDynamicTabs
Future-Outlier Dec 2, 2024
ce3ee15
change interval time
Future-Outlier Dec 2, 2024
d066231
test
Future-Outlier Dec 2, 2024
f9387ce
revert
Future-Outlier Dec 2, 2024
f14c3fa
test
Future-Outlier Dec 2, 2024
c33a909
nit
Future-Outlier Dec 2, 2024
8666c60
try dynamic containers
Future-Outlier Dec 2, 2024
93580d6
try dynamic containers v2
Future-Outlier Dec 2, 2024
bcaaabd
try dynamic containers v3
Future-Outlier Dec 2, 2024
a321700
debug
Future-Outlier Dec 2, 2024
6464fae
update
Future-Outlier Dec 2, 2024
884943c
nit
Future-Outlier Dec 3, 2024
d4b5b96
Refresh Botton
Future-Outlier Dec 3, 2024
406227c
fix
Future-Outlier Dec 3, 2024
1e77f54
lint
Future-Outlier Dec 3, 2024
d70a2d5
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Dec 3, 2024
7fc6393
test new refresh
Future-Outlier Dec 3, 2024
6980140
lint
Future-Outlier Dec 3, 2024
c87a342
Revert back html code, collaborating with Lyon
Future-Outlier Dec 5, 2024
f609760
lint
Future-Outlier Dec 5, 2024
473ae11
nit
Future-Outlier Dec 5, 2024
008fe52
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Dec 12, 2024
f0b9028
nit
Future-Outlier Dec 16, 2024
d48efa9
update
Future-Outlier Dec 17, 2024
6b55930
better code
Future-Outlier Dec 17, 2024
b5912fb
update
Future-Outlier Dec 17, 2024
a681ccd
some notes for giving user params builder deck enabled
Future-Outlier Dec 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions flytekit/bin/entrypoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,11 @@ def _dispatch_execute(
input_proto = utils.load_proto_from_file(_literals_pb2.LiteralMap, local_inputs_file)
idl_input_literals = _literal_models.LiteralMap.from_flyte_idl(input_proto)

# # TEST DECK
# if task_def is not None and not getattr(task_def, "disable_deck", True):
# _output_deck(task_def.name.split(".")[-1], ctx.user_space_params)
# print("@@@ OUTPUT DECK @@@")

# Step2
# Invoke task - dispatch_execute
outputs = task_def.dispatch_execute(ctx, idl_input_literals)
Expand Down
2 changes: 1 addition & 1 deletion flytekit/deck/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@
SourceCodeRenderer
"""

from .deck import Deck, DeckField
from .deck import Deck, DeckField, DummyDeck
from .renderer import MarkdownRenderer, SourceCodeRenderer, TopFrameRenderer
63 changes: 62 additions & 1 deletion flytekit/deck/deck.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,40 @@

OUTPUT_DIR_JUPYTER_PREFIX = "jupyter"
DECK_FILE_NAME = "deck.html"
DUMMY_DECK_HTML = """
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will make it automatically refresh the page.. probably need that on the empty deck but not sure how to do the realtime behavior once it has content (you don't want to refresh while the user is browsing around or clicking through various tabs)

Suggested change
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="refresh" content="5" >

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COOL, will try it, thank you

<title>Flytekit Status</title>
<style>
body {
display: flex;
justify-content: center;
align-items: center;
height: 100vh;
margin: 0;
font-family: Arial, sans-serif;
background-color: #f0f0f0;
}
.message {
background-color: #ffffff;
padding: 20px;
border: 1px solid #cccccc;
border-radius: 5px;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
text-align: center;
}
</style>
</head>
<body>
<div class="message">
<p> Flyte decks have not been created yet. </p>
</div>
</body>
</html>
"""


class DeckField(str, enum.Enum):
Expand Down Expand Up @@ -86,6 +120,27 @@ def name(self) -> str:
def html(self) -> str:
return self._html

@classmethod
def publish(cls):
task_name = FlyteContextManager.current_context().user_space_params.task_id.name
new_user_params = FlyteContextManager.current_context().user_space_params
_output_deck(task_name, new_user_params)


class DummyDeck(Deck):
"""
The DummyDeck class is designed
"""

def __init__(self):
name = "dummy_deck"
html = DUMMY_DECK_HTML
super().__init__(name, html)

@property
def html(self) -> str:
return self._html


class TimeLineDeck(Deck):
"""
Expand Down Expand Up @@ -154,7 +209,13 @@ def _get_deck(
Get flyte deck html string
If ignore_jupyter is set to True, then it will return a str even in a jupyter environment.
"""
deck_map = {deck.name: deck.html for deck in new_user_params.decks}

deck_map = {deck.name: deck.html for deck in new_user_params.decks if deck.name != "dummy_deck"}

# If deck_map is empty after filtering, add DummyDeck
if not deck_map:
deck_map = {"dummy_deck": DUMMY_DECK_HTML}

nav_htmls = []
body_htmls = []

Expand Down
Loading