Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: upload model on podman machine on WSL to speed loading #204

Merged
merged 10 commits into from
Mar 13, 2024

Conversation

lstocchi
Copy link
Contributor

@lstocchi lstocchi commented Jan 31, 2024

So, it should be ready.
When you are on WSL the model gets uploaded to the podman machine. This makes us loses like 20sec during the start of application but it makes us gain minutes when the model is actually loaded by the model service.

So to test it,

  1. run an application
  2. when it reaches the "application is started" state, if you see the model service logs, it should be already running with the model loaded
  3. if you check inside the podman machine you should see the model
    podman machine ssh ls

If the user system is no Windows, it works as before. Qemu will be added on another PR

@lstocchi lstocchi force-pushed the saveModelonPodmanMachine branch from 274b4f5 to f944d7c Compare February 20, 2024 11:20
@lstocchi lstocchi changed the title fix: speed up model service on WSL fix: upload model on podman machine on WSL to speed loading Feb 20, 2024
@lstocchi lstocchi force-pushed the saveModelonPodmanMachine branch from f944d7c to 5b4daf4 Compare February 20, 2024 12:42
@lstocchi lstocchi marked this pull request as ready for review February 20, 2024 17:06
@lstocchi lstocchi requested a review from a team as a code owner February 20, 2024 17:06
Copy link
Contributor

@jeffmaury jeffmaury left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only concern I have is that it uses the default Podman machine which may be stopped and deployments are performed against the first Podman machine that is running

@lstocchi
Copy link
Contributor Author

@jeffmaury updated

@lstocchi lstocchi requested a review from jeffmaury February 26, 2024 15:43
@lstocchi lstocchi linked an issue Feb 26, 2024 that may be closed by this pull request
@lstocchi lstocchi force-pushed the saveModelonPodmanMachine branch 2 times, most recently from d088ce3 to c957f6d Compare February 29, 2024 17:43
@lstocchi lstocchi force-pushed the saveModelonPodmanMachine branch from c957f6d to 3529ee7 Compare March 11, 2024 11:19
* SPDX-License-Identifier: Apache-2.0
***********************************************************************/

export interface ProgressiveEvent {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why you renamed from ProgressEvent to ProgressiveEvent ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they are two different things.
Ai studio had the DownloadEvent. This PR is adding an upload action which would require a similar event and so i renamed the DownloadEvent to ProgressiveEvent. It's like downloadEvent and uploadEvent = progressiveEvent

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like Progress and Progressive are too close and could lead to some confusion, we might want to find something different to improve readability.

BaseEvent instead of DownloadEvent and keep ProgressEvent to talk about the real progress.

or

TransferEvent if we want to keep some action/wording of what is happening

I have a preference for BaseEvent, as we could create its own IBaseEvent class, and keep it separate from the logic of the Downloader/Uploader

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

progressing = happening or developing gradually or in stages. Look to me that it expresses well what it does and it can be reused.
By using TrasferEvent we are binding it only to transferring actions which may not be true in future, no?
Maybe ExecutingProgressiveEvent instead of ProgressProgressiveEvent?

Copy link
Contributor

@axel7083 axel7083 Mar 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is the Download event

export interface DownloadEvent {
  id: string;
  status: 'error' | 'completed' | 'progress' | 'cancelled';
  message?: string;
}

This seems very basic and generic, I still think something short and easy to read like BaseEvent would be more suitable than ExecutingProgressiveEvent... but this is just my opinion, if other people are fine with your choice I open to accept it !

@lstocchi lstocchi force-pushed the saveModelonPodmanMachine branch 2 times, most recently from 04aaf81 to b30b71d Compare March 11, 2024 16:03
@lstocchi
Copy link
Contributor Author

@jeffmaury rebased

@lstocchi lstocchi requested a review from axel7083 March 11, 2024 17:06
@axel7083
Copy link
Contributor

While starting an Application I got the following

AI Studio Pod details
image image

Is this expected ?

@lstocchi
Copy link
Contributor Author

Is this expected ?

No, have you done something in particular? I'll try now

@lstocchi
Copy link
Contributor Author

Did you push your commit ?

🤦 no, sorry. Pushed now

@axel7083
Copy link
Contributor

axel7083 commented Mar 12, 2024

Did you push your commit ?

🤦 no, sorry. Pushed now

lucas-pr.mp4

Maybe my repo is too old ? (will retry after I ate)

@lstocchi
Copy link
Contributor Author

lstocchi commented Mar 12, 2024

Maybe my repo is too old ? (will retry after I ate)

You can try, the error seems related to the podman machine though 🤔
What if you run podman machine list --format json? Does it work fine? Do you have any value in the podman cli path setting?

I'll add a check to prevent this kind of error

@axel7083
Copy link
Contributor

Maybe my repo is too old ? (will retry after I ate)

You can try, the error seems related to the podman machine though 🤔 What if you run podman machine list --format json? Does it work fine? Do you have any value in the podman cli path setting?

I'll add a check to prevent this kind of error

PS C:\Users\axels> podman machine list --format json
[
    {
        "Name": "podman-machine-default",
        "Default": false,
        "Created": "2023-09-23T19:18:48.434351+02:00",
        "Running": true,
        "Starting": false,
        "LastUp": "2024-03-12T12:55:57.0740947+01:00",
        "Stream": "35",
        "VMType": "wsl",
        "CPUs": 12,
        "Memory": "689012736",
        "DiskSize": "72994521088",
        "Port": 64772,
        "RemoteUsername": "user",
        "IdentityPath": "C:\\Users\\axels\\.ssh\\podman-machine-default",
        "UserModeNetworking": false
    }
]

@lstocchi
Copy link
Contributor Author

Maybe my repo is too old ? (will retry after I ate)

You can try, the error seems related to the podman machine though 🤔 What if you run podman machine list --format json? Does it work fine? Do you have any value in the podman cli path setting?
I'll add a check to prevent this kind of error

PS C:\Users\axels> podman machine list --format json
[
    {
        "Name": "podman-machine-default",
        "Default": false,
        "Created": "2023-09-23T19:18:48.434351+02:00",
        "Running": true,
        "Starting": false,
        "LastUp": "2024-03-12T12:55:57.0740947+01:00",
        "Stream": "35",
        "VMType": "wsl",
        "CPUs": 12,
        "Memory": "689012736",
        "DiskSize": "72994521088",
        "Port": 64772,
        "RemoteUsername": "user",
        "IdentityPath": "C:\\Users\\axels\\.ssh\\podman-machine-default",
        "UserModeNetworking": false
    }
]

Only that machine? I wonder why you don't have it set to default. Podman desktop should ask you to set the active machine as the default one.

@axel7083
Copy link
Contributor

axel7083 commented Mar 12, 2024

Only that machine? I wonder why you don't have it set to default. Podman desktop should ask you to set the active machine as the default one.

Apparently it is not, but this case should be handle I guess

@lstocchi lstocchi force-pushed the saveModelonPodmanMachine branch from 4a41457 to b582f66 Compare March 13, 2024 09:46
@lstocchi
Copy link
Contributor Author

Apparently it is not, but this case should be handle I guess

I was able to replicate your case by using rootful machine.
So now i use the running connection name to recreate the podman machine, it should work.
Whenever you have time, please give it another try. Thanks

@lstocchi lstocchi requested a review from axel7083 March 13, 2024 09:47
throw new Error('No podman machine is running');
}
// check if model already loaded on the podman machine
let existsRemote = true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In another PR, that would be better to have this as a callable method, so we could allow from the model list to upload it to the podman machine manually by the user. We would also probably want to be able to delete them

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I agree 👍

@axel7083
Copy link
Contributor

Apparently it is not, but this case should be handle I guess

I was able to replicate your case by using rootful machine. So now i use the running connection name to recreate the podman machine, it should work. Whenever you have time, please give it another try. Thanks

Once uploaded, the model is soo fast to start !! Love it.

Copy link
Contributor

@jeffmaury jeffmaury left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Great improvment

@lstocchi lstocchi requested a review from axel7083 March 13, 2024 11:16
Copy link
Contributor

@axel7083 axel7083 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just tested and works fine ! Great job 👏 !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Upload model on podman machine so model service starts quickly - WSL
3 participants