Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a yml file to build the docker image #74

Merged
merged 19 commits into from
Oct 28, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
af49849
Added a yml file to build the docker image
Unique-Usman Aug 11, 2024
c2149da
Removed -it, and also save the docker image as an artifcat
Unique-Usman Aug 13, 2024
863e7f7
Added sample data for running the docker image, also added the script…
Unique-Usman Sep 26, 2024
1fa952c
Fixed the float not subscriptable error
Unique-Usman Sep 26, 2024
7294bde
Generated the testing images during docker building and added confirm…
Unique-Usman Sep 26, 2024
80c1663
Removed erroneus file
Unique-Usman Sep 26, 2024
602f17d
Improve the github actions to install all the required library for ru…
Unique-Usman Sep 26, 2024
33acf99
Added nibabel to the requirement
Unique-Usman Sep 26, 2024
dc9265d
Added debugging output options to the github action
Unique-Usman Sep 26, 2024
7b2f0b8
Added debugging output options to the github action
Unique-Usman Sep 26, 2024
ee2eb99
Change the name of the ivim_image.nii.gz to ivim_simulation.nii.gz
Unique-Usman Sep 26, 2024
a4136f8
used docker buildx for save the docker image as tar file and also mov…
Unique-Usman Oct 1, 2024
a6ddfb2
Revert from archiving to saving then archiving
Unique-Usman Oct 1, 2024
ea69e18
Added Docker cleanup
Unique-Usman Oct 21, 2024
428095e
Change the base image to alpine linux
Unique-Usman Oct 21, 2024
6fee6ed
Change the version of libssl to 3
Unique-Usman Oct 21, 2024
1f6da96
Remove pushing of the image and used slim instead of alpine
Unique-Usman Oct 21, 2024
17b98f8
Merge branch 'main' into main
Unique-Usman Oct 21, 2024
12ae83f
Avoid uploading the docker image
Unique-Usman Oct 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions .github/workflows/docker-build-and-run.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,24 +25,27 @@ jobs:
run: |
docker build -t tf2.4_ivim-mri_codecollection -f Docker/Dockerfile .
Unique-Usman marked this conversation as resolved.
Show resolved Hide resolved

- name: Save Docker image to a tarball
- name: Save and compress Docker image
run: |
docker save -o tf2.4_ivim-mri_codecollection.tar tf2.4_ivim-mri_codecollection
docker save tf2.4_ivim-mri_codecollection | gzip > tf2.4_ivim-mri_codecollection.tar.gz

- name: Upload Docker image artifact
Unique-Usman marked this conversation as resolved.
Show resolved Hide resolved
uses: actions/upload-artifact@v4
with:
name: docker-image
path: tf2.4_ivim-mri_codecollection.tar
path: tf2.4_ivim-mri_codecollection.tar.gz
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not seeing why, but the artifact is empty. I wonder if it's too big? If there's no way to save it perhaps it could be split or maybe we just forget about saving the file for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will check this and verify.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@etpeterson, I think it is there, where are you checking
Screenshot From 2024-10-01 05-29-09

Copy link
Contributor

@etpeterson etpeterson Oct 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A whole docker image is not 198 bytes. If you download it and unzip it you'll find it empty. Also, no space left on device:
image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@etpeterson, any suggested solution ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could try a cleaning
- name: Cleanup Docker run: | docker system prune -a --force

You could also try alpine linux base. I think it's smaller but might not work correctly.

If neither of those works then let's build and test but not push.


- name: Run Docker container
run: |
docker run --rm --name TF2.4_IVIM-MRI_CodeCollection \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should confirm that images are actually generated. Maybe not the exact values in the images, but even that the files we expect exist.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, exactly.

-v ${{ github.workspace }}:/usr/src/app \
-v ${{ github.workspace }}:/usr/app/output \
tf2.4_ivim-mri_codecollection brain.nii.gz brain.bvec brain.bval
tf2.4_ivim-mri_codecollection Docker/ivim_simulation.nii.gz Docker/ivim_simulation.bvec Docker/ivim_simulation.bval

- name: Clean up
- name: Clean up artifacts and Docker image
run: |
docker rmi tf2.4_ivim-mri_codecollection
rm tf2.4_ivim-mri_codecollection.tar
docker rmi tf2.4_ivim-mri_codecollection || true
rm -f tf2.4_ivim-mri_codecollection.tar.gz
rm -f ${{ github.workspace }}/f.nii.gz
rm -f ${{ github.workspace }}/dp.nii.gz
rm -f ${{ github.workspace }}/d.nii.gz
31 changes: 31 additions & 0 deletions Docker/generate_signal_docker_test.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of this file? It looks like it generates an image but I don't see it being used for anything. I was expecting to see the simulated image and associated files generated by this.

Copy link
Contributor Author

@Unique-Usman Unique-Usman Sep 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the code I used for generating the signal which I was using for the docker. I did not make use of it because I already generated the images. I will restructured it to generate the images on the fly thereby making use of it in the build process.

Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import numpy as np
import nibabel as nib
from utilities.data_simulation.GenerateData import GenerateData

def save_nii(data, filename='ivim_image.nii.gz'):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's already a save_nifti_file in the wrapper.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sure.

"""
Save the data as a NIfTI file (.nii.gz)
"""
nii_img = nib.Nifti1Image(data, affine=np.eye(4))
nib.save(nii_img, filename)

# Set random seed for reproducibility
np.random.seed(42)

# Create GenerateData instance
gd = GenerateData()

# Generate random input data
shape = (10, 10, 5)
f_in = np.random.uniform(low=0, high=1, size=shape)
D_in = np.random.uniform(low=0, high=1e-3, size=shape)
Dp_in = np.random.uniform(low=0, high=1e-1, size=shape)
S0 = 1000 # Setting a constant S0 for simplicity
bvals = np.array([0, 50, 100, 500, 1000])
bvals_reshaped = np.broadcast_to(bvals, shape)

# Generate IVIM signal
images = gd.ivim_signal(D_in, Dp_in, f_in, S0, bvals_reshaped)

# Save the generated image as a NIfTI file
save_nii(images)
1 change: 1 addition & 0 deletions Docker/ivim_simulation.bval
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0 50 100 500 1000
3 changes: 3 additions & 0 deletions Docker/ivim_simulation.bvec
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
1 0 0
0 1 0
0 0 1
Binary file added Docker/ivim_simulation.nii.gz
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and the associated bval and bvec files are inside the docker folder, so they're getting added to the docker image. We don't want that. Also, shouldn't they be generated on the fly rather than be pre-generated?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will generate it on the fly, I thought it makes sense having it constant.

Binary file not shown.
4 changes: 3 additions & 1 deletion WrapImage/nifti_wrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,9 @@ def save_nifti_file(data, output_file, affine=None, **kwargs):
For saving the 3d nifti images of the output of the algorithm
"""
if affine is None:
affine = np.eye(data.ndim + 1)
affine = np.eye(4)
else:
affine = np.array(affine.reshape(4, 4))
output_img = nib.nifti1.Nifti1Image(data, affine , **kwargs)
nib.save(output_img, output_file)

Expand Down
6 changes: 3 additions & 3 deletions utilities/data_simulation/GenerateData.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ def exponential_signal(self, D, bvalues):
bvalues : list or array of float
The diffusion (b-values)
"""
assert D >= 0, 'D must be >= 0'
assert np.all(D >= 0), 'all values in D must be >= 0'
return self._op.exp(-self._op.asarray(bvalues, dtype='float64') * D)

def multiexponential_signal(self, D, F, S0, bvalues):
Expand Down Expand Up @@ -116,7 +116,7 @@ def linear_signal(self, D, bvalues, offset=0):
offset : float
The signal offset
"""
assert D >= 0, 'D must be >= 0'
assert np.all(D >= 0), 'every value in D must be >= 0'
data = -D * np.asarray(bvalues)
return data + offset

Expand Down Expand Up @@ -144,4 +144,4 @@ def multilinear_signal(self, D, F, S0, bvalues, offset=0):
signal += f * self.linear_signal(d, bvalues)
signal *= S0
signal += offset
return signal
return signal
Loading
Loading