Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix28/add an entry point #30

Open
wants to merge 22 commits into
base: master
Choose a base branch
from

Conversation

Hierosme
Copy link

Conver by it Pull Request

  • Migrate to pyproject.toml
  • Create image-go-nord entry point

Usage

Installation - Normal

pip install .

Installation - Normal with extra AI

pip install .[AI]

Installation - Develloper mode

pip install -e .

Installation - Develloper mode with extra AI

pip install -e .[AI]

Entry point

image-go-nord --help

@Hierosme
Copy link
Author

I working on the entry point, and Make as Ready for review when i have something it work....
It can be all ready mergable, but the entry point is not ready...

@Wabri
Copy link
Member

Wabri commented Sep 16, 2024

Seems awesome, thanks! 🎉

@Hierosme
Copy link
Author

Hierosme commented Sep 18, 2024

Hello everyone look the entry point and instalaltion are testable

it can be test with by clone my repo and install it like that

mkdir DIR_WHERE_TEST
cd DIR_WHERE_TEST
python3 -m venv venv
source venv/bin/activate
git clone https://github.com/Hierosme/ImageGoNord-pip.git
git checkout FIX28/Add_an_entry_point
pip install -e .[AI]

You can use image-go-nord --help

usage: image-go-nord [-h] [--avg] [--ai] [--blur] [--quantize] [--resize WEIGHT HEIGHT] [--base64] [--reset-palette] [--add ADD] [-i] source [source ...] target

A tool to convert any RGB image or video to any theme or color palette input by the user

positional arguments:
  source                Pathname of a filename or directory, note it should have one source for process something
  target                A pathname of an existing or nonexistent file or directory, used for the output when a single file is copied. Note if it finish by '/' or '\' it will be consider like a directory

options:
  -h, --help            show this help message and exit
  --avg                 Avg algorithm and less colors it not enable it will use pixel-by-pixel approach
  --ai                  Process image by using a PyTorch model 'PaletteNet' for recoloring the image
  --blur                Enable blur
  --quantize            Enable quantization digital image processing. That reduces the number of distinct colors in an image while maintaining its overall visual quality.
  --resize WEIGHT HEIGHT
                        Resize the image during preprocessing phase
  --base64              Enable base64 convertion
  --reset-palette       Rest the palette to zero color, you can add colors with multiple --add call
  --add ADD             Add color also by hex code ex: '#FF0000' or name ex: 'POLAR_NIGHT', 'SNOW_STORM'. Note it can be a file path it contain you palette color, one hexadecimal base 16 peer line ex:
                        #FFFFFF .--add can be call more of one time
  -i, --interactive     Write a prompt for confirmation about: start processing, overwrite a existing file or create a target directory

Still to fixe:

  • Migrate tests under tests directory and plug it with pyproject.toml
  • --interactive can be disable (for debug periode)
  • --quantize i not plug
  • --base64 is not plug

Good working command exemple:

image-go-nord ./directory/ ./
image-go-nord --avg ./directory/ ./
image-go-nord --ai --blur --resize 1920 1080 ./directory/ ./
image-go-nord --ai --blur ./image.jpg ./
image-go-nord --reset-palette --add '#123456' --add './palette.txt' ./image.jpg ./

breff look to work ....

@TheJoin95
Copy link
Member

It's start looking nice and nicer!

@Hierosme Hierosme marked this pull request as ready for review September 18, 2024 22:17
@Hierosme
Copy link
Author

Hierosme commented Sep 18, 2024

Look work to work.

All the difuculty is push near argparse who is forced to true pass by properties setter where the logic is apply...

Normaly the code is correct... i still not understand how bas64 work , then cant implement it ....

image-go-nord --help
usage: image-go-nord [-h] [--avg] [--ai] [--blur] [--quantize] [--base64] [--resize WEIGHT HEIGHT] [--reset-palette] [--add ADD] [-i] [-y] SOURCE [SOURCE ...] TARGET

A tool to convert any RGB image or video to any theme or color palette input by the user. By default the algorithm is pixel-by-pixel and will be disable by --avg or --ai usage.

positional arguments:
  SOURCE                a pathname of an existing file or directory, note: you can chain source like SOURCE [SOURCE ...] in that case TARGET will be consider as directory.
  TARGET                a pathname of an existing or nonexistent file or directory, note: if nonexistent TARGET finish by '/' or '\' it will be consider as directory and will be create if necessary. (no
                        panik if the directory all ready exist it will be use as expected, in that case '/' or '\' is optional).

options:
  -h, --help            show this help message and exit
  --avg                 enable avg algorithm and less colors, if not enable the default is pixel-by-pixel approach, note: that option is disable by --ai usage.
  --ai                  process image by using a PyTorch model 'PaletteNet' for recoloring the image, note: that disable pixel-by-pixel and avg algorithms.
  --blur                enable blur
  --quantize            enable quantization digital image processing, it reduces the number of distinct colors in an image while maintaining its overall visual quality.
  --base64              enable base64 convertion during processing phase.
  --resize WEIGHT HEIGHT
                        resize the image during pre-processing phase.
  --reset-palette       reset the palette to zero color, you can add colors with multiple --add ADD calls.
  --add ADD             add color by hex16 code '#FF0000', name: 'AURORA', 'FROST', 'POLAR_NIGHT', 'SNOW_STORM' or an existing file path it contain a color palette, one hex base 16 peer line ex: #FFFFFF
                        . note: --add ADD can be call more of one time, no trouble to mixe them.
  -i, --interactive     write a prompt for confirmation about: start processing, overwrite a existing file, by default no questions is asking. note: during prompt if response is 'N', a filename will be
                        found automatically.
  -y, --yes             automatically by pass question by confirm with 'Y', that mean yes to continue and yes to overwrite existing files, note: by default prompt questions.

@TheJoin95
Copy link
Member

Could you add some tests and update the readme to add some documentation about its utilization?

To me is looking quite good, I'm don't know if it's still worth maintain the image-go-nord cli project by this PR. WDYT @Wabri ?

This package will come with the API library and the CLI entry point.

@Hierosme
Copy link
Author

Sure i can add some tests (with unittest)
And updade Readme and ChangeLogs.

In general for optain a 100% test cover code i start from the frist line of code by create tests.
Here it can be really hard to Moke stdout (but that my job) , i certainly have to trasnforme part of the code for be a bit testable.

@TheJoin95
Copy link
Member

Sure, let's say if we can aim for a good 70-80% of test coverage that would be super!

@Hierosme
Copy link
Author

Hierosme commented Sep 20, 2024

I'm on it,

ImageGoNord-pip (FIX28/Add_an_entry_point) [1]> pytest
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0
rootdir:  ImageGoNord-pip
configfile: pyproject.toml
testpaths: tests
plugins: mock-3.14.0, cov-5.0.0
collected 7 items                                                                                                                                                                                          

tests/test_GoNord.py::test_resize_image_with_size PASSED                                                                                                                                             [ 14%]
tests/test_GoNord.py::test_resize_image PASSED                                                                                                                                                       [ 28%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_lookup_file PASSED                                                                                                                            [ 42%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_lookup_file_supported_input_format PASSED                                                                                                     [ 57%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_size PASSED                                                                                                                                   [ 71%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_source PASSED                                                                                                                                 [ 85%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_target PASSED                                                                                                                                 [100%]

---------- coverage: platform linux, python 3.12.6-final-0 -----------
Name                                        Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------
ImageGoNord/GoNord.py                         281    198    30%   21-23, 28-30, 41-43, 182, 202-213, 216, 220-221, 225-226, 230, 234, 238, 254-258, 303-306, 322-324, 340, 347, 354, 390-401, 404-437, 440-447, 465-521, 545-579, 592, 611-623, 653-665, 691-708, 727-738, 756-765, 789-827
ImageGoNord/__init__.py                         4      0   100%
ImageGoNord/__main__.py                       261    135    48%   288-290, 293-320, 324-326, 331-340, 344-401, 404-429, 432-440, 443-505, 508, 511-531, 535-554, 558
ImageGoNord/models/PaletteNet/__init__.py       0      0   100%
ImageGoNord/models/__init__.py                  0      0   100%
ImageGoNord/palettes/Nord/__init__.py           0      0   100%
ImageGoNord/palettes/__init__.py                0      0   100%
ImageGoNord/utility/ConvertUtility.py          34     29    15%   33, 57-84, 102-107
ImageGoNord/utility/__init__.py                 0      0   100%
ImageGoNord/utility/model.py                   95     63    34%   11-12, 16, 25-29, 32-37, 41, 45-47, 57, 61, 64, 70-71, 80-88, 91-96, 99, 108-113, 116-141
ImageGoNord/utility/palette_loader.py          32     24    25%   26-31, 51-52, 72-75, 95-98, 118-121, 143-154
ImageGoNord/utility/quantize.py                17     15    12%   25-46
-------------------------------------------------------------------------
TOTAL                                         724    464    36%


============================================================================================ 7 passed in 11.32s ============================================================================================

@Hierosme Hierosme marked this pull request as ready for review September 20, 2024 16:16
@Hierosme
Copy link
Author

Hierosme commented Sep 20, 2024

That is tested

I have disable run main and processing because interactive mode make lot of trouble to pytest.
That is well tested and by really test something the entire projet is now cover to 77%

Enjoy cher

platform linux -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0
rootdir: ImageGoNord-pip
configfile: pyproject.toml
testpaths: tests
plugins: mock-3.14.0, cov-5.0.0
collected 15 items                                                                                                                                                                                         

tests/test_GoNord.py::test_resize_image_with_size PASSED                                                                                                                                             [  6%]
tests/test_GoNord.py::test_resize_image PASSED                                                                                                                                                       [ 13%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_ask_to_continue PASSED                                                                                                                        [ 20%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_lookup_file PASSED                                                                                                                            [ 26%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_lookup_file_get_next_non_existing_filename PASSED                                                                                             [ 33%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_lookup_file_into PASSED                                                                                                                       [ 40%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_lookup_file_supported_input_format PASSED                                                                                                     [ 46%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_post_processing PASSED                                                                                                                        [ 53%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_pre_processing_palette PASSED                                                                                                                 [ 60%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_preprocessing PASSED                                                                                                                          [ 66%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_print_summary PASSED                                                                                                                          [ 73%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_processing PASSED                                                                                                                             [ 80%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_size PASSED                                                                                                                                   [ 86%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_source PASSED                                                                                                                                 [ 93%]
tests/test_ImageGoNordCLI.py::TestImageGoNordCLI::test_target PASSED                                                                                                                                 [100%]

---------- coverage: platform linux, python 3.12.6-final-0 -----------
Name                                        Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------
ImageGoNord/GoNord.py                         281    106    62%   21-23, 28-30, 41-43, 182, 211, 230, 256, 303-306, 322-324, 409-410, 413-414, 418, 434, 440-447, 479, 482-483, 553-556, 561-571, 574, 611-623, 653-665, 691-708, 727-738, 756-765, 789-827
ImageGoNord/__init__.py                         4      0   100%
ImageGoNord/__main__.py                       223      0   100%
ImageGoNord/models/PaletteNet/__init__.py       0      0   100%
ImageGoNord/models/__init__.py                  0      0   100%
ImageGoNord/palettes/Nord/__init__.py           0      0   100%
ImageGoNord/palettes/__init__.py                0      0   100%
ImageGoNord/utility/ConvertUtility.py          34     28    18%   57-84, 102-107
ImageGoNord/utility/__init__.py                 0      0   100%
ImageGoNord/utility/model.py                   95      1    99%   41
ImageGoNord/utility/palette_loader.py          32     14    56%   26-31, 51-52, 143-154
ImageGoNord/utility/quantize.py                17      7    59%   29, 31-35, 37
-------------------------------------------------------------------------
TOTAL                                         686    156    77%

@TheJoin95
Copy link
Member

We are open to suggestions in terms of tests, do you have any? At the moment it's not in a CI/CD, would it be great to have it as a GitHub action for example.

Maybe @Wabri you can open an issue on it if you agree

@Hierosme
Copy link
Author

I can give you my personnal point:

I use unittest couple with green for my dailly work. Thats is sooooooooo fast for ping/pong correction / test / correction. And green have help me many time for found a trouble where pytest see nothing by default.

pytest support unittest out of the box. (Witout plugin need), it's what i have use for ImageGoNord-pip,

The design of pytest force to use plugins, then more you want options more you load plugins, and that finish to be too slow for a developper it have to do correction / test / correction / test for the entire day/night.

pytest really a good tool when it have html, json repport to do, it require more config, more plugin them more time, but it make good job wll.

Then i recommand unnittest at all, and i consider the display can be done by both green or pytest, it depends of project. Look like i never import pytest in my tests files due to slow pytest design (plugin approch).

@Hierosme
Copy link
Author

Hoooo i have forget to say.
With it PR the version change:

version = "1.1.1"

I have increase minor version for not let PIP make a mistake during Upgrade.
A new package should be gnerate for pypi

@Wabri
Copy link
Member

Wabri commented Sep 23, 2024

I want to point that @BugliL was working on the new cli and advance testing in this pr: Schroedinger-Hat/ImageGoNord-cli#25

He has done a impressive job on the mocking testing, see it here https://github.com/Schroedinger-Hat/ImageGoNord-cli/pull/25/files#diff-4000edb933a4a79d4e67889094ffb9fb9c57383090399e6d75e864ce1e37e8fd

I've no hard opinion on this, maybe we shoud do a separate discussion on it.

@Hierosme
Copy link
Author

I have look the code.
I note lot of / separator in hard it should be replace by os.sep for permit cross platform usage.

The use of unittest is really correct and compatible with the code on my PR.

I recommand to limit the size of a PR (number of feature), because it make un-mergable result.
I dont say my code is better i speack about impact of a click on Merge button.

My PR is mergable as it with only what i have describ on top of the PR creation.

I can help to merge the work of @BugliL we no trouble. (But in on other PR)

@Hierosme
Copy link
Author

Look strange to have a CLI on both repository.
i'm really confuse because i have respond to an existing Isssue on ImageGoNord-pip repository

I have no trouble to merge our base code.

I reppeat the @BugliL PR is too big for me mergable as minor change...
The better in opensource project is the ultra small PR...

What is the goal of thte PR:

  • Wdd new palette -> how many ?
  • fixe code on the backend for how many point ?
  • create a CLI
  • add documentation

From my point it shuold be 4 PR to be mergable.

But here we cumulate a responssability range trouble .

  • How many package pip is suppose to install when a user want to install?
  • From whish repository ?
  • Which CI/CD publish the package in pip ?

Hop it help.

I have no trouble to help for a merge code, and have no trouble to trash my code too.

@BugliL
Copy link
Contributor

BugliL commented Sep 27, 2024

Hi Hierosme,
I think that this PR is great! This project need it!
We needed those files and a starting configuration for the project.
I think that the code that you have done is great too! I whish you could contribute more to the project.

The difference between the gihub projects ImageGoNord-pip and ImageGoNord-cli is that this is the library, the other is to intended to be only the CLI.
With this separation we can evolve both projects in parallel so that the client could be implemented with more features (es. a GUI, folder managing, ....) without affecting the actual library.
So, who wants to use the library can use it without the need to install the CLI even if is small. I think that one of next improvements could be to pip the CLI, we need to do that too!

Yes I think that my PR is big, that is not a minor change, that is a big breaking change, but if you look the old code was not maintainable at all.
I think that my PR is a fresh start with argparse so we could improve as much as we need.

I think the best here is to move the CLI code of this branch to the ImageGoNord-cli project, and merge the other features of this PR.
We can work together to improve the CLI and make it much better than it is now.

@Hierosme
Copy link
Author

Hi @BugliL , hi everyone,

If i understand well,

  • The CLI i have code for ImageGoNord-pip as "entry point" should be migrate to ImageGoNord-cli,
  • Keep the ImageGoNord-pip myproject.toml file here for pip distribution of the Librairy
  • Help a bit for ImageGoNord-pip for CI/CD in direction to pypi
  • A merge of our code can be done under ImageGoNord-cli repository

Look correct plan,

The CLI i provide with it PR is 100% tested and yet require a myproject.toml, if i come with it code on ImageGoNord-cli it will cause a un-mergable code.
For merge it with ImageGoNord-cli we should choose which code will have the precedence, then integrate only the valuable difference.

We have code the same target CLI entry point without exchange about the architechture , the valuable code of my entry point is the sources / targets management. The rest is just an implementation of docs/exemple.py.
The palette management is not so bad, but can be improuve.

From my point: (Just a personal point, nothing realtive to Truth)

  • TUI (ncurses) or GUI (GTK, QT, Kivy etc ...) should be done an separted repository
  • The CLI should be provide as minimal entrypoint of the library without need of a second python package as dependency.
  • It entry point should be cross platforme
  • If need a dependencies it should for Ncurses ot Graphic binding where cross platform can be relative to the final target implementation.

Here i not understand really what is the next step ... i have no trouble to help, but the only thing i identify is the need to create a myproject.toml file into ImageGoNord-cli repository and put depency into dirrection of ImageGoNord-pip

As explain before i think it will contribute to create bad experience to ImageGoNord users, the ImageGoNord-pip CLI have 19Ko size, it normally shoud not require a dedicated repository...

Thanks for you response, and sorry to push hard like that to merge my code. My goal is not to create a battle with the both CLI. or to denegrate you work.

Regards

@BugliL
Copy link
Contributor

BugliL commented Sep 29, 2024

Thanks for you response, and sorry to push hard like that to merge my code. My goal is not to create a battle with the both CLI. or to denegrate you work.

Don't worry, nobody is denegrating any work, we all want to cooperate to make ImageGoNord better 😃

As explain before i think it will contribute to create bad experience to ImageGoNord users, the ImageGoNord-pip CLI have 19Ko size, it normally shoud not require a dedicated repository...

As mentioned before, it's just not about space but modularity. We can evolve both, the library and the client without affecting each other. If both stay in the same package, even a small change in the library would change a version in the CLI. If the CLI stay separated could be have a fixed requirement version of the library and would work even the library completely change.

Here i not understand really what is the next step ...

In my opinion, the first objective should be the merge of part of the functionalities that you added here, specific:

  • .gitignore additional paths
  • ImageGoNord/GoNord.py improvements here
  • ImageGoNord/__init__.py specific imports here
  • pyproject.toml with all configuration
  • setup.py
    and off course the CHANGELOG.md with changes.

The client in the __main__.py file and relative tests should be saved for later to be merged in the ImageGoNord-cli repository. I think this is a good starting point 😄

@TheJoin95
Copy link
Member

I do agree with @BugliL , let's split things first so that we can merge the changes here then open a PR on the cli repository.
It's an amazing work and I believe it will enhance the project :)

@Hierosme
Copy link
Author

Look a bit like a refactoring.

From my point it type refactoring should be address by multiple PR and if possible from main/master branch of both repository. The __main__.py file and tests are specifict to it PR, a natural place to save it is here on GitHub , by merge the code.

Why not simply merge and schedule the refactoring as it should?

We dont speack about a misstake introduse by the PR code, then for me if no error is on the code it should be merge.
It PR is for #28 , and it PR cover the description of it issue.

Yes by close #28 issue with it PR it look to introduse perturbation ... but that not invalide the issue and the PR.
Refactoring, or reduse perturbation introduse by my code should be address in the future by future request.

Schedule a refactoring by multiple PR, is a good practice because without that i'll let you choose where save my work, and let you choose the better moment to integrate it...

I propose to merge, and create Issue on both project for cover the refactoring.

@TheJoin95
Copy link
Member

Can we merge this? @BugliL @Wabri I don't have a really strong opinion on why not merging. If everyone agree, I'll let @BugliL and @Hierosme sync and create the following issues

@Wabri
Copy link
Member

Wabri commented Sep 30, 2024

I've not strong opinion too, we can use the cli in here just for the sake of test of the pip package and use the cli repository to enhance the usability of the pip package. In those cases we should tell somewhere in the docs that this cli is just for test purpose.

For me no problem with the merge!

@BugliL BugliL self-requested a review October 2, 2024 22:10
Copy link
Contributor

@BugliL BugliL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this CLI is much more than what is needed to have a [__main__.py](http://__main__.py/) entry point. This is good if we want to use the library calling it directly but I think we should continue all CLI improvements on the other repository.
I will ask you just to add few lines in docs, so that people that will read the code or print the help could know that exists a different project with CLI code.

After this edit I think that we can create subsequent issues and go on to the other project 😄 🚀

README.md Show resolved Hide resolved
Co-authored-by: Lorenzo Bugli <[email protected]>
Copy link
Member

@TheJoin95 TheJoin95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the missing doc lines. I'm approving this PR, so we can move forward.
@Wabri if you agree we can merge this after the #33 so that I can create two different release.

Please @BugliL and @Hierosme I'll let you create the roadmap/issues for the next part !

Huge contribution, thanks again @Hierosme !

@Hierosme
Copy link
Author

Hierosme commented Oct 4, 2024

Thanks @TheJoin95

The migration to myproject.toml is a big improvement.
In any case by install the package you get image-go-nord CLI Console entry point.

I waiting for a merge before any change

@TheJoin95
Copy link
Member

Please rebase, then we can merge

@BugliL BugliL self-requested a review October 6, 2024 17:57
Copy link
Contributor

@BugliL BugliL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@TheJoin95
Copy link
Member

Pleaser rebase and change the destination branch ref to be "main"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants