Skip to content

Commit

Permalink
Major Refactor
Browse files Browse the repository at this point in the history
  • Loading branch information
@MrTyton
MrTyton committed May 27, 2024
1 parent 20f913f commit 1ba1f0f
Show file tree
Hide file tree
Showing 44 changed files with 4,871 additions and 1,572 deletions.
28 changes: 28 additions & 0 deletions .VSCodeCounter/2024-05-20_23-45-31/details.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Details

Date : 2024-05-20 23:45:31

Directory c:\\Users\\Joshua\\Documents\\GitHub\\Hub\\Automated-FFDL

Total : 13 files, 3793 codes, 119 comments, 1213 blanks, all 5125 lines

[Summary](results.md) / Details / [Diff Summary](diff.md) / [Diff Details](diff-details.md)

## Files
| filename | language | code | comment | blank | total |
| :--- | :--- | ---: | ---: | ---: | ---: |
| [calibre_info.py](/calibre_info.py) | Python | 47 | 13 | 13 | 73 |
| [config.toml](/config.toml) | TOML | 16 | 0 | 2 | 18 |
| [defaults.ini](/defaults.ini) | Ini | 3,177 | 38 | 1,080 | 4,295 |
| [fanfic_info.py](/fanfic_info.py) | Python | 46 | 7 | 5 | 58 |
| [fanficdownload.py](/fanficdownload.py) | Python | 18 | 6 | 13 | 37 |
| [ff_logging.py](/ff_logging.py) | Python | 25 | 1 | 4 | 30 |
| [ff_logging_test.py](/ff_logging_test.py) | Python | 27 | 0 | 5 | 32 |
| [personal.ini](/personal.ini) | Ini | 81 | 0 | 19 | 100 |
| [pushbullet_notification.py](/pushbullet_notification.py) | Python | 31 | 8 | 6 | 45 |
| [regex_parsing.py](/regex_parsing.py) | Python | 71 | 2 | 11 | 84 |
| [requirements.txt](/requirements.txt) | pip requirements | 26 | 0 | 1 | 27 |
| [url_ingester.py](/url_ingester.py) | Python | 55 | 6 | 13 | 74 |
| [url_worker.py](/url_worker.py) | Python | 173 | 38 | 41 | 252 |

[Summary](results.md) / Details / [Diff Summary](diff.md) / [Diff Details](diff-details.md)
15 changes: 15 additions & 0 deletions .VSCodeCounter/2024-05-20_23-45-31/diff-details.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Diff Details

Date : 2024-05-20 23:45:31

Directory c:\\Users\\Joshua\\Documents\\GitHub\\Hub\\Automated-FFDL

Total : 0 files, 0 codes, 0 comments, 0 blanks, all 0 lines

[Summary](results.md) / [Details](details.md) / [Diff Summary](diff.md) / Diff Details

## Files
| filename | language | code | comment | blank | total |
| :--- | :--- | ---: | ---: | ---: | ---: |

[Summary](results.md) / [Details](details.md) / [Diff Summary](diff.md) / Diff Details
2 changes: 2 additions & 0 deletions .VSCodeCounter/2024-05-20_23-45-31/diff.csv
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
"filename", "language", "", "comment", "blank", "total"
"Total", "-", , 0, 0, 0
19 changes: 19 additions & 0 deletions .VSCodeCounter/2024-05-20_23-45-31/diff.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Diff Summary

Date : 2024-05-20 23:45:31

Directory c:\\Users\\Joshua\\Documents\\GitHub\\Hub\\Automated-FFDL

Total : 0 files, 0 codes, 0 comments, 0 blanks, all 0 lines

[Summary](results.md) / [Details](details.md) / Diff Summary / [Diff Details](diff-details.md)

## Languages
| language | files | code | comment | blank | total |
| :--- | ---: | ---: | ---: | ---: | ---: |

## Directories
| path | files | code | comment | blank | total |
| :--- | ---: | ---: | ---: | ---: | ---: |

[Summary](results.md) / [Details](details.md) / Diff Summary / [Diff Details](diff-details.md)
22 changes: 22 additions & 0 deletions .VSCodeCounter/2024-05-20_23-45-31/diff.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
Date : 2024-05-20 23:45:31
Directory : c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL
Total : 0 files, 0 codes, 0 comments, 0 blanks, all 0 lines

Languages
+----------+------------+------------+------------+------------+------------+
| language | files | code | comment | blank | total |
+----------+------------+------------+------------+------------+------------+
+----------+------------+------------+------------+------------+------------+

Directories
+------+------------+------------+------------+------------+------------+
| path | files | code | comment | blank | total |
+------+------------+------------+------------+------------+------------+
+------+------------+------------+------------+------------+------------+

Files
+----------+----------+------------+------------+------------+------------+
| filename | language | code | comment | blank | total |
+----------+----------+------------+------------+------------+------------+
| Total | | 0 | 0 | 0 | 0 |
+----------+----------+------------+------------+------------+------------+
15 changes: 15 additions & 0 deletions .VSCodeCounter/2024-05-20_23-45-31/results.csv
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
"filename", "language", "Ini", "Python", "TOML", "pip requirements", "comment", "blank", "total"
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\calibre_info.py", "Python", 0, 47, 0, 0, 13, 13, 73
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\config.toml", "TOML", 0, 0, 16, 0, 0, 2, 18
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\defaults.ini", "Ini", 3177, 0, 0, 0, 38, 1080, 4295
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\fanfic_info.py", "Python", 0, 46, 0, 0, 7, 5, 58
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\fanficdownload.py", "Python", 0, 18, 0, 0, 6, 13, 37
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\ff_logging.py", "Python", 0, 25, 0, 0, 1, 4, 30
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\ff_logging_test.py", "Python", 0, 27, 0, 0, 0, 5, 32
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\personal.ini", "Ini", 81, 0, 0, 0, 0, 19, 100
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\pushbullet_notification.py", "Python", 0, 31, 0, 0, 8, 6, 45
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\regex_parsing.py", "Python", 0, 71, 0, 0, 2, 11, 84
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\requirements.txt", "pip requirements", 0, 0, 0, 26, 0, 1, 27
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\url_ingester.py", "Python", 0, 55, 0, 0, 6, 13, 74
"c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\url_worker.py", "Python", 0, 173, 0, 0, 38, 41, 252
"Total", "-", 3258, 493, 16, 26, 119, 1213, 5125
1 change: 1 addition & 0 deletions .VSCodeCounter/2024-05-20_23-45-31/results.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/personal.ini":{"language":"Ini","code":81,"comment":0,"blank":19},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/ff_logging.py":{"language":"Python","code":25,"comment":1,"blank":4},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/ff_logging_test.py":{"language":"Python","code":27,"comment":0,"blank":5},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/fanfic_info.py":{"language":"Python","code":46,"comment":7,"blank":5},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/pushbullet_notification.py":{"language":"Python","code":31,"comment":8,"blank":6},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/fanficdownload.py":{"language":"Python","code":18,"comment":6,"blank":13},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/config.toml":{"language":"TOML","code":16,"comment":0,"blank":2},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/requirements.txt":{"language":"pip requirements","code":26,"comment":0,"blank":1},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/url_ingester.py":{"language":"Python","code":55,"comment":6,"blank":13},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/defaults.ini":{"language":"Ini","code":3177,"comment":38,"blank":1080},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/url_worker.py":{"language":"Python","code":173,"comment":38,"blank":41},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/calibre_info.py":{"language":"Python","code":47,"comment":13,"blank":13},"file:///c%3A/Users/Joshua/Documents/GitHub/Hub/Automated-FFDL/regex_parsing.py":{"language":"Python","code":71,"comment":2,"blank":11}}
24 changes: 24 additions & 0 deletions .VSCodeCounter/2024-05-20_23-45-31/results.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Summary

Date : 2024-05-20 23:45:31

Directory c:\\Users\\Joshua\\Documents\\GitHub\\Hub\\Automated-FFDL

Total : 13 files, 3793 codes, 119 comments, 1213 blanks, all 5125 lines

Summary / [Details](details.md) / [Diff Summary](diff.md) / [Diff Details](diff-details.md)

## Languages
| language | files | code | comment | blank | total |
| :--- | ---: | ---: | ---: | ---: | ---: |
| Ini | 2 | 3,258 | 38 | 1,099 | 4,395 |
| Python | 9 | 493 | 81 | 111 | 685 |
| pip requirements | 1 | 26 | 0 | 1 | 27 |
| TOML | 1 | 16 | 0 | 2 | 18 |

## Directories
| path | files | code | comment | blank | total |
| :--- | ---: | ---: | ---: | ---: | ---: |
| . | 13 | 3,793 | 119 | 1,213 | 5,125 |

Summary / [Details](details.md) / [Diff Summary](diff.md) / [Diff Details](diff-details.md)
40 changes: 40 additions & 0 deletions .VSCodeCounter/2024-05-20_23-45-31/results.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
Date : 2024-05-20 23:45:31
Directory : c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL
Total : 13 files, 3793 codes, 119 comments, 1213 blanks, all 5125 lines

Languages
+------------------+------------+------------+------------+------------+------------+
| language | files | code | comment | blank | total |
+------------------+------------+------------+------------+------------+------------+
| Ini | 2 | 3,258 | 38 | 1,099 | 4,395 |
| Python | 9 | 493 | 81 | 111 | 685 |
| pip requirements | 1 | 26 | 0 | 1 | 27 |
| TOML | 1 | 16 | 0 | 2 | 18 |
+------------------+------------+------------+------------+------------+------------+

Directories
+--------------------------------------------------------------------------------+------------+------------+------------+------------+------------+
| path | files | code | comment | blank | total |
+--------------------------------------------------------------------------------+------------+------------+------------+------------+------------+
| . | 13 | 3,793 | 119 | 1,213 | 5,125 |
+--------------------------------------------------------------------------------+------------+------------+------------+------------+------------+

Files
+--------------------------------------------------------------------------------+------------------+------------+------------+------------+------------+
| filename | language | code | comment | blank | total |
+--------------------------------------------------------------------------------+------------------+------------+------------+------------+------------+
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\calibre_info.py | Python | 47 | 13 | 13 | 73 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\config.toml | TOML | 16 | 0 | 2 | 18 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\defaults.ini | Ini | 3,177 | 38 | 1,080 | 4,295 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\fanfic_info.py | Python | 46 | 7 | 5 | 58 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\fanficdownload.py | Python | 18 | 6 | 13 | 37 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\ff_logging.py | Python | 25 | 1 | 4 | 30 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\ff_logging_test.py | Python | 27 | 0 | 5 | 32 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\personal.ini | Ini | 81 | 0 | 19 | 100 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\pushbullet_notification.py | Python | 31 | 8 | 6 | 45 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\regex_parsing.py | Python | 71 | 2 | 11 | 84 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\requirements.txt | pip requirements | 26 | 0 | 1 | 27 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\url_ingester.py | Python | 55 | 6 | 13 | 74 |
| c:\Users\Joshua\Documents\GitHub\Hub\Automated-FFDL\url_worker.py | Python | 173 | 38 | 41 | 252 |
| Total | | 3,793 | 119 | 1,213 | 5,125 |
+--------------------------------------------------------------------------------+------------------+------------+------------+------------+------------+
5 changes: 4 additions & 1 deletion .github/workflows/docker-image.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,15 @@ jobs:
echo "CALIBRE_VERSION=$CALIBRE_VERSION" >> $GITHUB_ENV
FFF_VERSION=$(cat release-versions/fff.txt)
echo "FFF_VERSION=$FFF_VERSION" >> $GITHUB_ENV
S6_OVERLAY_VERSION=$(cat release-versions/s6.txt)
echo "S6_OVERLAY_VERSION=$S6_OVERLAY_VERSION" >> $GITHUB_ENV
- name: Print image tag
run: |
echo "Branch: $CI_ACTION_REF_NAME"
echo "Release Version: ${{ env.RELEASE_VERSION }}"
echo "Calibre Version: ${{ env.CALIBRE_VERSION }}"
echo "FFF Version: ${{ env.FFF_VERSION }}"
echo "S6 Overlay Version: ${{ env.S6_OVERLAY_VERSION }}"
- name: Login to DockerHub
uses: docker/[email protected]
with:
Expand All @@ -60,3 +62,4 @@ jobs:
VERSION=${{ env.RELEASE_VERSION }}
CALIBRE_RELEASE=${{ env.CALIBRE_VERSION }}
FFF_RELEASE=${{ env.FFF_VERSION }}
S6_OVERLAY_VERSION=${{ env.S6_OVERLAY_VERSION }}
22 changes: 15 additions & 7 deletions Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ FROM python:3-slim
ARG VERSION
ARG CALIBRE_RELEASE
ARG FFF_RELEASE
LABEL build_version="FFDL-Auto version:- ${VERSION} Calibre: ${CALIBRE_RELEASE} FFF: ${FFF_RELEASE}"
ARG S6_OVERLAY_VERSION
LABEL build_version="FFDL-Auto version:- ${VERSION} Calibre: ${CALIBRE_RELEASE} FFF: ${FFF_RELEASE} S6_OVERLAY_VERSION: ${S6_OVERLAY_VERSION}"

ENV PUID="911" \
PGID="911"
Expand Down Expand Up @@ -36,6 +37,15 @@ RUN echo "**** install calibre ****" && \
apt-get install -y calibre && \
dbus-uuidgen > /etc/machine-id

ADD https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz /tmp
RUN tar -C / -Jxpf /tmp/s6-overlay-noarch.tar.xz
ADD https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-x86_64.tar.xz /tmp
RUN tar -C / -Jxpf /tmp/s6-overlay-x86_64.tar.xz

RUN echo "*** Install Other Python Packages ***" && \
COPY requirements.txt /tmp/
RUN python3 -m pip install --no-cache-dir -r /tmp/requirements.txt

RUN echo "*** Install FFF ***" && \
if [ -z ${FFF_RELEASE} ]; then \
echo "FFF Using Default Release"; \
Expand All @@ -46,9 +56,6 @@ RUN echo "*** Install FFF ***" && \
fi


RUN echo "*** Install Other Python Packages ***" && \
python3 -m pip --no-cache-dir install pushbullet.py pillow

RUN echo "**** cleanup ****" && \
rm -rf \
/tmp/* \
Expand All @@ -57,10 +64,11 @@ RUN echo "**** cleanup ****" && \

COPY root/ /

RUN chmod -R +777 /etc/cont-init.d/
RUN chmod -R +777 /etc/services.d/

VOLUME /config

WORKDIR /config

RUN groupmod -o -g "$PGID" abc && usermod -o -u "$PUID" abc && chown -R abc:abc /app && chown -R abc:abc /config && chown -R abc:abc /root && chmod +x /app/run.sh

CMD ["sh", "/app/run.sh"]
ENTRYPOINT ["/init"]
93 changes: 92 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,101 @@ Automated Fanfiction Download using FanficFare CLI

This is a docker image to run the Automated FFF CLI, with pushbullet integration.

[FanFicFare](https://github.com/JimmXinu/FanFicFare)

[Dockerhub Link](https://hub.docker.com/r/mrtyton/automated-ffdl)

## How to Install
- [AutomatedFanfic](#automatedfanfic)
- [Calibre Setup](#calibre-setup)
- [Execution](#execution)
- [How to Install - Docker](#how-to-install---docker)
- [How to Run - Non-Docker](#how-to-run---non-docker)
- [Configuration](#configuration)
- [Email](#email)
- [Calibre](#calibre)
- [Pushbullet](#pushbullet)


## Calibre Setup

1. Setup the Calibre Content Server. Instructions can be found on the [calibre website](https://manual.calibre-ebook.com/server.html)
2. Make note of the IP address, Port and Library for the server. If needed, also make note of the Username and Password.

## Execution

### How to Install - Docker

1. Install the docker image with `docker pull mrtyton/automated-ffdl`
2. Map the `/config` volume to someplace on your drive.
3. After running the image once, it will have copied over default configs. Fill them out and everything should start working.

### How to Run - Non-Docker

1. Make sure that you have calibre, and more importantly [calibredb](https://manual.calibre-ebook.com/generated/en/calibredb.html) installed on the system that you're running the script on. `calibredb` should be installed standard when you install calibre.
2. Install [Python3](https://www.python.org/downloads/)
3. Clone the Repo
4. Run `python -m pip install -r requirements.txt`
5. Install [FanficFare](https://github.com/JimmXinu/FanFicFare/wiki#command-line-interface-cli-version)
6. Fill out the config.toml file
7. Navigate to `root/app` and run `python fanficdownload.py`

## Configuration

The config file is a [TOML](https://toml.io/en/) file that contains the script's specific options. Changes to this file will only take effect upon script startup.


### Email

In order for the script to work, you have to fill out the email login information.

```toml
[email]
email = ""
password = ""
server = ""
mailbox = ""
sleep_time = 60
```


- `email`: The email address, username only.
- `password`: The password to the email address. It is recommened that you use an app password (Google's page on [App Password](https://support.google.com/accounts/answer/185833?hl=en)), rather than your email's actual password.
- `server`: Address for the email server. For Gmail, this is going to be `imap.gmail.com`. For other web services, you'll have to search for them.
- `mailbox`: Which mailbox to check, such as `INBOX`, for the unread update emails.
- `sleep_time`: How often to check the email account for new updates, in seconds. Default is 60 seconds, but you can make this as often as you want. Recommended that you don't go too fast though, since some email providers will not be happy.

### Calibre

The Calibre information for access and updating.

```toml
[calibre]
path=""
username=""
password=""
default_ini=""
personal_ini=""
```

- `path`: This is the path to your Calibre database. It's the location where your Calibre library is stored on your system. This can be either a directory that contains the `calibre.db` file, or the URL/Port/Library marked down above, such as `https://192.168.1.1:9001/#Fanfiction` This is the only argument that is **required** in this section.
- `username`: If your Calibre database is password protected, this is the username you use to access it.
- `password`: If your Calibre database is password protected, this is the password you use to access it.
- `default_ini`: This is the path to the [default INI configuration file](https://github.com/JimmXinu/FanFicFare/blob/main/fanficfare/defaults.ini) for FanFicFare.
- `personal_ini`: This is the path to your [personal INI configuration file](https://github.com/JimmXinu/FanFicFare/wiki/INI-File) for FanFicFare.

For both the default and personal INI, any changes made to them will take effect during the next update check, it does not require a restart of the script.

### Pushbullet

This script has an _optional_ [Pushbullet](https://pushbullet.com) integration, in case you want to get phone notifications when an update has occurred. The system will also send a notification if it fails to update a story, for whatever reason.

```toml
[pushbullet]
enabled = false
api_key = ""
device = ""
```

- `enabled`: Whether or not to enable the pushbullet notifications
- `api_key`: Your [[Pushbullet API Key](https://docs.pushbullet.com/#authentication)]
- `device`: If you want to send the notification to a specific device rather than the entirety of the pushbullet subscriptions, you can specify which device here with the device name.
Loading

0 comments on commit 1ba1f0f

Please sign in to comment.