Webgrab+

[dfigus]

Originally posted by @obert0 in #103 (comment)

Out of curiosity why did you remove WebGrabPlus from your addon?
Thank you!

Hi @obert0,

to answer your question from the #103 thread:

I removed Webgrab+ with #77 and I think it had multiple issues:

• It was installed during the initial startup - causing a long runtime
• It was only copying the SiteIniPack files for one country in a hardcoded fashion without a way to configure it
• The SiteIniPack was never updated
• It was installed in the TVHeadend Config folder which bloated the addon-on backups and also the TVHeadend config backup on config migrations

As I also never used it (I just use the internal DVB EPG grabber) , I removed it. If there is a need I would rather add it as part of the Dockerfile so it's backed into the docker image. Are you using it? How do you pull in the correct SiteIniPack for your country? If you help me to understand how you are using it, I'm happy to implement something 😄 .

[obert0]

Hi dfigus,
Actually I have setup Tvheadend, wg++ and comskip on a dedicated raspberry pi. I use wg++ because one of my OTA EPG dont work and I can't get recording from this station.

• I dont see issue with runtime because if the folder exist it skip the installation. My only concern is if you have a big guide it take a long time to tvheadend to process it.
• It is not really hardcoded because it is stored in config file so you can update it easily and it survive rebuild /update. It is also essential to have acces to that folder to edit wg+ config file.
• For a backup perspective I think you only need the active siteini in your config. So you can have only few siteini.

Thank you for your time with this addon!

[dfigus]

Hi @obert0,

• It is not really hardcoded because it is stored in config file so you can update it easily and it survive rebuild /update. It is also essential to have acces to that folder to edit wg+ config file.
• For a backup perspective I think you only need the active siteini in your config. So you can have only few siteini.

In the original addon the siteini is only copied from India, see code here:
https://github.com/GauthamVarmaK/addon-tvheadend/blob/011abf2aa5b42e9893a2cb085e1dceb7db85d6b1/tvheadend/rootfs/etc/s6-overlay/s6-rc.d/init-tvheadend/run#L28
cp siteini.pack/India/* siteini.user/
This is what I meant with "hardcoded". So did you use the siteini from India? Or did you manually copy your proper siteini files? And what about the config you mentioned? Is it similar to the config (ini file) of comskip?

Furthermore I could not spot wg+ in the list of EPG Grabber Modules on TVH, but maybe I overlooked it.

I would see the following a better solution:

• Install wg+ as part of the Dockerfile
• Pull siteini files based on a new addon config option
• Create a new addon config option for the wg+ config file

What else would you see could be improved?

[obert0]

Hi @dfigus,

Actually the whole currect siteini is download from WG++:
&& wget http://webgrabplus.com/sites/default/files/download/ini/SiteIniPack_current.zip \
I don't really understand why any files should be move to siteini.user folder. I look a little and I don't find the use of this folder. Yes the config file for wg++ is similar to the comskip ini file. It should be adjust to the specific channel you want to grab and must be edit by each user.
The guide here "http://www.webgrabplus.com/documentation/interact-your-pvr-software/tvheadend", is to make the link between TvHeadend and WG++.
This line:
&& wget -O /usr/bin/tv_grab_wg++ http://www.webgrabplus.com/sites/default/files/tv_grab_wg.txt \
add the file that TvHeadend use to map the guide of wg++. Then this line:
&& sed -i 's|~/.wg++/guide.xml|/config/tvheadend/wg++/guide.xml|g' /usr/bin/tv_grab_wg++ \
replace the path to point the guide in the "config/tvheanded/wg++" folder.
With those files in place TvHeandend should see the wg++ graber if a guide.xml is accessible.

• Install wg+ as part of the Dockerfile --> I agree!
• Pull siteini files based on a new addon config option --> This cleanup can be done by the user's. siteini are files that should be update sometime.
• Create a new addon config option for the wg+ config file --> This can be a really nice way to do. But the config can be complicated in some cases. See this link as reference: http://www.webgrabplus.com/documentation/configuration/webgrabconfigxml. I think a basic config with only simple channel line like this:
  <channel site="tv-gids.be" site_id="4" update="i" xmltv_id="MP_NatGeo">NATIONAL GEOGRAPHIC</channel>
  can be add in YAML. If more customisation is needed it must be done by editing the file after the installation.

I think this could be a really good start!

[obert0]

Hi @dfigus did you still plan to add WG++ on this addon? It can be a really great improvement.

[dfigus]

yes, i still plan it, but I did not have the time to do it so far.

[obert0]

I understand! Thank you!

[dfigus]

@obert0 I implemented a first version in the latest edge build (repo URL https://github.com/dfigus/hassio-addons-edge). Can you have a look at it? I didn't write any docu, yet, but that's on my todo. My first tests with local builds looked good, but as I don't need WG++ I don't really have a good test environment.

[obert0]

Ok I succed to log into TVH and load my config manyally. I should wait tomorrow to generate the guide du to new hardware detection... I keep you in touch, but it seam to work. The wg++ config is correctly written inside the folder.

[obert0]

Hi @dfigus, I finish the test and it work as expect. I succed to manually run the guide with the config I enter in the addon config. I link the guide to the channel and everything show up as expect.
The only thing I didn't test is the automatic guide regen by the cron file.

[dfigus]

@obert0 Thanks for testing this 👏 . Then let's wait for one more day to check if the cron job kicks in. You should also see log entries from the WG++ job. In my test setup I tested this with the job scheduled every 5 min, but would be good if you can double check this as well before releasing.

[obert0]

Good idea, I delete the guide and rerun it for a single day. I change it back for a week and I will let it run.

[obert0]

Hi @dfigus, The cron job lauch the command as expected.
I face a issue because the .wg++ is rebuilt at each addon startup, but I think it's a problem from my EPG source. When I run the guide a folder robots is created with files from each site that is grabed. Sometime some line in my robots is added with disallow that make the grab fail. This forum talk about this issue: http://www.webgrabplus.com/comment/21586.
This issue is one reason why I use a named pipe to manually remove the disallow line. If theres no robots from my epg provider the robot file is created with the dissalow line and the first time I run wg++ it will fail.
As I sais I think it's not a common problem and I can deal with it. Everything else work perfectly!

[obert0]

Hey I have a idea to update a HA sensor when a new recording end. But I will need the homeassistant_api enable in the config. Can you enable that or it has some repercution?
Thank you!

[dfigus]

I added the HA API with with the latest v7.0.0b1 release. Can you also have a look at the DOCs? I included a bit of WG++ documentation. If you have further suggestions, please let me know or create a PR 😀 .

[obert0]

Thank you very much! The documentation is perfect.
Thank you for your good work this is a solid add-on!

[dfigus]

@obert0 I released v7 including WG++ 🎉