EPUB3/HTML support #9

bertfrees · 2016-03-15T14:42:50Z

If SBS also intends to use EPUB3 as input format, pipeline-mod-sbs should also have an epub3-to-pef script and corresponding HTML translator.

mixa72 · 2016-03-15T14:56:04Z

Yes, it's quite likely that we will also use EPUB3 as input format in the long run.

bertfrees · 2016-07-26T18:41:34Z

bertfrees · 2016-10-10T14:34:25Z

There are a number of SBS-specific extensions to DTBook for which we need to find an alternative in EPUB 3. I will create a table here with the mapping and we can discuss the problematic ones.

[table moved to wiki page]

bertfrees · 2016-10-10T16:07:14Z

@egli Can I get that pointer to the Nordic EPUB specification?

bertfrees · 2016-10-18T10:53:39Z

Nordic markup guidelines for EPUB 3: http://nlbdev.github.io/nordic-epub3-dtbook-migrator/resources/2014-31_-_EPUB_3_Requirements_2015-1_rev-01.pdf
Nordic markup guidelines for DTBook: http://nlbdev.github.io/nordic-epub3-dtbook-migrator/resources/dtbook_markup_requirements_2011-2_v_1.docx

bertfrees · 2016-10-26T17:26:50Z

@egli The progress can be seen on branch sbs-9. I'm waiting with merging it because I still can't run all the tests at once even after increasing the memory.

bertfrees · 2016-10-26T17:29:09Z

If at some point we want to let Mischa try it, and I haven't found a real solution to the problem yet, we could merge it but with the new tests disabled.

egli · 2016-10-31T09:30:25Z

We do have an EPUB test document that was produced in India

bertfrees · 2016-11-01T10:25:36Z

@egli and @mixa72 Please have a look at my table above, I've updated it. Maybe you have some more ideas.

mixa72 · 2016-11-03T09:36:11Z

That looks pretty good. It is interesting how many standards there are for the different purposes. Compared to our current DTBook, EPUB3 will apparently involve a lot more namespaces and the terminology will be a varied mix. So whatever you decide for the elements brl:select, brl:running-line, brl:toc-line, brl:time is fine by me since it's not possible to find a uniform naming anyway.
BTW: AFAIK @brl:class is indeed only used for SBSForm.

bertfrees · 2016-11-03T12:20:59Z

Coherentness is indeed something we need to carefully think about. You need to work with this every day so your opinion is important. At the same using standards is also important, and last but not least, compatibility with the Nordic guidelines. Changing the Nordic guidelines is possible but apparently a slow process.

The Nordic guidelines have apparently chosen to use "class" for some semantics instead of a custom "epub:type" prefixed with "nordic:". I'm not sure what the motives were. However they do use epub:types that are available in either the default or the z3998 vocabulary. Moreover, they do have a "nordic:" prefix but they only use it for some of the metadata, not for epub:types.

Nordic's use of class is not always appropriate in my opinion, but I think we have to live with this. It's also hard to avoid the mix of different attributes and prefixes because this is just how EPUB works, and because of the compatibility requirement with Nordic. What we could do to simplify things a bit is to not use our own "sbs:" prefix and use classes instead. This is semantically not optimal, but at least it creates some coherentness with the Nordic guidelines. In addition, we can try to completely eliminate "brl:" elements and attributes.

egli · 2016-11-03T15:09:17Z

I would not take the Nordic guidelines as the be-all-end-all truth. While they are useful and most likely will define the shape of the EPUB we will get from our providers I would also be forward looking and improve things where you think it makes sense.

bertfrees · 2016-11-03T15:11:36Z

We could of course have a converter from "Nordic EPUB 3" to "SBS EPUB 3". But this makes interchanging files a bit difficult unless we have the conversion in the two directions.

mixa72 · 2016-11-03T15:41:34Z

Doing the markup with Oxygen is very user-friendly. DTBooks can be validated against both our inhouse minimal schema and the classic DTD. The most important feature is that the editor displays a list with all the possible elements at any place in the document (auto completion). If Oxygen also behaves like that with EPUB3 files then I don't see any problems for the users. It will take some time to learn and memorize the new markup, that's obvious, but after a while everybody will get used to it.

egli · 2016-11-22T13:12:25Z

I talked to @mixa72 about this yesterday and the consensus seems to be that the actual names of the elements that we will use in the EPUB are not so important to the transcribers, as long as oXygen does the auto completion.

bertfrees · 2016-11-22T13:20:46Z

Yes that's what Mischa said last time. But still we should think it through. What about the things where I have put question marks?

mixa72 · 2016-11-22T14:34:55Z

By me it's ok if you use the following for EPUB3:
brl:class --> @Class (no prefix)

brl:select --> brl:select (or solution with span)
brl:when-braille --> brl:when-braille (or solution with span)
brl:literal[@brl:grade=...] --> brl:literal[@brl:grade=...] (or solution with span)
brl:otherwise --> brl:otherwise (or solution with span)

brl:running-line --> brl:running-line
brl:toc-line --> brl:toc-line
brl:volume[@brl:grade=...] --> br[@Class='braille-volume-break-grade-...']

brl:time --> brl:time (if we keep brl:date; if we use sbs:date instead, I'd also prefer sbs:time)

But I'm open to accept anything as long as there is no loss in functionality with respect to the actual system.

bertfrees · 2016-11-29T15:26:21Z

Okay.

bertfrees · 2016-12-06T11:45:24Z

An importance remark that was made in our call today is that what we use as authoring format does not need to be standards compliant, as long as what we distribute or exchange with Nordic countries is standards compliant. So it is no problem if the authoring format has really SBS-specific things such as brl:select in it as long as we remove it when distributing/exchanging. The same can be said about the whole markup. In theory we could have two completely separate types of EPUB. One with all the brl:* that we are used to in DTBook, and one that is standards compliant, and conversion scripts to go from one format to the other and back.

bertfrees · 2017-03-31T08:38:05Z

Test suite works again (#52 (comment)).

bertfrees · 2017-05-05T16:11:06Z

All the existing unit tests pass now. I'm going to merge the sbs-9 branch even though some things might not work yet, and even though the exact EPUB 3 format (see wiki page) hasn't been decided yet.

We can move the issue back to "Backlog" if Mischa finds issues, or if we want to make changes to the EPUB 3 format.

mixa72 · 2018-05-14T15:32:15Z

I found some issues in the EPUB3 output. I first created a file as DTB and an identical one as EPUB. Here are the differences I found. Possibly my markup is wrong, please take a look at it.
test_epub3_html.zip

 Output from DTB                   Output from EPUB


                               |
         *H:TSV7Z34X           |           *H:TSV7Z34X
         -----------           |           -----------
                               |
           7]7 B+D             |             7]7 B+D
                               |
 TO'CL*E ................ #*A  |   TO'CL*E ................ #*A
                               |
          ZW3T7 B+D            |            ZW3T7 B+D
                               |
 TO'CL*E ............... #,,C  |   TO'CL*E ............... #,,C
                               |
         ::::::::::::          |           ::::::::::::
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
         *H:TSV7Z34X       >I  |           *H:TSV7Z34X       >I
p                              |  p
p                              |  p
       HEAD*G VOLUME #A        |         HEAD*G VOLUME #A
       ----------------        |         ----------------
                               |
 SPAN-+SW7 ---                 |   SPAN-+SW7 ---
 SPAN-+SW7'#A -                |   SPAN-+SW7'#A -
 SPAN-BO'X ---                 |   SPAN-BO'X ---
 LI-BRL-'CLA^                  |   LI-BRL-'CLA^                             <-- brl:class not working in EPUB (css was specified, but has no effect)
 'A-PA&REF #A                  |   'A-PA&REF #A
 BRL-HOMOGRAPH W<]UBE          |   BRL-HOMOGRAPH W<]UBE
 BRL-'V-F?M $S                 |   BRL-'V-F?M S                             <-- brl:v-form not working in EPUB
 BRL-NUM                       |   BRL-NUM                                  <-- brl:num not working in EPUB
   'C)D*AL #E                  |   'C)D*AL #E
   ?D*AL #?                    |   ?D*AL #E.                                <-- brl:num not working in EPUB
   ROMAN >II.                  |   ROMAN II.                                <-- brl:num not working in EPUB
   PHONE #JDC.CCC.CB.CB        |   PHONE #JDC !, #CCC #CB #CB               <-- brl:num not working in EPUB
   ISBN #IGH.C.DIB.BDJGB.G     |   ISBN #IGH-#C-#DIB-#BDJGB-#G              <-- brl:num not working in EPUB
   MEASURE #D'DL               |   MEASURE #D DL                            <-- brl:num not working in EPUB
   FRA'CTJ #C/                 |   FRA'CTJ #C!,#D                           <-- brl:num not working in EPUB
   MI'XED #H#A;                |   MI'XED #H #A!,#B                         <-- brl:num not working in EPUB
 BRL-PLA'CE M+NH3M             |   BRL-PLA'CE M+NH3M
 BRL-SYE'CT KZ                 |   BRL-SYE'CT BASIS Q KZ                    <-- brl:select not working in EPUB
 BRL-[PH                       |   BRL-[PH
   _[PH                        |   [PH                                      <-- brl:emph not working in EPUB
   '(S*GLE'QUO(')              |   S*GLE'QUO(                               <-- brl:emph not working in EPUB
   ('QUO()                     |   'QUO(                                    <-- brl:emph not working in EPUB
   IGN?E                       |   IGN?E
 _#I,)       R/N*GL*E      #A  |                                            <-- brl:running-line not working in EPUB (1 was selected)
p                              |  p
p                              |  p
         *H:TSV7Z34X           |           *H:TSV7Z34X
         -----------           |           -----------
                               |
          ZW3T7 B+D            |            ZW3T7 B+D
                               |
 TO'CL*E ............... #,,C  |   TO'CL*E ............... #,,C
                               |
         ::::::::::::          |           ::::::::::::
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
         *H:TSV7Z34X       >I  |           *H:TSV7Z34X       >I
p                              |  p
p                              |  p
       HEAD*G VOLUME #B        |         HEAD*G VOLUME #B
       ----------------        |         ----------------
                               |
 BRL-A'C'CCTS-SPAN R"EDUIT     |   BRL-A'C'CCTS-SPAN R"EDUIT
   D"%TAIQ"%                   |   D"%TAIQ"%
 BRL-'COMPUT7 '$WWW.SBS.CH     |   BRL-'COMPUT7 WWW.SBS.'4                  <-- brl:computer not working in EPUB
 BRL-DA( #,=AJ#BJJD            |   BRL-DA( #AG.AJ.BJJD                      <-- brl:date not working in EPUB
 BRL-TIME #E.AE                |   BRL-TIME #JE":#JE                        <-- brl:time not working in EPUB
 BRL-NAME K1FM+N               |   BRL-NAME K1FMN                           <-- brl:name not working in EPUB
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
                               |
 _#AJ,;      R/N*GL*E      #C  |
p                              |  p

bertfrees · 2018-05-14T15:44:56Z

OK thanks for the heads up!

bertfrees · 2018-07-09T13:57:48Z

@mixa72 What is supposed to happen with brl:class? As far as I remember this had something to do with macro's in dtbook2sbsform, which I guess would translate to CSS in the new system. If you want to select an element with a brl:class in CSS you should do it like this:

@namespace brl url(http://www.daisy.org/z3986/2009/braille/);
brl|class~='myclass' {
   ...
}

bertfrees · 2018-07-09T14:05:03Z

OK I see what you are trying to do. You put this in the EPUB:

<style>
@namespace xml "http://www.w3.org/XML/1998/namespace";
@namespace brl url(http://www.daisy.org/z3986/2009/braille/);

li[brl|class='myclass'] {
   margin-left:2;
}
   </style>

The problem is that this CSS is not enabled unless you specify the "apply-document-specific-stylesheets" option (why is currently not available in the SBS version of the script).

mixa72 · 2018-07-09T14:53:49Z

@bertfrees Thanks for the hint with the syntax. However, it appears that any css instruction in the style Element is ignored by the system. I even tried

      @namespace xml "http://www.w3.org/XML/1998/namespace";
      @namespace brl url(http://www.daisy.org/z3986/2009/braille/);
      li{
        margin-top:2 !important;
      }

but nothing changes. Is that possible?

bertfrees · 2018-07-09T15:04:43Z

Well, there are two problems. Firstly, like I said above you need the "apply-document-specific-stylesheets". (I will add it.) Secondly, you need to add type="text/css" to the style to make it work. (Preferably also add media="embossed" to make the style not influence the rendering on screen).

mixa72 · 2018-07-09T15:04:48Z

I seem to understand it now: as "apply-document-specific-stylesheets" is disabled now, I'll have to test the brl:class attribute via external stylesheet (scss), right?

bertfrees · 2018-07-09T15:05:38Z

Indeed.

bertfrees · 2018-07-09T15:08:26Z

However I think there is another issue, which might also explain why the elements like brl:v-form, brl:num etc. don't work. I'm investigating it now.

bertfrees · 2018-07-09T15:18:33Z

Never mind, forget that last comment.

bertfrees · 2018-07-09T15:30:29Z

OK so I've added the "apply-document-specific-stylesheets" option and that solves the brl:class issue.

All the other issues are because brl:* elements are not valid in HTML and as a result the prefixes are removed in the load step. (brl:* attributes are also invalid but here the prefixes are retained). A solution is to make the translator and the style sheets work regardless of whether the "brl:" prefix is present. But better is of course to create valid HTML, for example by using epub:type or class attributes.

Another issue I found in your EPUB is that it uses <list type="pl">. In EPUB use <ul style="list-style-type: none"> instead. NLB has a "list-style-type-none" class for it:

.list-style-type-none {
    list-style-type: none;
}

mixa72 · 2018-07-09T20:35:41Z

OK. I'll adjust my EPUB accordingly. Thanks!

mixa72 · 2018-07-10T07:14:45Z

BTW is the apply-document-specific-stylesheets option also visible in the GUI or just available in the background?

bertfrees · 2018-07-10T07:42:46Z

Yes it will be visible in the GUI.

bertfrees · 2018-07-19T13:37:25Z

Done.

I had to make some small adjustments to the EPUB in order to make it behave exactly as the DTBook: see chapter.xhtml.

mixa72 · 2018-07-20T08:56:29Z

EPUB3 to PEF Conversion works now. All the above mentioned inline elements are translated as in the DTB to PEF Conversion. CSS Support for stylesheets inside EPUB3 also works. Thanks.

The embedded braille rendition from the EPUB3 to EPUB3 conversion differs a bit from the output in the PEF in that some inline elements are not translated accordingly:
brl:num (ordinal, phone, isbn, measure, fraction, mixed)
em (strong) (brl:emph)
brl:date
brl:time
brl:name
The brl:select element should only render the braille in the corresponding grade (not each literal element).
The rest looks good.

bertfrees · 2018-07-20T09:14:26Z

Yes, I haven't applied the fix to the epub3-to-epub3 script yet. We'll track that in issue #58.

mixa72 · 2018-07-20T09:48:00Z

OK. Then I'll close this issue now.

bertfrees added the prio:2 label Jul 26, 2016

bertfrees added this to the 2016/Q4 milestone Jul 26, 2016

bertfrees added the L label Jul 26, 2016

bertfrees added the 1 - Next label Sep 7, 2016

bertfrees self-assigned this Sep 7, 2016

bertfrees added 2 - In progress and removed 1 - Next labels Oct 6, 2016

bertfrees added 4 - Done 2 - In progress and removed 2 - In progress 4 - Done labels Oct 26, 2016

bertfrees assigned egli Oct 31, 2016

bertfrees mentioned this issue Nov 18, 2016

Sbs titlepage tests sbsdev/pipeline#1

Closed

bertfrees mentioned this issue Jan 20, 2017

Script for e-braille production (EPUB 3 with braille rendition) #58

Closed

bertfrees added 4 - Done and removed 2 - In progress labels May 5, 2017

bertfrees added 1 - Next 4 - Done and removed 4 - Done 1 - Next labels Jul 16, 2018

mixa72 closed this as completed Jul 20, 2018

EPUB3/HTML support #9

EPUB3/HTML support #9

Comments

bertfrees commented Mar 15, 2016 • edited Loading

mixa72 commented Mar 15, 2016

bertfrees commented Jul 26, 2016 • edited Loading

bertfrees commented Oct 10, 2016 • edited Loading

bertfrees commented Oct 10, 2016

bertfrees commented Oct 18, 2016

bertfrees commented Oct 26, 2016

bertfrees commented Oct 26, 2016

egli commented Oct 31, 2016

bertfrees commented Nov 1, 2016

mixa72 commented Nov 3, 2016

bertfrees commented Nov 3, 2016 • edited Loading

egli commented Nov 3, 2016

bertfrees commented Nov 3, 2016 • edited Loading

mixa72 commented Nov 3, 2016

egli commented Nov 22, 2016

bertfrees commented Nov 22, 2016

mixa72 commented Nov 22, 2016

bertfrees commented Nov 29, 2016

bertfrees commented Dec 6, 2016 • edited Loading

bertfrees commented Mar 31, 2017

bertfrees commented May 5, 2017 • edited Loading

mixa72 commented May 14, 2018 • edited Loading

bertfrees commented May 14, 2018

bertfrees commented Jul 9, 2018

bertfrees commented Jul 9, 2018

mixa72 commented Jul 9, 2018

bertfrees commented Jul 9, 2018

mixa72 commented Jul 9, 2018

bertfrees commented Jul 9, 2018

bertfrees commented Jul 9, 2018

bertfrees commented Jul 9, 2018

bertfrees commented Jul 9, 2018

mixa72 commented Jul 9, 2018

mixa72 commented Jul 10, 2018

bertfrees commented Jul 10, 2018

bertfrees commented Jul 19, 2018 • edited Loading

mixa72 commented Jul 20, 2018

bertfrees commented Jul 20, 2018

mixa72 commented Jul 20, 2018

bertfrees commented Mar 15, 2016 •

edited

Loading

bertfrees commented Jul 26, 2016 •

edited

Loading

bertfrees commented Oct 10, 2016 •

edited

Loading

bertfrees commented Nov 3, 2016 •

edited

Loading

bertfrees commented Nov 3, 2016 •

edited

Loading

bertfrees commented Dec 6, 2016 •

edited

Loading

bertfrees commented May 5, 2017 •

edited

Loading

mixa72 commented May 14, 2018 •

edited

Loading

bertfrees commented Jul 19, 2018 •

edited

Loading