Home

Implementation Guidelines

The following instructions are meant to assist in implementing other course packs into the new in-house authoring and publishing process of the Department of Methodology at the LSE. The instructions here are based on my experience of implementing the course pack MY451. Depending on the contents of whatever book or course pack you want to implement, this guide may exceed your needs or be incomplete. Implementation into this process requires heavy one-time editing. For instructions on how to author new or the existing MY451 course pack, see the readme.md in this project's folder.

The steps of implementing an existing course pack written in Latex are:

0. Create a GitHub account.
1. Have the admin of the course packs (e.g. Ken Benoit) make a duplicate of the already implemented MY451 course pack and add you as a contributor to that new project repository.
2. Download or "clone" a copy of that new repository onto your computer.
3. Install Pandoc and TexLive on your computer.
4. Prepare the existing Latex manuscript files of the course pack you are implementing for conversion into Markdown.
5. Convert the Latex manuscript files into Markdown.
6. Copy the newly produced Markdown manuscript files into your copy of the ready MY451 course pack, replacing the existing manuscript files. Make figures available in pdf, png.
7. Prepare the newly converted Markdown files for implementation.
8. Configure the settings files in the project folder.
9. Upload or "push" the project folder to the online repository.
10. Have the admin of that repository activate it for automatic processing by Travis CI.

---

0. Create a GitHub account

Go to https://github.com/ and look for the "Sign Up" button. Sign up.

1. Ask admin to duplicate existing course pack, get access

The prototype of the new in-house authoring and publishing process is course pack MY451 and stored online on GitHub. Ask its owner (presently Ken Benoit) (1) to duplicate that repository, (2) to rename it according to the course pack you want to implement, and (3) grant your GitHub account authoring access to that newly created repository. The administrator can find instructions on how to duplicate a repository here.

2. Download or "clone" a copy of that new repository onto your computer

Once the new project folder or "repository" for the course pack you want to implement is set up and you have authoring access to it, download or "clone" a working copy of it onto your computer by following these steps:

1. Sign in to GitHub and GitHub Desktop before you start to clone.
2. On GitHub, navigate to the main page of the repository.
3. Under your repository name, click Clone or download.
4. Click Open in Desktop to clone the repository and open it in GitHub Desktop.
5. In GitHub Desktop, after verifying the name and location on your hard drive where you'd like to clone the repository, click Clone.

You now have a working copy of the online project folder on your computer. The copy on your computer and the online repository at github.com are linked through the GitHub Desktop app. That means, when you make changes to the copy on your computer you can upload or "push" to the online repository. In your GitHub Desktop you should see something like this:

The very left sidebar shows your repositories. The black bar across the top shows a timeline of updates, so-called commits, to your project folder. The large section of the window on the bottom right shows changes to any file in the project once you have made them and not yet updated, or committed them. The section of the window between the left-hand sidebar and the large right-hand section lists those files that you have made changes to and not yet committed them.

Commits or updates are meant to enable version control of a project. Every updating of your project, every "commit", requires a short description of the changes you've made. It is best practice to commit regularly. Every unitary set of changes you make to the project should get their own commit and retraceable description.

3. Install Pandoc and TexLive on your computer

I assume the manuscript files of the existing course pack you are implementing are written in Latex and have the file extension .tex. This new authoring and publishing process requires the manuscript files to be in another file format, specifically Markdown. That means you have to convert the .tex files into .Rmd files. You can do so with the help of a program called Pandoc.

You can find instructions on how to install Pandoc here: http://pandoc.org/installing.html

Make sure to also install MikTex for Windows or MacTex for Mac as instructed on Pandoc's website.

The Pandoc version used for the MY451 course pack prototype was 1.17.2. You can choose to use the same version. Alternatively, you can use newer or the latest version of Pandoc. Doing so may or may not produce better conversion results as Pandoc is constantly developed further. That may also make some of the manuscript preparation instructions here obsolete and require you to figure out yourself which changes you will need to make to the manuscript files so that they display correctly in the output in the end.

4. Prepare the existing Latex manuscript files of the course pack you are implementing for conversion into Markdown

The following Latex code will not convert. Some of it may convert properly in future versions of Pandoc. Some describe functions not included in Markdown and will therefore never convert.

Before making changes to your original manuscript files, make sure you have backups of them!

Thus, remove the following Latex code, including its arguments inside the brackets, from your manuscript files before converting them into Markdown:

• \hspace{}, \hspace*{}
• \vspace{}, \vspace*{}
• \rule[]{}{}
• \underline{}
• [*ex] and similar
• \footnotesize{}
• \begin{small}, \end{small}

Excursion: use "regular expressions" or Regex to automate simple mass code changes

Regex is a way to program your text editors search and replace function to find code following patterns your prescribe. Not all text editors have that feature. Atom text editor does.

Regex has its own syntax. A resource for me has been this online emulator that allows you to learn and understand the Regex syntax: https://regex101.com/r/cH8vN2/5

Some general tips:

• Some symbols take on a function in the Regex syntax. To use them literally place a backslash in front of them.
• The syntax for include anything at this point in the search string is .*.
• If you want to take unknown parts of the string you are searching for to use them in the replacement, use (.*). In the replacement string you can call up the contents of (.*) with $1. If you use more than one of this catch-all expression, then $2, $3, $4 in the replacement string correspond to the position of (.*) in the search string.

So for example to remove instances of \underline{} from your .tex manuscript files, make sure that "Use Regex" is activated in the search field (Cmd+Shift+F) of Atom editor by clicking on the .* button on the right of the search field. Then enter the following in the search field of the Atom editor:

\\underline{.*}

Leave the replace field empty. Now click "Find" and "Replace" instances of code you want to remove.

Here's an example from below, where you'll want to keep an unknown part of the expression you are replacing:

\ref{} in the already converted Markdown file will need to become \@ref().

To do so with Regex, enter the following:

• search field: \\ref{(.*)}
• replace field: \@ref($1)

One caveat: if the same line in the manuscript includes other instances of symbols included in your search string, Regex may take and replace more code than you want it to. It's therefore a good idea to be careful with the "Replace all" function, especially across multiple manuscript files.

5. Convert the Latex manuscript files into Markdown

Assuming you have installed Pandoc and either MikTex or MacTex, open your command prompt (Windows) or terminal (Mac) and navigate your prompt to where your Latex manuscript files are (google "basic command line commands" "Windows" or "Mac" to find out how to use the "change directory" command).

Once you are in the folder where your Latex manuscript files are, execute the following command:

pandoc -s -S --atx-headers --parse-raw -t markdown INPUTFILE.tex -o OUTPUTFILE.Rmd

Do so with every single .tex file and have the .Rmd filenames begin with numbers according to their chapter position in your book. Except the very first book section. The very first manuscript file needs to be named "index.Rmd".

6. Copy the newly produced Markdown manuscript files into your new project folder, replacing the existing manuscript files

Replace the existing manuscript files of our project folder with the newly converted .Rmd files.

6b. Make figures available in pdf, png, copy in images/ folder

Image files for figures go in the images/ folder inside the project folder.

Make sure to make them available both as pdf and png with the otherwise same filename.

Assuming a figure you made is available in .eps format or .pdf, you can produce a pdf or png file using your operating system's standard picture viewer. On Mac OS X, for example, open an image file with the Preview app and click "File" > "Export..."" > then choose output format PNG > select path to the images/ folder inside the project folder > click "Save".

7. Prepare the newly converted Markdown files for implementation

In this section you will find indications of code you need to remove from or replace in your newly converted Markdown manuscript files. Some of that code may not occur in your manuscript files. Some code that needs modification in your project may not be listed here because the manuscript files of the MY451 prototype did not include it.

First and simple change: in index.Rmd, remove title page code before the first header and enter information in metadata block instead. Follow the pattern below. Remember to insert a blank line after the metadata block closes with ---:

---
title: "MY451 Introduction to Quantitative Analysis"
author:
    - "Jouni Kuha"
    - "Department of Methodology"
    - "London School of Economics and Political Science"
date: "2016-17"
site: bookdown::bookdown_site
documentclass: book
link-citations: yes
github-repo: kbenoit/coursepack-bookdown
description: "This course is intended for those with little or no past training in quantitative methods. The course is an intensive introduction to some of the principles and methods of statistical analysis in social research. Topics covered in MY451 include descriptive statistics, basic ideas of inference and estimation, contingency tables and an introduction to linear regression models."
---

# Course information

This course is intended for...

Code to remove including arguments

• remove \newpage unless practical
• remove \center{}

Code to modify while keeping arguments

• replace \emph{} with *
• replace \textbf{} with **
• replace .\ with .

Tables

• remove \multicolumn{}{}{}, rearrange table accordingly
• alignment: if a top-row cell is empty, a \ can be set to determine the text alignment of the cells in that column

Footnotes

Footnotes will be placed at the bottom of the manuscript text and numbered within each manuscript file. In order to guarantee the proper numbering and displaying of footnotes across chapter files they need to be made inline.

• copy the text of a footnote
• search for its link in the text that has the form [^1]
• copy the text inside the brackets and prepend the caret: end of sentence.^[Footnote text] Beginning of next sentence
• once you've done that will every footnote, remove the remaining code chunks from the bottom of the manuscript

Labels / identifiers / permalinks / anchors

Section headers

• section header labels {#label} remain

• for headers deeper than level 3, that is, beginning with 4-6 hashes, append {-} to suppress section numbering in all outputs

Tables

The caption of a table may include the table's label in html code inside <span> tags.

• the label name goes at the very beginning of the caption in this format: (\#tab:unique-label)
• remove the excessive html code at the end of the caption

Figures

The caption of a figure may include the figure's label in html code inside <span> tags.

• the label name goes at the very beginning of the caption in this format: (\#fig:unique-label)
• remove the excessive html code at the end of the caption

Equations

Equations remain written in Latex. The label command inside those chunks of Latex code can stay the same. But make sure to

• prepend eq: to label name so that

\label{unique-label}

becomes

\label{eq:unique-label}

Usually equations found inside markdown files are bracketed by $ for inline equations and $$ for separate ones. Equations that have labels, need to be automatically numbered, and need to be referenced:

• replace the opening and closing dollar sign(s) with \begin{equation} and \end{equation} respectively, and
• ending of numbered equation and beginning of new one (latex \end{equation}\begin{equation}) needs to be separated by a blank line

Paragraphs

Labelling paragraphs is not supported in this process. Their only use is to call them with a \pageref and pagerefs are no longer supported given that two of the three output formats of this process are un-paginated.

I recommend leaving them in place. They usually are just ignored in the compilation process. Plus, when you remove the pagerefs from your manuscript files you may still want to find the paragraphs they were referencing in order to provide refs to the nearest section header instead.

Referencing labels / identifiers / permalinks / anchors

• turn \ref{} into \@ref()

  • search \\ref\{(.*)\}, replace with \@ref($1) using Regex

• insert tab: in references to tables: \@ref(tab:label)

• insert fig: in references to figures: \@ref(fig:label)

• insert eq: in references to equations: \@ref(eq:label)

• replace \pageref

  • insert reference to closest section header instead à la at the beginning/middle/end of Section \@ref(label)

• replace references to paragraphs ((p-), (#p-)) with references to closest section header

Change names inside labels and references

This process allows only alphanumeric characters and the dash - in label names. Thus you may have to:

• turn underscores into dashes inside labels and references
  • search \\@ref\((.*)_(.*)\), replace with \@ref($1-$2), repeat until no search results
  • search \(#tab:(.*)_(.*)\), replace with (#tab:$1-$2), repeat until no search results
  • search \(#fig:(.*)_(.*)\), replace with (#fig:$1-$2), repeat until no search results
  • search \\label{eq:(.*)_(.*)}, replace with \label{eq:$1-$2}, repeat until no search results
  • search {#(.*)_(.*)}, replace with {#$1-$2}, repeat until no search results

Code to re-insert

• put instances of   where \hspace{} used to be in .tex original (usually inside tables)

8. Configure the settings files in the project folder

Change titles and paths or URLs in the following settings files in your project folder:

• _bookdown.yml
• _output.yml

In _deploy.sh:

• enter your email address in the respective field
• update the path to the online repository to accurately point to your new project's online repository on GitHub

9. Upload or "push" the project folder to the online repository

Once you have made and saved changes to any file in the project folder, they will be shown as "uncommitted changes" in your GitHub Desktop app.

In order for you to be able to upload your changes your GitHub account needs to be registered as a contributor to the project's repository. Ask the project repository's administrator for that privilege.

To upload the changed files, which is also called "pushing" changes, go to your GitHub Desktop app. Select this project's repository on the left-hand side where your repositories are listed. Enter a brief description of the changes you made. Click the "Commit" button at the bottom. Depending on your settings GitHub Desktop may upload your changes automatically after clicking "Commit" or you may need to press the "Sync" button at the top right of the app manually.

10. Have the admin of that repository activate it for automatic processing by Travis CI

As a registered contributor to your new project's online repository you can see the automatic compilation process of your project folder into a website, pdf, and ebook on https://travis-ci.com/ or https://travis-ci.org/ after signing in.

Troubleshooting

All the compiling of Markdown source files into a website, and pdf and ebook downloadables happens on https://travis-ci.com/. As a registered contributor to your new project's online repository you can see the automatic compilation process of your project folder into a website, pdf, and ebook on https://travis-ci.com/ or https://travis-ci.org/ after signing in. Look for "My Repositories" to find the compilation process or "build" and its log. It is in this log where you will find indications of why a build failed. Most commonly errors will be output by Pandoc. Below is a selection of errors I encountered during the development of the prototype.

• Tex package needed to compile was missing, fix by including it in your project's .travis.yml:

before_install:
  - tlmgr install <missing-package>

Pandoc may return errors when trying to produce a pdf. This is usually due to minor formatting issues that keep it from interpreting math correctly. Below are some of the errors I encountered and how I fixed them. The error message usually contains the string that caused the problem. Simply search where and in which file that string occurs. If it's not clear from the error message what exactly is causing the error, google it. <tex.stackexchange.com> usually has the answer.

*! Package amsmath Error: Multiple \label's:

This error occurred when more than one \label was declared inside an eqnarray equation array. To remedy that I turned the equation array into separate display equations from

\begin{eqnarray}
FORMULA 1, \label{LABEL 1} \\
FORMULA 2, \label{LABEL 2} \\
FORMULA 3, \label{LABEL 3}
\end{eqnarray}

into

\begin{equation}FORMULA 1, \label{LABEL 1} \\\end{equation}
\begin{equation}FORMULA 2, \label{LABEL 2} \\\end{equation}
\begin{equation}FORMULA 3, \label{LABEL 3}\end{equation}

• ! Package amsmath Error: \hat allowed only in math mode.

Simply remove the line break so that

The ``$\; \hat{\;}
\; $'' or ``hat'' is often used to denote an estimate of a

becomes

The ``$\; \hat{\;}\; $''
or ``hat'' is often used to denote an estimate of a

-- Tobias Pester, September 2016