Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

documents and reorganizes the compiler code - part 2 #97

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions hsp/compiler/compiler.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ var treebuilder = require("./treebuilder/index");
var jsgenerator = require("./jsgenerator/index");

/**
* Compile a template and return a JS compiled string and a list of errors
* @param template {String} the template file content as a string
* @param fileName {String} the name of the file being compiled (optional - used for error messages)
* @param includeSyntaxTree {Boolean} if true, the result object will contain the syntax tree generated by the compiler
* @param bypassJSvalidation {Boolean} if true, the validation of the generated JS file (including non-template code) is
* bypassed - default:false
* Compiles a template and return a JS compiled string and a list of errors.
* @param {String} template the template file content as a string.
* @param {String} dirPath the directory path.
* @param {String} fileName the name of the file being compiled (optional - used for error messages).
* @param {Boolean} includeSyntaxTree if true, the result object will contain the syntax tree generated by the compiler.
* @param {Boolean} bypassJSvalidation if true, the validation of the generated JS file (including non-template code) is bypassed - default:false.
* @return {JSON} a JSON structure with the following properties:
* errors: {Array} the error list - each error having the following structure:
* description: {String} - a message describing the error
Expand Down
51 changes: 50 additions & 1 deletion hsp/compiler/jsgenerator/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,52 @@
# JS Generator #

TODO
The **jsgenerator** is the third and last step of the compilation process.
It generates the final compiled template as a javascript string, using the syntax tree built previously.

## Input ##
The syntax tree, e.g.:
```
{
"type": "template",
"name": "test",
"args": ["person"],
"content": [
{type:"element", name:"div", closed:false, attributes:[
{name:"title", type:"text", value:"Some text"},
{name:"id", type:"expression", "category": "objectref", "bound": true, "path": [ "person", "id" ]},
{name:"class", type:"textblock", content:[
{type:"expression", "category": "objectref", "bound": true, "path": [ "person", "gender" ]},
{type:"text", value:" "},
{type:"expression", "category": "objectref", "bound": true, "path": [ "person", "category" ]}
]}
],"content": [
{ "type": "textblock", "content": [
{"type": "text","value": "Hello "},
{"type": "expression", "category": "objectref", "bound": true, "path": [ "person", "name" ]},
{"type": "text","value": "! "}
]}
]}
]
}
```

## Output ##
The compiled template, e.g.:
```
n.elt( "div",
{e1:[1,2,"person","id"],e2:[1,2,"person","gender"],e3:[1,2,"person","category"]} ,
{"title":"Some text","id":["",1],"class":["",2," ",3]},
0,[
n.$text({e1:[1,2,"person","name"]},["Hello ",1,"! "])
]
)
```

## Algorithm ##

The first part of this step is the actual generation.
It is done thanks to the TemplateWalker class (which extend TreeWalker).
It walks the SyntaxTree built previously and manages each node with a processor, depending on its type.
Each processor generates a javascript string for the node and its children.

Once the generation is done, the resulting javascript string is validated with the *acorn* parser.
89 changes: 55 additions & 34 deletions hsp/compiler/jsgenerator/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,23 @@ var HEADER = module.exports.HEADER = HEADER_ARR.join('\r\n');
var HEADER_SZ = HEADER_ARR.length;

/**
* TODO
* Generates the JS compiled string and a list of errors.
* @param {Object} res the result of the previous steps of the compilation.
* @param {String} template the template file content as a string.
* @param {String} dirPath the directory path.
* @param {String} fileName the name of the file being compiled (optional - used for error messages).
* @param {Boolean} includeSyntaxTree if true, the result object will contain the syntax tree generated by the compiler.
* @param {Boolean} bypassJSvalidation if true, the validation of the generated JS file (including non-template code) is bypassed - default:false.
* @return {JSON} a JSON structure with the following properties:
* errors: {Array} the error list - each error having the following structure:
* description: {String} - a message describing the error
* line: {Number} - the error line number
* column: {Number} - the error column number
* code: {String} - a code extract showing where the error occurs (optional)
* code: {String} the generated JavaScript code
* syntaxTree: {JSON} the syntax tree generated by the parser (optional - cf. parameters)
* lineMap: {Array} array of the new line indexes: lineMap[3] returns the new line index for line 4 in
* the orginal file (lineMap[0] is always 0 as all line count starts at 1 for both input and output values)
*/
exports.generate = function(res, template, fileName, dirPath, includeSyntaxTree, bypassJSvalidation) {
res.code = '';
Expand Down Expand Up @@ -68,41 +84,46 @@ exports.generate = function(res, template, fileName, dirPath, includeSyntaxTree,
};

/**
* TODO
* Validates a javascript string using the jsvalidator module, and generates an error report if not valid.
* @param {String} code the javascript string.
* @param {Object} lineMap the line mapping between the source template and the compiled one
* @return {Object} a result map
*/
function _validate (code, lineMap) {
var r = jsv.validate(code);
var result = {isValid: r.isValid};
if (!r.isValid) {
var validationResult = jsv.validate(code);
var result = {isValid: validationResult.isValid};
if (!validationResult.isValid) {
// translate error line numbers
var err, ln, lm = lineMap;
for (var i = 0, sz = r.errors.length; sz > i; i++) {
err = r.errors[i];
ln = err.line;

err.line = -1; // to avoid sending a wrong line in case of pb
for (var j = 0, sz2 = lm.length; sz2 > j; j++) {
if (lm[j] === ln) {
err.line = j; // original line nbr
var error, lineNumber;
for (var i = 0; i < validationResult.errors.length; i++) {
error = validationResult.errors[i];
lineNumber = error.line;

error.line = -1; // to avoid sending a wrong line in case of pb
for (var j = 0; j < lineMap.length; j++) {
if (lineMap[j] === lineNumber) {
error.line = j; // original line nbr
break;
}
}
}
result.errors = r.errors;
result.errors = validationResult.errors;
}
return result;
}

/**
* Generate an error script to include in the template compiled script in order to show errors in the browser when the
* script is loaded
* Generate an error script to include in the template compiled script in order to show errors in the browser when the script is loaded
* @param {Array} errors the errror list
* @param {String} fileName the name of the file being compiled
* @return {String} the javascript snippet to be included
*/
function _getErrorScript (errors, fileName) {
var r = '';
var result = '';
if (errors && errors.length) {
r = ['\r\nrequire("hsp/rt").logErrors("', fileName, '",', JSON.stringify(errors, null), ');\r\n'].join("");
result = ['\r\nrequire("hsp/rt").logErrors("', fileName, '",', JSON.stringify(errors, null), ');\r\n'].join("");
}
return r;
return result;
}

/**
Expand All @@ -114,29 +135,29 @@ function _generateLineMap (res, file) {
if (res.errors && res.errors.length) {
return;
}
var st = res.syntaxTree, templates = [];
var syntaxTree = res.syntaxTree, templates = [];
// identify the templates in the syntax tree
for (var i = 0; st.length > i; i++) {
if (st[i].type === 'template') {
templates.push(st[i]);
for (var i = 0; i < syntaxTree.length; i++) {
if (syntaxTree[i].type === 'template') {
templates.push(syntaxTree[i]);
}
}

var nbrOfLinesInCompiledTemplate = 5;
var lm = [], sz = file.split(/\n/g).length + 1, pos = HEADER_SZ, tpl;
var lineMap = [], pos = HEADER_SZ, template;
var pos1 = -1; // position of the next template start
var pos2 = -1; // position of the next template end
var tplIdx = -1; // position of the current template

for (var i = 0; sz > i; i++) {
for (var i = 0; i < (file.split(/\n/g).length + 1); i++) {
if (i === 0 || i === pos2) {
// end of current template: let's determine next pos1 and pos2
tplIdx = (i === 0) ? 0 : tplIdx + 1;
if (tplIdx < templates.length) {
// there is another template
tpl = templates[tplIdx];
pos1 = tpl.startLine;
pos2 = tpl.endLine;
template = templates[tplIdx];
pos1 = template.startLine;
pos2 = template.endLine;
if (pos2 < pos1) {
// this case should never arrive..
pos2 = pos1;
Expand All @@ -146,23 +167,23 @@ function _generateLineMap (res, file) {
tplIdx = pos1 = pos2 = -1;
}
if (i === 0) {
lm[0] = 0;
lineMap[0] = 0;
}
i++;
}
if (i === pos1) {
for (var j = pos1, max = pos2 + 1; max > j; j++) {
for (var j = pos1; j < (pos2 + 1); j++) {
// all lines are set to the template start
lm[i] = pos;
lineMap[i] = pos;
i++;
}
pos += nbrOfLinesInCompiledTemplate;
i -= 2; // to enter the i===pos2 clause at next step
} else {
lm[i] = pos;
lineMap[i] = pos;
pos++;
}
}

return lm;
return lineMap;
}
40 changes: 22 additions & 18 deletions hsp/compiler/jsgenerator/jsvalidator/validator.js
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
var acorn = require("acorn/acorn");

/**
* Validate a JavaScript string Return a JSON structure with 'valid' and 'errors' properties e.g. {valid:false,
* errors:[{msg:'...',lineInfoTxt:'...',lineInfoHTML:'...',loc:{line:2,column:30}'}]}
* Validates a JavaScript string
* @param {String} input the Javascript string
* @return {Object} JSON structure with 'valid' and 'errors' properties e.g. {valid:false, errors:[{msg:'...',lineInfoTxt:'...',lineInfoHTML:'...',loc:{line:2,column:30}'}]}
*/
module.exports.validate = function (input) {
var res = {
var result = {
isValid : true
};

Expand All @@ -17,45 +18,48 @@ module.exports.validate = function (input) {
forbidReserved : true
});
} catch (ex) {
res.isValid = false;
res.errors = [formatError(ex, input)];
result.isValid = false;
result.errors = [formatError(ex, input)];
}

return res;
return result;
};

/**
* Format the error as an error structure with line extract information
* Formats the error as an error structure with line extract information.
* @param {Object} error the exception.
* @param {String} input the Javascript string.
* @return {Object} the structured error.
*/
function formatError (err, input) {
var msg = err.toString().replace(/\s*\(\d*\:\d*\)\s*$/i, ''); // remove line number / col number
function formatError (error, input) {
var message = error.toString().replace(/\s*\(\d*\:\d*\)\s*$/i, ''); // remove line number / col number

var bm = ('' + input.slice(0, err.pos)).match(/.*$/i);
var am = ('' + input.slice(err.pos)).match(/.*/i);
var before = bm ? bm[0] : '';
var after = am ? am[0] : '';
var beforeMatch = ('' + input.slice(0, error.pos)).match(/.*$/i);
var afterMatch = ('' + input.slice(error.pos)).match(/.*/i);
var before = beforeMatch ? beforeMatch[0] : '';
var after = afterMatch ? afterMatch[0] : '';

// Prepare line info for txt display
var cursorPos = before.length;
var errChar = (after.length) ? after.slice(0, 1) : 'X';
var lineStr = before + after;
var lncursor = [];
for (var i = 0, sz = lineStr.length; sz > i; i++) {
for (var i = 0; i < lineStr.length; i++) {
lncursor[i] = (i === cursorPos) ? '^' : '-';
}
var lineInfoTxt = lineStr + '\r\n' + lncursor.join('');

// Prepare line info for HTML display
var lineInfoHTML = ['<span class="code">', before, '<span class="error" title="', msg, '">', errChar, '</span>',
var lineInfoHTML = ['<span class="code">', before, '<span class="error" title="', message, '">', errChar, '</span>',
after.slice(1), '</span>'].join('');

return {
description : msg,
description : message,
lineInfoTxt : lineInfoTxt,
lineInfoHTML : lineInfoHTML,
code : lineStr,
line : err.loc ? err.loc.line : -1,
column : err.loc ? err.loc.column : -1
line : error.loc ? error.loc.line : -1,
column : error.loc ? error.loc.column : -1
};

}
Loading