Source map support plan?

[samwgoldman]

Sorry if this isn't the correct venue for this kind of discussion. If there is a better place to have this discussion (like a mailing list), please let me know.

Source maps are files that define mappings from a generated output to one or many constituent inputs. This mapping can be used to determine the original source location (line and column) given an generated source location. For example, given a stack trace for an error in generated javascript, the source location of the originating coffee script could be located.

Another use case is finding the original name of something. For example, if a minification caused a token to be rewritten into something smaller, an error message containing the minified token could be rewritten to instead show the original token.

Some browsers have included support for source maps. The CoffeeScriptRedux project has some support for generating source maps. Google's Closure Compiler has vey good support for source maps. UglifyJS2 also has preliminary source map support.

The story with individual source map transformations is pretty straightforward. Given one or many input files, the transformation generates the one or many output files, each with its own source map.

With multiple layers of transformations, however, things get a little more complicated. The transformation needs to take as input any source files and their corresponding source maps in order to generate output with source maps that include the information from the input mappings. As far as I know, UglifyJS2 is the only project that has started to support input source maps.

With rake-pipeline's current filter API, I don't believe that multiple layers of transformations are possible. Ideally, source map support could be a library add-on, but I think that the rake-pipeline API needs to change in order to support source maps.

I would like to come up with a plan to support source maps. I am happy to help, if a consensus can be reached.

My proposal:

A rake-pipeline filter takes as input an array of FileWrapper objects. I would like to augment this object with an optional source_map property. If this property is not present, the file is the considered to be the original source file.

A filter that supports source maps would be responsible for reading the input FileWrappers' source maps and generating output FileWrappers having source maps.

The DSL would also need some way to indicate where the source map for any input file can be found, and where the output source map(s) should be placed.

[wagenet]

This would definitely be a nice feature.

[samwgoldman]

I have time to put together a proof of concept, if that would help. Perhaps something simple, like the concat filter?

[wagenet]

A POC seems nice to me.

[joefiorini]

FYI, I have a working original_inputs property on FileWrapper that returns the original inputs an output was created from.. Would it help this effort if I sent a pull request for it?

[samwgoldman]

I don't think that will be necessary, but I'll keep it in mind. The source map itself keeps track of that information, so perhaps the property could be computed from the source map. Too soon to tell.

[joefiorini]

It sounds similar to what you proposed here:

A rake-pipeline filter takes as input an array of FileWrapper objects. I would like to augment this object with an optional source_map property. If this property is not present, the file is the considered to be the original source file.

Instead of a source_map property on Filter I have an original_inputs property on FileWrapper. Now that I think about it, it probably wouldn't be terribly difficult to do this on the root Pipeline instead.

FYI, I'm not using this for source maps, I needed it for something else. If it's completely unrelated, that's fine, but if there's anything I could do to make my work a little more useful I'm all for it. What do you think @samwgoldman?

[samwgoldman]

I see the similarity, but I'm not sure how the pieces fit together yet. Once I dive into the code I will have a better idea. Sorry I don't have a clear answer, but if it appears to me that the work you have done will help, I will not hesitate to pull it in.

[ahawkins]

Why not just do something like this in your filters? https://github.com/wycats/rake-pipeline-web-filters/blob/master/lib/rake-pipeline-web-filters/minispade_filter.rb#L54