Add first class karma support

[zpratt]

It's possible to get yadda and karma working together, but I have not settled on a clean way to accomplish this. Having the two working harmoniously together would be very valuable in my opinion. I've created a hack to show that it's possible: https://github.com/zpratt/yadda-karma-example

[cressie176]

Thanks Zack, I'll take a look.

[zpratt]

In my opinion, what is really needed is to implement something similar to the FileSearch class, except it will need to make an ajax call to the karma server to fetch the feature file. Aside from that, I don't see anything specific that yadda needs, given that there is a couple browserify plugins for karma which handle the dirty work of making things available to the runner. If I have time, I'll try to put together something and submit a pull request. Thanks for your consideration.

[cressie176]

I agree. Some REST based mechanism would be ideal. Something like...

GET /
{ "features": "/features" }

GET /features 
[ { "title": "12 bottles of beer", "uri": "/features/12-bottles-of-beer" }, { "title": "another feature", "uri": "/features/another-feature"}]

GET /features/10-green-bottles
{ "title": "12 bottles of beer", annotations: { pending: true },  scenarios: [ ... ]  }

I don't know how feasible writing something like this is in Karma - whether it just serves static files or whether it allows you to execute code on the server.

[zpratt]

I was thinking of something that would just use an XHR to request the feature file from the karma server instance. To my knowledge, the web server that karma starts up is primarily for static files. Production code and tests get injected as script tags inside of an IFRAME (if you're not using the requirejs plugin) and the files point to a specific path on the karma web server instance. I updated the example on my repo to use jquery to fetch the files from karma. I understand that you do not wish to pull in additional dependencies, so I think implementing an alternative to FileSearch that just uses the XHR (which will be provided by the browser instance that karma starts) to request the feature file.

Two components enable this:

• A browserify plugin for karma (there are two that I know of)
• The fact that, when running tests with karma, yadda is running in browser instance and not a node.js instance

[jeffreytgilbert]

+1 for interest, though I'm still working out my testing stack, so i'm not sure if this is superfluous or if it would benefit the testing workflow speed wise. I understood karma to be a fast test runner with ample coverage for most things, but I'm definitely still married to yadda for the gherkin support.

[cressie176]

I agree it's worth doing. Just not getting the time for any biggish features atm :(

[simoami]

I also think Yadda is great for what it does: "gherkin parsing" and there's a lot that can be covered in this area.

[jeffreytgilbert]

Yep. That's what we use it for. When we finish the startup portion of getting this into a CI flow and finish our grunt watch feature files to generate stubbed test code support, i'll have to consider looking into what the benefits are of adopting karma and see if it warrants the investment on our end to make it happen.

[AndyAtTogether]

I'd also be interested in this

[zpratt]

For my purposes, I have decided to go a different direction. We have begun to adopt the commonjs module structure for our front-end modules, and as a result, I have worked out a solution to use yadda with jsdom to run our acceptance tests outside the context of a browser. This eliminates the problem of reading feature files from the filesystem.

[jeffreytgilbert]

We use yadda for parsing feature files and then pass those instructions off to forks of mocha that run over sauce connect vms. It's slow but worth it. Karma support would still be beneficial.

[ghost]

By karma environment I wasn't able to use fs at least not the sync functions yadda currently uses in the FileSearch. I tried it with browserify using the brfs transformation, but I got a weird error message about statically calling fs functions. So fs in the browser does not work currently and we need a workaround.

I don't know whether yadda supports parsed feature serialization, or the parser calls the steps directly.

1. In the first case we can process the feature files with nodejs and send the json serialized data to the browser, where yadda could call the steps. It is possible to send data in the karma.conf.js and read it with window.__karma__.config in the browser.
2. In the second case we can serve but not include the .feature files with karma. After that we can filter them from window.__karma__.files, get them with XHR and parse them with yadda in the browser.
3. Another possible solution to write a karma plugin which can preprocess the .feature files and do the first solution completely in the background. I am not familiar with karma plugins, but I guess it is not that hard to write one.

I don't support the REST API proposed by @cressie176 . I don't like the idea of including the whole jquery just to use XHR, it's just a few lines of vanilla js. I think this requires a karma-yadda project. I assume it works the same way by webpack and browserify, so we don't have to worry about that. We might need require.js support, but it is not that popular nowadays I think.

[ghost]

I implemented the XHR workaround which appears to work with browserify. It was the simplest to do, because I know almost nothing about how yadda works in the background. I got success message, but I don't get a list of features. I am not sure It requires some modifications on the following files:

• shims/index.js - add a shim for browser

var Platform = require('../Platform');

module.exports = (function () {

    var platform = new Platform();

    var shims = {
        node: function () {
            return {
                fs: require('fs'),
                path: require('path'),
                process: process
            };
        },
        phantom: function () {
            return {
                fs: require('./phantom-fs'),
                path: require('./phantom-path'),
                process: require('./phantom-process')
            };
        },
        browser: function () {
            if (!window.__karma__) {
                console.warn("No karma environment.");
                return {};
            }
            return {
                fs: require('./karma-fs'),
                path: require("path"),
                process: {}
            };
        }
    };

    function get_shim() {
        if (platform.is_phantom()) return shims.phantom();
        if (platform.is_node()) return shims.node();
        if (platform.is_browser()) return shims.browser();
        return {};
    }

    return get_shim();
})();

• parsers/FeatureFileParser.js - using the shim fs in the parser as well

var FeatureFileParser = function(language) {

    // Requiring fs locally so it doesn't break component
    var fs = require('../shims/index').fs;

    var FeatureParser = require('./FeatureParser');
    var parser = new FeatureParser(language);

    this.parse = function(file, next) {
        var text = fs.readFileSync(file, 'utf8');
        var feature = parser.parse(text);
        return next && next(feature) || feature;
    };
};

module.exports = FeatureFileParser;

• shim/karma-fs.js - mocking the used fs functions in browser environment

module.exports = (function () {

    var p = require("path"); // supported by browserify

    var KarmaFileSystem = function () {
        this.files = this.parseFilePaths(window.__karma__.files);
        this.directories = this.parseDirectoryPaths(this.files);
    };
    KarmaFileSystem.prototype = {
        constructor: KarmaFileSystem,
        parseFilePaths: function (karmaFiles) {
            var karmaBase = /^\/base/;
            return Object.keys(karmaFiles).filter(function (karmaPath) {
                return karmaBase.test(karmaPath);
            }).map(function (karmaPath) {
                return karmaPath.replace(karmaBase, ".");
            });
        },
        parseDirectoryPaths: function (files) {
            var directoryRegistry = {};
            var registerAllParentDirectory = function (relativePath) {
                var directory = p.dirname(relativePath);
                directoryRegistry[directory] = true;
                if (directory != ".")
                    registerAllParentDirectory(directory);
            };
            files.forEach(registerAllParentDirectory);
            return Object.keys(directoryRegistry);
        },
        normalize: function (path) {
            var normalizedPath = p.normalize(path);
            if (normalizedPath != ".")
                normalizedPath = "./" + normalizedPath;
            return normalizedPath;
        },
        isDirectory: function (path) {
            path = this.normalize(path);
            return this.directories.indexOf(path) != -1;
        },
        isFile: function (path) {
            path = this.normalize(path);
            return this.files.indexOf(path) != -1;
        },
        readDirectory: function (directory) {
            directory = this.normalize(directory);
            if (!this.isDirectory(directory))
                throw new Error("Directory does not exist: " + directory);
            var contains = function (path) {
                return p.dirname(path) == directory;
            };
            return this.directories.filter(contains).concat(this.files.filter(contains)).map(function (path) {
                return p.basename(path);
            });
        },
        readFile: function (file) {
            file = this.normalize(file);
            if (!this.isFile(file))
                throw new Error("File does not exist: " + file);
            var url = file.replace(/^./, "/base");
            var xhr = new XMLHttpRequest();
            xhr.open("get", url, false);
            xhr.send();
            return xhr.responseText;
        }
    };

    var karmaFileSystem = new KarmaFileSystem();

    var fs = {};

    fs.existsSync = function (path) {
        return karmaFileSystem.isDirectory(path) || karmaFileSystem.isFile(path);
    };

    fs.readdirSync = function (path) {
        return karmaFileSystem.readDirectory(path)
    };

    fs.statSync = function (path) {
        return {
            isDirectory: function () {
                return karmaFileSystem.isDirectory(path);
            }
        }
    };

    fs.readFileSync = function (path, encoding) {
        return karmaFileSystem.readFile(path);
    };

    fs.read = function (path) {
        return karmaFileSystem.readFile(path);
    };

    return fs;
})();

It's far from stable, there are no tests, but it is working now. It stores the available paths in a ./dir/subdir/file format in two arrays currently. I am not sure whether this is the proper format, maybe the dir/subdir/file would be better. Anyways it is a good starting point to support karma.

[ghost]

Karma is now supported with Browserify. I ended up with sync XHR, since it is simpler than writing a Karma plugin. I'll might add non-Browserify Karma support later. The current version uses path-browserify, so most probably it is not compatible with webpack and certainly not compatible with module libs like require.js.

[cressie176]

LGTM. OK to close?

[ghost]

@cressie176 I think it should be opened until Karma is not supported with webpack and probably with require.js. So this is only a partial solution. Btw. how do you create the UMD release files? I did not find an npm build script. I'll need that to test the lib with require.js.

[ghost]

Adding Webpack support appears to be harder than I thought. Can anyone give info about how to configure webpack with karma?

I tried with this one, but it did not find the ./lib/fn.js somehow and it appears to load the phantomjs shims instead of the karma shims. I guess webpack is somewhat different than browserify.

module.exports = function (config) {
    config.set({
        plugins: [
            "karma-webpack",
            "karma-phantomjs-launcher",
            "karma-mocha"
        ],
        frameworks: ["mocha"],
        files: [
            "lib/**/*.js",
            {pattern: "lib/**/!(*.js)", included: false},
            "test/**/*.js",
            {pattern: "test/**/!(*.js)", included: false}
        ],
        preprocessors: {
            "lib/**/*.js": ["webpack"],
            "test/**/*.js": ["webpack"]
        },
        client: {
            mocha: {
                reporter: "html",
                ui: "bdd"
            }
        },
        webpack: {
            quiet: true,
            output: {libraryTarget: "commonjs"},
            externals: [
                {
                    "yadda": "./lib/index"
                }
            ],
            resolve: {
                modulesDirectories: [
                    'bower_components',
                    'node_modules'
                ]
            }
        },
        browsers: ["PhantomJS"],
        reporters: ["progress"],
        port: 9876,
        colors: true,
        logLevel: config.LOG_INFO,
        autoWatch: false,
        captureTimeout: 6000,
        singleRun: true
    });
};

[ghost]

Sorry guys, I don't have enough time/motivation to finish the Webpack support. Browserify support is enough for me, I think Webpack users should finish this. Good luck!

[cressie176]

No problem. Thanks for your contributions.

[ghost]

The phantom env check https://github.com/acuminous/yadda/blob/master/lib/Platform.js#L43 is not perfect by karma because karma runs the code in an iframe by default and the karma phantom launcher does not write the phantom properties, like window._phantom into this frame. Here is some fix:

var phantom = global._phantom;
if (!phantom && global.__karma__ && global.frameElement)
    phantom = global.parent._phantom;
var isPhantom = !!phantom;

Because of this the fs and path shims were never tested in a phantomjs environment. I'll check them asap.

Another interesting thing, that browserify does not support dynamic require calls. So something like this will fail by karma+browserify:

var Yadda = require("yadda");
Yadda.plugins.mocha.StepLevelPlugin.init();
var English = Yadda.localisation.English;
var path = Yadda.shims.path;

new Yadda.FeatureFileSearch("./features").each(function (file) {

    featureFile(file, function (feature) {

        var stepsFile = String("./" + path.dirname(file) + "/step_definitions/" + path.basename(file, ".feature")).split("\\").join("/");

        var library = English.library();

        var defineSteps = require(stepsFile);
        defineSteps.call(library);

        var yadda = Yadda.createInstance(library);

        scenarios(feature.scenarios, function (scenario) {
            steps(scenario.steps, function (step, done) {
                yadda.run(step, done);
            });
        });
    });
});

I fast fix to add a function to the end of the file, which is never called but requires the js files one by one. Another possible solution to use require_globify and a glob pattern for the same, but it did not work for me. So even the browserify support is not complete yet.