Test phrases

[1ec5]

Added a convenience method for tokenizing a phrase, plus tests and test fixtures against that method.

Fixes #143.

/cc @mcwhittemore @lyzidiamond

[yuryleb]

It's not related to this PR directly but is it possible also to change arguments order of new tokenize(instruction, replaceTokens, language) function to move language argument first as in all other index.js's methods?

[1ec5]

@yuryleb, thanks for the suggestion. I opened #149 with that change.

[mcwhittemore]

I'm on the fence here.

Having used tokenize and compile to achieve this, I agree that a better way to access phrases would be nice. Accessing the JSON and then falling back to english is a bit much to then also need to send the language code to tokenize.

That said, compiling the steps as part of this process seems like we're going too far here. Part of me wonders how often this will be used vs just using compile and tokenize.

That said, I could see this being helpful if you were trying to reduce a set of steps into a non-one-to-one list of instructions so maybe its worth adding...

[1ec5]

I understand your hesitation around the additional compilePhrase() method. I ended up adding it mainly because it was necessary for unit testing phrase generation. I’d be fine with considering it a private method and not documenting it for clients (although it still needs to be exported for the tests).

[mcwhittemore]

Can't the tests just call compile and then tokenize?

[1ec5]

Can't the tests just call compile and then tokenize?

Good point. Done.

[mcwhittemore]

Cool. I think this ready to merge.