Nokogiri License is not recognized but it still includes valid License.txt

[stefan-kolb]

License file: https://github.com/sparklemotion/nokogiri/blob/master/LICENSE.txt
Somehow the algorithm has problems with the file contents.

Caused by line breaks possibly:

to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,

https://github.com/versioneye/crawl_r/blob/master/lib/versioneye/crawlers/license_crawler.rb, line 370

The matcher should be made more robust to line breaks.

[timgluz]

@reiz This issue should go away when you switch to LicenseMatcher as it uses wordvectors and cos-similarity to calculate best match und it supports all the SPDX licenses + custom licenses;