Compiling to web assemby #1323

maxiplay · 2018-12-27T09:45:42Z

Had you try to compile it to WebAssembly ?

It would be perfect to avoid a high load on server. Each user computing the spellcheck locally.

There is now compiler from java

https://github.com/appcypher/awesome-wasm-langs#java

danielnaber · 2018-12-27T10:02:57Z

I'm not aware of any recent attempts, but if you're willing to give it a try that would be great!

danielnaber · 2018-12-27T22:14:07Z

Also see https://forum.languagetool.org/t/using-teavm-to-run-lt-under-javascript/490 for an old discussion about that

maxiplay · 2018-12-28T05:48:10Z

Thx for the link !

I test it and it seems the compilation of the Hunspell.java class is in error.

It does not surprise me. It seems that LanguageTool use jna to execute native code so it's a non sense to want to transform it to native LLVM.

Error optimizing program for method org.languagetool.rules.spelling.hunspell.Hunspell.<init>(Ljava/lang/String;)V: var @this as this var @libDir as libDir var @urgh as urgh var @is as is var @fos as fos var @lib as lib var @buf as buf var @e as e var @libDir_1 as libDir var @libDir_2 as libDir var @libDir_3 as libDir var @is_1 as is $0 goto $1 $1 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 88 nop invoke java.lang.Object.()V@this at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 42 @2 := null field org.languagetool.rules.spelling.hunspell.Hunspell.hsl @this := @2 asLorg/languagetool/rules/spelling/hunspell/HunspellLibrary;at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 216 @3 := new java.util.HashMap invokejava.util.HashMap.()V@3 field org.languagetool.rules.spelling.hunspell.Hunspell.map @this := @3 asLjava/util/HashMap;at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 90 if @libDir === null then goto $2 else goto $3 $2 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 90 @libDir_1 := null @4 := invokeStaticorg.languagetool.rules.spelling.hunspell.Hunspell.libNameBare()Ljava/lang/String;at unknown location goto $4 $3 @libDir_2 := nullCheck @libDir at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 90 @5 := new java.lang.StringBuilder invokejava.lang.StringBuilder.()V@5 @6 := invokejava.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder;@5, @libDir_2 @7 := '/' @8 := invokejava.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder;@6, @7 at unknown location @85 := nullCheck @6 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 90 @9 := invokeStaticorg.languagetool.rules.spelling.hunspell.Hunspell.libName()Ljava/lang/String;@10 := invokejava.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder;@8, @9 at unknown location @86 := nullCheck @8 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 90 @11 := invokejava.lang.StringBuilder.toString()Ljava/lang/String;@10 at unknown location @87 := nullCheck @10 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 90 goto $4 $4 @12 := phi @4 from $2, @11 from $3 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 90 field org.languagetool.rules.spelling.hunspell.Hunspell.libFile @this := @12 asLjava/lang/String;at unknown location goto $11 $5 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 122 if @40 === null then goto $6 else goto $7 $6 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 122 @40 := null at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 125 throw @39 $7 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 125 @40 := nullCheck @40 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 123 invokeVirtualjava.io.FileOutputStream.close()V@40 @69 := new java.lang.NoClassDefFoundError @70 := 'Class not found: java.io.FileOutputStream' invokejava.lang.NoClassDefFoundError.(Ljava/lang/String;)V@69, @70 throw @69 catch java.io.IOException goto $19 $8 @39 := exception @40 := phi @fos from $14, @36 from $14, @51 from $15 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 123 goto $16 $9 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 123 @is_1 := nullCheck @is at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 106 @fos := null at unknown location goto $14 $10 @is := null at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 99 @59 := new java.lang.UnsatisfiedLinkError @75 := new java.lang.NoClassDefFoundError @76 := 'Class not found: java.lang.UnsatisfiedLinkError' invokejava.lang.NoClassDefFoundError.(Ljava/lang/String;)V@75, @76 throw @75 $11 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 92 @13 := field org.languagetool.rules.spelling.hunspell.Hunspell.libFile @this asLjava/lang/String;@14 := classOfLorg/languagetool/rules/spelling/hunspell/HunspellLibrary;@15 := invokeStaticcom.sun.jna.Native.loadLibrary(Ljava/lang/String;Ljava/lang/Class;)Ljava/lang/Object;@13, @14 @16 := cast @15 toLorg/languagetool/rules/spelling/hunspell/HunspellLibrary;field org.languagetool.rules.spelling.hunspell.Hunspell.hsl @this := @16 asLorg/languagetool/rules/spelling/hunspell/HunspellLibrary;at unknown location goto $12 catch java.lang.UnsatisfiedLinkError goto $13 $12 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 129 return $13 @urgh := exception at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 96 nop @18 := invokeStaticorg.languagetool.rules.spelling.hunspell.Hunspell.libName()Ljava/lang/String;field org.languagetool.rules.spelling.hunspell.Hunspell.libFile @this := @18 asLjava/lang/String;at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 97 @19 := classOfLorg/languagetool/rules/spelling/hunspell/Hunspell;@20 := new java.lang.StringBuilder invokejava.lang.StringBuilder.()V@20 @21 := '/' @22 := invokejava.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder;@20, @21 @23 := field org.languagetool.rules.spelling.hunspell.Hunspell.libFile @this asLjava/lang/String;@24 := invokejava.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder;@22, @23 at unknown location @95 := nullCheck @22 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 97 @25 := invokejava.lang.StringBuilder.toString()Ljava/lang/String;@24 at unknown location @96 := nullCheck @24 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 97 @is := invokejava.lang.Class.getResourceAsStream(Ljava/lang/String;)Ljava/io/InputStream;@19, @25 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 98 nop if @is !== null then goto $9 else goto $10 $14 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 108 nop @28 := 'jna' @29 := new java.lang.StringBuilder invokejava.lang.StringBuilder.()V@29 @30 := '.' @31 := invokejava.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder;@29, @30 @32 := field org.languagetool.rules.spelling.hunspell.Hunspell.libFile @this asLjava/lang/String;@33 := invokejava.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder;@31, @32 at unknown location @98 := nullCheck @31 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 108 @34 := invokejava.lang.StringBuilder.toString()Ljava/lang/String;@33 at unknown location @99 := nullCheck @33 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 108 @lib := invokeStaticjava.io.File.createTempFile(Ljava/lang/String;Ljava/lang/String;)Ljava/io/File;@28, @34 @81 := new java.lang.NoClassDefFoundError @82 := 'Class not found: java.io.File' invokejava.lang.NoClassDefFoundError.(Ljava/lang/String;)V@81, @82 throw @81 catch java.io.IOException goto $15 catch goto $8 $15 @e := exception @51 := phi @fos from $14, @36 from $14 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 118 nop @52 := new java.lang.Error @53 := new java.lang.StringBuilder invokejava.lang.StringBuilder.()V@53 @54 := 'Failed to create temporary file for ' @55 := invokejava.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder;@53, @54 @56 := field org.languagetool.rules.spelling.hunspell.Hunspell.libFile @this asLjava/lang/String;@57 := invokejava.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder;@55, @56 at unknown location @55 := nullCheck @55 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 118 @58 := invokejava.lang.StringBuilder.toString()Ljava/lang/String;@57 at unknown location @57 := nullCheck @57 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 118 invokejava.lang.Error.(Ljava/lang/String;Ljava/lang/Throwable;)V@52, @58, @e throw @52 catch goto $8 $16 at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 121 invokejava.io.ByteArrayInputStream.close()V` @is
at unknown location
goto $17
catch java.io.IOException goto $18
$17
at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 121
goto $5
$18
at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 121
goto $5
$19
at 'org/languagetool/rules/spelling/hunspell/Hunspell.java' 121
goto $6

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------`

maxiplay · 2018-12-28T05:53:07Z

I created a test repo https://github.com/maxiplay/LanguageTool-WebAssembly

maxiplay · 2018-12-28T06:00:07Z

Now I excluded the hunspell native part

    <dependency>
      <groupId>org.languagetool</groupId>
      <artifactId>language-fr</artifactId>
      <version>4.4</version>

      <exclusions>
        <exclusion>
          <groupId>org.languagetool</groupId>
          <artifactId>hunspell-native-libs</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

And i just initalize the french version

public class Client {
    public static void main(String[] args) throws IOException {

        Language french = new French();

 
        HTMLDocument document = HTMLDocument.current();
        HTMLElement div = document.createElement("div");
        div.appendChild(document.createTextNode("Welcome to Language Tool"));
        document.getBody().appendChild(div);
    }
}

And i get this error relative to java.lang.System.getSecurityManager()

[ERROR] Failed to execute goal org.teavm:teavm-maven-plugin:0.5.1:compile (web-client) on project teaVM: Unexpected error occured: Error rendering class org.xml.sax.helpers.NewInstance. See a cause for details: Can't provide name for method as it was not found: java.lang.System.getSecurityManager()Ljava/lang/SecurityManager; -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

maxiplay · 2018-12-28T06:01:11Z

I invited the creator of TeaVM to contribute

maxiplay · 2018-12-28T06:21:56Z

Do you have a full developer documentation of LanguageTool ?
Contributions to the core code would be more frequent with a documentation about it

maxiplay · 2018-12-28T06:35:37Z

I created an online writing software for writer that use actively LanguageTool.

https://www.writecontrol.fr

If the project grow contributions could be frequent.

danielnaber · 2018-12-28T11:26:53Z

Maybe it's a good idea to first make a cleanup, ensure everything still compiles and the tests work, and then introduce teaVM? I've tried that here: 2230484 (the cleanup part - only languages left are EN and FR). Can you try to build on that, i.e. add teaVM?

BTW, this seems to be a commercial teaVM alternative: https://www.leaningtech.com/cheerpj

danielnaber · 2018-12-28T11:30:40Z

Alternatives to teaVM are also relevant because teaVM is "not for taking your large existing codebase in Java or Kotlin and producing JavaScript." (http://teavm.org/docs/intro/overview.html)

maxiplay · 2018-12-28T15:28:04Z

Ok thx for the cleanup.
I tried and test failed for french because a test need hunspell.

I re-add hunspell module for checking and now test fail with this error : org.languagetool.commandline.AbstractSecurityTestCase$ExitException: There is no escape!

maxiplay · 2018-12-28T15:28:38Z

Will test with https://www.leaningtech.com/cheerpj

maxiplay · 2018-12-28T15:59:11Z

On master branch all test seems good excepted for http server but it seems to be an encoding problem. Perhaps relative to my windows machine.

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.834 s <<< FAILURE! - in org.languagetool.server.UserDictTest
[ERROR] testHTTPServer(org.languagetool.server.UserDictTest)  Time elapsed: 7.829 s  <<< FAILURE!
java.lang.AssertionError:

Expected: is "[Schöckl]"
     but: was "[Sch?ckl]"
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.junit.Assert.assertThat(Assert.java:956)
        at org.junit.Assert.assertThat(Assert.java:923)
        at org.languagetool.server.UserDictTest.runTests(UserDictTest.java:89)
        at org.languagetool.server.UserDictTest.testHTTPServer(UserDictTest.java:67)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
        at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
        at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344)
        at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417)

danielnaber · 2018-12-28T17:05:11Z

Could you push your changes to https://github.com/maxiplay/LanguageTool-WebAssembly so I can easily reproduce them? The test indeed seems to have an encoding issue, I think we hardly ever run the tests on Windows.

maxiplay · 2018-12-28T17:36:23Z

For the encoding problem I cloned the languageTool repo and checkout to master branch. No change to source code. I am on windows 10, jdk 8

For the hunspell error relative to french, I was on LanguageTool repo on the "webassembly" branch. it was the branch you do the cleanup

I read the documentation about cheerpj and successfuly test the getting started.
cheerpj need a jar to compile into js. So I directly tested the languagetool.jar (4.4) from the dowload page of languageTool.

The compilation crash with forbiden path error. I think there is perhaps a relation with the lib folder and the need to have a "fat" jar instead.

I will try to create a very simple test with english and language-core in a unique jar.

maxiplay · 2018-12-28T23:23:02Z

Just to succeed to compile a test project with cheerpJ :)

it's in this repo https://github.com/maxiplay/LanguageTool-WebAssembly on the cheerpj branch.

maxiplay · 2018-12-29T09:00:18Z

It takes 36 seconds to start (with browser cache) to launch a spell check.

It load 50MB the first time composed of a javascript compiled JavaSE and language-en dependency.

During 36 seconds it takes 20% of my CPU.
The loading perfs are mitigated. It's very cool that it works but it heavy weight

danielnaber · 2018-12-29T09:19:10Z

Thanks for the update. It's amazing that it works at all :-) Is it faster once everything is loaded? How long does checking a single sentence take once everything is loaded?

danielnaber · 2018-12-29T10:37:11Z

Here's a discussion about how to make the file size smaller: leaningtech/cheerpj-meta#14 - summary: it's not supported yet...

maxiplay · 2018-12-29T17:32:52Z

Need to improve the test project to say you that.
Need also to verify that it works also with French or other languages. It seems to me that the compilation failed with french but I will verify.

What is the reason why the French version need Hunspell and not the English version ?

First I will follow this page to improve the startup page.
There are good tips at this page :
https://github.com/leaningtech/cheerpj-meta/wiki/Startup-time-optimization

danielnaber · 2018-12-29T18:22:26Z

What is the reason why the French version need Hunspell and not the English version ?

Actually, I don't remember. Maybe this was the reason: #850 (comment). So a version for French without hunspell could be possible, but it might lack some words or so. For other languages, like German, living without hunspell is more complicated as they need support for compounds.

maxiplay · 2018-12-29T21:16:53Z

Ok thx for the response.

I will try to test Proguard following this documentation.

https://github.com/leaningtech/cheerpj-meta/wiki/Startup-time-optimization

Fully test the application, making sure that all possible scenarios are executed and all needed classes are loaded

Do you have a resource (Like a test) that give me an full scenario of spellchecking ? Grammar, hunspell, etc.

The goal is to reduce significally the size of the generated jar just providing the necessary bytcode to do spellcheck.

danielnaber · 2018-12-29T21:39:36Z

Do you have a resource (Like a test) that give me an full scenario of spellchecking ? Grammar, hunspell, etc.

All the tests combined (as in running mvn test in LT's top-level directory) should have a very good code coverage. But just checking a text should also execute almost all code for the language of that text.

I think one key to keeping the results small is to work on one language a time. A lot of what makes LT so large is the fact that the resulting artifacts support all languages.

maxiplay · 2018-12-29T22:27:21Z

Yes that was my idea. Separate languages.
But just with english the loading is 50MB. (Java SE part ~ 25MB)
Want to test Proguard just to see what will be the result but I'm afraid that is approximative.

I tested the pack-jar option but it need to have a fat jar and loose incremental loading. The complete donwload up to 80MB to launch.

I tested french and actually it compile except 2 files.

`Failure compiling META-INF/versions/9/javax/xml/bind/ModuleUtil.class
command: C:\Users\Maxime\Documents\writecontrol\LanguageTool-WebAssembly\cheerpj_1.3\bin\cheerpj "" -llvm -llvm-exceptions -cp C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\rt.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\language-fr-4.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\languagetool-core-4.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-lang3-3.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\guava-21.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jna-4.4.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-fsa-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-fsa-builders-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-speller-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-stemming-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\hppc-0.7.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\segment-2.0.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-logging-1.1.1.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\language-detector-0.6.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jsonic-1.2.11.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\annotations-12.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\lucene-core-5.5.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\lucene-backward-codecs-5.5.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\berkeleylm-1.1.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-databind-2.9.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-annotations-2.9.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-core-2.9.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\unit-api-1.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\indriya-1.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\uom-lib-common-1.1.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-api-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-runtime-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-core-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\txw2-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\istack-commons-runtime-3.0.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\stax-ex-1.7.8.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\FastInfoset-1.2.13.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\javax.activation-api-1.2.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\slf4j-api-1.7.25.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\xgboost-predictor-0.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jafama-2.1.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-csv-1.6.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\hunspell-native-libs-2.9.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\annotations-12.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\berkeleylm-1.1.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-csv-1.6.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-lang3-3.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-logging-1.1.1.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\FastInfoset-1.2.13.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\guava-21.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\hppc-0.7.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\hunspell-native-libs-2.9.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\indriya-1.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\istack-commons-runtime-3.0.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-annotations-2.9.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-core-2.9.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-databind-2.9.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jafama-2.1.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\javax.activation-api-1.2.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\javax.servlet-api-3.1.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-api-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-core-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-runtime-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jna-4.4.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jsonic-1.2.11.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\language-detector-0.6.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\language-fr-4.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\languagetool-core-4.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\lucene-backward-codecs-5.5.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\lucene-core-5.5.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-fsa-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-fsa-builders-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-speller-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-stemming-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\segment-2.0.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\slf4j-api-1.7.25.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\stax-ex-1.7.8.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\txw2-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\unit-api-1.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\uom-lib-common-1.1.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\xgboost-predictor-0.3.0.jar.dir\ META-INF/versions/9/javax/xml/bind/ModuleUtil.class
Loading class META-INF/versions/9/javax/xml/bind/ModuleUtil and dependencies

Could not open class java/lang/Module

Could not open class java/lang/invoke/StringConcatFactory

Unsupported LLVM generation for opcode ba

In function getClassesFromContextPath

SUPPORT CONSTANT 8Assertion failed!

Program: C:\Users\Maxime\Documents\writecontrol\LanguageTool-WebAssembly\cheerpj_1.3\bin\cheerpj

File: CheerpJCompilerMethod.cpp, Line 2241

Expression: false
`

And

`Failure compiling module-info.class
command: C:\Users\Maxime\Documents\writecontrol\LanguageTool-WebAssembly\cheerpj_1.3\bin\cheerpj "" -llvm -llvm-exceptions -cp C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\rt.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\language-fr-4.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\languagetool-core-4.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-lang3-3.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\guava-21.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jna-4.4.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-fsa-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-fsa-builders-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-speller-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-stemming-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\hppc-0.7.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\segment-2.0.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-logging-1.1.1.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\language-detector-0.6.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jsonic-1.2.11.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\annotations-12.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\lucene-core-5.5.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\lucene-backward-codecs-5.5.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\berkeleylm-1.1.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-databind-2.9.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-annotations-2.9.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-core-2.9.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\unit-api-1.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\indriya-1.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\uom-lib-common-1.1.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-api-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-runtime-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-core-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\txw2-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\istack-commons-runtime-3.0.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\stax-ex-1.7.8.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\FastInfoset-1.2.13.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\javax.activation-api-1.2.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\slf4j-api-1.7.25.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\xgboost-predictor-0.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jafama-2.1.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-csv-1.6.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\hunspell-native-libs-2.9.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\annotations-12.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\berkeleylm-1.1.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-csv-1.6.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-lang3-3.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\commons-logging-1.1.1.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\FastInfoset-1.2.13.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\guava-21.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\hppc-0.7.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\hunspell-native-libs-2.9.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\indriya-1.2.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\istack-commons-runtime-3.0.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-annotations-2.9.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-core-2.9.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jackson-databind-2.9.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jafama-2.1.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\javax.activation-api-1.2.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\javax.servlet-api-3.1.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-api-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-core-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jaxb-runtime-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jna-4.4.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\jsonic-1.2.11.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\language-detector-0.6.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\language-fr-4.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\languagetool-core-4.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\lucene-backward-codecs-5.5.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\lucene-core-5.5.4.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-fsa-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-fsa-builders-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-speller-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\morfologik-stemming-2.1.5.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\segment-2.0.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\slf4j-api-1.7.25.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\stax-ex-1.7.8.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\txw2-2.3.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\unit-api-1.0.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\uom-lib-common-1.1.jar.dir:C:\Users\Maxime\AppData\Local\Temp\tmplewadpgu\xgboost-predictor-0.3.0.jar.dir\ module-info.class
Loading class module-info and dependencies

Assertion failed!

Program: C:\Users\Maxime\Documents\writecontrol\LanguageTool-WebAssembly\cheerpj_1.3\bin\cheerpj

File: ClassFile.h, Line 213

Expression: false
`

English version compile except other file but the text spellching worked.

French Version get a runtime exception during the spellchecking. Will need to try to get the stack trace to understand what is going on.

maxiplay · 2018-12-30T16:31:23Z

I printed the stackTrace with catch.

try{

           List<RuleMatch> matchs = langTool.check("Les enfant aiment la soupe");
           System.out.println(matchs);

       }catch(Exception e){

           System.out.println(e);

           System.out.println(e.getMessage());
           System.out.println(e.getCause());

           StringWriter errors = new StringWriter();
           e.printStackTrace(new PrintWriter(errors));

           System.out.println( errors.toString());

        }

And get

java.lang.RuntimeException: java.lang.RuntimeException: Could not check sentence (language: French): 'Les enfant aiment la soupe'
	at org.languagetool.JLanguageTool.performCheck(Unknown Source)
	at org.languagetool.JLanguageTool.check(Unknown Source)
	at com.writecontrol.ltwa.Client.main(Unknown Source)
	at internal.internal(Unknown Source)
Caused by: java.lang.RuntimeException: Could not check sentence (language: French): 'Les enfant aiment la soupe'
	at org.languagetool.JLanguageTool$TextCheckCallable.getOtherRuleMatches(Unknown Source)
	at org.languagetool.JLanguageTool$TextCheckCallable.call(Unknown Source)
	... 4 more
Caused by: java.lang.UnsupportedOperationException: Unknown OS/arch: linux/cheerpj
	at org.languagetool.rules.spelling.hunspell.Hunspell.libNameBare(Unknown Source)
	at org.languagetool.rules.spelling.hunspell.Hunspell(Unknown Source)
	at org.languagetool.rules.spelling.hunspell.Hunspell.getInstance(Unknown Source)
	at org.languagetool.rules.spelling.hunspell.HunspellRule.init(Unknown Source)
	at org.languagetool.rules.spelling.hunspell.HunspellRule.match(Unknown Source)
	at org.languagetool.JLanguageTool.checkAnalyzedSentence(Unknown Source)
	... 6 more

That say this specific message relative to Hunspell lib

Unknown OS/arch: linux/cheerpj

danielnaber · 2018-12-30T16:51:19Z

Seems hunspell is still active. As a first step - which will remove spell checking altogether - you could try commenting out the line new FrenchCompoundAwareHunspellRule(messages, this, userConfig, altLanguages), in French.java

maxiplay · 2018-12-30T18:04:30Z

Yes it works. Thx !
In my app I use Ckeditor (Wysiwyg text editor) and the editor is able to use the internal spellcheck of browser. So I can combine grammar checking of language tool and spellchecking on the browser.
So thx for the support !

I will do now a check about performance.

maxiplay · 2018-12-30T20:01:46Z

I tested perfs with a text list and calculate the time to process
Performances are relatively slow relative to java "native" version.

What is interesting is with CheerpJ the CPU is never above 35% whereas with Java Native the CPU reach 75%.

Here the code for the test

    public static void main(String[] args) throws IOException {

        Language language = new French();
        

        JLanguageTool langTool = new JLanguageTool(language);

        List<String> listText = getListText();

        for (String text : listText) {

            check(text, langTool);

        }


    }


    public static void check(String text, JLanguageTool langTool) {

        try {

            long time = System.currentTimeMillis();

            List<RuleMatch> matchs = langTool.check(text);

            time = System.currentTimeMillis() - time;


            System.out.println(time);

        } catch (Exception e) {

            System.out.println(e);

            System.out.println(e.getMessage());
            System.out.println(e.getCause());

            StringWriter errors = new StringWriter();
            e.printStackTrace(new PrintWriter(errors));

            System.out.println(errors.toString());

        }
    }

konsoletyper · 2019-01-02T15:42:32Z

You better start with compilation to JavaScript, WebAssembly support is experimental in TeaVM. And, according to my experience, WebAssembly gives no advantages compared to JavaScript.
You better use most recent version of TeaVM (0.6.0-dev-650). You can even build your local TeaVM from source code, which will provide even more recent build.
You should use proper formatting (e.g. three backticks) when providing any output from Maven or TeaVM, so that it would be readable output.
You can describe how you actually using TeaVM (I don't have any idea of your setup), even share your project, so that I could reproduce the problem myself.

danielnaber · 2019-01-21T09:43:21Z

@maxiplay Are you still working on this? I'd be very interested in any updates.

maxiplay · 2019-01-23T16:26:28Z

I don't work anymore on it.
The last update of test project with performance tests is here

https://github.com/maxiplay/LanguageTool-WebAssembly

maxiplay · 2019-01-23T16:28:27Z

The conclusion is it can work without using Hunspell in pure java. And the performance are relatively slow because first you need to load a fake JSE + Language tool at each page loading. And performance during spellcheck are relatively slow too.

maxiplay mentioned this issue Dec 28, 2018

Help to build the better open source spellchecker in browser konsoletyper/teavm#373

Open

danielnaber added a commit that referenced this issue Dec 28, 2018

remove some code that teamVM might have issues with (#1323)

2230484

maxiplay mentioned this issue Dec 29, 2018

Impossible to use pack-jar option leaningtech/cheerpj-meta#57

Closed

maxiplay mentioned this issue Dec 30, 2018

LanguageTool - Compilation failed for certains class and runtime exception leaningtech/cheerpj-meta#58

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compiling to web assemby #1323

Compiling to web assemby #1323

maxiplay commented Dec 27, 2018

danielnaber commented Dec 27, 2018

danielnaber commented Dec 27, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018 •

edited

Loading

maxiplay commented Dec 28, 2018

danielnaber commented Dec 28, 2018

danielnaber commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

danielnaber commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 29, 2018

danielnaber commented Dec 29, 2018

danielnaber commented Dec 29, 2018

maxiplay commented Dec 29, 2018

danielnaber commented Dec 29, 2018

maxiplay commented Dec 29, 2018

danielnaber commented Dec 29, 2018

maxiplay commented Dec 29, 2018

maxiplay commented Dec 30, 2018

danielnaber commented Dec 30, 2018

maxiplay commented Dec 30, 2018

maxiplay commented Dec 30, 2018

konsoletyper commented Jan 2, 2019

danielnaber commented Jan 21, 2019

maxiplay commented Jan 23, 2019

maxiplay commented Jan 23, 2019

Compiling to web assemby #1323

Compiling to web assemby #1323

Comments

maxiplay commented Dec 27, 2018

danielnaber commented Dec 27, 2018

danielnaber commented Dec 27, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018 • edited Loading

maxiplay commented Dec 28, 2018

danielnaber commented Dec 28, 2018

danielnaber commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

danielnaber commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 28, 2018

maxiplay commented Dec 29, 2018

danielnaber commented Dec 29, 2018

danielnaber commented Dec 29, 2018

maxiplay commented Dec 29, 2018

danielnaber commented Dec 29, 2018

maxiplay commented Dec 29, 2018

danielnaber commented Dec 29, 2018

maxiplay commented Dec 29, 2018

maxiplay commented Dec 30, 2018

danielnaber commented Dec 30, 2018

maxiplay commented Dec 30, 2018

maxiplay commented Dec 30, 2018

konsoletyper commented Jan 2, 2019

danielnaber commented Jan 21, 2019

maxiplay commented Jan 23, 2019

maxiplay commented Jan 23, 2019

maxiplay commented Dec 28, 2018 •

edited

Loading