Unable to compile Felix #171
Comments
|
Oh dear. Ocaml messes it up again. I'm using this: The OCaml toplevel, version 4.13.0+dev0-2020-10-19 You have a hyper new version 5.x Ocaml which has changed things I have not noticed because I'm running and older version. BTW yes opam doesn't seem to work for me either. The compiler is complaining about this: on the second last line. It's complaint is very strange. The symbol This can be fixed by just using the I cannot fix this without building Ocaml version 5.x because I need to reproduce the error and then see if I can fix it. The problem has to be fixed in the lexer generator program to generate a valid Ocaml 5.1 lexer. The other issue, the alert on module Unix is unrelated and the advice given is rubbish since Felix is not using any of that stuff: it builds using the compiler directly. No dune, ocamlfind, or any other crap. This is a pain, because if I adjust the build script to work for Ocaml 5 it won't work for Ocaml 4. Grr. OK .. Ill try to get Ocaml 5 now. |
|
The github website says this: "Owing to the large number of changes, the initial 5.0 release will be more experimental than usual. It is recommended that all users wanting a stable release use the 4.14 release which will continue to be supported and updated while 5.0 reaches feature and stability parity. Similarly, if you need one of the ports not yet supported in the 5.0 release you must use the 4.14 release." You have 5.1 which is already more advanced than these comments suggest. Wonder where you got it? Did you build from the github repo? |
|
OOps .. wrong that file is editable ... |
|
OK tried a fix .. works with Ocaml 4.x. Used "set" function instead of assignment syntax. |
Thank you. So.....I should re-download this repo and try it again?
I reported an issue to the opam repo, and suggested they update their instructions. I got yelled at and the ticket was closed for "being offensive". I hadn't been rude. I've noticed a lot of open source maintainers are like that: any criticism whatsoever is considered offensive, and the ticket is closed.
Yes. I had to. Bloody opam doesn't work. What other option is there, other than to build it by hand? Believe me, I would much rather just download a release and install it. But Felix hasn't released since 2019, and OCaml doesn't appear to have any options other than opam - which doesn't work. |
|
Btw, your website is down. Just FYI. |
|
I'm afraid it still will not compile: |
|
I know the website is down. The owner is uncontactable. I need another server but i cannot afford to rent one. It has to be a raw machine preferably running some kind of Unix because I'm using flx_web, a web server written in Felix that automatically displays felix packages you can see in src/packages of the repo, and automatically hyperlinks Felix and C++. Ok I fixed that next bug I think (the patch compiles on Ocaml 4 at least). |
|
FYI: there are lots of options other than opam. For example with Debian you can apt-get ocaml-nox. You can also build the 4.14 branch of the repo. I personally had so much trouble with opam I just build from source myself. My build is actually from the trunk, but it's old I guess. |
|
Opam should work. I'm on MacOS and it used to be fine. However when I try to get other packages it refuses and thats actually the packages being out of date and not building, rather than opam failing. The problem with Ocaml is the core developers are really only interested in the type system not industrial strength tooling, and they're idiot copy-left anti-microsoft and refuse to support Windows. So if you have trouble on a Linux platforms .. OMG don't try to build it on Windows. Felix itself works fine on Windows once Ocaml is built and you can actually build the compiler but, even though i have a Windows box, I cannot maintain the Windows binding because my stupid iPhone I used for internet won't accept a wifi connection from it. I can rewrite most of the compiler in Felix or even (ARGGGGHHHH in C++) but not the parser. Dypgen is the best parser ever written, but it's not maintained. Felix relies on its ability to dynamically update itself on the fly. |
|
Compilation still fails: I also tried using To be honest, I'm surprised you don't do regular releases as AppImages or Docker containers. Would make it much easier. |
|
I have no idea how to stuff around with Docker or AppImages (never even heard of that). That's just one more layer of idiot dependencies I am trying to avoid. At the moment to get Felix moving you need, obviously, C++, you need Ocaml, and you need Python 3.x and that's ALL you need (well apart from basic tools like make and shell and git). Everything else, such as ocaml packages like Dypgen, and C++ packages like Google RE2 are bundled directly in the system. It looks like the Debian build is also way WAY out of date which is not surprising. Probably the Ubuntu build is more up to date. The problems we're having here all stem from one really bad change to Ocaml which is the result of a long running stupidity: in Ocaml strings were originally mutable. The problem was the compiler assumed string literals were immutable .. so the type system as unsound. To fix this, they added a new type bytes which was mutable, and made strings immutable, which broke every program that used mutable string buffers .. including other standard Ocaml library packages like Lexing. Lex buffers are intrinsically mutable (they get reloaded obviously), but when you change them to bytes that breaks everything using it. I fixed a whole lot of stuff already. I have my own Dypgen clone inside Felix. But whilst the older compilers have some tolerance, it seems the 5.x family doesn't. They should have fixed the dang compiler, not messed with every user program in existence that used strings. |
|
ok i'm building Ocaml from trunk now. |
|
AppImage is a way to bundle your product and its dependancies into a single executable. I use it myself, for the Linux edition of my product. |
|
Will updating the Felix sources to be compatible with OCaml 5 break compatibility with OCaml 4? |
No idea. Possibly. Felix doesn't require any C modules in Ocaml, but the 5.x family is a totally rewrite of the run time, especially the garbage collector, so Ocaml can finally run multicore. It's really hard to know what might break. It probably still has bugs. The problems will be mainly on Windows. It uses a mingw variant to build the Ocaml run time, but Felix itself doesn't bother supporting crud, and only supports the Microsoft compiler. Unfortunately C++ ABI is dependent on the compiler as well as the OS, unlike C. So basically a Windows developer either needs to have the right mingw installed AS WELL as Visual Studio, or they just need to use Bash for Windows which uses Ubuntu. So the answer is .. probably going to Ocaml 5.x will make the system unbuildable on Windows but I have no way to find out. Microsoft owns GitHub but I cannot get their own CI build services to run their own compiler. Ideally I'd build Felix on GitHub for Linux, Mac, and Windows, and you could just install a tarball and bobs your uncle. I can't use Applimages .. my box is an old Macbook Pro running the latest MacOS. |
|
Okay, now I'm confused. I cloned OCaml 4.14, built and installed it. Then I cloned Felix again, and managed to get most of the way through compiling it. Then it failed: If I'm using the exact version of OCaml recommended for building Felix............I don't understand why it's still broken. |
|
Nothing to do with Ocaml by the look of it. That's an error in Felix building the compiler driver flx, which is written Felix. And its due to change I did two days ago. I missed that bug. Here's why: The first time you build Felix, you build a bootstrap version using script written in Python. The build tool is called There is a BUG in bootflx I didn't catch because normally I just rebuild Felix using Felix rather than rebuilding from scratch. I'll fix that now. |
|
I have to wonder though......as interesting as Felix looks, and as helpful as you are being......if it's dependant on a broken compiler for a broken language (which also has unfriendly maintainers!) and is also dependant on a parser that is no longer being maintained.............is it really wise for me to write software in this language? |
|
The problem with the parser is due to Ocaml team breaking stuff. I wish I had written the compiler in Haskell now but I don't know Haskell. But once the parser is built it should be fine. I don't need to ever change the parser except when Ocaml team breaks it. The Felix parser dynamically loads the grammar using Dypgen. The only hassle might be if Ocaml breaks their Marshal module because Felix uses that to save and reload the parsing automaton, to avoid rebuilding the parser every time you run the compiler. So the only real pain is getting the compiler to actually build. Once it builds, it should be fine because it doesn't depend on anything nasty at all. It's all very basic Felix file to C++ file stuff. The last bug you found is actually because Felix is working fine: that's an error in Felix code you found and it's my bad and already fixed. |
|
I'm going to try to build Felix with Ocaml 5.x trunk. The problem is without opam, its very hard to switch to another Ocaml version and make sure it builds with 4.14 as well. Opam can do this with a single command. You are going to find bugs in Felix espeically if you push it and use some of the latest experimental stuff. These will be like:
In the build, you may find a couple of regression test cases fail. In particular if you don't have the SDL2 multi-media stuff installed, or it isn't where Felix expects to find it, the GUI test cases will fail. That's not a bug, it's just because you haven't got SDL2 support: SDL2 is not bundled with Felix. The build system doesn't detect if you have a valid install of SDL2. |
|
Still fails with 4.14: |
Thank you. I appreciate that. |
|
OK that's interesting. This is a run time error in the compiler. It's saying the standard Ocaml module Filename, function chop_suffix is given an invalid string eg a filename without a suffix to chop. Which is a bit weird because it works for me using 4.13 and Linux and Mac use the same Unix filenames for everything for .. will suffices. I sure wish I knew what caused the error exactly. Unfortunately dynamic exception handling is a very bad idea (which is why Felix doesn't have it). However:
is the only use of it. Stuff knows why I do not get this error. I'll have to add a special test to trap the problem for you to run. |
|
LOL! The result of that operation is never used anyhow. |
|
Just on the "should you use Felix to write software" issue: my assessment is that it should be good to write support tools. For production software I'd be more wary: you need to gain confidence not only in the system but that you will get support when there's an issue. The flip side is you're obviously an expert developer so as you become familiar with the system you'll be able to resolve a lot of issues yourself and also help modify and extend the system's capabilities. Try doing that with other software, open source or not! So the issues to consider are at least partly political. You have to trade off some problems arising due to the small number of users against the fact that a small number of users also means YOU get the maximum support to get the system to do things that YOU want to do .. and you'll find the flexibility in the system .. such as user defined grammar and the ability to bind to C++ almost seamlessly .. increasingly useful for doing what Felix was designed for: automation. If you're doing something repeatedly enough to recognise "engineering patterns" you should be able to get your language to abstract away the boilerplate, and if you can't, you can then ask, how you would modify that system in order to be able to do so. There are a lot of papers on different kinds of polymorphism and a lot of the systems are well described with formal semantics etc, so quite a lot can be implemented without too much effort: the real problem is deciding which things to implement. One of the things you cannot do easily with Felix at the moment is write real-time programs. This is because it uses a garbage collector. The fix for that is not a minor patch: I'm woking on real time CSP and the design seems to work, but getting Felix to use it AS WELL as the GC is a hard problem: the real-time channels and continuation objects are quite different (ref counted instead of garbage collected). That's an example of something that cannot be made to work in 24 hours. |
|
Did you fix the thing that isn't used? Should I retry the compilation?
Thanks. I've had quite a bit of experience in certain areas of development, but I'm not sure I'd call myself an expert. What makes you think so?
Absolutely! This is one reason I detest C++ with a passion.
Oh yes? Wasn't aware of that facility. It's hard enough even to find a language with UFCS, or with user-definable infix operators. Am I to assume Felix has both?
Not 100% sure what you mean by that. Just that the GC takes time to operate, causing tiny delays every so often, like .NET does? It's not often that this becomes a major issue, in my experience. |
|
If you look in The bootstrap grammar that is hard baked into Felix consists of core comment lexing and a the "grammar of the grammar specifications". So the bootstrap parser cannot parse Felix, it can only parse Felix grammar specifications. It parses these specifications into modules called DSSLs or Domain Specific Sub-Lanuages" which when opened cause the parse automaton to be rebuilt on the fly (that is at compile time) and then THAT parser is used to parse your "Felix code". So if you can write an EBNF grammar for something you want to do, provided it fits in with the existing grammar sanely, you just extended the language syntax. TO extend the semantics, we look further and the first thing we see is that your grammar productions have action codes that say how to build an AST from productions .. and you find NO that is not actually an AST code .. that is executable R5RS Scheme. So you can write "grammar macros" in Scheme! The Scheme, when executed, must generate an S-expression which the Ocaml module The actual S-expressions the Ocaml sex2flx module recognises aren't documented separately from the actual code. Felix uses a very specific abstract machine model represented with Ocaml code and all the complexity reduces down to this model. Basically has Executable instructions which have arguments that can be Executable Expressions, and which has specific Types and few other things. The Ocaml names are
and this virtual machine code is stashed into symbol tables. |
|
SO: a real time program is one which must complete in a fixed amount of time (no matter what the inputs are) OR more commonly, a program which loops forever doing a job with the same property as above. The time in seconds that the code has to complete is a hard bound. If the program fails to do this, the program has failed and the code is incorrect (in that it failed to meet its temporal boundary condition). For example, my use case: on an Apple platform the operating system is able to deliver sound data in blocks in real time, where the sound say comes from the computer microphone, and it can deliver sound data to the speaker in real time too. So any audio processing you are doing on a block of data must complete before the next one is delivered. Sound is sampled at 44.1KHz and you can choose a block size, typically 128 or 256 floating point numbers, which determines the period of a processing cycle. So you have to ask the OS to give you a real time thread and promise you a certain amount of CPU time in any given time span the length of the period. On MacOS you can do this. You can in Unix too if you have RT-extensions. There are RT versions of Linux too. What this means is that a GC is fine if it can use an amount of time every cycle which is bounded so you can allow for it. And that's more or less impossible with the Felix GC. It's a standard mark/sweep world/stop algorithm. World stop is a real pain because there is no portable way to stop a thread. On MacOS there is no way at all. On Linux, you can send a signal but that's close to useless because the GC thread has to be able to capture the machine stack because that with the machine registers pushed onto it, is the root for the collector. |
|
yeah i made a patch but only you can tell if it's actually a fix .. |
I'm afraid I understood very little of that post. Parsers and grammars have always confused me. Recently I tried writing a toy Lisp parser for experience. I found it very difficult and confusing.
If it can't be done.....how is Felix available for those platforms? On second thought.....I probably wouldn't understand the explanation. Compiler design is a couple layers below my experience level. |
|
Example of grammar: That's pretty easy to understand except that there is one more piece of information you need: which defines some of the precedences needed. It's saying multiplication is bound tighter syntactically than addition so This definition, for your convenience, is already in the standard library. The fact Felix doesn't know how to add integers is a major powerup! It forces the developer, that's me, to find a way to let the user do it. If the user can do that, then the user can do a whole lot of other things the same way. |
|
I got Felix compiled and installed, but I'm getting a strange error when I run it: The |
|
You're running as root?? Ouch. I'd advise against that but you can just create it: |
|
I mean, the error occurs irrespective of the existence of the directory. And yeah, I run as root. I figure I can accidentally delete all my files just as easily as a limited user, because they're owned by me. Only additional risk of being root is that I might break the operating system. And if I do, I can just run a massive Ruby script I wrote that reinstalls the whole machine from the ground up... |
|
Running as a limited user doesn't help with the |
|
The test line is: That should work and print "Hello World". You can also do which tells where Felix is installed if it isn't the default. Since I don't install it, the install directory is set to the build target in the repository clone. To make that work, however, you need to set |
|
Thanks for the help. I will play about with it and see how I get on. |
|
What exactly is the error? Below is correct and expected: |
|
I mean, the error I posted ( It was like this: See? Running |
|
I would have expected the usage message to be printed every time no arguments were supplied. And the cache warning to only be printed once. |
|
So: Felix puts EVERYTHING into the cache. So you can run Notice, I have a shared library there not an executable. You'll find a If you want an executable: If you want it in the current directory: The The Sometimes .. stuff is so radically changed the tool gives up and rebuilds everything from scratch. The way this is forced is by .. deleting the cache! |
|
Oh, the usage message should be printed every time, it's just a quirk that the cache was deleted because it was out of date (even though there's nothing in it .. that's precisely what makes it out of date). Since Felix autocompiles everything, things like the parser automaton are also built on demand. And it will be rebuilt if you change the grammar! So all this autobuilding (without any actual felix program to compile) has somehow become a higher priority than printing the usage message. When the system gets changed and/or rebuilt, running the hello world program is the authoritative way to finalise the build. You don't want to reparse and rebind the standard library every time! EDIT: this one is a "Wont Fix". :-) |
So....running the same source again should be quicker the second time? Felix skips the compilation phase if the source hasn't changed? Are Is it possible to compile against
The failure to print the usage? Or....? |
|
Also, what is a |
|
Yes, if you The *.par file is a cache of the AST produced by parsing the file. There are other cached files in $HOME/.felix/cache/text. Not sure splitting the cache into binary and text was a good idea. You will see the generated C++: The .dep and .d files record dependency information. The *.resh file contains the list of packages required to C++ compile and link. The *.rtti file contains tables the compiler generates which tell the garbage collector where all the pointers in a type are. The generated C++ is split into a few files: the _ctors file contains constructors. The If you generate an executable it is already linked to ALL the Felix run time and startup code, so it is entirely portable (except for any system enforced shared libraries. On MacOS the C library is always dynamically linked). |
|
Linux platforms always enforce Given that Felix is transcoded to CPP before being compiled, I would expect this to be reasonably straightforward to implement. |
|
Is that Felix does allow you to specify compiler and linker switches on the Relevant here are: although note these are not well tested. The spirit of the system is that you can use the exact same shell commands on Linux, MacOS and Windows to do the exact same thing, so using these switches will break that property. There MAY be another way to do this however: using the metadata database query system. Something like The advantage of the meta data method is it doesn't require changing the build tool The In the flx model, instances of this interface are constructed by code in a plugin. As it happens Felix can The point is the system is deliberately restricted in its capabilities so they're well defined, and the interactions between them are well defined, AND, those operational definitions are platform independent (in that they work on all current toolchains). So the whole idea is to avoid using anything platform specific. However as mentioned Windows ALSO has a static link C library. MacOS does not deliberately. Apple uses Apart from the bad interaction with shared libraries, your executable would be very platform dependent, it would not work on different versions of Linux that used different OS calling conventions. It breaks the tradition for Unix that the OS interface is via C, not assembler. In any case you can try using the -L and -l switches. I can't because I run MacOS. |
|
BTW there is a mailing list, send a message to it I'll add you: [email protected] |
|
Done! |
|
Well...that didn't seem to work... |
|
Your email is going to the group. But you're not subscribed. Try to subscribe. I seem to be the owner of the group but I am not getting the "Add Members" option i should be. I had an issue with passwords etc before. |
|
Use group: [email protected] because I can manage that. |
Having installed
ocaml(manually, because theopaminstructions on their website don't work), I followed the Felix installation instructions:The build failed, and I got a bunch of error messages (and warnings!)
Environment:
The text was updated successfully, but these errors were encountered: