-
Notifications
You must be signed in to change notification settings - Fork 109
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add the AST json file generation feature for external analysis purpose #840
base: master
Are you sure you want to change the base?
Conversation
Teal Playground URL: https://840--teal-playground-preview.netlify.app |
Hi! Thanks for the initiative in contributing! The AST format is internal to the compiler and hence it is not stable. For this reason I don't feel super comfortable in exposing it for third-party tools that would rely on it at this time. The output produced by |
Here is an example of what I want from a teal file (I used tutorial example). When you mean not stable, is that it is that the structure is changing often with new features coming? |
It means that is an internal data structure owned by the compiler. That means there is no public guarantee that it will stay stable across releases. If it changes often or rarely, that is less of the point; the point is that it can change at any time for any reason (e.g. internal refactoring) and it is not considered a bug or a breaking change. The I'm on my phone, sorry, I can't easily check out a zip file right now, but I'd be glad if you could summarize what is the kind of information you need (other than a full dump of the AST). Is it a list of public symbols exposed by a module and their types? From your description at the top of the message it sounds unlikely that you need every local variable in order to do language bindings. |
Also, generally spoken lua ends up embedded in another language like rust rather than the other way around. Wouldn't it be easier to generate the teal types and function definition based on what is defined in rust? I know a few projects that add typing information to mlua to be collected for definition file creation.(I made one of them which can directly create definition files for teal among other things). That sounds an easier way to do things than going from teal to rust. |
No I am not going from teal to rust but well embedding teal-lua script in Rust. First being able to get all teal variables (even local ones) and types will allow me to do hot reload. Second the interest of Teal in my project is also to be able to know which exact type of variables (object) and not any table so that I know which Rust object to bind with it and dynamically. It works well manually for now by making the list of variables and their types but now I need a way to get this automatically from teal... For example, from this teal code: local var1: number = 1 I need this: Right now I know how to get it from the AST, but maybe I need to dig in the functionalities of command "types"... |
Ok I think I understand a bit more the structure of the json return from the "types" command. Can you just tell me what means the second number from the symbols, 7 in this example (from a teal program with first line being "local testTypeNumber1 = 1.0"): As well, when we create local variables in a function, we find it in the symbols but I don't get how we can find in which function it is declared... |
The four entries are line, column, symbol name and symbol type code. The symbol type codes then map to entries in the
How are you scanning your functions for hot reloading? Are you using the Lua |
@jclavier44 Keep in mind that being a source-to-source compiler, sometimes Teal does create additional temporary variables in the output Lua that do not exist in the source Teal. (It does very little of that currently, mainly for providing compatibility code, such as |
I am currently working on locals backups and restore, but yes I plan to use lua injection like debug.getlocal and debug.setlocal to handle locals in functions/programs. |
That's available from Lua itself, via debug.getlocal. |
In order to use Teal in a project using mlua and rust, I need to extract all variables name and type from my teal files in order to automatically link lua with a typed language (Rust in my case).
I saw we have the AST result during generation in 'tl' file.
So I have added a function to convert it into json so that a 'FILE_NAME_ast.json' is generated if the config "gen_ast" is set at a number greater than 0. I have used a number for this config key in order to use it as a parameter for the max depth of the AST json generation (For the moment with gen_ast = 10 is getting enough info from the AST, but we will be able to increase it if need).
I will publish soon on my github account the node.js program which extract variables from Teal files and will add the link to "awesome-teal".