Consider merging eval, eval-ast and macroexpand #587

asarhaddon · 2021-08-21T18:15:44Z

Hello.

A suggestion may simplify the process, or at least most existing implementations as far as I have experimented.

Please be indulgent for the argumentation, english is not my native language.

EVAL should switch on the type of ast. The fact that list evaluation is normally a function call would be easyer to understand. In the current process, eval-ast is confusing because it has two distinct purposes:
- evaluate all elements of a sequence. This deserves a subprogram (with a better name) in some languages, but for most existing languages with native map, following the process and calling eval-ast leads to unneeded boxing and unboxing.
- the part of EVAL outside the apply phase. In languages with a case/switch statement or pattern matching, this leads to unneeded complexity because the call to eval-ast breaks the natural destructuring of ast.
the right place for macro expansion should be the apply phase. The only differences between a function call and a macro call are that the arguments are not evaluated and that EVAL (or better, a TCO continuation) is called on the result.
A strong argument for this suggestion is that, while debugging, macroexpand becomes redundant with a display of the EVAL arguments at the start of the TCO loop, which is quite useful and well understood far sooner. Moreover, the display would print two lines for each macro evaluation, while the current macroexpand does a loop before printing anything.

Mixing the suggestions, the EVAL structure becomes:

switch on the type of ast. All cases are already described in the process for non-list types and empty lists.
if ast is a non-empty list, evaluate its first argument (by the way, the result may be a macro even if the first argument is not a symbol).
switch on the type of the evaluated first argument. If it is a macro, apply it on the unevaluated arguments and continue the TCO loop. If it is a function, apply it on the evaluated arguments and return the result (on some implementations, there are different kinds of functions, but this only extends the switch cases and does not add complexity in the EVAL structure).

The text was updated successfully, but these errors were encountered:

…ages * Merge eval-ast and eval into a single conditional. * Print "EVAL: $ast" at the top of EVAL. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). Each change was already applied by some implementations. With the two last changes, macroexpand and quasiquoteexpand special forms are not needed anymore. Not tested locally: common-lisp, julia, io, logo, lua, matlab, vhdl, vimscript If macroexpand is kept: * an optional test should be added for `(macroexpand ())`, which currently fails on several implementations, * it should only apply one macro at a time. The purpose being debugging, hiding an iteration does not help. Moreover, the loop is currently untested (and probably wrong in nim).

…ages * Merge eval-ast and eval into a single conditional. * Print "EVAL: $ast" at the top of EVAL. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). Each change was already applied by some implementations. With the two last changes, macroexpand and quasiquoteexpand special forms are not needed anymore. If macroexpand is kept: * an optional test should be added for `(macroexpand ())`, which currently fails on several implementations, * it should only apply one macro at a time. The purpose being debugging, hiding an iteration does not help. Moreover, the loop is currently untested (and probably wrong in nim).

…ages * Merge eval-ast and eval into a single conditional. * Print "EVAL: $ast" at the top of EVAL. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). Each change was already applied by some implementations. With the two last changes, macroexpand and quasiquoteexpand special forms are not needed anymore. If macroexpand is kept: * an optional test should be added for `(macroexpand ())`, which currently fails on several implementations, * it should only apply one macro at a time. The purpose being debugging, hiding an iteration does not help. Moreover, the loop is currently untested (and probably wrong in nim). These tests fail: * objpascal: I have no such error with a more recent compiler * groovy: stack overflow during test of recursive functions. Did work after first push. * elm: most probably unrelated, this directory is unchanged

See issue kanaka#587. * Merge eval-ast and eval into a single conditional. * Print "EVAL: $ast" at the top of EVAL. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). * Remove macroexpand and quasiquoteexpand special forms. These tests fail: * objpascal: I have no such error with a more recent compiler * groovy: stack overflow during test of recursive functions. Did work after first push. * elm: most probably unrelated, this directory is unchanged While reformatting process/step*.txt, use a pseudo-syntax inspired by pattern matching. The changes have not been propagated to: * ada 2-9 * awk 2-9 * bash 2-9 * basic 2-A * bbc-basic 2-9 * c 2-9 * c.2 2-9 * chuck 2-A * clojure 2-9 * coffee 2-9 * common-lisp 2-9 * cpp 2-9 * crystal 2-A * cs 2-9 * d 2-9 * dart 2-9 * elisp 2-9 * elixir 2-9 * elm 2-9 * erlang 2-9 * es6 2-9 * factor 2-9 * fantom 2-9 * fennel 2-9 * forth 2-A * fsharp 2-9 * gnu-smalltalk 2-9 * go 2-9 * groovy 2-9 * guile 2-9 * haskell 2-9 * haxe 2-9 * hy 2-A * io 2-A * janet 2-A * java 2-9 * java-truffle 2-A * jq 2-A * js 2-9 * julia 2-9 * kotlin 2-9 * livescript 2-A * logo 2-9 * lua 2-A * make 2-9 * matlab 2-A * miniMAL 2-9 * nasm 2-9 * nim 2-9 * objc 2-A * objpascal 2-9 * ocaml 2-9 * perl 2-9 * perl6 2-9 * php 2-9 * picolisp 2-9 * pike 2-9 * plpgsql 2-A * plsql 2-A * powershell 2-A * ps 2-9 * python 2-9 * python.2 2-9 * r 2-9 * racket 2-9 * rexx 2-A * rpython 2-9 * ruby 2-9 * rust 2-9 * scala 2-A * scheme 2-9 * skew 2-9 * sml 2-9 * swift 2-A * swift3 2-A * swift4 2-A * swift5 2-A * tcl 2-9 * ts 2-9 * vala 2-A * vb 2-9 * vhdl 2-9 * vimscript 2-9 * wasm 2-A * wren 2-9 * xslt 2-A * yorick 2-9 * zig 2-A

See issue kanaka#587. * Merge eval-ast and eval into a single conditional. * Print "EVAL: $ast" at the top of EVAL. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). * Remove macroexpand and quasiquoteexpand special forms. These tests fail: * objpascal: I have no such error with a more recent compiler * groovy: stack overflow during test of recursive functions. Did work after first push. * elm: most probably unrelated, this directory is unchanged While reformatting process/step*.txt, use a pseudo-syntax inspired by pattern matching. The changes have not been propagated to: * ada 2-9 * awk 2-9 * bash 2-9 * basic 2-A * bbc-basic 2-9 * chuck 2-A * clojure 2-9 * coffee 2-9 * common-lisp 2-9 * cpp 2-9 * crystal 2-A * cs 2-9 * d 2-9 * dart 2-9 * elixir 2-9 * elm 2-9 * erlang 2-9 * es6 2-9 * factor 2-9 * fantom 2-9 * fennel 2-9 * forth 2-A * fsharp 2-9 * gnu-smalltalk 2-9 * go 2-9 * groovy 2-9 * guile 2-9 * haxe 2-9 * hy 2-A * io 2-A * janet 2-A * java 2-9 * java-truffle 2-A * jq 2-A * js 2-9 * julia 2-9 * kotlin 2-9 * livescript 2-A * logo 2-9 * lua 2-A * make 2-9 * matlab 2-A * miniMAL 2-9 * nasm 2-9 * nim 2-9 * objc 2-A * objpascal 2-9 * perl6 2-9 * php 2-9 * picolisp 2-9 * pike 2-9 * plpgsql 2-A * plsql 2-A * powershell 2-A * ps 2-9 * python.2 2-9 * r 2-9 * racket 2-9 * rexx 2-A * rpython 2-9 * rust 2-9 * scala 2-A * scheme 2-9 * skew 2-9 * sml 2-9 * swift 2-A * swift3 2-A * swift4 2-A * swift5 2-A * tcl 2-9 * ts 2-9 * vala 2-A * vb 2-9 * vhdl 2-9 * vimscript 2-9 * wasm 2-A * wren 2-9 * xslt 2-A * yorick 2-9 * zig 2-A

See issue kanaka#587. * Merge eval-ast and eval into a single conditional. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). * Print "EVAL: $ast" at the top of EVAL if DEBUG-EVAL exists in the MAL environment. * Remove macroexpand and quasiquoteexpand special forms. * Use pattern-matching style in process/step*.txt. c: also fixe defmacro so that it does not mutate functions. elm: the test failure is most probably unrelated. groovy: sometimes fail, but not on each rebuild. nasm: fails some new soft tests, but the issue is unreproducible when running the interpreter manually. objpascal: I cannot reproduce the error with a more recent compiler. ocaml: also remove metadata from symbols. ruby.2: step9 was failing with REGRESS=1, the fix is now backported from stepA.

See issue kanaka#587. * Merge eval-ast and eval into a single conditional. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). * Print "EVAL: $ast" at the top of EVAL if DEBUG-EVAL exists in the MAL environment. * Remove macroexpand and quasiquoteexpand special forms. * Use pattern-matching style in process/step*.txt. Unrelated fixes: c: also fixe defmacro so that it does not mutate functions. ocaml: also remove metadata from symbols. ruby.2: step9 was failing with REGRESS=1, the fix is now backported. Unresolved issues: c.2: unable to reproduce with gcc 11.12.0. elm: the directory is unchanged. groovy: sometimes fail, but not on each rebuild. nasm: fails some new soft tests, but the issue is unreproducible when running the interpreter manually. objpascal: unreproducible with fpc 3.2.2. ocaml: unreproducible with 4.11.1. perl6: unreproducible with rakudo 2021.09.

See issue kanaka#587. * Merge eval-ast and eval into a single conditional. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). * Print "EVAL: $ast" at the top of EVAL if DEBUG-EVAL exists in the MAL environment. * Remove macroexpand and quasiquoteexpand special forms. * Use pattern-matching style in process/step*.txt. Unrelated fixes: c: fix defmacro! so that it does not mutate functions. python.2: fix argv in step9 and add with-meta reader macro. ocaml: remove metadata from symbols. ruby.2: step9 was failing with REGRESS=1, the fix is now backported. Unresolved issues: c.2: unable to reproduce with gcc 11.12.0. elm: the directory is unchanged. groovy: sometimes fail, but not on each rebuild. nasm: fails some new soft tests, but the issue is unreproducible when running the interpreter manually. objpascal: unreproducible with fpc 3.2.2. ocaml: unreproducible with 4.11.1. perl6: unreproducible with rakudo 2021.09.

See issue kanaka#587. * Merge eval-ast and eval into a single conditional. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). * Print "EVAL: $ast" at the top of EVAL if DEBUG-EVAL exists in the MAL environment. * Remove macroexpand and quasiquoteexpand special forms. * Use pattern-matching style in process/step*.txt. Unresolved issues: c.2: unable to reproduce with gcc 11.12.0. elm: the directory is unchanged. groovy: sometimes fail, but not on each rebuild. nasm: fails some new soft tests, but the issue is unreproducible when running the interpreter manually. objpascal: unreproducible with fpc 3.2.2. ocaml: unreproducible with 4.11.1. perl6: unreproducible with rakudo 2021.09. Unrelated fixes: c: fix defmacro! so that it does not mutate functions. dart: fix recent errors and warnings logo: prevent macros from mutating functions python.2: fix argv in step9 and add with-meta reader macro. ocaml: remove metadata from symbols. ruby.2: step9 was failing with REGRESS=1, the fix is now backported. vb: prevent macro from mutating functions, allow (keyword :a). Improve the logo implementation. Encapsulate all representation in types.lg and env.lg, unwrap numbers. Replace some manual iterations with logo control structures. Reduce the diff between steps. Use native iteration in env_get and env_map Rewrite the reader with less temporary strings. Reduce the number of temporary lists (for example, reverse iteration with butlast requires O(n^2) allocations). It seems possible to remove a few exceptions: GC settings (Dockerfile), NO_SELF_HOSTING (IMPLS.yml) and step5_EXCLUDES (Makefile.impls) .

See issue kanaka#587. * Merge eval-ast and eval into a single conditional. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). * Print "EVAL: $ast" at the top of EVAL if DEBUG-EVAL exists in the MAL environment. * Remove macroexpand and quasiquoteexpand special forms. * Use pattern-matching style in process/step*.txt. Unresolved issues: c.2: unable to reproduce with gcc 11.12.0. elm: the directory is unchanged. groovy: sometimes fail, but not on each rebuild. nasm: fails some new soft tests, but the issue is unreproducible when running the interpreter manually. objpascal: unreproducible with fpc 3.2.2. ocaml: unreproducible with 4.11.1. perl6: unreproducible with rakudo 2021.09. Unrelated fixes: c: fix defmacro! so that it does not mutate functions. dart: fix recent errors and warnings logo: prevent macros from mutating functions minimal: prevent defmacro! from mutating functions ocaml: remove metadata from symbols. python.2: fix argv in step9 and add with-meta reader macro. ruby.2: step9 was failing with REGRESS=1, the fix is now backported. vb: prevent macro from mutating functions, allow (keyword :a). Improve the logo implementation. Encapsulate all representation in types.lg and env.lg, unwrap numbers. Replace some manual iterations with logo control structures. Reduce the diff between steps. Use native iteration in env_get and env_map Rewrite the reader with less temporary strings. Reduce the number of temporary lists (for example, reverse iteration with butlast requires O(n^2) allocations). It seems possible to remove a few exceptions: GC settings (Dockerfile), NO_SELF_HOSTING (IMPLS.yml) and step5_EXCLUDES (Makefile.impls) .

See issue kanaka#587. * Merge eval-ast and eval into a single conditional. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). * Print "EVAL: $ast" at the top of EVAL if DEBUG-EVAL exists in the MAL environment. * Remove macroexpand and quasiquoteexpand special forms. * Use pattern-matching style in process/step*.txt. Unresolved issues: c.2: unable to reproduce with gcc 11.12.0. elm: the directory is unchanged. groovy: sometimes fail, but not on each rebuild. nasm: fails some new soft tests, but the issue is unreproducible when running the interpreter manually. objpascal: unreproducible with fpc 3.2.2. ocaml: unreproducible with 4.11.1. perl6: unreproducible with rakudo 2021.09. Unrelated fixes: c: fix defmacro! so that it does not mutate functions. dart: fix recent errors and warnings forth: prevent macro from mutating functions. logo: prevent macros from mutating functions minimal: prevent defmacro! from mutating functions ocaml: remove metadata from symbols. python.2: fix argv in step9 and add with-meta reader macro. ruby.2: step9 was failing with REGRESS=1, the fix is now backported. vb: prevent macro from mutating functions, allow (keyword :a). Improve the logo implementation. Encapsulate all representation in types.lg and env.lg, unwrap numbers. Replace some manual iterations with logo control structures. Reduce the diff between steps. Use native iteration in env_get and env_map Rewrite the reader with less temporary strings. Reduce the number of temporary lists (for example, reverse iteration with butlast requires O(n^2) allocations). It seems possible to remove a few exceptions: GC settings (Dockerfile), NO_SELF_HOSTING (IMPLS.yml) and step5_EXCLUDES (Makefile.impls) .

See issue kanaka#587. * Merge eval-ast and eval into a single conditional. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). * Print "EVAL: $ast" at the top of EVAL if DEBUG-EVAL exists in the MAL environment. * Remove macroexpand and quasiquoteexpand special forms. * Use pattern-matching style in process/step*.txt. Unresolved issues: c.2: unable to reproduce with gcc 11.12.0. elm: the directory is unchanged. groovy: sometimes fail, but not on each rebuild. nasm: fails some new soft tests, but the issue is unreproducible when running the interpreter manually. objpascal: unreproducible with fpc 3.2.2. ocaml: unreproducible with 4.11.1. perl6: unreproducible with rakudo 2021.09. Unrelated changes: Reduce diff betweens steps. Prevent defmacro! from mutating functions: c forth logo miniMAL vb. dart: fix recent errors and warnings ocaml: remove metadata from symbols. Improve the logo implementation. Encapsulate all representation in types.lg and env.lg, unwrap numbers. Replace some manual iterations with logo control structures. Reduce the diff between steps. Use native iteration in env_get and env_map Rewrite the reader with less temporary strings. Reduce the number of temporary lists (for example, reverse iteration with butlast requires O(n^2) allocations). It seems possible to remove a few exceptions: GC settings (Dockerfile), NO_SELF_HOSTING (IMPLS.yml) and step5_EXCLUDES (Makefile.impls) .

See issue #587. * Merge eval-ast and eval into a single conditional. * Expand macros during the apply phase, removing lots of duplicate tests, and increasing the overall consistency by allowing the macro to be computed instead of referenced by name (`((defmacro! cond (...)))` is currently illegal for example). * Print "EVAL: $ast" at the top of EVAL if DEBUG-EVAL exists in the MAL environment. * Remove macroexpand and quasiquoteexpand special forms. * Use pattern-matching style in process/step*.txt. Unresolved issues: c.2: unable to reproduce with gcc 11.12.0. elm: the directory is unchanged. groovy: sometimes fail, but not on each rebuild. nasm: fails some new soft tests, but the issue is unreproducible when running the interpreter manually. objpascal: unreproducible with fpc 3.2.2. ocaml: unreproducible with 4.11.1. perl6: unreproducible with rakudo 2021.09. Unrelated changes: Reduce diff betweens steps. Prevent defmacro! from mutating functions: c forth logo miniMAL vb. dart: fix recent errors and warnings ocaml: remove metadata from symbols. Improve the logo implementation. Encapsulate all representation in types.lg and env.lg, unwrap numbers. Replace some manual iterations with logo control structures. Reduce the diff between steps. Use native iteration in env_get and env_map Rewrite the reader with less temporary strings. Reduce the number of temporary lists (for example, reverse iteration with butlast requires O(n^2) allocations). It seems possible to remove a few exceptions: GC settings (Dockerfile), NO_SELF_HOSTING (IMPLS.yml) and step5_EXCLUDES (Makefile.impls) .

kanaka · 2024-08-06T17:39:20Z

I think this is resolved now with the merging of #592. Remaining implementations are listed in #657

This incorporates various updates to the diagrams. * The largest change is the incorporation of the new process where eval_ast is integrated into eval and the macroexpand function and has been incorporated directly into the eval TCO loop and no longer requires a separate function or special form. For the most part, this change was discussed in #587 and implemented in #592. Instead of a special form, there is now a "apply macro" box in the eval block. The "apply macro" box has a TCO line (but not env creation line unlike the "apply" box. * Change the "eval_ast" sub-block to "evaluate" and convert the list of evaluated items to individual blocks. Add a "list" block and connect this to the "apply" block (rather than connecting eval_ast to eval). * Change the "symbol lookup" arrow to point to the "symbol" box in the "evaluate" block instead of pointing to the "evaluate" block. * Update the "create env" arrows to point to the child env box instead of the root/REPL env box. * Add try/catch TCO line. * Remove the `or` macro and `gensym` function from stepA since this has been dropped for a while.

asarhaddon mentioned this issue Sep 28, 2021

Implement suggestions from issue #587 in stepA of several languages #592

Closed

kanaka closed this as completed Aug 6, 2024

OldLiu001 mentioned this issue Aug 9, 2024

First VBScript implementation #624

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consider merging eval, eval-ast and macroexpand #587

Consider merging eval, eval-ast and macroexpand #587

asarhaddon commented Aug 21, 2021

kanaka commented Aug 6, 2024

Consider merging eval, eval-ast and macroexpand #587

Consider merging eval, eval-ast and macroexpand #587

Comments

asarhaddon commented Aug 21, 2021

kanaka commented Aug 6, 2024