Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minted and lstlisting environment should be parsed as verbatim #77

Closed
pddg opened this issue Feb 18, 2024 · 6 comments · Fixed by #78
Closed

minted and lstlisting environment should be parsed as verbatim #77

pddg opened this issue Feb 18, 2024 · 6 comments · Fixed by #78

Comments

@pddg
Copy link
Contributor

pddg commented Feb 18, 2024

Environment

  • Node.js v20.11.0
  • @unified-latex/unified-latex-utils-parser v1.6.0

Background

minted and lstlisting is a popular environment in LaTeX to show source code.
It will contains some special chars for LaTeX and it should not be parsed normally.

https://www.overleaf.com/learn/latex/Code_Highlighting_with_minted
https://www.overleaf.com/learn/latex/Code_listing#Using_listings_to_highlight_code

In other words, they are one of a verbatim environment.

Expected behavior

unified-latex parses them as verbatim environment.

Actual behavior

unified-latex parses them normaly. It will parsed as normal latex source when the syntax of embedded source code matches LaTeX's one.

I uses following command to show AST:

#!/usr/bin/env node

import * as fs from "fs";
import { getParser } from "@unified-latex/unified-latex-util-parse";

const content = fs.readFileSync("sample.tex");
const parser = getParser();
const parsedAst = parser.parse(content.toString());

console.log(JSON.stringify(parsedAst, undefined, "  "));

And sample.tex is as follows:

\begin{minted}{latex}
    \section{section title}

    this is an embedded source.
\end{minted}
AST of sample.tex
{
  "type": "root",
  "content": [
    {
      "type": "environment",
      "env": "minted",
      "content": [
        {
          "type": "group",
          "content": [
            {
              "type": "string",
              "content": "latex",
              "position": {
                "start": {
                  "offset": 15,
                  "line": 1,
                  "column": 16
                },
                "end": {
                  "offset": 20,
                  "line": 1,
                  "column": 21
                }
              }
            }
          ],
          "position": {
            "start": {
              "offset": 14,
              "line": 1,
              "column": 15
            },
            "end": {
              "offset": 21,
              "line": 1,
              "column": 22
            }
          }
        },
        {
          "type": "whitespace",
          "position": {
            "start": {
              "offset": 21,
              "line": 1,
              "column": 22
            },
            "end": {
              "offset": 26,
              "line": 2,
              "column": 5
            }
          }
        },
        {
          "type": "macro",
          "content": "section",
          "position": {
            "start": {
              "offset": 26,
              "line": 2,
              "column": 5
            },
            "end": {
              "offset": 34,
              "line": 2,
              "column": 13
            }
          },
          "_renderInfo": {
            "breakAround": true,
            "namedArguments": [
              "starred",
              null,
              "tocTitle",
              "title"
            ]
          },
          "args": [
            {
              "type": "argument",
              "content": [],
              "openMark": "",
              "closeMark": ""
            },
            {
              "type": "argument",
              "content": [],
              "openMark": "",
              "closeMark": ""
            },
            {
              "type": "argument",
              "content": [],
              "openMark": "",
              "closeMark": ""
            },
            {
              "type": "argument",
              "content": [
                {
                  "type": "string",
                  "content": "section",
                  "position": {
                    "start": {
                      "offset": 35,
                      "line": 2,
                      "column": 14
                    },
                    "end": {
                      "offset": 42,
                      "line": 2,
                      "column": 21
                    }
                  }
                },
                {
                  "type": "whitespace",
                  "position": {
                    "start": {
                      "offset": 42,
                      "line": 2,
                      "column": 21
                    },
                    "end": {
                      "offset": 43,
                      "line": 2,
                      "column": 22
                    }
                  }
                },
                {
                  "type": "string",
                  "content": "title",
                  "position": {
                    "start": {
                      "offset": 43,
                      "line": 2,
                      "column": 22
                    },
                    "end": {
                      "offset": 48,
                      "line": 2,
                      "column": 27
                    }
                  }
                }
              ],
              "openMark": "{",
              "closeMark": "}"
            }
          ]
        },
        {
          "type": "parbreak",
          "position": {
            "start": {
              "offset": 49,
              "line": 2,
              "column": 28
            },
            "end": {
              "offset": 55,
              "line": 4,
              "column": 5
            }
          }
        },
        {
          "type": "string",
          "content": "this",
          "position": {
            "start": {
              "offset": 55,
              "line": 4,
              "column": 5
            },
            "end": {
              "offset": 59,
              "line": 4,
              "column": 9
            }
          }
        },
        {
          "type": "whitespace",
          "position": {
            "start": {
              "offset": 59,
              "line": 4,
              "column": 9
            },
            "end": {
              "offset": 60,
              "line": 4,
              "column": 10
            }
          }
        },
        {
          "type": "string",
          "content": "is",
          "position": {
            "start": {
              "offset": 60,
              "line": 4,
              "column": 10
            },
            "end": {
              "offset": 62,
              "line": 4,
              "column": 12
            }
          }
        },
        {
          "type": "whitespace",
          "position": {
            "start": {
              "offset": 62,
              "line": 4,
              "column": 12
            },
            "end": {
              "offset": 63,
              "line": 4,
              "column": 13
            }
          }
        },
        {
          "type": "string",
          "content": "an",
          "position": {
            "start": {
              "offset": 63,
              "line": 4,
              "column": 13
            },
            "end": {
              "offset": 65,
              "line": 4,
              "column": 15
            }
          }
        },
        {
          "type": "whitespace",
          "position": {
            "start": {
              "offset": 65,
              "line": 4,
              "column": 15
            },
            "end": {
              "offset": 66,
              "line": 4,
              "column": 16
            }
          }
        },
        {
          "type": "string",
          "content": "embedded",
          "position": {
            "start": {
              "offset": 66,
              "line": 4,
              "column": 16
            },
            "end": {
              "offset": 74,
              "line": 4,
              "column": 24
            }
          }
        },
        {
          "type": "whitespace",
          "position": {
            "start": {
              "offset": 74,
              "line": 4,
              "column": 24
            },
            "end": {
              "offset": 75,
              "line": 4,
              "column": 25
            }
          }
        },
        {
          "type": "string",
          "content": "source",
          "position": {
            "start": {
              "offset": 75,
              "line": 4,
              "column": 25
            },
            "end": {
              "offset": 81,
              "line": 4,
              "column": 31
            }
          }
        },
        {
          "type": "string",
          "content": ".",
          "position": {
            "start": {
              "offset": 81,
              "line": 4,
              "column": 31
            },
            "end": {
              "offset": 82,
              "line": 4,
              "column": 32
            }
          }
        }
      ],
      "position": {
        "start": {
          "offset": 0,
          "line": 1,
          "column": 1
        },
        "end": {
          "offset": 95,
          "line": 5,
          "column": 13
        }
      }
    }
  ],
  "position": {
    "start": {
      "offset": 0,
      "line": 1,
      "column": 1
    },
    "end": {
      "offset": 96,
      "line": 6,
      "column": 1
    }
  }
}

#37 seems to introduce support for macros of them, but not for environments.

@siefkenj
Copy link
Owner

Hmm. That is very strange. Are you using a current version or are you using the playground (I don't recall when I last updated the playground...)?

minted support was added a while ago to the grammar, so it should be working as a verbatim environment:

verbatim_minted_environment "verbatim minted environment"

@pddg
Copy link
Contributor Author

pddg commented Feb 18, 2024

Oh. I try to prepare a reproduction procedure.

@pddg
Copy link
Contributor Author

pddg commented Feb 18, 2024

@siefkenj
I am using v1.6.0, which seems to be the latest.
https://www.npmjs.com/package/@unified-latex/unified-latex-util-parse

I can reproduce it in my environment by following procedures:

mkdir repro-issue77
cd repro-issue77

cat << 'EOF' > package.json
{
  "name": "unified-latex-issue77",
  "version": "1.0.0",
  "type": "module",
  "scripts": {},
  "bin": {
    "tex2ast": "./tex2ast.js"
  },
  "dependencies": {
    "@unified-latex/unified-latex-util-parse": "1.6.0"
  }
}
EOF

cat << 'EOF' > tex2ast.js
#!/usr/bin/env node

import * as fs from "fs";
import { getParser } from "@unified-latex/unified-latex-util-parse";

const content = fs.readFileSync("sample.tex");
const parser = getParser();
const parsedAst = parser.parse(content.toString());

console.log(JSON.stringify(parsedAst, undefined, "  "));
EOF

cat << 'EOF' > sample.tex
\begin{minted}{latex}
    \section{section title}

    this is an embedded source.
\end{minted}
EOF

# I uses v20.11.0
node -v

npm install
npx tex2ast

I may have misunderstood how to initialize the parser. If so, please point it out.

@siefkenj
Copy link
Owner

It appears you're right! The code is in the grammar, but it was never tested. I will look into it.

@siefkenj
Copy link
Owner

Should be fixed in v1.6.1

@FelixZY
Copy link

FelixZY commented May 3, 2024

@siefkenj I'm still having problems in 1.7.1 using prettier-plugin-latex:

Example document:

\begin{minted}{csharp}
using System;

public class Program
{
  int a = 3;
  bool b = true;
  public static void Main()
  {
    // Write "Hello World" to the console
    Console.WriteLine("Hello World");
  }
}
\end{minted}

Expected

No change

Actual

\begin{minted}
	{csharp} using System;

	public class Program { int a = 3; bool b = true; public static void Main() { // Write "Hello World" to the console Console.WriteLine("Hello World"); } }
\end{minted}

Note: I have explicitly locked @unified-latex/unified-latex-ctan and @unified-latex/unified-latex-prettier to 1.7.1 (as well as 1.4.0, 1.6.0 and 1.6.1) to work around #96 . All of these versions seem affected and do not produce the expected output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants