Skip to content

Commit

Permalink
Merge pull request #98 from siefkenj/pretext
Browse files Browse the repository at this point in the history
Report unsupported macros by katex and expand user-define commands

* Break Up LaTeX Source on Section Macros (#101)

Break up a latex source by divisions (`\section` , `\subsection`, etc.) and turn those divisions into environments. E.g., `\section{foo}XXX\section{bar}YYY` becomes 

```
\begin{_section}[foo]XXX\end{_section}
\begin{_section}[bar]YYY\end{_section}
```

This is an intermediate step in the PreTeXt conversion.

* [WIP] author-info (#100)

Add function to extract information from the `\author{...}` tag that appears before `\begin{document}...`

* Fix the skipped test cases in the PreTeXt package (#103)

Necessary macro subs and environment subs to create valid PreTeXt from LaTeX.

* Added LaTeX to PreTeXt Plugin to the CLI package (#113)

* started adding pretext package to cli

* Upgrade deps

* Rebuild documentation

---------

Co-authored-by: Renee <[email protected]>
Co-authored-by: Bo-Y-G <[email protected]>
  • Loading branch information
3 people authored Aug 20, 2024
2 parents a05437d + 0aabe43 commit d29363b
Show file tree
Hide file tree
Showing 51 changed files with 4,272 additions and 1,112 deletions.
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# unified-latex Changelog

### v1.8.0

- Added initial PreTeXt conversion support
- Upgraded deps
- Added `amsart` macros
- Consume the whitespace after special character macros when expanding ligatures. For example `\o y` produces `øy` instead of `ø y`
- Fix signatures of `\hyphenation`


### v1.7.1

- Types fix for `@unified-latex/unified-latex-types`
Expand Down
1,985 changes: 917 additions & 1,068 deletions package-lock.json

Large diffs are not rendered by default.

22 changes: 11 additions & 11 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
"trailingComma": "es5"
},
"devDependencies": {
"@types/node": "^20.12.12",
"@types/node": "^20.16.1",
"@types/prettier": "^2.7.3",
"@types/shelljs": "^0.8.15",
"esbuild": "^0.21.2",
Expand All @@ -44,16 +44,16 @@
"remark-gfm": "^3.0.1",
"remark-parse": "^10.0.2",
"remark-stringify": "^10.0.3",
"rimraf": "^5.0.7",
"rimraf": "^6.0.1",
"shx": "^0.3.4",
"ts-morph": "^22.0.0",
"typedoc": "^0.25.13",
"typescript": "^5.4.5",
"vite": "^5.2.11",
"vite-plugin-dts": "^3.9.1",
"vite-plugin-static-copy": "^1.0.4",
"vitest": "^1.6.0",
"wireit": "^0.14.4"
"ts-morph": "^23.0.0",
"typedoc": "^0.26.6",
"typescript": "^5.5.4",
"vite": "^5.4.2",
"vite-plugin-dts": "^4.0.3",
"vite-plugin-static-copy": "^1.0.6",
"vitest": "^2.0.5",
"wireit": "^0.14.7"
},
"dependencies": {
"@types/color": "^3.0.6",
Expand All @@ -75,7 +75,7 @@
"unified-engine": "^10.1.0",
"unified-lint-rule": "^2.1.1",
"unist-util-position": "^4.0.4",
"vfile-reporter-json": "^3.2.0",
"vfile-reporter-json": "^3.3.0",
"vfile-reporter-position": "^0.1.7",
"vfile-reporter-pretty": "^6.1.1"
},
Expand Down
6 changes: 6 additions & 0 deletions packages/unified-latex-cli/libs/unified-args/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import {
} from "unified-engine";
import { unifiedLatexToHast } from "@unified-latex/unified-latex-to-hast";
import { unifiedLatexToMdast } from "@unified-latex/unified-latex-to-mdast";
import { unifiedLatexToPretext } from "@unified-latex/unified-latex-to-pretext";
import { options, Options } from "./options";
import { availableLints } from "../lints";
import { statsJsonPlugin, statsPlugin } from "../stats";
Expand Down Expand Up @@ -168,6 +169,11 @@ export function unifiedArgs(cliConfig: Options) {
config.plugins.push([remarkStringify as any]);
}

if (config.pretext) {
config.plugins.push([unifiedLatexToPretext]);
config.plugins.push([prettyPrintHtmlPlugin]);
}

/**
* Handle complete run.
*
Expand Down
2 changes: 2 additions & 0 deletions packages/unified-latex-cli/libs/unified-args/options.ts
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ export function options(flags: string[], configuration: Options) {
),
html: config.html,
markdown: config.markdown,
pretext: config.pretext,
} as EngineOptions & {
help: boolean;
helpMessage: string;
Expand All @@ -172,6 +173,7 @@ export function options(flags: string[], configuration: Options) {
macro: { name: string; signature: string }[];
html: boolean;
markdown: boolean;
pretext: boolean;
};
}

Expand Down
7 changes: 7 additions & 0 deletions packages/unified-latex-cli/libs/unified-args/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,13 @@ export const schema: Option[] = [
type: "boolean",
default: false,
},
{
long: "pretext",
description:
"Convert the output to PreTeXt. Note, you should expand/replace any macros not recognized by the converter",
type: "boolean",
default: false,
},
{
long: "stdout",
description: "[Don't] write the processed file's contents to stdout",
Expand Down
8 changes: 4 additions & 4 deletions packages/unified-latex-ctan/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ import the `.js` file. To explicitly access the commonjs export, import the `.cj

# Constants

| Name | Type | Description |
| :---------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------- |
| `environmentInfo` | `{ cleveref: EnvInfoRecord; exam: EnvInfoRecord; geometry: EnvInfoRecord; hyperref: EnvInfoRecord; latex2e: EnvInfoRecord; ... 10 more ...; multicol: EnvInfoRecord; }` | Info about the environments for available ctan packages. `latex2e` contains&#xA;the standard environments for LaTeX. |
| `macroInfo` | `{ cleveref: MacroInfoRecord; exam: MacroInfoRecord; geometry: MacroInfoRecord; hyperref: MacroInfoRecord; latex2e: MacroInfoRecord; ... 10 more ...; multicol: MacroInfoRecord; }` | Info about the macros for available ctan packages. `latex2e` contains&#xA;the standard macros for LaTeX. |
| Name | Type | Description |
| :---------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------- |
| `environmentInfo` | `{ amsart: EnvInfoRecord; cleveref: EnvInfoRecord; exam: EnvInfoRecord; geometry: EnvInfoRecord; hyperref: EnvInfoRecord; ... 11 more ...; multicol: EnvInfoRecord; }` | Info about the environments for available ctan packages. `latex2e` contains&#xA;the standard environments for LaTeX. |
| `macroInfo` | `{ amsart: MacroInfoRecord; cleveref: MacroInfoRecord; exam: MacroInfoRecord; geometry: MacroInfoRecord; hyperref: MacroInfoRecord; ... 11 more ...; multicol: MacroInfoRecord; }` | Info about the macros for available ctan packages. `latex2e` contains&#xA;the standard macros for LaTeX. |
229 changes: 229 additions & 0 deletions packages/unified-latex-to-pretext/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
<!-- DO NOT MODIFY -->
<!-- This file was autogenerated by build-docs.ts -->
<!-- Edit the docstring in index.ts and regenerate -->
<!-- rather than editing this file directly. -->
# unified-latex-to-pretext

## What is this?

Functions to convert `unified-latex` Abstract Syntax Tree (AST) to a XAST (xml-like)
tree in the [PreTeXt](https://pretextbook.org/) format.

## When should I use this?

If you want to convert LaTeX to PreTeXt for further processing with the PreTeXt compiler.

## Controlling the PreTeXt output

This plugin comes with presets for several common LaTeX macros/environments, but you probably want to
control how various macros evaluate yourself. For example, you may have used `\includegraphics` with `pdf`s
in your LaTeX source by want the output to reference different files.
You can accomplish this by passing `macroReplacements` (for environments, there is the similarly-named
`environmentReplacements`) to the plugin.

For example,

```typescript
import { unified } from "unified";
import rehypeStringify from "rehype-stringify";
import { htmlLike } from "@unified-latex/unified-latex-util-html-like";
import { printRaw } from "@unified-latex/unified-latex-util-print-raw";
import { unifiedLatexToPretext } from "@unified-latex/unified-latex-to-pretext";
import { unifiedLatexFromString } from "@unified-latex/unified-latex-util-parse";
import { getArgsContent } from "@unified-latex/unified-latex-util-arguments";

const convert = (value) =>
unified()
.use(unifiedLatexFromString)
.use(unifiedLatexToPretext, {
macroReplacements: {
includegraphics: (node) => {
const args = getArgsContent(node);
const path = printRaw(
args[args.length - 1] || []
).replace(/\.pdf$/, ".png");
return htmlLike({
tag: "img",
attributes: { src: path },
});
},
},
})
.use(rehypeStringify)
.processSync(value).value;

console.log(convert(`\\includegraphics{foo.pdf}`));
```

`macroReplacements` and `environmentReplacements` functions can return any unified-latex `Node`, but
using the `htmlLike` utility function will return nodes that get converted to specific HTML. See `htmlLike`'s
documentation for more details.

## Install

```bash
npm install @unified-latex/unified-latex-to-pretext
```

This package contains both esm and commonjs exports. To explicitly access the esm export,
import the `.js` file. To explicitly access the commonjs export, import the `.cjs` file.

# Plugins

## `unifiedLatexToPretext`

Unified plugin to convert a `unified-latex` AST into a `xast` AST representation of PreTeXt source.

### Usage

`unified().use(unifiedLatexToPretext[, options])`

#### options

```typescript
PluginOptions
```

### Type

`Plugin<PluginOptions[], Ast.Root, Xast.Root>`

```typescript
function unifiedLatexToPretext(
options: PluginOptions
): (tree: Ast.Root, file: VFile) => Xast.Root;
```

## `unifiedLatexWrapPars`

Unified plugin to wrap paragraphs in `\html-tag:p{...}` macros.
Because `-` and `:` cannot occur in regular macros, there is no risk of
a conflict.

### Usage

`unified().use(unifiedLatexWrapPars[, options])`

#### options

```typescript
PluginOptions
```

### Type

`Plugin<PluginOptions[], Ast.Root, Ast.Root>`

```typescript
function unifiedLatexWrapPars(options: PluginOptions): (tree: Ast.Root) => void;
```

## `xmlCompilePlugin`

Unified plugin to convert a `XAST` AST to a string.

### Usage

`unified().use(xmlCompilePlugin)`

### Type

`Plugin<void[], Root, string>`

```typescript
function xmlCompilePlugin(): void;
```

# Functions

## `attachNeededRenderInfo(ast)`

Attach `renderInfo` needed for converting some macros into their
katex equivalents.

```typescript
function attachNeededRenderInfo(ast: Ast.Ast): void;
```

**Parameters**

| Param | Type |
| :---- | :-------- |
| ast | `Ast.Ast` |

## `convertToPretext(tree, options)`

Convert the `unified-latex` AST `tree` into an HTML string. If you need
more precise control or further processing, consider using `unified`
directly with the `unifiedLatexToPretext` plugin.

For example,

unified()
.use(unifiedLatexFromString)
.use(unifiedLatexToPretext)
.use(rehypeStringify)
.processSync("\\LaTeX to convert")

```typescript
function convertToPretext(
tree: Ast.Node | Ast.Node[],
options: PluginOptions
): string;
```

**Parameters**

| Param | Type |
| :------ | :----------------------- |
| tree | `Ast.Node \| Ast.Node[]` |
| options | `PluginOptions` |

## `wrapPars(nodes, options)`

Wrap paragraphs in `<p>...</p>` tags.

Paragraphs are inserted at

* parbreak tokens
* macros listed in `macrosThatBreakPars`
* environments not listed in `environmentsThatDontBreakPars`

```typescript
function wrapPars(
nodes: Ast.Node[],
options: {
macrosThatBreakPars?: string[];
environmentsThatDontBreakPars?: string[];
}
): Ast.Node[];
```

**Parameters**

| Param | Type |
| :------ | :-------------------------------- |
| nodes | `Ast.Node[]` |
| options | <span color='gray'>Omitted</span> |

# Constants

| Name | Type |
| :------------------------------------- | :------------------------------------------------------------------ |
| `KATEX_SUPPORT` | `{ macros: any; environments: any; }` |
| `katexSpecificEnvironmentReplacements` | `Record<string, (node: Ast.Environment) => Ast.Node \| Ast.Node[]>` |
| `katexSpecificMacroReplacements` | `Record<string, (node: Ast.Macro) => Ast.Node \| Ast.Node[]>` |

# Types

## `PluginOptions`

```typescript
export type PluginOptions = HtmlLikePluginOptions & {
/**
* A boolean where if it's true then the output won't be wrapped in the <pretext><article> ... etc. tags.
* If it's false (default), a valid and complete PreTeXt document is returned.
*/
producePretextFragment?: boolean;
};
```
Loading

0 comments on commit d29363b

Please sign in to comment.