Exclude frontmatter (yaml or toml) in markdown or html files

[danielfdickinson]

With static site generators one often has front matter in one's Markdown files (and some, like Hugo, also support 'html' files (that is a chunk of html to go in body), also with front matter). Such front matter is typically yaml, toml, or json, with some type of delimeter (--- in the case yaml, +++ for toml when using Hugo) before and after the front matter.

Is there a reliable way to omit this from spell checking via cspell.json?

I'm thinking some kind of multi-line regexp might get the job done (although I find regexp easy to get wrong and overlook some corner case), but a few quick searches (surprisingly) don't turn up results for me (I see results for 'sed' but that is not the right regex format for CSpell).

EDIT: I have a regexp, and two replacement questions:

1. What does this regexp miss?
2. Is there a workaround for the case where a file begins with '---' but doesn't have a closing '---' causes CSpell to hang?

The regexp I came up with:

	"ignoreRegExpList": [
		"/^(\\+\\+\\+|---)(\\s|\\S)*?\\n(\\+\\+\\+|---)\\n/g"
	],

EDIT: Fixed the regexp to be more reasonable.

EDIT:

Is there way to ignoreRegExpList for only specific files?

[Jason3S]

@danielfdickinson,

I'll answer the easy question first.

cspell-dicts has a lot of cspell-ext.json examples. Here are a few to take a look at:

• python/cspell-ext.json
• typescript/cspell-ext.json
• latex/cspell-ext.json

My suggestion would be to define a pattern: PatternFrontMatter.

Once you have that, you can use it wherever ignoreRegExpList is allowed:

"overrides": [{"filename": "website/**/*.md", "ignoreRegExpList": ["PatternFrontMatter"]}]

"languageSettings":[{"languageId": "markdown", "ignoreRegExpList": ["PatternFrontMatter"]}]

[danielfdickinson]

Thank you. I wasn't sure if 'ignoreRegExpList' was available for 'overrides'. There isn't an explicit example.

On reflection, I presume that most things that can be in the main block can be in an 'overrides', and likewise 'languageSettings'?

[Jason3S]

@danielfdickinson,

Documentation is a challenge.

cspell.config.yaml

ignoreRegExpList: []
patterns: []
languageSettings:
  - languageId: markdown
    patterns: [] # patterns with the same name will replace patterns defined at the root.
    ignoreregExpList: [] # These are in addition to the ones defined at the root
overrides:
  - filename: "**/*.md"
    patterns: [] # patterns with the same name will replace patterns defined at the root.
    ignoreregExpList: [] # These are in addition to the ones defined at the root
    languageSettings:
      - languageId: markdown
        patterns: [] # patterns with the same name will replace patterns defined at the root and overrides
        ignoreregExpList: [] # These are in addition to the ones defined at the root and overrides

Configuration is additive:

[Jason3S]

@danielfdickinson,

I suggest using a site like regex101.com. Here is an example: https://regex101.com/r/PeXKZN/2

Note: detecting front matter is difficult because it only occurs at the beginning of the file. ignoreRegExpList is designed to search the whole file.

Note: Work is in progress to add parsers to help with the spell checking context: https://github.com/streetsidesoftware/cspell/tree/main/rfc/rfc-0003%20parsing%20files

[danielfdickinson]

Thanks for that. I did use regex101.com but I'm too used to working with far more limited regex versions where things I expect to work don't, so I tend to forget what a full regex language can do.

I did modify your sample a little, and came up with: https://regex101.com/r/ze3C6Q/1

"ignoreRegExpList": [
  "/^(-{3}|[+]{3})$(\\s|\\S)*?^\\1$/gm"
],

(the [\1], while it works wasn't my favourite). It looks like the new version is marginally faster on some files, and both definitely beat the version I came up with initially.

[Jason3S]

I suggest using a named pattern:

"patterns": [{ "name": "match_markdown_frontmatter", "pattern": "/^(-{3}|[+]{3})$[\\s\\S]*?^\\1$/gm" }],
"languageSettings": [{ "languageId": "markdown", "ignoreRegExpList": ["match_markdown_frontmatter"] }]

[danielfdickinson]

That is actually what I did with my cspell.json. I just did the regex by itself here for illustration.

At the risk of showing how badly I've missed things in the documentation, here is my work in progress tests/config/cspell.json (I do that and use --config tests/config/cspell.json to reduce clutter in the project root). I am thinking for the markdown and html I should modify to use languageId. In the meantime:

{
	"$schema": "https://raw.githubusercontent.com/streetsidesoftware/cspell/main/cspell.schema.json",
	"allowCompoundWords": false,
	"dictionaryDefinitions": [
		{
			"addWords": true,
			"locale": "en,en-ca",
			"name": "project-words",
			"path": "./words-project.txt"
		},
		{
			"addWords": true,
			"locale": "fr,fr-ca",
			"name": "project-words-fr",
			"path": "./words-fr-project.txt"
		},
		{
			"addWords": true,
			"locale": "en,en-ca",
			"name": "tech-words",
			"path": "./dfd-wordlists/words-tech.txt"
		},
		{
			"addWords": true,
			"locale": "en,en-ca",
			"name": "dfd-words",
			"path": "./dfd-wordlists/words-dfd.txt"
		}
	],
	"ignorePaths": [
		"../../go.*",
		"**/exampleSite/go.*",
		"**/tests/config/cspell.json",
		"**/tests/config/words-*.txt",
		"**/tests/config/dfd-wordlists/**"
	],
	"ignoreRegExpList": [],
	"ignoreWords": [],
	"import": [
		"@cspell/dict-fr-fr/cspell-ext.json",
		"@cspell/dict-fr-reforme/cspell-ext.json"
	],
	"language": "en,en-gb,en-us",
	"overrides": [
		{
			"filename": [
				"**/exampleSite/**/fr/*.md",
				"**/exampleSite/**/fr/**/*.md"
			],
			"ignoreRegExpList": [
				"frontMatter"
			],
			"language": "fr"
		},
		{
			"filename": [
				"**/exampleSite/**/fr/*.md",
				"**/exampleSite/**/fr/**/*.md"
			],
			"includeRegExpList": [
				"frontMatter"
			],
			"language": "en,fr"
		},
		{
			"dictionaries": [
				"css",
				"html",
				"project-words",
				"tech-words"
			],
			"filename": [
				"**/tests/config-action/htmlvalidate*.json"
			]
		},
		{
			"dictionaries": [
				"companies",
				"dfd-words",
				"project-words",
				"tech-words"
			],
			"filename": [
				"**/config.toml",
				"**/theme.toml",
				"**/i18n/fr.toml",
				"**/exampleSite/config.toml",
				"**/.github/**/*.yml",
				"**/.pre-commit-config.yaml"
			],
			"language": "en,en-gb,fr,fr-90"
		},
		{
			"dictionaries": [
				"dfd-words",
				"project-words",
				"tech-words"
			],
			"filename": [
				"**/archetypes/*",
				"**/layouts/**/*.html",
				"**/README.md",
				"**/docs/*.md"
			]
		},
		{
			"dictionaries": [
				"dfd-words",
				"project-words",
				"tech-words"
			],
			"filename": [
				"**/content/en/*.md",
				"**/content/en/*.html",
				"**/content/en/**/*.md",
				"**/content/en/**/*.html"
			],
			"includeRegExpList": [
				"frontMatter"
			]
		}
	],
	"patterns": [
		{
			"name": "frontMatter",
			"pattern": "/^(-{3}|[+]{3})$(\\s|\\S)*?^\\1$/gm"
		}
	],
	"version": "0.2"
}

[Jason3S]

Nicely done.

A couple of things that might help:

• globRoot - you might want to set it to be "globRoot": "../..". By default, all globs (except those starting with **) are relative to the config location.
• In the case of dir/**/*.md, the ** is zero or more. So it will match both dir/a.md and dir/sub/b.md.