Feature Request: Pagination extension (A4 format content)

[clemente-xyz]

Description

Extension which purpose is to emulate the content into an A4 page format, and automatically split the content for the user when the max height is exceeded.

Until now, Tiptap has no extensions that allow the editor to enter to this "page" format.

Use Case

Many apps with more conventional users (mainly in industries such as lawyers, legal guardians, any anyone which has to work with lots of paper) still need online editors that emulate the content into an A4 page format. Many need to print it out or export it to PDF to be sent to recipients that need to document in A4 format.

Type

New extension

[clemente-xyz]

I have already done some work for my Nextjs app, but it still a bit buggy:

import { Extension } from '@tiptap/core';
import { Node, mergeAttributes } from '@tiptap/core';
import { Node as PMNode } from '@tiptap/pm/model';
import { Plugin, PluginKey, TextSelection } from '@tiptap/pm/state';

export const PageNode = Node.create({
  name: 'page',
  group: 'block',
  content: 'block+',
  defining: true,
  isolating: true,

  parseHTML() {
    return [
      {
        tag: 'div[data-page]',
      },
    ];
  },

  renderHTML({ HTMLAttributes }) {
    return ['div', mergeAttributes(HTMLAttributes, { 'data-page': true, class: 'page' }), 0];
  },

  addNodeView() {
    return () => {
      const dom = document.createElement('div');
      dom.setAttribute('data-page', 'true');
      dom.classList.add('page');
      dom.style.height = '297mm'; // A4 height in mm
      dom.style.width = '210mm'; // A4 width in mm
      dom.style.padding = '25.4mm';
      dom.style.border = '1px solid #ccc';
      dom.style.background = 'white';
      dom.style.overflow = 'hidden';
      dom.style.position = 'relative';

      const contentDOM = document.createElement('div');
      dom.appendChild(contentDOM);

      return {
        dom,
        contentDOM,
      };
    };
  },
});

export const PaginationExtension = Extension.create({
  name: 'pagination',

  addProseMirrorPlugins() {
    return [
      new Plugin({
        key: new PluginKey('pagination'),
        appendTransaction: (transactions, oldState, newState) => {
          const paginationMeta = 'pagination';
          const lastTransaction = transactions[transactions.length - 1];
          const isPaginationTransaction = lastTransaction.getMeta(paginationMeta);

          // Avoid infinite loops and unnecessary processing
          if (isPaginationTransaction || !lastTransaction.docChanged) {
            return null;
          }

          const { schema } = newState;
          const pageType = schema.nodes.page;

          if (!pageType) {
            return null;
          }

          // Collect content nodes, flattening existing pages
          const contentNodes: PMNode[] = [];

          newState.doc.forEach((node) => {
            if (node.type === pageType) {
              node.forEach((child) => contentNodes.push(child));
            } else {
              contentNodes.push(node);
            }
          });

          // Partition content nodes into pages
          const pages = [];
          let currentPageContent: PMNode[] = [];
          let currentHeight = 0;
          const pageHeight = (297 - 25.4 * 2) * 3.77953; // A4 height in mm minus padding, converted to px (1 mm = 3.77953 px)
          const lineHeight = 24; // Line height in px

          for (const node of contentNodes) {
            const nodeHeight = estimateNodeHeight(node, lineHeight);

            if (currentHeight + nodeHeight > pageHeight && currentPageContent.length > 0) {
              // Start a new page
              pages.push(pageType.create({}, currentPageContent));
              currentPageContent = [node];
              currentHeight = nodeHeight;
            } else {
              currentPageContent.push(node);
              currentHeight += nodeHeight;
            }
          }

          if (currentPageContent.length > 0) {
            pages.push(pageType.create({}, currentPageContent));
          }

          const newDoc = schema.topNodeType.create(null, pages);

          // Compare the content of the documents
          if (newDoc.content.eq(newState.doc.content)) {
            return null; // No changes, skip transaction
          }

          const tr = newState.tr.replaceWith(0, newState.doc.content.size, newDoc.content);
          tr.setMeta(paginationMeta, true);

          // Map the selection from oldState to the new document
          const { selection } = oldState;
          const mappedSelection = selection.map(tr.doc, tr.mapping);

          if (mappedSelection) {
            tr.setSelection(mappedSelection);
          } else {
            // Fallback to a safe selection at the end of the document
            tr.setSelection(TextSelection.create(tr.doc, tr.doc.content.size));
          }

          return tr;
        },
      }),
    ];
  },
});

function estimateNodeHeight(node: PMNode, lineHeight: number): number {
  if (node.isTextblock) {
    const lines = node.textContent.split('\n').length || 1;
    return lines * lineHeight;
  } else if (node.type.name === 'image') {
    return 200;
  } else {
    return lineHeight;
  }
}

[bdbch]

Nice job. I was working on something similar at some point but didn't find enough time to put into this in general.

The biggest task is actually calculating if content on a page fits the page's boundaries (in size including margin) and if content needs to be moved to a new page or should be merged back into a previous page.

Do you have a few more insights into this?

[Cassielxd]

You can refer to this implementation https://github.com/Cassielxd/CassieEditor

[clemente-xyz]

@Cassielxd your implementation looks nice, but the code is super hard to follow and adapt to React. I actually submitted an issue on your repo asking for a React extension/adaptation to it: Cassielxd/CassieEditor#5

I spent a lot of time trying to adapt it, but no case

[clemente-xyz]

@bdbch yes. Right now the content split into nodes is working super good, but only for paragraphs, lists, and other simple nodes. The big issue comes with more complex HTML elements such as tables. It's not an easy job to traverse the table element and make a split. Something similar with images.

There are also some buggy behaviors when deleting content and passing to previous page nodes, which I still cant figure out how to solve

[bdbch]

It's tricky indeed. Cases like this are currently not really possible with Tiptap/Prosemirror as we need to strictly abide to the DOM being correct.

Tables could work even though they're complex to map and calculate through (specially if nested with more complex elements).

One idea was to just not split those and keep them in one while pushing them to a new page but that's not the 1:1 behavior we get from other editors and also brings up a new edge case where a node is larger than a page (for example a very long table).

[clemente-xyz]

Tomorrow I'll post an update of my code. I managed to accurately compute node heights and distribute them accordingly, as well as be able to parse initial content that don't come in "page nodes" format.

Do you have any code to share of your implementation?

I really think we can come with something out 🚀

[clemente-xyz]

Ok, so these are my updates. Its working fine (still a bit buggy, but in general, looking good).

import { Extension } from '@tiptap/core';
import { Node, mergeAttributes } from '@tiptap/core';
import { keymap } from '@tiptap/pm/keymap';
import { Node as PMNode } from '@tiptap/pm/model';
import { Plugin, PluginKey, Selection, TextSelection } from '@tiptap/pm/state';
import { EditorView } from '@tiptap/pm/view';

export const PageNode = Node.create({
  name: 'page',
  group: 'block',
  content: 'block*',
  defining: true,
  isolating: false,

  parseHTML() {
    return [
      {
        tag: 'div[data-page]',
      },
    ];
  },

  renderHTML({ HTMLAttributes }) {
    return ['div', mergeAttributes(HTMLAttributes, { 'data-page': true, class: 'page' }), 0];
  },

  addNodeView() {
    return () => {
      const dom = document.createElement('div');
      dom.setAttribute('data-page', 'true');
      dom.classList.add('page');
      dom.style.height = '297mm'; // A4 height in mm
      dom.style.width = '210mm'; // A4 width in mm
      dom.style.padding = '25.4mm'; // 1 inch padding
      dom.style.border = '1px solid #ccc';
      dom.style.background = 'white';
      dom.style.overflow = 'hidden';
      dom.style.position = 'relative';

      const contentDOM = document.createElement('div');
      dom.appendChild(contentDOM);

      return {
        dom,
        contentDOM,
      };
    };
  },
});

export const PaginationExtension = Extension.create({
  name: 'pagination',
  addProseMirrorPlugins() {
    return [
      keymap({
        Enter: (state, dispatch) => {
          const { from, to } = state.selection;

          // Proceed only if dispatch is provided and we're working with a valid range
          if (dispatch && from === to) {
            const tr = state.tr;

            // Get the resolved position in the document
            const $pos = state.doc.resolve(from);

            // Ensure that the position is within a valid block (paragraph)
            if ($pos.parent.type.name === 'paragraph') {
              // Create a new empty paragraph node
              const paragraph = state.schema.nodes.paragraph.create();

              // Insert the empty paragraph at the cursor's current position
              tr.insert(from, paragraph);

              // Find the nearest valid cursor position inside the new paragraph
              const newSelection = Selection.near(tr.doc.resolve(from + 1), 1);

              // Set the selection to be inside the new paragraph
              tr.setSelection(newSelection);

              // Dispatch the transaction properly
              dispatch(tr);

              return true;
            }
          }

          return false;
        },
      }),
      PaginationPlugin,
    ];
  },
});

export const PaginationPlugin = new Plugin({
  key: new PluginKey('pagination'),
  view() {
    let isPaginating = false;

    return {
      update(view: EditorView, prevState) {
        if (isPaginating) {
          return;
        }

        const { state } = view;
        const { schema } = state;
        const pageType = schema.nodes.page;

        if (!pageType) {
          return;
        }

        const docChanged = !view.state.doc.eq(prevState.doc);
        console.log('view.state.doc: ', view.state.doc);
        const initialLoad = prevState.doc.content.size === 0 && state.doc.content.size > 0;

        let hasPageNodes = false;
        state.doc.forEach((node) => {
          if (node.type === pageType) {
            hasPageNodes = true;
          }
        });

        if (!docChanged && hasPageNodes && !initialLoad) {
          return;
        }

        isPaginating = true;

        // Collect content nodes and their old positions
        const contentNodes: { node: PMNode; pos: number }[] = [];
        state.doc.forEach((node, offset) => {
          if (node.type === pageType) {
            node.forEach((child, childOffset) => {
              contentNodes.push({ node: child, pos: offset + childOffset + 1 });
            });
          } else {
            contentNodes.push({ node, pos: offset + 1 });
          }
        });

        // Measure node heights
        const MIN_PARAGRAPH_HEIGHT = 20; // Adjust as needed for your styling
        const nodeHeights = contentNodes.map(({ pos, node }) => {
          const dom = view.nodeDOM(pos);
          if (dom instanceof HTMLElement) {
            let height = dom.getBoundingClientRect().height;
            if (height === 0) {
              if (node.type.name === 'paragraph' || node.isTextblock) {
                // Assign a minimum height to empty paragraphs or textblocks
                height = MIN_PARAGRAPH_HEIGHT;
              }
            }
            return height;
          }
          return MIN_PARAGRAPH_HEIGHT; // Default to minimum height if DOM element is not found
        });

        // Record the cursor's old position
        const { selection } = view.state;
        const oldCursorPos = selection.from;

        // Build the new document and keep track of new positions
        const pages = [];
        let currentPageContent: PMNode[] = [];
        let currentHeight = 0;
        const pageHeight = ((297 - 25.4 * 2) / 25.4) * 96;

        const oldToNewPosMap: { [key: number]: number } = {};
        let cumulativeNewDocPos = 0;

        for (let i = 0; i < contentNodes.length; i++) {
          const { node, pos: oldPos } = contentNodes[i];
          const nodeHeight = nodeHeights[i];

          if (currentHeight + nodeHeight > pageHeight && currentPageContent.length > 0) {
            const pageNode = pageType.create({}, currentPageContent);
            pages.push(pageNode);
            cumulativeNewDocPos += pageNode.nodeSize;
            currentPageContent = [];
            currentHeight = 0;
          }

          if (currentPageContent.length === 0) {
            cumulativeNewDocPos += 1; // Start of the page node
          }

          // Record the mapping from old position to new position
          const nodeStartPosInNewDoc =
            cumulativeNewDocPos + currentPageContent.reduce((sum, n) => sum + n.nodeSize, 0);
          oldToNewPosMap[oldPos] = nodeStartPosInNewDoc;

          currentPageContent.push(node);
          currentHeight += Math.max(nodeHeight, MIN_PARAGRAPH_HEIGHT);
        }

        if (currentPageContent.length > 0) {
          const pageNode = pageType.create({}, currentPageContent);
          pages.push(pageNode);
        }

        const newDoc = schema.topNodeType.create(null, pages);

        // Compare the content of the documents
        if (newDoc.content.eq(state.doc.content)) {
          isPaginating = false;
          return;
        }

        const tr = state.tr.replaceWith(0, state.doc.content.size, newDoc.content);
        tr.setMeta('pagination', true);

        // Map the cursor position
        let newCursorPos = null;
        for (let i = 0; i < contentNodes.length; i++) {
          const { node, pos: oldNodePos } = contentNodes[i];
          const nodeSize = node.nodeSize;

          if (oldNodePos <= oldCursorPos && oldCursorPos <= oldNodePos + nodeSize) {
            const offsetInNode = oldCursorPos - oldNodePos;
            const newNodePos = oldToNewPosMap[oldNodePos];
            newCursorPos = newNodePos + offsetInNode;
            break;
          }
        }

        if (newCursorPos !== null) {
          const $pos = tr.doc.resolve(newCursorPos);
          let selection;

          if ($pos.parent.isTextblock) {
            // The position is valid for a TextSelection
            selection = Selection.near($pos);
          } else if ($pos.nodeAfter && $pos.nodeAfter.isTextblock) {
            // Move into the next textblock
            selection = Selection.near(tr.doc.resolve($pos.pos + 1));
          } else if ($pos.nodeBefore && $pos.nodeBefore.isTextblock) {
            // Move into the previous textblock
            selection = Selection.near(tr.doc.resolve($pos.pos - 1), -1);
          } else {
            // Find the nearest valid cursor position
            selection = Selection.findFrom($pos, 1, true) || Selection.findFrom($pos, -1, true);
          }

          if (selection) {
            tr.setSelection(selection);
          } else {
            // Fallback to a safe selection at the end of the document
            tr.setSelection(TextSelection.create(tr.doc, tr.doc.content.size));
          }
        } else {
          tr.setSelection(TextSelection.create(tr.doc, tr.doc.content.size));
        }

        view.dispatch(tr);

        isPaginating = false;
      },
    };
  },
});

Small demo:

Screen.Recording.2024-10-14.at.11.45.57.mov

As you can see, there are some bugs when deleting content (the cursor sometimes skips the empty content and starts deleting), and others, but I think is a good start

@bdbch can you have a look? I think stills need some improvements to be able to release a formal extension

[caseybeck10]

Great job, this is the first well-documented pagination solution that I've seen online. I echo your need for a comprehensive pagination plugin. As you said, there are so many sectors in which being able to see editable content in pages is essential.

Have you tested cases in which the number of pages reaches 100, 200 or higher? That's generally where the existing solutions begin to slow down.

[clemente-xyz]

not yet. Did you test it out? Im struggling with some minor issues:

1. Deleting content from one page to the previous dont make the cursor jump to the previous page (needs to be done manually).
2. Initial content that comes in "flat format" (not paginated into page nodes) is not properly being parsed into page nodes (all loads into a single page and I have to press Enter or edit the content to make the proper page splitting).
3. Tables not being properly handled: they should be split into multiple tables when overflowing a page content, and right now they are treated a a single big element, hence moved completely to a new page (leaving available empty content on the previous page with no use)

Would be nice to hear some feedback on those issues, as well as recommendations of changes!

[caseybeck10]

I've copied the code and played around with it, but haven't been able to thoroughly test it as of yet. You're right, there are definitely some issues with pasting content, specifically when the pasted text overflows onto the next page.

I'll take a good look at the code tonight and see what I can do. Would you be interested in placing the up-to-date code in a public github repo? I'm in need of a pagination solution for my app & have been researching the current landscape for a while now. I'd certainly be interested in helping build a robust one.

If you haven't seen already; there's a recent forum post that outlines a paginated JS editor built on prosemirror called BadonWriter: https://discuss.prosemirror.net/t/a-new-text-editor-with-pagination/6667/7

There's a live demo on the website. Might be worth playing around with to get some inspiration. The author mentioned that they'd be open-sourcing in the near future. May be worth talking to them too?

[clemente-xyz]

oh this is great. I just posted a reply so the authors can have a look in here. Hope the project gets open sourced and is easy to lookup for references on how to improve this Tiptap extension

@bdbch any updates from your side?

[clemente-xyz]

Hi guys. Any progress by your side? For me has been a hell, and right now trying out a Collabora Online server to just embedded it to the app

[jamAL108]

Hey can you please share where to integrate this pagination Extension as a i tried doing so in extension array but that's not working

[hugs7]

@jamAL108 I believe you need to add the PageNode to the extension array as well.
There isn't a React example in the docs but you can figure this out from the Vue example: https://tiptap.dev/docs/editor/extensions/custom-extensions/node-views/examples