Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Parallelize documentation post-processing #448

Open
bdice opened this issue Nov 8, 2023 · 1 comment
Open

[FEA] Parallelize documentation post-processing #448

bdice opened this issue Nov 8, 2023 · 1 comment
Labels
? - Needs Triage Need team to review and classify feature request New feature or request

Comments

@bdice
Copy link
Contributor

bdice commented Nov 8, 2023

Documentation deployment takes more than one hour to run. It may make sense to parallelize the CPU operations that post-process the documentation files. I would guess that this is the slow step that would benefit from parallelization:

for FILE in $(grep "${JTD_SEARCH_TERM}\|${DOXYGEN_SEARCH_TERM}\|${PYDATA_SEARCH_TERM}" -rl \
--include=\*.html \
--exclude-dir=stable \
--exclude-dir=nightly \
--exclude-dir=latest \
--exclude-dir=legacy \
--exclude-dir=cudf-java \
${FOLDER_TO_CUSTOMIZE} ); do
python ${SCRIPT_SRC_FOLDER}/customize_doc.py $(realpath ${FILE})
echo "" # line break for readability
done

@bdice bdice added ? - Needs Triage Need team to review and classify feature request New feature or request labels Nov 8, 2023
@bdice
Copy link
Contributor Author

bdice commented Nov 8, 2023

This may become irrelevant with the ongoing work to unify rapids.ai and docs.rapids.ai.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
? - Needs Triage Need team to review and classify feature request New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant