feat: Add the stdlib_diff tool to compare gno and go standard libraries

[Villaquiranm]

continuing the work started on #1425
thanks FloRichardAloeCorp for the great job on this issue 👍
closes #1310

Contributors' checklist...

• Added new tests, or not needed, or not feasible
• Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory
• Updated the official documentation or not needed
• No breaking changes were made, or a BREAKING CHANGE: xxx message was included in the description
• Added references to related issues and PRs
• Provided any useful hints for running manual tests
• Added new benchmarks to generated graphs, if any. More info here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

[Villaquiranm]

@thehowl I took a look of this, the current status:
1- Not done yet
2- I think it is already done, I can select the code and the + & - are not selected
3- I was thinking on implementing this using semaphores ? like that I could have the entire code on the same file as it is right now. If you think is necessary to use channels I could do that too.

4 and 5 are fixed on eaf6a87
Only for 4 I think there are some folder we should avoid ? For exemple cmd & vendor as they have a lot of files and I think they are not related to standard lib ?

Thank you for your contribution!

Sorry for the wait, this is an initial to improve. Nice work, but you can improve.

Some functionality missing on the generated report:

1- Formatting is still very funky. Use CSS to render tabs correctly; they should render as 4 spaces.

2- Ideally, for the added/deleted lines, make the + and - symbols non selectable. This makes it so that if I select the lines in either side of the page I can copy and paste the code without copying the symbols.

3- I believe the report's performance could be improved. Consider using the library I mention in the comments (diffmatchpatch) and parallelizing execution. You can parallelize execution by having a goroutine trying to first of all find all of the directories in the source and destination, and then sending off to a channel individual package names.

4- The tool does not currently perform diffing recursively. Ie. creating the report on my stdlibs it does not generate a diff for package regexp/syntax. Consider searching using something like filepath.WalkDir.

5- On the report, please make the lines which are == greyed out (putting css opacity: 0.75 probably works)

[thehowl]

@Villaquiranm

1. Formatting: you're right, + / - are not selectable. Can you add line numbers as well, and align each column? (line number, + / - / no sign, code line), making sure that only the code is selectable?
2. For the diff'ing algo and performance, I actually found this library when I was working on a side project some time ago. I believe it's the best for this use case. diffmatchpatch, which I linked in the original PR, is not ideal as it is for "character-by-character" diff (ie. what Google uses for google docs); not for the kind of diffs we use in programming.
   There are also likely some other "easy" performance wins. As you mentioned, we should skip directories which entirely don't exist in their Gno counterpart (in the index, we can just show cmd in red and say missing in gno or missing in go for libs like std).
   Parallelism may not be necessary; let's see what single-thread performance we can get, and if it's still slow after my suggestions use pprof to find out what's taking the most time. I think we can get <5s performance on most machines pretty easily even without it.
3. Can you make sure the index lists sub-directories as well?

[jefft0]

Removed the "review team" label because this is already reviewed by thehowl.

[thehowl]

There are some more improvements overall, but I think with these few changes we're good to merge and iterate. Thanks, and sorry for the delay in the review!

[thehowl]

btw, can you modify the gh-pages workflow? (.github/workflows/gh-pages)

so it creates the report automatically on each push. Maybe put the report in the subdir stdlib_diff, so it eventually shows up on https://gnolang.github.io/gno/stdlib_diff

I suggest you also try it out on a fork and link to a sample output, so we can check the workflow works correctly ahead of the merge.

[Gno2D2]

🛠 PR Checks Summary

🔴 Maintainers must be able to edit this pull request (more info)

Manual Checks (for Reviewers):

• SKIP: Do not block the CI for this PR

Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

🔴 Maintainers must be able to edit this pull request (more info)
🟢 The pull request head branch must be up-to-date with its base (more info)

☑️ Contributor Actions:

1. Fix any issues flagged by automated checks.
2. Follow the Contributor Checklist to ensure your PR is ready for review.
   • Add new tests, or document why they are unnecessary.
   • Provide clear examples/screenshots, if necessary.
   • Update documentation, if required.
   • Ensure no breaking changes, or include BREAKING CHANGE notes.
   • Link related issues/PRs, where applicable.

☑️ Reviewer Actions:

1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.

📚 Resources:

• Report a bug with the bot.
• View the bot’s configuration file.

Debug

Automated Checks

Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 The pull request was created from a fork (head branch repo: TERITORI/gno)

Then

🔴 Requirement not satisfied
└── 🔴 Maintainer can modify this pull request

The pull request head branch must be up-to-date with its base (more info)

If

🟢 Condition met
└── 🟢 On every pull request

Then

🟢 Requirement satisfied
└── 🟢 Head branch (TERITORI:feat/stdlib_diff) is up to date with base (master): behind by 0 / ahead by 48

Manual Checks

**SKIP**: Do not block the CI for this PR

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

• Any user with comment edit permission

[Villaquiranm]

btw, can you modify the gh-pages workflow? (.github/workflows/gh-pages)

so it creates the report automatically on each push. Maybe put the report in the subdir stdlib_diff, so it eventually shows up on https://gnolang.github.io/gno/stdlib_diff

I suggest you also try it out on a fork and link to a sample output, so we can check the workflow works correctly ahead of the merge.

@thehowl I followed your recommendations, here is the result :) thanks
https://villaquiranm.github.io/gno/stdlib_diff/

[thehowl]

TERITORI#24

[Villaquiranm]

@thehowl Merged your PR sorry for having this extra step :)
next time Will do it from my personal github account

[thehowl]

no worries :)