Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write the catalog files at the end of the process #1250

Open
4 of 5 tasks
lordrip opened this issue Jul 15, 2024 · 2 comments · Fixed by #1312, #1519, #1530 or #1531
Open
4 of 5 tasks

Write the catalog files at the end of the process #1250

lordrip opened this issue Jul 15, 2024 · 2 comments · Fixed by #1312, #1519, #1530 or #1531
Labels
build Build and GitHub related issues catalog Catalog-related issues
Milestone

Comments

@lordrip
Copy link
Member

lordrip commented Jul 15, 2024

Please describe the task that needs to be done

Currently, the catalog generation workflow looks like the following:

  1. Fetch the catalog information
  2. Write each piece to disk as soon as it completes
  3. The final result gets assembled into the index.jsonfile

The idea is to generate everything in memory and as the last step, write everything to file. This to be able to run assertions against the output more easily.

Todo

@lordrip lordrip added catalog Catalog-related issues build Build and GitHub related issues labels Jul 15, 2024
@lordrip lordrip added this to the 2.2.0 milestone Jul 16, 2024
@lordrip lordrip self-assigned this Aug 12, 2024
lordrip added a commit to lordrip/kaoto that referenced this issue Aug 13, 2024
The first batch for @kaoto/camel-catalog tests

fix: KaotoIO#1250
@lordrip lordrip removed their assignment Aug 13, 2024
@lordrip lordrip reopened this Aug 13, 2024
@lordrip lordrip closed this as completed Aug 13, 2024
@lordrip lordrip reopened this Aug 14, 2024
@lordrip
Copy link
Member Author

lordrip commented Aug 14, 2024

Hi @shivamG640, I have a proposal for the catalog-generator test:

Extract each of these methods to individual classes, under a dedicated package
The method could not receive any parameter, but instead, return an index entry that will be collected by the CatalogGeneratorBuilder class

By doing this, we can assert over the output of each class while also being able to mock them when needed to test the main class.

We should be able to postpone writing the files as a much as possible, up to the generate command.

@lordrip
Copy link
Member Author

lordrip commented Aug 14, 2024

@mhempleman As you already noticed, we're adding tests for the catalog generator, and as part of this effort, we're thinking of reorganizing a few classes to make them more testable.

We're collecting proposals for this issue, and since you're contributing to the catalog generator, we wanted to give you a heads-up. This won't affect

I don't think it will affect #1311, since whatever is decided, it should be implemented on a step-by-step basis.

That being said, if you have a proposal or feedback about what changes would be nice to implement to make the catalog generator more robust and testable, please let us know so we can incorporate your feedback.

@shivamG640 shivamG640 assigned shivamG640 and unassigned shivamG640 Aug 20, 2024
@lordrip lordrip modified the milestones: 2.2.0, 2.3.0 Sep 13, 2024
@lordrip lordrip self-assigned this Oct 2, 2024
lordrip added a commit to lordrip/kaoto that referenced this issue Oct 2, 2024
At this time, we're adding tests for the `ConfigBean` class and the
`GenerateCommandTest` class.

fix: KaotoIO#1250
lordrip added a commit to lordrip/kaoto that referenced this issue Oct 2, 2024
At this time, we're adding tests for the `ConfigBean` class and the
`GenerateCommandTest` class.

fix: KaotoIO#1250
lordrip added a commit to lordrip/kaoto that referenced this issue Oct 2, 2024
At this time, we're adding tests for the `ConfigBean` class and the
`GenerateCommandTest` class.

fix: KaotoIO#1250
@lordrip lordrip closed this as completed in 3907213 Oct 2, 2024
@lordrip lordrip reopened this Oct 2, 2024
@lordrip lordrip modified the milestones: 2.3.0, 2.4.0 Oct 2, 2024
lordrip added a commit to lordrip/kaoto that referenced this issue Oct 3, 2024
lordrip added a commit to lordrip/kaoto that referenced this issue Oct 3, 2024
lordrip added a commit to lordrip/kaoto that referenced this issue Oct 3, 2024
lordrip added a commit to lordrip/kaoto that referenced this issue Oct 3, 2024
lordrip added a commit to lordrip/kaoto that referenced this issue Oct 3, 2024
lordrip added a commit to lordrip/kaoto that referenced this issue Oct 3, 2024
@lordrip lordrip closed this as completed in eff2de5 Oct 3, 2024
lordrip added a commit to lordrip/kaoto that referenced this issue Oct 3, 2024
Since the tests were migrated to the new `catalog-generator` package, is
safe to remove this package.

fix: KaotoIO#1250
@lordrip lordrip reopened this Oct 4, 2024
@lordrip lordrip removed their assignment Oct 4, 2024
@lordrip lordrip reopened this Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment