Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Time Partitioner not working as expected for GCS #381

Open
shaileshjha061 opened this issue Jun 28, 2022 · 1 comment
Open

[BUG] Time Partitioner not working as expected for GCS #381

shaileshjha061 opened this issue Jun 28, 2022 · 1 comment
Labels
type/bug

Comments

@shaileshjha061
Copy link

shaileshjha061 commented Jun 28, 2022 •
edited
Loading

Data Sent to GCS Bucket is not in the correct date folder.
I have configured timePartitionDuration: 1d It was working fine but after a few days dates folder was not being created (say after 2022-06-23 folders had not been created) and the data was being sent to the 2022-06-23 folder.
Data being sent to GCS.

To Reproduce
I am using
Function Mesh version: 0.1.11-rc3
Pulsar IO Cloud storage sink connector Version: 2.9.2.21

Expected behavior
Data Sent to GCS Bucket should be in the respective data folder.

Screenshots
Screenshot 2022-06-28 204511

Additional context
Add any other context about the problem here.
@alpreu
Copy link
Contributor

alpreu commented Oct 11, 2022

Hi @shaileshjha061, can you confirm whether your records Message objects have a publishTime associated?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alpreu @shaileshjha061