Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] EVENT logging masks all inputs #4926

Open
2 tasks
dlvenable opened this issue Sep 9, 2024 · 0 comments
Open
2 tasks

[BUG] EVENT logging masks all inputs #4926

dlvenable opened this issue Sep 9, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@dlvenable
Copy link
Member

dlvenable commented Sep 9, 2024

The EVENT mask is intended to be a signal to the logger to mask only Data Prepper Event objects. However, it currently operates like a SENSITIVE marker which should mask all fields (except exceptions see #3375).

For example, add_entries has this exception.

LOG.error(EVENT, "Error adding entry to record [{}] with key [{}], metadataKey [{}], value_expression [{}] format [{}], value [{}]",
recordEvent, entry.getKey(), entry.getMetadataKey(), entry.getValueExpression(), entry.getFormat(), entry.getValue(), e);

It results in logs like the following:

Error adding entry to record [******] with key [******], metadataKey [******], value_expression [******] format [******], value [******] java.lang.ClassCastException: null

Solution:

Fix the SensitiveArgumentMaskingConverter to only mask Event objects when the marker is EVENT rather than SENSITIVE.

Also, we should audit our usages of EVENT. For each usage, we should consider 1) Can we keep it as EVENT as is? 2) Do we need to remove some fields to keep as EVENT; or 3) Do we need to make it SENSITIVE?

Tasks

  • Fix the SensitiveArgumentMaskingConverter
  • Audit and update usages of the DataPrepperMarkers.EVENT marker
@dlvenable dlvenable added bug Something isn't working untriaged and removed untriaged labels Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Development

No branches or pull requests

1 participant