Plume data ingestion duplicate data

[Riyad-boop]

Pulling data using new method (changing offset every 2 days) I think is causing an overlap in data, which is causing the plots to look like the one below. We need a way to check for duplicate data or maybe some correction needs to be applied to the data pulling method.
The image at the bottom shows there exists duplicate data (8000 rows) so the root cause is most likely how the offset is being used.

[Riyad-boop]

Redesign plume data fetching code to get data with null values