Jump to content

Specifying a dataframe’s timezone to ensure .push() yields matching timestamps between Workbench and source dataframe


Emilio Conde
 Share

Recommended Posts

  • Seeq Team

Timezone mismatches can oftentimes arise when using the .push() function with a dataframe. To ensure the dataframe’s timezone matches with the source workbench, we can use pandas tz_localize() function. See an example of encountering and addressing this issue while pushing a csv dataset into workbench below.

 

Step 1:

Complete imports

Screen Shot 2021-08-27 at 3.58.40 PM.png

 

Step 2:

Load in csv file as a dataframe. When you want to push data, it must have an index with a datetime data type. That's why we used the parse_dates and index_col arguments for Pandas.read_csv(). Note my csv file’s date/time column is named TIME(unitless), hence the arguments within parse_dates and index_col.

Screen Shot 2021-08-27 at 4.00.29 PM.png

 

*** Note the dates in Out[5] all are -06:00***

 

If I simply moved forward to .push(), I’d see the following results:

Screen Shot 2021-08-27 at 4.05.08 PM.png

Screen Shot 2021-08-27 at 8.36.32 AM.png

 

The original data’s dates are not properly aligned with my worksheet, which is in US/Eastern. Instead, I should use the tz_localize() function on my index before pushing. See Step 3.

 

Step 3:

Use the tz_localize() function on your index to first remove any native timezone from the dataframe, then again to assign the timezone of interest to the dataframe. 

Screen Shot 2021-08-27 at 4.15.39 PM.png

 

*** Note the dates in Out[8] now are all -04:00***

 

Finally, I can proceed to push the data into Seeq. You can now see that the timestamps of my data in workbench matches with their original timestamps.

Screen Shot 2021-08-27 at 4.21.48 PM.png

Screen Shot 2021-08-27 at 8.57.19 AM.png

 

 

 

 

 

Edited by Emilio Conde
Included incorrect images
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...