Rolling 'training window' for reference profile etc

[Ruben]

I am interested in creating a reference profile based on a rolling 'training window'. For example, I want to create a reference profile for a boundary based on the last 7 days of data, or the data for the same month last year. How would I go about doing this?

Thanks

[Lindsey.Wilcox]

Hi Ruben-

Are you referring to the Training Window in the Reference Profile tool?  If so, the Reference Profile tool is designed to create a static profile, so it is not possible to define a dynamic Training Window (like the last 7 days).

However, the latest version of Seeq (R21.0.41) introduced the now() operator in Formula, which may be used to compute dynamic, single-value boundaries.  Here is an example of how this may be used.

Given a temperature signal, lets create upper and lower boundaries as the (avg )+/- (2*std. dev.) of the past 7 days.  The now() operator us used to define the time period over which the average and standard deviation are calculated

1. First, lets calculate the average of the temperature signal over the past 7 days:

2. Next calculate the standard deviation of the temperature signal over the past 7 days:

3. Now create the upper and lower limits:

 

Please let me know if you have any additional questions.

 

Thanks,

Lindsey

[Thorsten Vogt]

Hi Ruben,

maybe you can use a rolling training window in the following way. For this demo I want to create a reference profile for the Compressor Power when the stage of the compressor is != OFF. The rolling training window shall be the last 7 days as you said.

1. Add the data to the display pane

2. Create a new signal for the compressor power, that takes the data of the last 7 days and moves this data to some fixed date range (e.g. 01/01/2019):

3. Repeat this step for compressor stage signal:

4. Create a condition that identifies the times when the compressor is on during training window:

Your display now should look like this when you move to 01/01/2019:

5. Create a condition that identifies the times when the compressor is on (NOT during training). We will use this for the reference profile to be applied to:

6. Create the reference profile based on the training window. I had to shorten the training window by two minutes at end due to an error message ( Training window for reference table must be certain at 'captureReferenceSlice',).

The result looks like this:

 

 

To simulate a change of time you can modify the formulas for the training data like this  You can see that the shape of the profile changes:

 

I am not sure if this workaround may cause problems in some situations, as it seems the error message in step 6 is thrown because there is no sample at the end of the training window. Therefore this solutions has to be adjusted to your needs.

 

Regards,

Thorsten

 

[Ben Johnson]

This is a creative solution, but I wouldn't depend on it. It's exposed a bug in the delay() function regarding uncertainty. The offset that is being passed to delay() is "uncertain". That means it's possible the value could change. But delay() is not reflecting the uncertainty of the offset in its output properly (the whole signal should be uncertain), which makes it possible to use the delayed signal in the reference profile.

You'll see the negative effects of this after a few days - the portion of the signal that was cached using an older training window won't get updated to reflect the more recent training window.

[Thorsten Vogt]

Hello Ben,

does "uncertain" in this context mean the values could change because of getting the data dynamically by using now()?

If I change the code from my example above without using now() and changing to a fixed capsule the visualization changes (2. lane):

 

Is there any way to get around this issue?

Regards,

Thorsten

[Ben Johnson]

Yes. It may seem odd to call now() "uncertain" - don't we have accurate clocks? It really means that the value will change, and any computation using that value is likely to get get different results in the future so it shouldn't be cached. 

The reference series feature is really designed to create static profiles - the performance impact of having to recompute the profile every time that it is used was considered too great. So I don't think there's a workaround.

Perhaps you could describe your use case where a dynamic profile has significant value over the static profile? Are the profiles expected to change significantly over time such that recent behavior is more relevant than a training window(s) that was identified as the "golden batch"? If there are interesting use cases, perhaps it's worth filing a feature request as a result.

[Thorsten Vogt]

In case of the reference profile I don't have a use case.But I would like to know if this uncertainty resulting in using now() is a problem for predicting a signal into the future? In my case I have a signal that depends on two other signals. In Seeq example data I created a demo that creates a model of the "Temperature" signal by using "Wet Bulb" and "Relative Humidity". To get predicted values for the forecast of  5 days I take the values of the humidity for the last 24hs and repeat it the next 5 days. I then use the values of Wet Bulb from the last 24hs and repeat them also over the next 5 days with rising them by 1°C per day:

The prediction for the temperature signal is build upon this two signals:

[Ben Johnson]

Predicting into the future is a great use of now(), and it's uncertainty is a key part of keeping the predicted data from polluting the cache.

I'm sure you're working toward your own use case, but here's a different approach that's a bit more concise (and uses some newer functions). I'm sure we agree that yesterday's temperature has no prediction value for future, but that's just the demo data. It seems you're just looking for a golden batch to repeat with some adjustment into the future

// extract a single good day to repeat, then repeat it every day
$repeatShape = referenceTable(5min)
  .addRow($temp, capsule("2019-05-15"))
  .repeatOver(days(), ReferenceTableStat.Average)
// this is 5F/day rise since now
$slopeLine = timeSince(now(), 1d).setUnits('F') * 5
// stitch known with future
$temp.forecastSplice($repeatShape + $slopeLine )