Seeq function to aggregate last X days of valid data

[Ruby Han]

Hello,

I am wondering if there is a Seeq function to determine the last X days of valid data. For instance, in the last 7 days, 2 of those days are downtime or invalid data while 5 days are valid data. So hence, I would like to expand the reference window size to more than the set 7 days in order to include 7 days of actual valid data. Is there currently an out of the box function or a workaround?

Thank you.

[Siti Tay]

Hi Ruby Han,

Step 1. Create capsules for every 7 valid samples ($toCapsulesbyCount) for every sample : 

*Note that the function toCapsulesByCount() is available in Seeq versions R54+

$signal.toCapsulesByCount(7, 10d)

Step 2. Use signal from condition to determine the X days of 7days valid data : 

If you would like to aggregate every 7 valid samples, you can use aggregateByCount  function which also introduced in version R54. 
Example using this function can be found here : 

Hope this help.

Edited April 4, 2023 by Siti Tay

[Ruby Han]

Thanks for responding. I have not really tried your approach yet as I found a post that provided a solution that I was kind of looking for. This is the post URL: Replace Gaps in Data with an Average Value from Previous Time Frame - General Seeq Discussions - Seeq Community Website 

To summarize the post, it's basically imputing on the missing gaps with the average value for X time period which is essentially what I'm looking for. However, when I tried replicating the solution and modifying the 10 min in order to make it a dynamic parameter for $conditionForDataGaps.move(-10min,0min), I'm getting an error as the dynamic parameter is basically the capsule duration which is a variable signal type.

 

$w - brown trend

$capsuleDuration - yellow trend

$gapCondition - pink capsules

green trend is the signal I'm determining where $gapCondition is at.

@Chris Orr recommended a transform for the move parameter but I'm not knowledgeable enough to implement that here. 

I hope this clarifies my problem. Thanks!

 

[Joe Reckamp]

Hi Ruby,

Here's an example of the transform method:

$step.move(-30d,0)
     .transform($capsule -> {
                $capEnd = $capsule.endKey()
                capsule($capEnd-$durationsignal.valueAt($capEnd), $capEnd)
                })

Some notes about this:

You can only make a capsule that's within the capsule that goes into the transform. That's why I started with the .move(-30d,0). Basically what I'm doing there is saying I'd like to keep the end of the capsule (because that's what your new capsule will be based off of) and in worst case scenario, the capsule would be 30 days long so I move it 30 days to start. Adjust this as necessary to match your use case. Then inside the transform I'm  saying make a new capsule that ends where my previous capsule ended and starts the amount of time of the $durationsignal at the point where my original capsule ended before the end of the capsule. 

Hopefully this makes sense, but let me know if you have any questions or difficulties with implementing this.

[Ruby Han]

I'm getting this error. Not sure if I'm implementing this correctly.

[Joe Reckamp]

Hi Ruby,

You'll need to add a .removelongerthan() before the .transform. I don't know how long your $r could possibly be (+30 days for the move), but I'd guess something like .removelongerthan(90d) would be more than sufficient and could probably be even shorter.

[Ruby Han]

Thanks for the help!

@Emilio Conde is working on this with me and modifying the expression you've provided and we're close to what I'm trying to get at! Thank you!

[Emilio Conde]

Ruby and I implemented a solution that avoids the .transform() function. Details can be found here.