Jump to content

Shamus C

Super Seeqer
  • Posts

  • Joined

  • Last visited

  • Days Won


Everything posted by Shamus C

  1. I wanted to put together a quick guide on how to clear the cache for a particular signals inside of Seeq Workbench. Come common reasons for wanting to clear the cache on a signal Data source caching is turned on and you have changed a calculation in your source database You have added prior history to a tag and filled in a gap Steps to Clear the cache Open Item Properties Open Advanced Settings Click "Clear Cached Values"
  2. In R52+ we have replaced the concept of document owner with folders for each individual users. There are two slightly different methods for changing ownership depending on if the document orignates from a User's folder or the Corporate Drive If you want to change the owner of a document that has not been published to the corporate drive you need to move that document from the existing user's folder to your target new users. The gif below gives a quick overview of the process. If the document is in the Corporate Drive admins can edit the document and directly transfer ownership to another user. The document owner for Topics is important as the document owner will act as the user permissions to render all of the charts and graphics. The document owner thus must have access to all the datasources in a topic in order for it to render properly.
  3. We have had a couple of users ask for a method to create a table of timestamps and values for each of the samples in a signal. Below is a quick method to create such a table using the new features available in versions R53+ This method in general makes the most sense for finding the timestamps of discrete points but could be used for signal Step 1: Create a condition with a capsule for every sample point $signal.toCapsules() Step 2: Move to the table view and select the "Condition" option The general settings you are going to want to pick are Condition mode Headers = Start Time Columns Capsule Property = Value Final Product:
  4. There is a quick little trick to do this by combining two formulas together $signal.tocondition().tosignal() What this formula does is create a condition .tocondition() where each capsule starts when the value in your ID signal changes (eliminating duplicate values) and then transforms those capsules back into a signal using .tosignal() Let me know if this example helps solve your question
  5. Currently there is not a way inside of Seeq formula to access a signal or condition's metadata for use in a formula. This comes up when users would like to do things like plot a special property that has been added to a signal from an external system such as OSIsoft Pi AF. There is a simple way to add these properties as scalars inside of Seeq DataLab however using the example notebook below. The key pieces are to make sure to include the all_properties=True flag in the spy.search() command and then to define your new signal names in your metadata dataframe Push Signal Metadata as Scalars.ipynb
  6. Can you see the forecast signal ever crossing your empty threshold. You can visually mark your lower limit on the trend by creating a Scalar value in formula
  7. There is also a way to complete this inside of Seeq Formula. As a warning as rates change over the years these formulas could get longer and longer as you splice in new rate schedules against old rate schedules. Below is a formula for a rate schedule with a Winter Rate & Summer Off/Mid/High Rates. You could expand this to weekends and weekdays as well if needed $Summer = periods(6months, 1year, "2020-05-01T00:00:00","US/Pacific") $Winter = periods(6months, 1 year, "2020-11-01T00:00:00","US/Pacific").setProperty('Rate',0.10) $SummerHighPeak = (shifts(16, 5, "US/Pacific") and $Summer).setProperty('Rate',0.18) $SummerMidPeak = (shifts(14, 2,"US/Pacific") and $Summer).setProperty('Rate',0.16) $SummerOffPeak = ($Summer -($SummerHighPeak or $SummerMidPeak)).setProperty('Rate',0.12) $AllPeriods = CombineWith($Winter, $SummerHighPeak, $SummerMidPeak, $SummerOffPeak) $AllPeriods.toSignal('Rate').setUnits('$')
  8. You can also re-create the same effect at the past() operator in offline situations or in older versions by modifying the formula to remove the period of time that there is data in the original source signal $LowerLimit = 0% $AboveLowerLimit = $forecast > $LowerLimit $AboveLowerLimit - isvalid($original)
  9. Another common question through the support portal this morning that is of general interest To help with this example I am going to create a quick polynomial prediction using Data from Area C in the example set. Our target is going to be to try to predict compressor power as a function of all of the input weather signals If you wanted to re-create this prediction model in excel or another tool you need the coefficients from block #1 in the screenshot above and the y-intercept from block #2 in the screenshot. Inside of the workbench tool you will see rounded values for each of the coefficients and intercepts but the full values are available when you copy them to the clipboard by clicking the little button highlighted in red. To fill out the example in excel the formula will look like the following $temperature^2 * -0.000230 + $temperature * 0.0607 + $WB^2 * 0.000646 + $WB * -0.101 + 6.5946 A final point to mention here is that for multi-variable regressions with many input signals it is important to take a minute and evaluate the p-values listed in the coefficient table. If the p-values for any coefficient are above 0.05 it is best practices to rethink if that signal needs to be included in the model at all or if you may need to perform data data cleansing or re-alignment to create a better performing model. Good blog post on P-values - https://medium.com/analytics-vidhya/understanding-the-p-value-in-regression-1fc2cd2568af Great reference post on how to optimize regression models using time shifting -
  10. Sam, You can pretty easily forecast the value from now till you empty setpoint and then display the result as a Capsule on the screen as well as a Time/Duration in a Scorecard/Table and have all the results update in real-time For my example I am going to use some example data but this should look very similar for your use case Step 1 - Create a Forecasted Value - This function may not work on exactly as you expect on historical data depending on your datasource $signal.forecastLinear(1.5h,5d) //Train in the last 1.5 hours of data //Project 5 days into the future Step 2 - Create a condition that captures the time between now() and when you fall below your "empty" threshold. This step will only work for online data as it is using the past() operator $LowerLimit = 0% $AboveLowerLimit = $fv > $LowerLimit $AboveLowerLimit - past() Step 3 - Create a Scorecard to Quantify the time between now and full and display it in a table. This uses the "Condition" mode in the Tables view and a Condition scorecard type
  11. In Seeq version 52 and beyond we have introduced scheduled notebooks which opens up a new world of signals calculated using purely python and pushed into the seeq system on a schedule. These scheduled executions can be really powerful but also can place huge loads on the entire Seeq system if your notebooks are not designed with scheduled execution in mind. Attached is a notebook with some of our current thoughts on best practices to minimize the load on the system and not request and push the same sets of data over and over again with each notebook execution. General Calculation Plan Query our calculated signal name between now - Lookback length (1day) Find the last timestamp of our calculated signal Query the input signals to our calculation for the time range between that last timestamp and now Run our python script on our input signals to generate a new range of values for our calculated signal Push results back into Seeq for range between last timestamp and now Schedule next execution (15 min) Scenarios/methods you may want to avoid include pulling a month of data for all input signals, calculating values and then pushing results back on a 15 minute schedule. This will result in needlessly reading and writing the same data points over and over again. Example Scheduled Signal Calculation (1).ipynb
  12. Great question today from the support channel that I wanted to share along with a helpful spreadsheet. As much as I spend my days getting people out of Excel, sometimes it comes in really handy for creating signals from lookup tables. Attached to this Post is an excel sheet I put together where you can input a rate schedule and pricing tiers for Summer/Winter and On/Mid/Off peak pricing. The last sheet in the excel file is setup to be exported to CSV which can then be quickly imported back into Seeq for use in analysis and integration anywhere it would be helpful. Step 1 - In the excel sheet fill out the Lookup Tables sheet to match your utility rate schedule Step 2 - Export the "Output Table - Save as CSV" to a new CSV file Step 3 - Import to Seeq using the Import CSV tool making sure to fill out your timezone, Step interpolation, unit of measure ($) and under optional settings you may want to add the "Lenient daylight Savings" option to ignore those pesky clock changes. Electric Tier Pricing Worksheet.xlsx
  13. Starting in Seeq Version R52 Data Lab notebooks can be run on a schedule which opens up a world of new interesting possibilities. One of those possibilities is to create a simple script that pulls data from a Web API source and pushes it into the Seeq data cache on a schedule. This can be a great way to prototype out a data connection prior to building a full featured connector using the Connector SDK This example notebook pulls from the USGS which has information on river levels, temperatures, turbidity etc and pushes those signals for multiple sites into the Seeq system. The next logical step would be to make a notebook to organize these signals into an asset tree. Curious to see what this inspires other to do and to connect to. If there are additional public resources of interest put them in the thread for ideas. USGS Upload Example.ipynb
  14. Sometimes you want to clean up and remove an asset tree built using Seeq Data Lab from a Workbook so that you can restart or do something else. Below is the quickest way to remove an unwanted Tree Step 1 - In Workbench click in the info icon for the Asset Tree Step 2 - Copy the ID value to your clipboard. This may be located in a different place or under the advanced subsection in prior versions of Seeq Step 3 - Open the API Reference page from the Menu in the upper right corner Step 4 - Navigate down to the Trees/assets/{id} delete endpoint expand it, paste in the ID you copied in Step 2 and then click the Try it Out button You should receive a 200 confirmation response code and your Tree will no longer show up in your workbench analysis.
  15. Great question through the support portal today I am going to try to generalize for everyone. This requires joining two conditions and I am going to show how to do it in a using the simple tools and then how to combine it all together into one formula. Step 1 - Create a Value Search which matches you product code ABC. Once trick here is to search for "ABC*" that will return string results for anything starting with ABC and ending with any other series of characters. The * is part of Regex notation which you can use when searching strings inside the Value Search tool - https://seeq.atlassian.net/wiki/spaces/KB/pages/146637020/Regex+Searches Step 2 - Create a Condition for each batch using formula and the toCondition() function. This will create a capsule every time the value of our string signal changes. $mp.toCondition() Step 3 - Combine conditions together using the Composite Condition Tool and the intersection join Alternate Single Formula - Combine the steps in a multi-line formula $AllBatches = $mp.toCondition() $ProductRuns = $mp.contains("ABC") $AllBatches and $ProductRuns
  16. Yasmin, This should be pretty straight forward using the integral function in formula Step 1 - Create a condition that marks the start and end time for when you want to calculate the integral. This could be done with a value search or manual condition Step 2 - Use the integral function to calculate the new signal $signal.integral($condition.removeLongerThan(5d)).convertUnits('kWh')
  17. Question through the support channel Great use case which can be captured in a two quick steps Step 1 - Create a signal in the Formula tool with your custom aggregation. In this case we are going to use the periods() function to create capsules every 15 minutes and the delta() function to plot the change indensity from the start of the 15minute capsule to the end of the 15 minute capsule. Another alternative is to use the range() function which would return the absolute min - max of any values within the 15 minute interval. $myInterval = 15min $delta = $signal.aggregate(delta(),periods($myInterval),durationkey()) abs($delta) Step 2 - Create a Value Search to identify the periods of interest where the density change signal exceeds your desired threshold
  18. A follow-up on this post. As of Seeq version R48 there is no longer a limit on the number of capsules which can be displayed in chain view. So if you were interested in signal scrunching as a work around that is no longer needed. https://seeq.atlassian.net/wiki/spaces/KB/pages/736035141/What+s+New+in+R22.0.48 With a couple of our new functions this calculation is a lot simpler This requires a Manual Condition with a single capsule so that we know the starting point where we will calculate delays from. $delay = timesince($ManualCondition,1min,$Running) $signal.within($running).move(-$delay,30days)
  19. Final variation on this theme How do I calculates a running time since the last sample in my signal $cond = $VariableSampleRateSignal.toCondition().setMaximumDuration(30d) $cond.timeSince(1min)
  20. Unfortunately currently (R53) the ForecastLinear() operator does not allow for a calculated scalar to be passed in as a parameter to the function. However, I came up with this two step work around that should help get you started on your workflow Step 1 - Create a scalar for the time interval since your last step change In my example here I am going to use the time since the Example -> Area A -> Compressor Stage signal last changed values. You will need to create a similar calculation for your workflow $cond = $cs.toCondition().removeLongerThan(40h) //Create a condition with a capsule that starts each time you want to reset your training range $currentCap = $cond.toGroup(capsule(now()-1min,now())).pick(1) //Pick the most capsule closest to "Now" $currentCap.duration().convertUnits("min") //Plot the duration of the most recent capsule as a scalar value Step 2 - Manually generate linear forecast $trainingCapsule = capsule(now()-$Step1DurationScalar,now()+4h) //Create training range and distance you want to project signal into the future -- 4h in this example $training = group($trainingCapsule) //Convert capsule into a group for use in regression tool $timeSince = timeSince(condition(30d,$trainingCapsule),1min) //Create a signal which is a time counter since the start of your mode of interest -- 1min is sampling rate of new signal $model = $signal.regressionModelOLS($training,false,$timeSince).predict($timeSince) //Train model and project it into the future $signal.forecastSplice($model) //splice model into the orignal signal
  21. Unfortunately in the trend view there is not a way to stack the bars or create a legend. These two options are available in the Histogram tool but given your setup with a number of different signals I don't think there is a good way to configure your display for that tool. I will pass along your suggestions to our development team
  22. Quick method to calculate the median of multiple related signals over a time period $allSignals = combinewith($p1fr.move(1ns), $p2fr.move(2ns), $p3fr.move(3ns), $p4fr.move(4ns)) $allSignals.aggregate(median(),hours(),durationkey()) The first part of the formula combines all the samples from your signals of interest into a new single signal which has all of the datapoints from the first four signals. One trick with the combinewith() function is that it can't combine samples that occur at the exact same timestamp. Since I am using data from a csv file for this example where the timestamps for all 4 signals are exactly the same I use the move() function to slightly offset each of the signals by a nanosecond so that they all combine smoothly. The final part of the formula is the aggregation where we can calculate the median value of our new combined signals which has all the sample form our prior 4 signals. In this example I am calculating an hourly median but this can be changed to daily() or a calculated condition that makes the most sense for your application.
  23. The formula above is only focused on getting the single value of the time period between the very last sample and "Now". It uses the past() operator which was added in Seeq Version 51 so the error you are seeing is because you are probably on an earlier version. On your separate question about a signal that represents the time since the last sample this formula will give you a continuous signal that calculates things are you outlined in your table. You can replace the durationkey() parameter with either startkey() or endkey() depending on if you want the time between samples to be plotted at the beginning or end of the interval. $CapsEachSample = $signal.toCapsules() //create a capsule for the time between each sample in the orignal signal $CapsEachSample.aggregate(totalduration(),$CapsEachSample,durationkey()) .convertUnits("min") //convert the duration of the capsules into their own signal
  24. One additional option here is to hold the last valid value until a new one appears. The formula below allows you to mix in step interpolation characteristics for gaps into a linearly interpolated signal. $gaps = $signal.isNotValid().removeLongerThan(30d).move(-1ns,0) $holdSignal = $signal.aggregate(startValue(),$gaps,durationkey()) $signal.validValues().splice($holdSignal,$gaps)
  25. A generally interest question I want to document this morning Step 1 - Create a condition from the last sample till "Now" $caps = $MySignal.toCapsules() $timeSinceSample = past().subtract($caps) $timeSinceSample Step 2 - Create a Scorecard to Capture the duration of the condition created in Step 1 Make sure the Maximum Capsule Duration is longer than the longest gap you would expect in your signal. In this example it is 180 days
  • Create New...