Jump to content

John Cox

Seeq Team
  • Posts

    27
  • Joined

  • Last visited

  • Days Won

    11

John Cox last won the day on September 9

John Cox had the most liked content!

Personal Information

  • Company
    Seeq
  • Title
    Analytics Engineer
  • Level of Seeq User
    Seeq Intermediate

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

John Cox's Achievements

Apprentice

Apprentice (3/14)

  • Conversation Starter
  • First Post
  • Collaborator Rare
  • Week One Done
  • One Month Later

Recent Badges

16

Reputation

  1. Accurate data alignment can be the most challenging part of creating process calculations, finding correlations, and developing prediction models. It is an often overlooked data cleansing step that may be even more critical than smoothing noisy signals and removing data outliers. The need for data alignment stems from time delays present in the industrial process (related to physical transport and equipment/piping volumes, as well as lab measured data reported well after process operation completes). Another common data alignment need centers on comparing process metrics before/after process events of interest, which again involves time delays (or time shifts). There are at least four categories of data alignment use cases prevalent across the process manufacturing industries: Known time delay - The time delay resulting from the transportation of material at a given speed or velocity (across some distance in the process operation) can mask strong correlations between upstream/downstream signals or other signals separated in time, resulting in poor modeling results if not accounted for by a data alignment step. Variable time delay – Here the time delay is variable and a function of production speed, storage volumes, etc. but can be calculated based on measured/known process parameters. Before/after comparisons - Related to process experimentation and optimization, there is a recurring need to calculate and compare process metrics before and after some identified process event, such as a process feed being turned on, a unit restart, equipment maintenance, a process parameter or setpoint being adjusted, etc. Process and analytical (LIMS) data - When trying to correlate process signals with lab-measured analytical results, work is often needed to align the process signals and subsequent analytical results. In some cases, the alignment can be based on sequential, consistently reported lab data values. In other cases, more sophisticated logic, such as connecting process operation and lab results by matching id properties, is needed. Example methods for addressing each of these cases, using Seeq tools and Formula, are included below. Use Case Category #1: Known Time Delay In this specific example, the goal is to correlate a process signal (temperature) with a later reported lab result, but this use case also occurs frequently with continuous upstream (earlier in the process) and downstream signals (later in process), and the same time shifting methodology applies. It is known that the lab result is consistently reported 2 hours after the process operation which would most correlate. Therefore, the “Process Temperature” signal needs to be shifted 2 hours to the right in time and then sampled for each unique “Lab Measured Analytical Result” value. Analysis Steps 1. We shift the raw "Process Temperature" 2 hours to the right using Seeq Formula's move() function to create "Temperature Shifted 2 Hours to Right": $Temperature.move(2h) 2. We use Seeq Formula's resample() function to pick off values of the shifted temperature at the exact timestamp that new "Lab Measured Analytical Result" values are reported. The resample() function gives the ability to sample one signal based on the timestamps of the data values from another signal. The result is 1 temperature sample (green, Lane 1) per lab sample (Lane 2). $TemperatureShifted.resample($LabDataValues) 3. The value of the data alignment is obvious by comparing two XY plots of the lab result and the original, raw temperature/aligned temperature. The correlation between the process temperature and the lab result, hidden in the XY plot on the left, is obvious in the XY plot on the right, which shows the aligned temperature: Use Case Category #2: Variable Time Delay This use case is similar to the previous: correlating two measurements separated by a time delay, but with an additional complication. Here, the time delay is variable and is based on the physics of a material transport distance: changes in an upstream pressure physically take many hours to work their way through equipment/piping and influence a downstream analyzer signal. The physical time delay varies based on Production Line Speed. Analysis Steps 1. The transport distance is known (500 feet) and set up as a value using Seeq Formula. The user calculates the Time Delay between the 2 signals based on the Transport Distance / Production Line Speed. The resulting time delay fluctuates between 7 and 13 hours. Transport Distance Formula: 500.toSignal().setUnits('ft') Calculated Time Delay Formula: $TransportDistance/$LineSpeed 2. Using Seeq Formula's move() function, the Upstream Pressure can then be shifted (by the variable time delay calculated in Step 1) so that the effect of pressures changes upstream aligns in time with the resulting impact on the Downstream Analyzer. // Limit the maximum time delay to 20 hours $UpstreamPressure.move($CalculatedTimeDelay,20h) Changes in the upstream pressure (yellow signal) are now correctly aligned in time with their effect on the downstream analyzer (blue signal): Note that in this use case, we did not need to use the resample() function as we did with the discretely measured lab data in Use Case Category #1, as here we are working with continuously measured signals. Use Case Category #3: Before/After Comparisons An extremely common use case is to compare process metrics before/after some process event. The process event could be anything of interest: a process unit restart, equipment or control strategy modifications, periodic maintenance work, process experimentation, etc. The analysis begins by identifying the process events and then calculating the metrics over appropriate time ranges before and after each event. The alignment step typically involves creating a common time basis spanning before/after operation and moving the before/after calculated metrics to the same point in time for comparison. Analysis Steps 1. For this example we begin with a Pressure signal that ramps up over time as equipment run life increases and the equipment fouls. The equipment is shut down for Maintenance periodically and the process then restarts at a much lower pressure value. We use the Pressure signal and Seeq Formula to identify Equipment Maintenance periods based on the running delta of the pressure signal < 0: $Pressure.runningDelta().isLessThan(0) 2. Use Formula's grow() function to expand Equipment Maintenance to create a Before and After Maintenance time period which includes the desired time period (for example, 4 hrs) for calculating before/after pressures. This also gives a common time basis for a later alignment step. $EquipmentMaintenance.grow(4hr) Results are shown here in Chain View for the Before and After Maintenance capsules, which gives a nice visual of the pressure 4 hours before and after maintenance. 3. Average the Pressure signal over the first 2 hours of the Before and After Maintenance capsules. // Do a 2 hour avg pressure over the FIRST 2 hours // of the Before and After Maintenance capsules $Pressure.aggregate(average(),$BeforeAfterMaintenance.afterStart(2hr),middleKey()) 4. Use Signal from Condition to align the average pressure before maintenance (Step 3) at the middle of the Before and After Maintenance capsules. Signal from Condition is commonly used to find a value within a condition and move it to a specific location in time. In this case, we can use the Maximum statistic to find the correct "Pressure Avg (Before)" value, as there is only 1 value for each maintenance capsule. Using Signal from Condition in this way is a key technique for aligning before/after process values/metrics. We can see that the "Pressure Avg (Before)" has now been moved to the middle of the "Before and After Maintenance" capsules: 5. Repeat steps 3 and 4 to compute/align the average pressure after maintenance, and then move/align to the middle of the Before and After Maintenance capsules. 6. With the before/after pressure values aligned in time, we now use Formula to calculate the percent pressure reduction resulting from maintenance. // Calculate the % change relative to the // the Pressure Avg (Before) (($BeforeAvg-$AfterAvg)/$BeforeAvg*100).setUnits('%') Use Case Category #4: Process and Analytical (LIMS) Data In this example, we will align calculated process metric values from each batch (e.g., max temperature, total Chemical A flow added) with later reported lab results (often referred to as LIMS - Laboratory Information Management System data).This is a very common analytics need. Here, the “Product Impurity” lab results are reported at varying time intervals following batch completion, so a constant time delay alignment approach isn’t feasible. Zooming in, we can confirm that the lab results are reported (step to a new value) some time after the Process Batches complete. The reporting time is variable. Analysis Steps 1. The maximum temperature and totalized chemical added over each Process Batch are thought to correlate with (influence) the amount of final Product Impurity. So, we need to calculate the max T and total chemical A added per Process Batches capsule. Use Signal for Condition to do the calculations and place results at the end of each Process Batch. 2. We now need to work on alignment. To provide a basis for joining the process results to the corresponding lab results, we need to create "Product Impurity Results Capsules" for every value change in the reported Product Impurity. We use the Formula toCondition() function for this and store the numeric lab result in a capsule property named PctImpurity. $ProductImpurityData.toCondition('PctImpurity') Note: in the screenshot below, the highlighted Product Impurity Results capsule contains the lab result for the Process Batches capsule at the top left of the screen. 3. Now, use Composite Condition to "join" the Process Batches condition to the Product Impurity Results Capsules condition, so we have a time period that contains the process and lab results we want to align and correlate. We check the Inclusive options to create capsules from the start of Process Batches to the end of Product Impurity Results Capsules: The resulting yellow "joined" capsules in the screenshot below now span the time period of process operation and the eventual lab measured impurity result, for each individual batch. Investigating a zoomed time period, we can see the Process Batches start joined to the end of the resulting Product Impurity Results Capsules. 4. With a common capsule established, we align Max T, Total Chemical A, and Product Impurity at the middle of the "Process Batches to Impurity Result (joined)" capsules. For Max T and Total Chemical A, we use the 'Value at Start', and for Product Impurity, we use the 'Value at End'. Using Signal from Condition in this way is a key technique used in aligning process and lab data values. The results for a short time period look like this, and the "aligned" values can be used for further calculations or for creating a prediction model to predict Product Impurity based on Max T and Total Chemical A: Joining Process and Lab Values Based on a Matching Capsule Property With that example finished, let’s look at another common (and more complicated) scenario in this use case category: when analytical results can be reported inconsistently or out of sequence with process batches, a more advanced condition join using Seeq Formula, based on a matching id or other capsule property value, may be needed. In this example, a numeric batch id is a capsule property shared by the Process Batches and Lab Analytical Batches conditions: see the 437 and 438 capsule property values shown as labels on the blue and green capsules in the screenshot below. Using this batch id linkage, matching id Process/Lab capsules can be joined with a single line formula, and capsule properties from the separate process and lab conditions (e.g., see the 0.99 and 1.32 lab product impurity results shown on the blue capsules) can be preserved, all courtesy of enhanced “capsule matching by property” functionality introduced in Seeq R56 (link). Starting with the raw data and Process Batches and Lab Analytical Batches conditions and their capsule properties, we have already calculated the "Average Process Ratio for Batch" using Signal from Condition and located it at the start of the Process Batches capsules. We illustrate only the critical steps in this use case under Analysis Steps below: Analysis Steps - Joining Process and Lab Data on a Matching Capsule Property 1. We join the Process Batches (capsules) to Lab Analytical Batches, based on their BatchID/LabID property matching. We use the join() Formula function. The batch id numeric is stored as the "LabID" capsule property in the Lab Batches condition, so, prior to doing the join, we must rename it to have the same property name as the "BatchID" capsule property on the Process Batches condition. Note: the capsule property matching and keepProperties() are enhanced functionality options introduced in Seeq R56. // Join process to lab batches based on matching ID. // We need to rename the LabID capsule property to BatchID for the // lab capsules. // Use keepProperties() so that the resulting condition has // the Result capsule property (the lab measured product impurity value). $ProcessBatches.join($LabBatches.renameProperty('LabID','BatchID'), 4d, true, 'BatchId', keepProperties()) Inspecting the capsules and the batch id and product impurity capsule property values at the top of the trend, we see the "Process Batches Joined" capsules are linked based on matching ID, and the product impurity "Result" capsule property (the 0.99 and 1.32 values) is retained and now part of a capsule that starts at the beginning of each Process Batches capsule: 2. We now translate the "Process Batches Joined to Lab Batches on ID Match" Result capsule property into a "Lab Measured Product Impurity Aligned to Batch Start" signal, with values moved to the start of the Process Batches, at the exact location we have the Avg Process Ratio for Batch. $JoinedBatches.toSignal('Result',startKey()).toDiscrete() For this short time range, we can confirm the 0.99 value for the resulting impurity signal aligns correctly to BatchID 437, and the 1.32 value aligns correctly to Batch 438. As a result, we have now connected the lab measured product impurity result to each individual process batch, regardless of lab result timing, and with additional steps have aligned the Average Process Ratio and Lab Measured Product Impurity.
  2. A common industrial use case is to select the highest or lowest signal value among several similar measurements. One example is identifying the highest temperature in a reactor or distillation column containing many temperature signals. One of many situations where this is useful is in identifying the current "hot spot" location to analyze catalyst deactivation/performance degradation. When selecting the highest value over time among many signals, Seeq's max() Formula function makes this easy. Likewise, if selecting the lowest value, the min() Formula function can be used. A more challenging use case is to select the 2nd highest, 3rd highest, etc., among a set of signals. There are several approaches to do this using Seeq Formula and there may be caveats with each one. I will demonstrate one approach below. For our example, we will use a set of 4 temperature signals (T100, T200, T300, T400). Viewing the raw temperature data: 1. We first convert each of the raw temperature signals to step interpolated signals, and then resample the signals based on the sample values of a chosen reference signal that has representative, regular data samples (in this case, T100). This makes the later formulas a little simpler overall and provides slightly cleaner results when signal values cross each other. For the T100 step signal Formula: Note that the T200 step signal Formula includes a resample based on using 'T100 Step' as a reference signal: The 'T300 Step' and 'T400 Step' formulas are identical to that for T200 Step, with the raw T signals substituted. 2. We now create the "Highest T Value" signal using the max() function and the step version T signals: 3. To create the '2nd Highest T Value' signal, we use the splice() function to insert 0 values where a given T signal is equal to the 'Highest T Value'. Following this, the max() function can again be used but this time will select the 2nd highest value: 4. The process is repeated to find the '3rd Highest T Value', with a very similar formula, but substituting in values of 0 where a given T signal is >= the '2nd Highest Value': The result is now checked for a time period where there are several transitions of the T signal ordering: 5. The user may also want to create a signal which identifies the highest value temperature signal NAME at any given point in time, for trending, display in tables, etc. We again make use of the splice() function, to insert the corresponding signal name when that signal is equal to the 'Highest T Value': Similarly, the '2nd Highest T Sensor' is created, but using the '2nd Highest T Value': (The '3rd Highest T Sensor' is created similarly.) We now have correctly identified values and sensor names... highest, 2nd highest, 3rd highest: This approach (again, one possible approach of several) can be extended to as many signals as needed, can be adapted for finding low values instead of high values, can be used for additional calculations, etc.
  3. Hi Chase, Teddy has a great suggestion above on using Seeq's OData export. I also wanted to mention another option for your consideration: creating an Add-on tool in Seeq Workbench that has a button the operators click to pull the data. Seeq Add-ons use the Seeq Python Library module (SPy) with Seeq Data Lab. Add-ons can easily pull data into the Data Lab environment and create custom visualizations, calculations, charts, etc. using any Python library functionality, and all of this can be accessed from Seeq Workbench. This may or may not be a way to address your current need, but even if not, it may be a good solution for other applications in the future. There is a Seeq Add-on Gallery here: https://seeq12.github.io/gallery/ The following links provide additional information on how to create your own add-ons or install the open source add-ons: https://seeq.atlassian.net/wiki/spaces/KB/pages/2254536724/Add-ons
  4. Hi, Currently you can add a capsule property to a condition type table without using the New Metric, by using the Row or Column buttons. But, while you do have filtering and sorting capabilities on these capsule properties in the table, this doesn't give you the ability to color code the table cells for that property (New Metric with thresholds is needed for that). So yes, currently you would need to convert the capsule property to a signal first. There is a new feature request in our development system to do exactly what you want, so this may be added in the future. John
  5. The details and approach will vary depending on exactly where you are starting from, but here is one approach that will show some of the key things you may need. When you say you have 3 capsules active at any given time, I assume you mean 3 separate conditions. Assuming that is the case, you can create a "combined condition" out of all 9 conditions using Seeq Formula: // Assign meaningful text to a new capsule property // named TableDesription $c1 = $condition1.setProperty('TableDescription','High Temperature') $c2 = $condition2.setProperty('TableDescription','Low Pressure') and so forth... $c9 = $condition9.setProperty('TableDescription','Low Flow') // Create a capsule that starts 5 minutes from the current time $NowCapsule = past().inverse().move(-5min) // Combine and keep only the currently active capsules (touching "now") combineWith($c1,$c2,...,$c9).touches($NowCapsule) You can then go to Table View, and click the Condition table type. Select your "combined condition" in the Details Pane and add a capsule property (TableDescription) using the Column or Row button at the top of the display. You can then set your display time range to include the current time, and you should see the currently active capsules with the TableDescription text that you assigned in the Formula. You of course will want to vary the "-5min" value and the 'TableDescription' values per what is best for you. Your approach may be a little different depending on where you are starting from, but I think that creating capsule properties (text you want to see in your final table), combining into one condition, and using .touches($NowCapsule), may all be things you will need in your approach. Here are some screenshots from an example I put together, where I used "Stage" for the capsule property name:
  6. Hello Siang, Currently, simple type scorecard metrics can't be used in Seeq Formula. We have recently added some functionality for using condition type scorecard metrics in Formula, but that will not help you in this use case. Your request to get a display range capsule is a good one and is already on our feature request list for our development team. For now, your best method is to create a Manual Condition, and add a capsule to the Manual Condition that is your current display range, to create a Display Range Condition. You can then use the Display Range Condition with Signal from Condition to calculate a signal representing the average (or other desired statistics) over the display range. That signal can be used in additional Formulas and calculations. When you want to change your analysis to a new display range, you will of course need to take one extra step to edit the Manual Condition and update it to match the current display range. John
  7. Hi Bayden, I would try something like this in Seeq Formula, on the signals that change infrequently: $signal.validValues().toStep(30d) In place of the "30d" you would enter the maximum time that you want to connect data points and to see the lines drawn. Given you are working with setpoints, I would recommend converting them to step interpolation using .toStep(). If that doesn't get you to the final result that you want, you can add one further function: $signal.validValues().toStep(30d).resample(1d) The resample will add more frequent data values at the period you specify (change the "1d" to whatever works best for you). You do need to be careful that you aren't adding any incorrect or misleading values with this approach, based on your knowledge of the signals. Let us know if this helps. I believe you can get a solution you are happy with. John
  8. Hello Filip, When you want to add data to an already existing signal, you will want to find your original imported file in the Seeq Data tab, and click the edit button (circled in red below): You will then be able to drag a new file for import, and you will notice that you now have Replace and Append options (see an example I did below): You can choose to replace or append. I think this will solve your problem. If it does not, please let me know. Some additional information on adding new data to an existing imported signal can be found here in our Knowledge Base: Knowledge Base: CSV Import Replace and Append John
  9. Hi Brian, In more recent versions of Seeq, the max function used in that way (that specific syntax or form) only works with scalars. For your case, try $p53h2.max($p53h3).max($p53h4) That is the form needed with signals. Hope this helps! John
  10. Hi Robin, 1) To answer your earlier question, to avoid the time weighted standard deviation, all you need to do is change the signal to a discrete signal and then apply the same formula: $CanWeightSignal.toDiscrete() .aggregate(stdDev(), // std deviation statistic periods(10min,1min).inside($CanProductionCapsules), // do the calc over 10 min rolling window every minute // but only for 10 minute capsules fully inside CanProduction capsules endKey() // place the calculation result at the end of the 10 minute window , 0s) // max interpolation between data points, use 0s if you want discrete result 2) To answer your most recent question, to convert your current result to a stepped signal rather than the discrete results you now have on the trend, you can use a Formula function named .toStep(). Inside the parentheses, you enter the maximum amount of interpolation, the maximum amount of time you want to draw a line connecting your calculated results. For example, the function below sets a maximum interpolation between data points of 8 hours. .toStep(8hr) Hope this helps! John
  11. Hello Mohammed, The answer depends on exactly what you want to remove from the signal. In some cases, you may want to do a Value Search to find the peaks you want to remove, then use the .remove() function in Seeq Formula to create the new signal with the peaks removed. In some cases, all you may need is to apply a smoothing filter (for example, agileFilter() in Seeq Formula or one of the Signal Smoothing options under the Cleanse Tools), or perhaps use removeOutliers() in Seeq Formula. The documentation below provides a great summary of the many data cleansing tools in Seeq. I hope this information is what you are looking for!
  12. I do not have access to your original signal for testing, so my solution not handle all the unusual characteristics that may occur. But I believe the approach below will help you get going on this calculation. 1. I would first recommend you create a clean weight signal that uses the remove() function to eliminate small decreases in the weight signal due to measurement noise and equipment variation. (This first step makes step 2 work accurately, because in step 2 we will be looking for the weight to go above 2000, 3000, 4000, etc., and we don't want noise variation from say 3002 lb to 2999 lb at the next sample, to appear to be a step decrease.) Create a clean weight signal using the formula below. This should result in a weight signal that only increases during the normal rampups, but still resets to 0 when the next rampup processing begins. Note that the 1hr inside of toStep() is the maximum interpolation between data values; set this based on your preferences. 2. Create the Step Counting Signal by dividing the clean weight (step 1 result) by 1000 and using the floor function to generate an integer result. 3. For an example signal that I tested with, you can see the Step Counting Signal incrementing as each 1000 lb increment is exceeded. Depending on your final analysis goals, further calculations could of course be done to calculate/report the number of steps completed for each of your original purple capsules or to do other types of statistics.
  13. Hi Robin, I would use a Seeq formula similar to the one below, with the Can Weight signal and your existing Can Production capsules as inputs. The key functions used are aggregate(), periods(), and inside(). Note that you could also do the std deviation calculation using Signal from Condition, using a condition created in Formula with: periods(10min,1min).inside($CanProductionCapsules). $CanWeightSignal .aggregate(stdDev(), // std deviation statistic periods(10min,1min).inside($CanProductionCapsules), // do the calc over 10 min rolling window every minute // but only for 10 minute capsules fully inside CanProduction capsules endKey() // place the calculation result at the end of the 10 minute window , 0s) // max interpolation between data points, use 0s if you want discrete result Hope this helps! John
  14. Hi Sivaji, While this isn't easy to do in the current Seeq scatter plot view, below are two related forum posts that I would suggest you read. These may help with workarounds. Please note that your request (scatter plots with signals from multiple assets) is already a feature request in our system, and it will be added to the software at some point in the future.
  15. Hi Clinton, I would suggest you come to one of Seeq's open Office Hours and discuss this with one of our Analytics Engineers. They can likely get you started or make suggestions on how to approach this analysis. There are Office Hour slots available each day of the week. Please read more at this link: Seeq Office Hours
×
×
  • Create New...