Jump to content

Leaderboard

Popular Content

Showing content with the highest reputation since 03/29/2023 in all areas

  1. Tulip is one of the leading frontline operations platforms, providing manufacturers with a holistic view of quality, process cycle times, OEE, and more. The Tulip platform provides the ability to create user-friendly apps and dashboards to improve the productivity of your operations without writing any code. Integrating Tulip and Seeq allows Tulip app and dashboard developers to directly include best-in-class time series analytics into their displays. Additionally, Seeq can access a wealth of contextual information through Tulip Tables. Accessing Tulip Table Data in Seeq Tulip table data is an excellent source of contextual information as it often includes information not gathered by other systems. In our example, we will be using a Tulip Table called (Log) Station Activity History. This data allows us to see how long a line process has been running, the number of components targeted for assembly, actually assembled, and the number of defects. The easiest way to bring this into Seeq is as condition data. We will create one condition per station and each column will be a capsule property. This can be achieved with a scheduled notebook: import requests import json import pandas as pd # This method gets data from a tulip table and formats the data frame into a Seeq-friendly structure def get_data_from_tulip(table_id, debug): url = f"https://{TENANT_NAME}.tulip.co/api/v3/tables/{table_id}/records" headers = { "Authorization": AUTH_TOKEN } params = { "limit": 100, "sortOptions" : '[{"sortBy": "_createdAt", "sortDir": "asc"}]' } all_data = [] data = None while True: # Use for paginating the reqeusts if data: last_sequence = data[-1]['_sequenceNumber'] params['filters'] = json.dumps([{"field":"_sequenceNumber","functionType":"greaterThan","arg":last_sequence}]) # Make the API request response = requests.get(url, headers=headers, params=params) if debug: print(json.dumps(response.json(), indent=4)) # Check if the request was successful if response.status_code == 200: # Parse the JSON response data = response.json() all_data.extend(data) if len(data) < 100: break # Exit the loop if condition is met else: print(f"API request failed with status code: {response.status_code}") break # Convert JSON data to pandas DataFrame df = pd.DataFrame(all_data) df = df.rename(columns={'id': '_id'}) df.columns = df.columns.str.split('_').str[1] df = df.drop(columns=['sequenceNumber','hour'], errors='ignore') df['createdAt'] = pd.to_datetime(df['createdAt']) df['updatedAt'] = pd.to_datetime(df['updatedAt']) df = df.rename(columns={'createdAt': 'Capsule Start', 'updatedAt': 'Capsule End', 'duration': 'EventDuration'}) df = df.dropna() return df def create_metadata(station_data, station_name): print(f"DataFrame for station: {station}") print("Number of rows:", len(group)) metadata=pd.DataFrame([{ 'Name': station_name, 'Type': 'Condition', 'Maximum Duration': '1d', 'Capsule Property Units': {'status': 'string', 'id': 'string', 'station':'string', 'duration':'s'} }]) return metadata # This method splits the dataframe by station. Each Station will represent a condition in Seeq. def create_dataframe_per_station(all_data, debug): data_by_station = all_data.groupby('station') if debug: for station, group in data_by_station: print(f"DataFrame for station: {station}") print("Number of rows:", len(group)) display(group) return data_by_station # This method sends the data to Seeq def send_to_seeq(data, metadata, workbook, quiet): spy.push(data=data, metadata=metadata, workbook=workbook, datasource="Tulip Operations", quiet=quiet) data = get_data_from_tulip(TABLE_NAME, False) per_station = create_dataframe_per_station(data, False) for station, group in per_station: metadata = create_metadata(group, station) send_to_seeq(group, metadata, 'Tulip Integration >> Bearing Failure', False) The above notebook can be run on a schedule with the following command: spy.jobs.schedule('every 6 hours') This will pull the data from the Tulip Table into Seeq to allow for quick analysis. The notebook above will need you to provide a tenant, API key, and table name. It will also be using this REST API method to get the records. Once provided, this data will be pulled into a dataset called Tulip Operations and scoped to a workbook called Tulip Integration. We can now leverage the capsule properties to start isolating interesting periods of time. For example, using the formula $ea.keep('status', isEqualTo('RUNNING')) Where $ea is the Endbell Assembly condition from the Tulip Table. We can create a new condition keeping only the capsules where the state is running. Once a full analysis is created, Seeq content can be displayed in a Tulip App as an iFrame, allowing for the combination of Tulip and Seeq data: Data can be pushed back to Tulip using the create record API. This allows for Tulip Dashboards to contain Seeq Content:
    5 points
  2. Proportional-integral-derivative (PID) control loops are essential to the automation and control of manufacturing processes and are foundational to the financial gains realized through advanced process control (APC) applications. Because poorly performing PID controllers can negatively impact production capacity, product quality, and energy consumption, implementing controller performance monitoring analytics leads to new data-driven insights and process optimization opportunities. The following sections provide the essential steps for creating basic controller performance monitoring at scale in Seeq. More advanced CLPM solutions can be implemented by expanding the standard framework outlined below with additional metrics, customization features, and visualizations. Data Lab’s Spy library functionality is integral to creating large scale CLPM, but small scale CLPM is possible with no coding, using Asset Groups in Workbench. Key steps in creating a CLPM solution include: Controller asset tree creation Developing performance metric calculations and inserting them in the tree as formulas Calculating advanced metrics via scheduled Data Lab notebooks (if needed) Configuring asset scaled and individual controller visualizations in Seeq Workbench Setting up site performance as well as individual controller monitoring in Seeq Organizer Using these steps and starting from only the controller signals within Seeq, large scale CLPM monitoring can be developed relatively quickly, and a variety of visualizations can be made available to the manufacturing team for monitoring and improving performance. As a quick example of many end result possibilities, this loop health treemap color codes controller performance (green=good, yellow=questionable, red=poor): The key steps in CLPM implementation, summarized above, are detailed below. Note: for use as templates for development of your own CLPM solution, the associated Data Lab notebooks containing the example code (for Steps 1-3) are included as file attachments to this article. The code and formulas described in Steps 1-3 can be adjusted and expanded to customize your CLPM solution as desired. Example CLPM Solution: Detailed Steps STEP 1: Controller asset tree creation An asset tree is the key ingredient which enables scaling of the performance calculations across a large number of controllers. The desired structure is chosen and the pertinent controller tags (typically at a minimum SP, PV, OP and MODE) are mapped into the tree. For this example, we will create a structure with two manufacturing sites and a small number of controllers at each site. In most industrial applications, the number of controllers would be much higher, and additional asset levels (departments, units, equipment, etc.) could of course be included. We use SPy.trees functionality within the Data Lab notebook to create the basic structure: Controller tags for SP, PV, OP, and MODE are identified using SPy.search. Cleansed controller names are extracted and inserted as asset names within the tree: Next, the controller tags (SP, PV, OP, and MODE), identified in the previous Data Lab code section using SPy.search, are mapped into the tree. At the same time, the second key step in creating the CLPM solution, developing basic performance metrics and calculations using Seeq Formula and inserting them into the asset tree, is completed. Note that in our example formulas, manual mode is detected when the numerical mode signal equals 0; this formula logic will need to be adjusted based on your mode signal conventions. While this second key step could be done just as easily as a separate code section later, it also works nicely to combine it with the mapping of the controller tag signals: STEP 2: Developing performance metric calculations and inserting them in the tree as formulas Several key points need to be mentioned related to this step in the CLPM process, which was implemented using the Data Lab code section above (see previous screenshot). There are of course many possible performance metrics of varying complexity. A good approach is to start with basic metrics that are easily understood, and to incrementally layer on more complex ones if needed, as the CLPM solution is used and shortcomings are identified. The selection of metrics, parameters, and the extent of customization for individual controllers should be determined by those who understand the process operation, control strategy, process dynamics, and overall CLPM objectives. The asset structure and functionality provided with the Data Lab asset tree creation enables the user to implement the various calculation details that will work best for their objectives. Above, we implemented Hourly Average Absolute Controller Error (as a percentage based on the PV value) and Hourly Percent Span Traveled as basic metrics for quantifying performance. When performance is poor (high variation), the average absolute controller error and percent span traveled will be abnormally high. Large percent span traveled values also lead to increased control valve maintenance costs. We chose to calculate these metrics on an hourly basis, but calculating more or less frequently is easily achieved, by substituting different recurring time period functions in place of the “hours()” function in the formulas for Hourly Average Absolute Controller Error and Hourly Percent Span Traveled. The performance metric calculations are inserted as formula items in the asset tree. This is an important aspect as it allows calculation parameters to be customized as needed on an individual controller basis, using Data Lab code, to give more accurate performance metrics. There are many customization possibilities, for example controller specific PV ranges could be used to normalize the controller error, or loosely tuned level controllers intended to minimize OP movement could be assigned a value of 0 error when the PV is within a specified range of SP. The Hourly Average Absolute Controller Error and Hourly Percent Span Traveled are then aggregated into an Hourly Loop Health Score using a simple weighting calculation to give a single numerical value (0-100) for categorizing overall performance. Higher values represent better performance. Another possible approach is to calculate loop health relative to historical variability indices for time periods of good performance specified by the user. The magnitude of a loop health score comprised of multiple, generalized metrics is never going to generate a perfect measure of performance. As an alternative to using just the score value to flag issues, the loop health score can be monitored for significant decreasing trends to detect performance degradation and report controller issues. While not part of the loop health score, note in the screenshot above that we create an Hourly Percent Time Manual Mode signal and an associated Excessive Time in Manual Mode condition, as another way to flag performance issues – where operators routinely intervene and adjust the controller OP manually to keep the process operating in desired ranges. Manual mode treemap visualizations can then be easily created for all site controllers. With the asset tree signal mappings and performance metric calculations inserted, the tree is pushed to a Workbench Analysis and the push results are checked: STEP 3: Calculating advanced metrics via scheduled Data Lab notebooks (if needed) Basic performance metrics (using Seeq Formula) may be all that are needed to generate actionable CLPM, and there are advantages to keeping the calculations simple. If more advanced performance metrics are needed, scheduled Data Lab notebooks are a good approach to do the required math calculations, push the results as signals into Seeq, and then map/insert the advanced metrics as items in the existing asset tree. There are many possibilities for advanced metrics (oscillation index, valve stiction, non-linearity measures, etc.), but here as an example, we calculate an oscillation index and associated oscillation period using the asset tree Controller Error signal as input data. The oscillation index is calculated based on the zero crossings of the autocorrelation function. Note: the code below does not account for time periods when the controller is in manual, the process isn’t running, problems with calculating the oscillation index across potential data gaps, etc. – these issues would need to be considered for this and any advanced metric calculation. Initially, the code above would be executed to fill in historical data oscillation metric results for as far back in time as the user chooses, by adjusting the calculation range parameters. Going forward, this code would be run in a recurring fashion as a scheduled notebook, to calculate oscillation metrics as time moves forward and new data becomes available. The final dataframe result from the code above looks as follows: After examining the results above for validity, we push the results as new signals into Seeq Workbench with tag names corresponding to the column names above. Note the new, pushed signals aren’t yet part of the asset tree: There are two additional sections of code that need to be executed after oscillation tag results have been pushed for the first time, and when new controller tags are added into the tree. These code sections update the oscillation tag metadata, adding units of measure and descriptions, and most importantly, map the newly created oscillation tags into the existing CLPM asset tree: STEP 4: Configuring asset scaled and individual controller visualizations in Seeq Workbench The CLPM asset tree is now in place in the Workbench Analysis, with drilldown functionality from the “US Sites” level to the signals, formula-based performance metrics, and Data Lab calculated advanced metrics, all common to each controller: The user can now use the tree to efficiently create monitoring and reporting visualizations in Seeq Workbench. Perhaps they start by setting up raw signal and performance metric trends for an individual controller. Here, performance degradation due to the onset of an oscillation in a level controller is clearly seen by a decrease in the loop health score and an oscillation index rising well above 1: There are of course many insights to be gained by asset scaling loop health scores and excessive time in manual mode across all controllers at a site. Next, the user creates a sorted, simple table for average loop health, showing that 7LC155 is the worst performing controller at the Beech Island site over the time range: The user then flags excessive time in manual mode for controller 2FC117 by creating a Lake Charles treemap based on the Excessive Time in Manual Mode condition: A variety of other visualizations can also be created, including controller data for recent runs versus current in chain view, oscillation index and oscillation period tables, a table of derived control loop statistics (see screenshot below for Lake Charles controller 2FC117) that can be manually created within Workbench or added later as items within the asset tree, and many others. Inspecting a trend in Workbench (see screenshot below) for a controller with significant time in manual mode, we of course see Excessive Time in Manual Mode capsules, created whenever the Hourly Percent Time Manual Mode was > 25% for at least 4 hours in a row. More importantly, we can see the effectiveness of including hours().intersect($Mode!=0) in the formulas for Hourly Average Absolute Controller Error and Hourly Percent Span Traveled. When the controller is in manual mode, that data is excluded from the metric calculations, resulting in the gaps shown in the trend. Controller error and OP travel have little meaning when the controller is in manual, so excluding data is necessary to keep the metrics accurate. It would also be very easy to modify the formulas to only calculate metrics for hourly time periods where the controller was in auto or cascade for the entire hour (using Composite Condition and finding hourly capsules that are “outside” manual mode capsules). The ability to accurately contextualize the metric calculations, to the time periods where they can be validly calculated, is a key feature in Seeq for doing effective CLPM implementations. Please also refer to the “Reminders and Other Considerations” section below for more advanced ideas on how to identify time periods for metric calculations. STEP 5: Setting up site performance as well as individual controller monitoring in Seeq Organizer As the final key step, the visualizations created in Workbench are inserted into Seeq Organizer to create a cohesive, auto-updating CLPM report with site overviews as well as individual controller visuals. With auto-updating date ranges applied, the CLPM reports can be “review ready” for recurring meetings. Asset selection functionality enables investigative workflows: poorly performing controllers are easily identified using the “Site CLPM” worksheet, and then the operating data and metrics for those specific controllers can be quickly investigated via asset selection in the site’s “Individual Controllers” worksheet – further details are described below. An example “Site CLPM” Organizer worksheet (see screenshot below) begins with a loop health performance ranking for each site, highlighting the worst performing controllers at the top of the table and therefore enabling the manufacturing team to focus attention where needed; if monitoring hundreds of controllers, the team could filter the table to the top 10 or 20 worst performing controllers. The visualizations also include a treemap for controllers that are often switched to manual mode – the team can talk to operators on each crew to determine why the controllers are not trusted in auto or cascade mode, and then generate action items to resolve. Finally, oscillating controllers are flagged in red in the sorted Oscillation Metrics tables, with the oscillation period values also sorted – short oscillation periods may prematurely wear out equipment and valves due to high frequency cycling; long oscillation periods are more likely to negatively impact product quality, production rate, and energy consumption. Controllers often oscillate due to root causes such as tuning and valve stiction and this variability can often be eliminated once an oscillating controller has been identified. The oscillation period table can also be perused for controllers with similar periods, which may be evidence of an oscillation common to multiple controllers which is generating widespread process variation. An example “Individual Controllers” Organizer worksheet is shown below, where detailed operating trends and performance metrics can be viewed for changes, patterns, etc., and chain view can be used to compare controller behavior for the current production run versus recent production runs. Other controllers can be quickly investigated using the asset selection dropdown, and the heading labels (e.g., Beech Island >> 7LC155) change dynamically depending on which controller asset is selected. For example, the Beech Island 7LC155 controller which was identified as the worst performing controller in the “Site CLPM” view above, can be quickly investigated in the view below, where it is evident that the controller is oscillating regularly and the problem has been ongoing, as shown by the comparison of the current production run to the previous two runs: Reminders and Other Considerations As evident with the key steps outlined above, a basic CLPM solution can be rapidly implemented in Seeq. While Seeq’s asset and contextualization features are ideal for efficiently creating CLPM, there are many details which go into developing and maintaining actionable CLPM dashboards for your process operation. A list of reminders and considerations is given below. 1. For accurate and actionable results, it is vital to only calculate performance metrics when it makes sense to do so, which typically means when the process is running at or near normal production rates. For example, a controller in manual during startup may be expected and part of the normal procedure. And of course, calculating average absolute controller error when the process is in an unusual state will likely lead to false indications of poor performance. Seeq is designed to enable finding those very specific time periods when the calculations should be performed. In the CLPM approach outlined in this article, we used time periods when the controller was not in manual by including hours().intersect($Mode!=0) in the formulas for Hourly Average Absolute Controller Error and Hourly Percent Span Traveled. When the controller is in manual mode, that data is excluded from the metric calculations. But of course, a controller might be in auto or cascade mode when the process is down, and there could be many other scenarios where only testing for manual mode isn’t enough. In the CLPM approach outlined above, we intentionally kept things simple by just calculating metrics when the controller was not in manual mode, but for real CLPM implementations you will need to use a more advanced method. Here are a few examples of finding “process running” related time periods using simple as well as more advanced ways. Similar approaches can be used with your process signals, in combination with value searches on controller mode, for excluding data from CLPM calculations: A simple Process Running condition can be created with a Value Search for when Process Feed Flow is > 1 for at least 2 hours. A 12 Hours after Process Running condition can be created with a formula based on the Process Running Condition: $ProcessRunning.afterStart(12hr) A Process Running (> 12 hours after first running) condition can then be created from the first two conditions with the formula: $ProcessRunning.subtract($TwelveHrsAfterRunning) Identifying time periods based on the value and variation of the production rate is also a possibility as shown in this Formula: The conditions described above are shown in the trends below: 2. As previously mentioned, including metric formulas and calculations as items within the asset tree enables customization for individual controllers as needed, when controllers need unique weightings, or when unique values such as PV ranges are part of the calculations. It may also be beneficial to create a “DoCalcsFlag” signal (0 or 1 value) as an item under each controller asset and use that as the criteria to exclude data from metric calculations. This would allow customization of “process is running normally and controller is not in manual” on an individual controller basis, with the result common for each controller represented as the “DoCalcsFlag” value. 3. In the CLPM approach outlined above, we used SPy.trees in Data Lab to create the asset tree. This is the most efficient method for creating large scale CLPM. You can also efficiently create the basic tree (containing the raw signal mappings) from a CSV file. For small trees (<= 20 controllers), it is feasible to interactively create the CLPM asset tree (including basic metric calculation formulas) directly in Workbench using Asset Groups. The Asset Groups approach requires no Python coding and can be quite useful for a CLPM proof of concept, perhaps focused on a single unit at the manufacturing site. For more details on Asset Groups: https://www.seeq.org/index.php?/forums/topic/1333-asset-groups-101-part-1/). 4. In our basic CLPM approach, we scoped the CLPM asset tree and calculations to a single Workbench Analysis. This is often the best way to start for testing, creating a proof of concept, getting feedback from users, etc. You can always decide later to make the CLPM tree and calculations global, using the SPy.push option for workbook=None. 5. For long-term maintenance of the CLPM tree, you may want to consider developing an add-on for adding new controllers into the tree, or for removing deleted controllers from the tree. The add-on interface could also prompt the user for any needed customization parameters (e.g., PV ranges, health score weightings) and could use SPy.trees insert and remove functionality for modifying the tree elements. 6. When evidence of more widespread variation is found (more than just variability in a single controller), and the root cause is not easily identified, CLPM findings can be used to generate a list of controllers (and perhaps measured process variables in close proximity) that are then fed as a dataset to Seeq’s Process Health Solution for advanced diagnostics. 7. For complex performance or diagnostic metrics (for example, stiction detection using PV and OP patterns), high quality data may be needed to generate accurate results. Therefore, some metric calculations may not be feasible depending on the sample frequency and compression levels inherent with archived and compressed historian data. The only viable options might be using raw data read directly from the distributed control system (DCS), or specifying high frequency scan rates and turning off compression for controller tags such as SP, PV, and OP in the historian. Another issue to be aware of is that some metric calculations will require evenly spaced data and therefore need interpolation (resampling) of historian data. Resampling should be carefully considered as it can be problematic in terms of result accuracy, depending on the nature of the calculation and the signal variability. 8. The purpose of this article was to show how to set up basic CLPM in Seeq but note that many types of process calculations to monitor “asset health” metrics could be created using a similar framework. Summary While there are of course many details and customizations to consider for generating actionable controller performance metrics for your manufacturing site, the basic CLPM approach above illustrates a general framework for getting started with controller performance monitoring in Seeq. The outlined approach is also widely applicable for health monitoring of other types of assets. Asset groups/trees are key to scaling performance calculations across all controllers, and Seeq Data Lab can be used as needed for implementing more complex metrics such as oscillation index, stiction detection, and others. Finally, Seeq Workbench tools and add-on applications like Seeq’s Process Health solution can be used for diagnosing the root cause of performance issues automatically identified via CLPM monitoring. CLPM Asset Tree Creation.ipynb Advanced CLPM Metric Calculations.ipynb
    5 points
  3. A typical data cleansing workflow is to exclude equipment downtime data from calculations. This is easily done using the .remove() and .within() functions in Seeq formula. These functions remove or retain data when capsules are present in the condition that the user supplies as a parameter to the function. There is a distinct difference in the behavior of the .remove() and .within() functions that users should know about, so that they can use the best approach for their use case. .remove() removes the data during the capsules in the input parameter condition. For step or linearly interpolated signals, interpolation will occur across those data gaps that are of shorter duration than the signal's maximum interpolation. (See Interpolation for more details concerning maximum interpolation.) .within() produces data gaps between the input parameter capsules. No interpolation will occur across data gaps (no matter what the maximum interpolation value is). Let's show this behavior with an example (see the first screenshot below, Data Cleansed Signal Trends), where an Equipment Down condition is identified with a simple Value Search for when Equipment Feedrate is < 500 lb/min. We then generate cleansed Feedrate signals which will only have data when the equipment is running. We do this 2 ways to show the different behaviors of the .remove() and .within() functions. $Feedrate.remove($EquipmentDown) interpolates across the downtime gaps because the gap durations are all less than the 40 hour max interpolation setting. $Feedrate.within($EquipmentDown.inverse()) does NOT interpolate across the downtime gaps. In the majority of cases, this result is more in line with what the user expects. As shown below, there is a noticeable visual difference in the trend results. Gaps are present in the green signal produced using the .within() function, wherever there is an Equipment Down capsule. A more significant difference is that depending on the nature of the data, the statistical calculation results for time weighted values like averages and standard deviations, can be very different. This is shown in the simple table (Signal Averages over the 4 Hour Time Period, second screenshot below). The effect of time weighting the very low, interpolated values across the Equipment Down capsules when averaging the Feedrate.remove($EquipmentDown) signal, gives a much lower average value compared to that for $Feedrate.within($EquipmentDown.inverse()) (1445 versus 1907). Data Cleansed Signal Trends Signal Averages over the 4 Hour Time Period Content Verified DEC2023
    4 points
  4. Hi Jacob. you can use the within() function to include only the portions of the signal while the condition appears and then use this signal in your calculations: Regards, Thorsten
    3 points
  5. Here's an alternative method to getting the last X batches in the last 30 days: // return X most recent batches in the past Y amount of time $numBatches = 20 // X $lookback = 1mo // Y // create a rolling condition with capsules that contain X adjacent capsules $rollingBatches = $batchCondition.removeLongerThan($lookback) .toCapsulesByCount($numBatches, $lookback) // find the last capsule in the rolling condition that's within the lookback period $currentLookback = capsule(now()-$lookback, now()) $batchWindow = condition( $lookback, $rollingBatches.toGroup($currentLookback, CAPSULEBOUNDARY.ENDSIN).last() ) // find all the batches within the capsule identified // ensure all the batches are within the lookback period $batchCondition.inside($batchWindow) .touches(condition($lookback, $currentLookback)) This is similar to yours in that it uses toGroup, but the key is in the use of toCapsulesByCount as a way to get a grouping of X capsules in a condition. You can see an example output below. All capsules will show up as hollow because by the nature of the rolling 'Last X days' the result will always be uncertain.
    3 points
  6. Welcome to the Seeq Developer Club Forum! This forum is your go-to destination for all things related to developing Add-ons, API integrations, and leveraging Seeq's capabilities to enhance your projects. Whether you're a seasoned developer or just starting out, this is the place to discuss, share insights, and collaborate with fellow Seeq users. To ensure a positive and productive experience for everyone, please take note of the following guidelines: Public Forum: While we encourage open discussion and collaboration, please refrain from sharing sensitive or confidential information, as this is a public forum accessible to all. Respectful Dialogue: The goal of this forum is to foster respectful and constructive dialogue. Please refrain from behavior that may be considered harassing, threatening, or discriminatory. Resource Utilization: Before posting questions, explore our extensive documentation, tutorials, and the Seeq Support site. You may find answers to your queries or helpful resources that address your needs. Seeq Support - Developing Seeq Add-ons and Connectors Specificity is Key: When posting questions, provide as much detail as possible about your environment, the Seeq version you're using, and any error messages encountered. Screenshots or code snippets can greatly aid in understanding and resolving issues. Constructive Engagement: When responding to questions, aim to be helpful and constructive. Avoid personal attacks or dismissive comments and focus on providing valuable feedback and assistance. We're thrilled to have you as part of our developer community! Let's collaborate, innovate, and empower each other to unlock the full potential of Seeq. Thank you for your participation!
    3 points
  7. There are instances where it's desirable to add or remove Access Control to a user created datasource. For instance, you may want to push data from Seeq Datalab to a Datasource that is only available to a sub-set of users. Below is a tip how to add additional entries to a Datasource Access Control List (ACL) using `spy.acl`. 1) Identify Datasource ID If not already known, retrieve the ID for the Datasource. This can be done via the Seeq REST API (accessible via the "hamburger" menu in the upper right corner of your browser window). This is different from the datasourceId identifier and will contain a series of characters as shown below. It can be retrieved via the `GET/datasources` endpoint under the `Datasources` section. The response body will contain the ID: 2) Identify User and/or Group ID We will also want to identify the User and/or Group ID. Below is an example of how to retrieve a Group ID via the `UserGroups` GET/usergroups endpoint: The response body will contain the ID: 3) Use `spy.acl()` to add group to the Datasource Access Control List (ACL) Launch a Seeq Datalab Project and create a new ipynb Notebook Assign the two IDs to a variable datasource_id='65BDFD4F-1FA3-4EB5-B21E-069FA5A772DF' group_id='19F1B7DC-69BE-4402-AD3B-DF89B1B9A1A4' (Optional) - Check the current Access Control List for the Datasource using `spy.acl.pull()` current_access_control_df=spy.acl.pull(datasource_id) current_access_control_df.iloc[0]['Access Control'] Add the group_id to the Datasource ACL using `spy.acl.push()` spy.acl.push(datasource_id, { 'ID': group_id, 'Read': True, 'Write': True, 'Manage': False }, replace=False) Note I've set `replace=False`, which means the group will be appended to the current access list. If the desire is to the replace the entire existing ACL list, this can be toggled to `replace=True`. Similarly, you can adjust the read/write/manage permissions to fit your need. For more information on `spy.acl.push`, reference the `spy.acl.ipynb` document located in the SPy Documentation folder or access the docstring by executing help(spy.acl.push) (Optional) - To confirm the correct ID has been added, re-pull the ACL via `spy.acl.pull()` current_access_control_df=spy.acl.pull(datasource_id) current_access_control_df.iloc[0]['Access Control']
    2 points
  8. Starting in R63, you have the ability to "deep copy" worksheets to automatically create new, independent items. Details here: https://support.seeq.com/kb/latest/cloud/worksheet-document-organization
    2 points
  9. Working with Asset Groups or spy.assets.Tree() has been effective. However, as your project expands, the process of adding and managing additional assets has become cumbersome. It's time to elevate your approach by transitioning to an Asset Template for building your Asset Tree. In this guide, I'll share valuable tips and tricks for creating an asset tree from an existing hierarchy. If you already have an established asset tree from another source, such as PI AF, you can leverage it as a quick starting point to build your tree in Seeq. Whether you're starting with an existing asset tree or starting from scratch, these insights will help you seamlessly integrate your asset structure into Seeq Data Lab. Additionally, I'll cover essential troubleshooting techniques, focusing on maximizing ease of use when using Asset Templates. In a future topic, I'll discuss the process of building an Asset Tree from scratch using Asset Templates. Let's dive into the best practices for optimizing your asset management workflow with Seeq Data Lab. Why Templates over Trees? The choice between using Asset Templates and spy.assets.Tree() depends on the complexity of your project and the level of customization you need. Here are some reasons why you might want to use Asset Templates instead of spy.assets.Tree(): Object-Oriented Design: Asset Templates allow you to leverage object-oriented design principles like encapsulation, inheritance, composition, and mixins. This can increase code reuse and consistency, especially in larger projects. However, the learning curve is a bit longer if you do not come from a programming background. Customization: Asset Templates provide a higher level of customization. You can define your own classes and methods, which can be useful if you need to implement complex logic or calculations. You can more easily create complex hierarchical trees similar Asset Trees from PI AF. Handling Exceptions: Manufacturing scenarios often have many exceptions. Asset Templates can accommodate these exceptions more easily than spy.assets.Tree(). Here are some reasons why you might want to use spy.assets.Tree() instead of Asset Templates: Simplicity: spy.assets.Tree() is simpler to use and understand, especially for beginners or for simple projects. If you just need to create a basic hierarchical structure without any custom attributes or methods, spy.assets.Tree() is a good choice. You can always graduate to Asset Templates later. Flat Structure: If your data is already organized in a flat structure (like a CSV file), spy.assets.Tree() can easily convert that structure into an asset tree. Less Code: With spy.assets.Tree(), you can create an asset tree with just a few lines of code. You don't need to define any classes or methods. Okay, we are settled on Asset Templates. What's first? If you have an existing asset hierarchy with the attributes you want to use in your calculations, you can start there. I'll be using the Example >> Cooling Tower 1 as my example existing Asset Tree. We'll use the asset tree path for our spy.search criteria and use the existing asset hierarchy when building our Template. The attachment includes a complete working Jupyter notebook. You can upload it to a Data Lab project and test each step as described here. Okay, let's review the code section by section. # 1. Importing Libraries from seeq import spy import pandas as pd import re from seeq.spy.assets import Asset # 2. Checking and displaying the spy version version = spy.__version__ print(version) # 3. Setting the compatibility option so that you maximize the chance that SPy # will remain compatible with your notebook/script spy.options.compatibility = 189 # 4. Disply configuration for Pandas to show all columns in DataFrame output pd.set_option('display.max_colwidth', None) In this cell, the script initializes the Seeq Python (SPy) library as well as other required libraries, checks the spy version, sets a compatibility option, and configures pandas to display DataFrame output without column width truncation. # 1. Use spy.search to search for assets using the existing Asset Tree hierarchy. metadata = spy.search( {'Path': 'Example >> Cooling Tower 1 >> Area*'}, # MODIFY: Change the search path old_asset_format=False, limit=None, ) # 2. Split the "Path" column by the ">>" delimiter and create a new column "Modified_Path." metadata["Modified_Path"] = metadata['Path'] + ' >> ' + metadata['Asset'] split_columns = metadata['Modified_Path'].str.split(' >> ', expand=True) # 3. Rename the columns of the split DataFrame to represent different hierarchy levels. split_columns.columns = [f'Level_{i+1}' for i in range(split_columns.shape[1])] # 4. Concatenate the split columns with the original DataFrame to incorporate the hierarchy levels. metadata = pd.concat([metadata, split_columns], axis=1) # 5. Required for building an asset tree. This is the parent branch/name of the asset tree. metadata['Build Path'] = 'Asset Tree Descriptive Name' # MODIFY: Set the desired parent branch name # 6. The child branch of 'Build Path' and will be named after the "Level_2" column. metadata['Build Asset'] = metadata['Level_2'] # Ex: "Cooling Tower 1" This cell constructs the metadata used in our new asset tree. The spy.search on the existing hierarchy is used to retrieve the metadata for each attribute in the asset path. The existing tree path is "Example >> Cooling Tower 1 >> Area*". This will fetch all attributes under each of the Areas (e.g., Area A - K). However, it will not retrieve any attributes that may reside in "Example >> Cooling Tower 1. The complete path for each asset is the concatenation of the "Path" column and the "Asset" column. This "Modified Path" is how we will define each level of our new Asset Tree. We are splitting the Modified Path and inserting each level as a new column that will be referenced later when building the asset tree. In steps 5 and 6, we define the first two levels of the asset tree. In this example, the new path will start with "Asset Tree Descriptive Name >> Cooling Tower 1". By utilizing the 'Level_2' column in the 'Build Path,' if we had included all cooling towers in our spy.search, the new second level in the path could be 'Cooling Tower 2,' depending on the asset's location in the existing tree." Alternatively, if you prefer a flat tree, you can specify the name of "Level 2" as a string rather than referencing the metadata column. # This is the class will be called when using Spy.Asset.Build # This is the child branch of 'Build Asset' # Ex: Asset Tree Descriptive Name >> Cooling Tower 1 >> Area A # Can you also have attributes at these levels by adding @Asset.Attribute() within the class. class AssetTreeName(Asset): # MODIFY: AssetTreeName @Asset.Component() def Asset_Component(self, metadata): # MODIFY: Unit_Component return self.build_components( template=Level_4, # This the name of the child branch class metadata=metadata, column_name='Level_3' # Metadata column name of the desired asset; Ex: Area A ) # Example Roll Up Calculation # This roll up calculation will find the evaluate the temperature if each child branch (e.g., Area A, Area B, # Area C, etc.) and create a new signal of the maximum temperature of the group @Asset.Attribute() def Cooling_Tower_Max_Temperature(self, metadata): return self.Asset_Component().pick({ ## MODIFY: Unit_Component() if changed in @Asset.Component() above 'Name': 'Temperature Unique Raw Tag' # MODIFY:'Temperature Unique Raw Tag'; this is the Friendly name of the attribute created in Level_4 }).roll_up('maximum') # MODIFY: 'maximum' In this cell, we are building our first class in our Asset Template. In the context of Asset Templates, consider a class as a general contractor responsible for constructing the current level in your asset tree. It subcontracts the responsibility for the lower level to that level's class. TIn the class, you define attributes such as signals, calculated signals, conditions, scorecard metrics, etc. or components, which are the 'child' branches of this 'parent' branch. These components are built by a different class. This first class will be called when we use spy.assets.build to create the new asset tree. @Asset.Component() indicates that we are defining a method to construct a component. The method name is Asset_Component which will construct the child branch. The parameter 'template' defines the class that will construct the child branch, while 'column_name' specifies the name of each child branch. Here we are using "Level_3" which contains the Area name (e.g., Area A). @Asset.Attribute() indicates that we are defining a method that will be an attribute. More details about creating attributes will be discussed in the next class. However, the example provided is a roll up calculation which will be a signal equal to the maximum value of each of the child branches' "Temperature Unique Raw Tag" which will be defined in the next class. When using roll-up calculations, note that the method being called is the component method responsible for constructing the child branches, not the attribute residing in the child branches. The .pick() function utilizes the friendly name of the attribute to identify the attributes in the child branches. # This is the child branch of 'AssetTreeName' class class Level_4(Asset): @Asset.Attribute() # Raw Tag (Friendly Name is the same as the method name with '_' removed) # Use this method if you are highly confident there are no multiple matches in current or child branches def Temperature_Unique_Raw_Tag(self, metadata): # MODIFY: Temperature_Unique_Raw_Tag return metadata[metadata['Name'] == 'Temperature'] @Asset.Attribute() def Temperature_Possible_Duplicates_Raw_Tag(self, metadata): # MODIFY:Temperature_Possible_Duplicates_Raw_Tag # If multiple matching metadata is found, the first match is selected using .iloc[[0]]. # This also addresses duplicate attributes in child branches. # If selecting first match is not desired, improve regex search criteria # or add additional filters to select desired attribute (e.g, # metadata[metadata['Name'].str.contains(r'(?:Temperature)$') # & metadata[metadata['Description'].str.contains(r'(?:Discharge)')] # ) filtered = metadata[metadata['Name'].str.contains(r'(?:Temperature)$')] # MODIFY: r'(?:Temperature)$') return filtered.iloc[[0]] In our example, Level_4 is the lowest level in our asset tree. All our attributes related to an asset will reside here. In our example, our raw tags (i.e., no calculations) will reside in the same branch as our derived tags (i.e., calculated), which typically reference the raw tags in the formula parameter. Each attribute is defined by a method starting with @Asset.Attribute before defining the method. When not specified, the friendly name of the attribute is the same as the method name with the underscores removed. Example: The method name is Temperature_Unique_Raw_Tag, and the friendly name is "Temperature Unique Raw Tag". We reference the metadata dataframe with a search criteria to select the proper row for the attribute. There are several methods using pandas to select the correct row. Here we are using the "Name" to find the row with "Temperature" as the value. Spy will find each "Temperature" value for each asset. This method can result in an error if there are multiple matches. The second method "Temperature_Possible_Duplicates_Raw_Tag" explains how you can avoid this error by using .iloc[[0]] to select the first match found or how you can increase the complexity of your search criteria to further filter your results. # Calculated Tag (Signal) referencing an attribute in this class @Asset.Attribute() def Temperature_Calc(self, metadata): return { 'Name': 'Signal Friendly Name', # MODIFY: 'Signal Friendly Name' 'Type': 'Signal', # This is the same formula you would use in Workbench Formula 'Formula': '$signal.agileFilter(1min)', # MODIFY: "$signal.agileFilter(1min)"; 'Formula Parameters': {'$signal': self.Temperature_Unique_Raw_Tag()}, # MODIFY: '$signal' and self.Temperature_Unique_Raw_Tag() } # Calculated Tag (Signal) not previously referenced as an attribute @Asset.Attribute() def Wet_Bulb(self, metadata): filtered = metadata[metadata['Name'].str.contains(r'^Wet Bulb$')] # MODIFY: '^Wet Bulb$' return { 'Name': 'Signal Friendly Name 2', # MODIFY: 'Signal Friendly Name 2' 'Type': 'Signal', 'Formula': '$a.agileFilter(1min)', # MODIFY: '$a.agileFilter(1min)' 'Formula Parameters': {'$a': filtered.iloc[[0]]}, # MODIFY: '$a' } # Calculated Tag (Condition) @Asset.Attribute() def Temperature_Cond(self, metadata): return { 'Name': 'Condition Friendly Name', # MODIFY: 'Condition Friendly Name' 'Type': 'Condition', 'Formula': '$b > 90', # MODIFY: '$b > 90' 'Formula Parameters': {'$b': self.Temperature_Possible_Duplicates_Raw_Tag()}, # MODIFY: '$b' & self.Temperature_Possible_Duplicates_Raw_Tag() } # Scorecard Metric # See Asset Templates documentation for more examples of Scorecard Metrics @Asset.Attribute() def Temperature_Statistic_KPI(self, metadata): return { 'Type': 'Metric', 'Measured Item': self.Temperature_Unique_Raw_Tag(), # MODIFY: self.Temperature_Unique_Raw_Tag() 'Statistic': 'Average' # MODIFY: 'Average' } This is a continuation of our Level_4 class. Here we are creating derived attributes which use a formula or metric and reference other attributes in most cases. There are several different example attributes for calculated signals, conditions, and scorecard metrics. You can specify the friendly name in the "Name" parameter as an alternative method to using the name of the method. The value of the "Formula" is the exact same formula you use in Workbench. "Formula Parameters" is where you define your variables in your formula. The value of the "variable" key (i.e., $signal) is a method call to the attribute to be used in the formula (i.e., self.Temperature_Unique_Raw_Tag()). If you wish to specify a signal that has not been defined as an attribute, you can specify the value of the referenced variable using the same search criteria when specifying a new attribute. It is important to remember to change the "Type" when creating a condition or a scorecard metric, as this will cause an error if not properly specified. # Use this method if you want a different friendly name rather than the regex search criteria @Asset.Attribute() def Friendly_Name_Example(self, metadata): return self.select_metadata(metadata, name_contains = r'^Compressor Power$', friendly_name ='Power') #MODIFY: name_contains = r'Compressor Power', friendly_name = 'Power' # Use this method if you want the friendly name to be the same as regex search criteria # without the regex formatting @Asset.Attribute() def No_Friendly_Name_Example(self, metadata): return self.select_metadata(metadata, r'^Compressor Power$', None) # Example of a string attribute being passed through @Asset.Attribute() def String_Example(self, metadata): return self.select_metadata(metadata, r'Compressor Stage$', None) # Example of an attribute not found @Asset.Attribute() def No_Attribute_Exists(self, metadata): return self.select_metadata(metadata, r'Non-existing Attribute', None) # 1. Method below matches the metadata, perform attribute type detection, and applies proper formula def select_metadata(self, metadata, name_contains, friendly_name): filtered = metadata[metadata['Name'].str.contains(name_contains)] if not filtered.empty: # checks if metadata is a signal vs scalar and selects the first signal type if filtered['Type'].str.contains('signal', case=False).any() : signal_check = filtered[filtered['Type'].str.contains('signal', case=False) & ~filtered['Value Unit Of Measure'].str.contains('string', case=False)] filtered_signal = signal_check if len(signal_check) > 0 else filtered.iloc[[0]] selected_metadata = filtered_signal.iloc[[0]] else: selected_metadata = filtered.iloc[[0]] return self.determine_attribute_type(selected_metadata, name_contains, friendly_name) else: # This returns a signal with no values rather than not adding an attribute when # an attribute cannot be found. If this is not desired, simply comment out the # else statement. Having a signal is recommended to assist in asset swapping. return { 'Name' : friendly_name if friendly_name is not None else re.sub( r'\^|\(|\$|\(\?:|\)$|\)', '', name_contains), 'Type': 'Signal', 'Formula': 'Scalar.Invalid.toSignal()', } # 2. Determine if an attribute is a signal or scalar type def determine_attribute_type(self, metadata, name_contains, friendly_name): if metadata['Type'].str.contains('signal', case=False).any(): return self.generic_metadata_function(metadata, name_contains, 'Signal', friendly_name) elif metadata['Type'].str.contains('scalar', case=False).any(): return self.generic_metadata_function(metadata, name_contains, 'Scalar', friendly_name) return None # 3. Creates the metadata to create the attribute def generic_metadata_function(self, metadata, name_contains, formula_type, friendly_name): if friendly_name is None: # Regular expression pattern to remove caret (^), first "(", dollar ($), # non-capturing groups (?:), and trailing ")" pattern = r'\^|\(|\$|\(\?:|\)$|\)' friendly_name = re.sub(pattern, '', name_contains) # Metadata for signals includes strings. Required to separate strings to perform a # formula on non-string signals. Check if the signal is a "string" based on 'Value Unit Of Measure' if formula_type == 'Signal' and metadata['Value Unit Of Measure'].str.contains( 'string', case=False).any(): formula_type = 'String' # If signal, perform a calculation. If string or scalar, pass only the variable in formula. if formula_type == 'Signal': formula = '$signal.validValues()' # MODIFY: '$signal.validValues()' elif formula_type == 'String': formula = '$signal' else: formula = '$scalar' return { 'Name' : friendly_name, 'Type': formula_type, 'Formula': formula, 'Formula Parameters': ( {'$signal': metadata} if formula_type in ['Signal', 'String'] else {'$scalar': metadata}) } This is a continuation of the Level_4 class. This section is more of a trick than a tip. A common request is how to bulk apply a formula to all signals as an initial data cleaning step. This can be challenging if your existing asset tree contains signals, scalars, and/or strings with the same friendly name in the parent and/or child branches. I've seen this happen when an instrument does not exist for a particular asset. The attribute still exists in the asset tree but contains no data and is either a scalar or string. This method allows users to quickly perform the same calculations on all raw data signals. It will check for string, scalar, and signal types to determine if a calculation is required. If there are multiple matches, it will preferentially select a signal over a string or scalar. It will perform the appropriate calculation on the tag based on the type or return a blank signal if the attribute is not found. The blank signal is a necessary evil for asset swapping if the attribute is used in a calculation for other attributes. The formula should be adjusted to handle when there are no valid values in the attribute. @Asset.Attribute() def Friendly_Name_Example(self, metadata): return self.select_metadata(metadata, name_contains = r'^Compressor Power$', friendly_name ='Power') This attribute method has been modified to make a method call to select_metadata. It passes the metadata, name_contains, which is the search criteria for finding the name value in the metadata "Name" column, and a friendly name. In this example, the attribute will be named "Power". @Asset.Attribute() def No_Friendly_Name_Example(self, metadata): return self.select_metadata(metadata, r'^Compressor Power$', None) If the friendly name does not need to be different from the name_contains parameter without the regex formatting (i.e., special characters), it will name the attribute the search criteria name minus special characters. In the example above the attribute will be named "Compressor Power". This naming method is fairly simple and is not dynamic enough to insert result of the regex search. In those cases, the friendly_name should be specified. The subsequent method call to determine_attribute_type determines the attribute type (e.g., signal, string, scalar). If multiple matches are found, it will select the first signal type over a scalar or string. If attribute is not found, it creates an empty attribute. The final method call to generic_metadata_function determines the friendly name to be used and selects the correct formula for the proper attribute type. As I said earlier, this is more of a trick that address most common issues I have faced when dealing with complex asset trees and applying a common formula in bulk. This can also be applied even if you don't want to apply a formula to all signals. In that case, you would simply modify the formula to be "$signal" instead of "$signal.validValues()". You can still preferentially select signals types over string or scalar types. However, if referencing one of these attributes in a calculated attribute later, you will still have to use the attribute method name (e.g., self.Friendly_Name_Example() when referencing the 'Power' attribute in the formula parameter). build_df = spy.assets.build(AssetTreeName,metadata) # Check for success for each tag for an asset build_df.head(15) After creating our asset structure and attributes, you can run this cell to build our asset tree. I recommend visually checking for success in the 'Build Result' column of the dataframe, looking for 'Success' in each attribute for a single asset. workbookid = "WORKBOOK_ID" # MODIFY: "WORKBOOK_ID" # If you want the tree to be available globally use None as the workbookid push_results_df = spy.push(metadata=build_df, workbook=workbookid, errors='catalog') # Check for success at Push Result push_results_df.head(15) Once you are satisfied with your results or have resolved any errors, you can push your asset tree. If you wish to push your tree globally, change the value of the 'workbook' argument in spy.push to be set to None. I recommend doing this only after addressing all the issues in a workbook, which can serve as your sandbox environment. errors = push_results_df[push_results_df['Push Result'] != "Success"] errors The final step I perform is to check if any of my attributes were not successfully pushed. If there were no issues, the error dataframe should contain no rows. Well, if you have made it to the end, congrats! If you skipped to the end, that's fine too. In this guide, we delved into the world of Asset Templates and their role in streamlining the creation and management of asset trees within Seeq Data Lab. By leveraging object-oriented design principles, encapsulation, and inheritance, Asset Templates offer a powerful way to build complex hierarchical structures. Whether you're dealing with an existing asset tree or starting from scratch, the insights shared here aim to maximize the efficiency of your workflow. Download the attached Jupyter notebook and experiment with the code by modifying or adding new attributes or applying it to a different asset tree. Remember, the best way to grasp these concepts is to dive into the provided Jupyter notebook, make modifications, and witness the results firsthand. In the comments below, I will add some screenshots of common errors I come across when using spy.assets.build and what you should do to troubleshoot the errors. Tips Tricks for Existing Asset Trees.ipynb
    2 points
  10. Troubleshooting common errors when using Asset Templates: Spy.Assets.Build Error 1: No matching metadata row found Error - Key Phrase: "No matching metadata row found" for Temperature Unique Raw Tag on Level_4 class Fix - Modify the search criteria of the attribute shown in the error Error 2: Component dependency not built (calculated tag) Errors - Key phrases: Component Dependency not built - A calculated signal could not be created because the formula parameter referenced was not created. Attribute dependency not built - No matching metadata row found for the referenced formula parameter. Fix - Modify the search criteria of the attributes referenced by the calculated signals Error 3: Component dependency not built (branch not built) Error: Key Phrases: name 'Level 4' is not defined. Unit Component [on AssetTreeName class] component dependency not built. This is because the referenced template in def Unit_Component in the AssetTreeClass does not exist. The class name for the child branch is actually "Not_Level_4". Fix- Change the template to match the actual class name. Example: Code causing error: class AssetTreeName(Asset): @Asset.Component() def Unit_Component(self, metadata): return self.build_components( template=Level_4, metadata=metadata, column_name='Level_3' ) class Not_Level_4(Asset): @Asset.Attribute() def Temperature_Unique_Raw_Tag(self, metadata): return metadata[metadata['Name'] == 'Temperature Unique Raw Tag'].iloc[[0]] Corrected Code: class Level_4(Asset): @Asset.Attribute() def Temperature_Unique_Raw_Tag(self, metadata): return metadata[metadata['Name'] == 'Temperature Unique Raw Tag'].iloc[[0]] Error 4: Multiple attributes returned Error - Key Phrase: "Multiple attributes returned" - This indicates that your search criteria for the attribute returned multiple results. In this case, there a match in ">> Area A" and a match in ">> Area A >> Area A-Duplicate". Fix - Use .iloc[[0]] at the end of the metadata search criteria to select the first match. Original: metadata[metadata['Name'].str.contains(r'(?:Temperature)')] Revised: metadata[metadata['Name'].str.contains(r'(?:Temperature)')].iloc[[0]] Alternatively, you can add more criteria to the search to filter down to the desired tag: metadata[metadata['Name'].str.contains(r'(?:Temperature)')] & metadata[metadata['Level_4'].str.contains(‘Duplicate’) Error 5: ‘class name’ object has no attribute ‘attribute method name’ for calculated attributes Error: Key Phrases: "in Temperature_Cond ‘Formula Parameters’ lies the error. For the formula parameter, we have misspelled the attribute method name in the Level_4 class. Fix – check for misspelled methods or correct method is referenced Error 6: ‘class name’ object has no attribute ‘attribute name’ for roll up calculations Error: Key Phrases: "Cooling_Tower_Max_Temperature" is causing the error. For the roll up calculation, we are trying to reference the self.Temperature_Unique_Raw_Tag() which is in Level_4 which is the child branch. Since this method does not reside in the AssetTreeName class, it cannot find it. Example: Code causing error: class AssetTreeName(Asset): @Asset.Component() def Unit_Component(self, metadata): return self.build_components( template=Level_4, # This the name of the child branch class metadata=metadata, column_name='Level_3' # Metadata column name of the desired asset; Ex: Area A ) # Example Roll Up Calculation @Asset.Attribute() def Cooling_Tower_Max_Temperature(self, metadata): return self.Temperature_Unique_Raw_Tag().roll_up('maximum') Fix - Use the method that builds the child branch (Unit_Component) in the AssetTreeName class and then use the pick function using the friendly name of the tag to be used in the calculation. # Example Roll Up Calculation @Asset.Attribute() def Cooling_Tower_Max_Temperature(self, metadata): return self.Unit_Component().pick({ 'Name': 'Temperature Possible Duplicates Raw Tag' }).roll_up('maximum') Error 7: Spy.Push Errors In this example, I have tried to apply an agileFilter function onto a string attribute. This error can occur when an asset may have a string or scalar attribute exist instead of a signal attribute when a field instrument does not exist for this asset while a sibling asset would have a signal attribute and you want to apply the agileFilter. It can be difficult to determine which attributes are causing the issue. You can catalog the errors and allow spy.push to continue to push the Asset Tree without the errors and cataloging the errors. push_results_df = spy.push(metadata=build_df, workbook=workbookid, errors='catalog') The error will be cataloged in the "Push Result" column with same error shown in the example. To view all errors, you can filter the push result to see where the push result was not success. error = push_results_df[push_results_df['Push Result'] != "Success"] error
    2 points
  11. I was able to use the template by using the 'copy' function as below. So grabbed the document template (as described in the article you've shared above), made a copy of it and assigned to a new document. new_document_name = 'Executive View: September' new_document = template_document.copy(label=new_document_name) new_document.name = new_document_name Then defined the parameters: new_document.parameters = { "month": 'September', "[Image]My Visualization 1": 'fig1.png', "[Image]My Visualization 2": 'fig2.png', "First App Impact": None, "First App": None } Appended it to my existing topic: topic.documents.append(new_document) And finally, pushed my topic back to seeq: spy.workbooks.push(topic)
    2 points
  12. Hi Johannes, maybe one quick way you could try: First you can move the signal by a longer period than the 90 minutes to make sure the value of the signal is present at the timestamp of the whole hour. Here I used $signal.move(-100min) Next you can resample the moved signal: $movedSignal.resample(1h) Regards, Thorsten
    2 points
  13. There are times when you may need to calculate a standard deviation across a time-range using the data within a number of signals. Consider the below example. When a calculation like this is meaningful/important, the straightforward options in Seeq may not be mathematically representative to calculate a comprehensive standard deviation. These straightforward options include: Take a daily standard deviation for each signal, then average these standard deviations Take a daily standard deviation for each signal, then take the standard deviation of the standard deviations Create a real-time standard deviation signal (using stddev($signal1, $signal2, ... , $signalN)), then take the daily average or standard deviation of this signal While straightforward options may be OK for many statistics (max of maxes, average of averages, sum of totalizes, etc), a time-weighted standard deviation across multiple signals presents an interesting challenge. This post will detail methods to achieve this type of calculation by time-warping the data from each signal then combining each individually warped signal into a single signal. Similar methods are also discussed in the following two seeq.org posts: Two different methods to arrive at the same outcome will be explored. Both of these methods share the same Step 1 & 2. Step 1: Gather Signals of Interest This example will consider 4 signals. The same methods can be used for more signals, but note that implementing this solution programmatically via Data Lab may be more efficient when considering a high number of signals (>20-30). Step 2: Create Important Scalar Constants and Condition Number of Signals: The number of signals to be considered. 4 in this case. Un-Warped Interval: The interval you are interested in calculating a standard deviation (I am interested in a Daily standard deviation, so I entered 1d) Warped Interval: A ratio calculation of Un-Warped Interval / Number of Signals. This metric is detailing what the new time-range will be for the time-warped signals. I.e. given I have 4 signals considering a days worth of data of, each signal's day worth of data will be warped into 6 hour intervals Un-Warped Periods: This creates a condition with capsules spanning the original periods of interest. periods($unwarped_interval) Method 1: Create ONE Time-Shift Signal, and move output Warped Signals The Time Shift Signal will be used as a counter to condense the data in the period of interest (1 day for this example) down to the warped interval (6 hours for this example). 0-timeSince($unwarped_period, 1s)*(1-1/$num_of_signals) The next step is to use this Time Shift Signal to move the data within each signal. Note there is an integer in this Formula that steps with each signal applied to. Details can be viewed in the screenshots. $area_a.move($time_shift_signal, $unwarped_interval).setMaxInterpolation($warped_interval).move(0*$warped_interval) The last step is to combine each of these warped signals together. We now have a Combined Output that can be used as an input into a Daily Standard Deviation that will represent the time-weighted standard deviation across all 4 signals within that day. Method 2: Create a Time-Shift Signal per each Signal - No Need to move output Warped Signals This method takes advantage of 4 time-shift signals, one per signal. Note there is also an integer in this Formula that steps with each signal applied to. Details can be viewed in the screenshot. These signals take care of the data placement, where-as the data placement was taken care of using .move(N*$warped_interval) above. 0*$warped_interval-timeSince($unwarped_period, 1s)*(1-1/$num_of_signals) We can then follow Method 1 to use the time shift signals to arrange our signals. We just need to be careful to use each time shift signal, as opposed to the single time shift signal that was created in Method 1. As mentioned above, there is no longer a .move(N*$warped_interval) needed at the end of this formula. The last step is to combine each of these warped signals together, similar to Method 1. $area_a.move($time_shift_1, $unwarped_interval).setMaxInterpolation($warped_interval) Comparing Method 1 and Method 2 & Calculation Outputs The below screenshot shows how Methods 1 & 2 arrive at the same output Note the difference in calculated values. The Methods reviewed in this post most closely capture the true time-weighted standard deviation per day across the 4 signals. Caveats and Final Thoughts While this method is still the most mathematically correct, there is a slight loss in data at the edges. When combining the data in the final step, the beginning of $signal_2 falls at the end of $signal_1, and so on. There are some methods that could possibly address this, but this loss in samples should be negligible to the overall standard deviation calculation. This method is also heavy on processing, especially depending on the input signals' data resolution and as the overall number of signals being considered increases. It is most ideal to use this method if real-time results are not of high importance, and better fitting if the calculation outputs are input in an Organizer that displays the previous day's/week's/etc results.
    2 points
  14. Hello, Exports cannot currently be modified from the UI. That functionality will be available when R62 releases in the August-September time range. They can be modified manually through the API using the POST /export/id endpoint if you're comfortable with that, but I'd recommend the delete & recreate method for now. Thanks.
    2 points
  15. The challenge with specifying a shared folder by Path is that to the owner of the content, the workbook will show up in their own folder, while to the shared user it will show up in their Shared Folder. This means you'd have to specify a different path based on who was running the code. The easiest method would be to specify the workbook by ID. If you really need to specify by name, I'd suggest doing a spy.search or spy.workbooks.search to get the ID, and then use that as the argument in your push: Another alternative would be to have the workbook live in the Corporate folder. The docstring for spy.push shows how you can push to workbooks in the Corporate folder: f'{spy.workbooks.CORPORATE} >> MySubfolder >> MyWorkbook'
    2 points
  16. In Seeq's May 25 Webinar, Product Run and Grade Transition Analytics, we received a lot of requests during the Q&A asking to share more details about how the grade transition capsules were created and how they were used to build the histograms presented. So here is your step-by-step guide to building the condition that will make everything else possible! Defining your transition condition Step 1: Identifying product campaigns The "condition with properties" tool under "identify" can transform a string signal of product or grade code into a single capsule per campaign. In older versions of Seeq this same outcome can be achieved in formula using $gradeCodeStringSignal.toCondition('grade') Note: if you don't have a signal representing grade code, you're not out of luck, you can make one! Follow the general methodology here to search for set point ranges associated with different grades or combinations of set points that indicate a particular grade is being produced. Step 2: Create the transition condition encoded with the starting and ending product as a capsule property $t = $c.shrink(2min).inverse() //2 min is an arbitrary small amound of time to enable the inverse function to //capture gaps in the campaigns $t.removelongerthan(5min) // 5min is an arbitrarily small amount of time to bound calc (that is >2min * 2) .transform($capsule -> $capsule.setProperty('Transition Type', $pt.resample(1min).toscalars($capsule).first() + '-->' + $pt.resample(1min).toscalars($capsule).last())) .shrink(2min) Step 3: Identify time period before the transition Step 4: Identify time period after the transition Step 5: Optional - for display purposes only - Show the potential transition timeline (helpful for chain view visualizations) Step 6: Create a condition for on-spec data Here we use value search against spec limit signals. If you don't have spec limit signals already in Seeq, you can use the splice method discussed in the post linked in step 1 to define them in Seeq. Step 7: Combine condition for on-spec data and condition for time period before the transition Step 8: Combine condition for on-spec data and condition for time period after the transition Step 9: Join the end of the condition for on-spec before the transition to the start of the condition for on-spec after the transition to identify the full transition duration Your condition for product transitions is now officially ready for use in downstream applications/calculations! Working with your transition condition Option 1: Calculate and display the transition duration Use Signal from Condition to calculate the duration. The duration timestamp will generate a visual where the length of the line is proportional to the length of the transition. Increasing the line width in the customize panel will display the transition duration KPI in a gantt-like view. Option 2: Build some histograms Summarize a count of capsules like the number of times each distinct transition type occurred within the date range. Summarize a signal during each unique transition type, like the max/min/average transition duration for a distinct transition type. Option 3: Generate some summary tables Switch to the table view and toggle "condition" to view each transition as a row in the table with key metrics like total duration and viscosity delta between the two grades. Add the "transition type" property from the "columns" selector. If you run into hiccups using this methodology on your data and would like assistance from a Seeq or partner analytics engineer please take advantage of our support portal and daily office hours.
    2 points
  17. If you modify your wind_dir variable to $wind_dir = group( capsule(0, 22.5).setProperty('Value', 'ENUM{{0|N}}'), capsule(22.5, 67.5).setProperty('Value', 'ENUM{{1|NE}}'), capsule(67.5, 112.5).setProperty('Value', 'ENUM{{2|E}}'), capsule(112.5, 158.5).setProperty('Value', 'ENUM{{3|SE}}'), capsule(158.5, 202.5).setProperty('Value', 'ENUM{{4|S}}'), capsule(202.5, 247.5).setProperty('Value', 'ENUM{{5|SW}}'), capsule(247.5, 292.5).setProperty('Value', 'ENUM{{6|W}}'), capsule(292.5, 337.5).setProperty('Value', 'ENUM{{7|NW}}'), capsule(337.5, 360).setProperty('Value', 'ENUM{{8|N}}') ) You will get an ordered Y axis: This is how Seeq handles enum Signal values from other systems - it has some limitations, but it seems like it should work well for your use case.
    2 points
  18. Chromatography Transition Analysis in Seeq Many biopharmaceutical companies use Transition Analysis to monitor column integrity. Transition Analysis works by using a step change in the input conductivity signal and tracking the conductivity at the outlet of the column. The output conductivity transition can be analyzed via moment analysis to calculate the height of a theoretical plate (HETP) and the Asymmetry factor as described below. Step 1: Load Data and Find Transition Periods In order to perform this analysis in Seeq, we start by loading outlet conductivity and flow rate data for the column: Note: Depending on the density of the conductivity data, many companies find that some filtering of the data needs to be performed to get consistent results when performing the above differential equations. The agilefilter operator in Formula can be helpful to perform this filtering if needed: $cond.agileFilter(0.5min) Once the data is loaded, the next step is to find the transition periods. The transition periods can be found using changes in the signal such as a delta or derivative with Value Searches have also been applied. Combine the periods where the derivative is positive and the flow is low to identify the transitions. Alternative methods using the Profile Search tool can also be applied. Step 2: Calculate HETP As the derivatives are a function of volume instead of time, the next step is to calculate the volume using the following formula: Volume = $flow.integral($Trans) The dC/dV function used in the moment formulas can then be calculated: dCdV = runningDelta($Cond) / runningDelta($vol) Using that function, the moments (M0 through M2) can be calculated: M0 = ($dCdV*runningDelta($vol)).aggregate(sum(), $Trans, middleKey()) M1 = (($vol*$dCdV)*runningDelta($vol)).aggregate(sum(), $Trans, middleKey()) M2 = (($dCdV*($vol^2))*runningDelta($vol)).aggregate(sum(), $Trans, middleKey()) The moments are then used to calculate the variance: Variance = ($M2/$M0) - ($M1/$M0)^2 Finally, the HETP can be calculated: HETP = ((columnlength*$variance)/($M1/$M0)^2) In this case, the column length value needs to be inserted in the units desired for HETP (e.g. 52mm). The final result should look like the following screenshot: Alternatively, all of the calculations can be performed in a single Formula in Seeq as shown in the code below: $vol = $flow.integral($Trans) $dCdV = runningDelta($cond) / runningDelta($vol) $M0 = ($dCdV*runningDelta($vol)).aggregate(sum(), $Trans, middleKey()) $VdCdV = $vol*$dCdV $M1 = ($VdCdV*runningDelta($vol)).aggregate(sum(), $Trans, middleKey()) $V2dCdV = $dCdV*$vol^2 $M2 = ($V2dCdV*runningDelta($vol)).aggregate(sum(), $Trans, middleKey()) $variance = ($M2/$M0) - ($M1/$M0)^2 (52mm*$variance)/(($M1/$M0)^2) //where 52mm is the column length, L Step 3: Calculate Asymmetry Asymmetry is calculated by splitting the dC/dV peak by its max value into a right and left side and comparing the volume change over those sides. This section assumes you have done the calculations to get volume and dC/dV calculated already as performed for HETP in Step 2 above. The first step for Asymmetry is to determine a minimum threshold for dC/dV to begin and end the peaks. This is often done by calculating a percentage of the difference between the maximum and minimum part of the transition period (e.g. 10%): $min = $dCdV.aggregate(minValue(), $Trans, durationKey()) $max = $dCdV.aggregate(maxValue(), $Trans, durationKey()) $min + 0.1*($max - $min) The Deviation Search tool can then be used to identify the time when dC/dV is greater than the 10% value obtained above. Next, the maximum point of the dC/dV peaks can be determined by calculating the derivative of dC/dV in the Formula tool: $dCdV.derivative() The derivative can then be searched for positive values (greater than 0) with the Value Search tool to identify the increasing part of the dC/dV curve. Finally, a Composite Condition intersecting the positive dC/dV derivative and the transition values above 10% of the curve will result in the identification of the left side of the dC/dV curve: The right side of the dC/dV curve can then be determined using Composite Condition with the A minus B operator to subtract the positive dC/dV derivative from the transition above 10%: The change in volume can then be calculated by aggregating the delta in volume over each side of the peak using Formula: $Vol.aggregate(delta(), $LeftSide, middleKey()).aggregate(maxValue(), $Trans, middleKey()) Finally, the Asymmetry ratio can be calculated by dividing the volume change of the right side of the peak divided by the volume change of the left side of the peak. $VolRightSide/$VolLeftSide The final view should look similar to the following: Similar to HETP, all of the above formulas for Asymmetry may be calculated in a single formula with the code below: $vol = $flow.integral($Trans) $dCdV = (runningDelta($cond) / runningDelta($vol)).agileFilter(4sec) //calculate 10%ile of dCdV $min = $dCdV.aggregate(minValue(), $Trans, durationKey()) $max = $dCdV.aggregate(maxValue(), $Trans, durationKey()) $dCdV10prc = $min + 0.1*($max - $min) //Deviation search for when dCdV is above the 10%ile $deviation1 = $dCdV - $dCdV10prc $Above10 = valueSearch($deviation1, 1h, isGreaterThan(0), 0min, true, isLessThan(0), 0min, true) //Calculate filtered derivative of dCdV $dCdVderiv = $dCdV.derivative() //Value Search for Increasing dCdV (positive filtered derivative of dCdV) $dCdVup = $dCdVderiv.validValues().valueSearch(40h, isGreaterThan(0), 30s, isLessThanOrEqualTo(0), 0min) //Composite Conditions to find increasing left side above 10% and right side $LeftSide = $Above10.intersect($dCdVup) $RightSide = $Above10.minus($dCdVup) //Find change in volume over left side and right sides, then divide b/a $VolLeftSide = $Vol.aggregate(delta(), $LeftSide, middleKey()).aggregate(maxValue(), $Trans, middleKey()) $VolRightSide = $Vol.aggregate(delta(), $RightSide, middleKey()).aggregate(maxValue(), $Trans, middleKey()) $VolRightSide/$VolLeftSide Optional Alteration: Multiple Columns It should be noted that oftentimes the conductivity signals are associated to multiple columns in a chromatography system. The chromatography system may switch between two or three columns all reading on the same signal. In order to track changes in column integrity for each column individually, one must assign the transitions to each column prior to performing the Transition Analysis calculations. Multiple methods exist for assigning transitions to each column. Most customers generally have another signal(s) that identify which column is used. This may be valve positions or differential pressure across each column. These signals enable a Value Search (e.g. “Open” or “High Differential Pressure”) to then perform a Composite Condition to automatically assign the columns in use with their transitions. Alternatively, if no signals are present to identify the columns, the columns can be assigned manually through the Custom Condition tool or assigned via a counting system if the order of the columns is constant. An example of Asymmetry calculated for multiple columns is shown below: Content Verified DEC2023
    2 points
  19. Hi Leah - Would you mind submitting a support ticket via our portal at https://seeq.atlassian.net/servicedesk/customer/portal/3? If you can view the dashboard and have shared it with your colleagues via the share menu, yet they get this error, we will want to investigate a bit more.
    1 point
  20. Hello Bill, if you do not have Data Lab than you can use the REST API of Seeq. I created an example using Powershell: $baseurl = "https://myseeqinstance" $headers = @{ "accept" = "application/vnd.seeq.v1+json" "Content-Type" = "application/vnd.seeq.v1+json" } $body = @{ "authProviderClass" = "Auth" "authProviderId" = "Seeq" "code" = $null "password" = "<password>" "state" = $null "username" = "<username>" } | ConvertTo-Json $url = $baseurl + "/api/auth/login" $response = Invoke-WebRequest -Method Post -Uri $url -Headers $headers -body $body $headers = @{ "accept" = "application/vnd.seeq.v1+json" "x-sq-auth" = $response.Headers.'x-sq-auth' } $url = $baseurl + "/api/users?sortOrder=email%20asc&offset=0&limit=200" $response = Invoke-RestMethod -Method Get -Uri $url -Headers $headers $selectedAttributes = $response.users | Select-Object -Property firstname, lastname, email $selectedAttributes | Export-Csv -Path "userlist.csv" -NoTypeInformation The script uses the /auth/login endpoint to authenticate the user and retrieves the list of users (limited to 200) using the /users endpoint. You would need to replace the value in the first line with the URL of your Seeq system (eg. https://mycompany.seeq.site) and specify the credentials by replacing <password> and <username> inside the script. The list of users will be written to the file userlist.csv inside the directory from which the powershell script is run. Let me know if this works for you. Regards, Thorsten
    1 point
  21. Yes-- this bug is addressed in SPy v190.5, which will be released to PyPI by the end 2023.
    1 point
  22. Hello Baishun, You are correct. When using the Histogram Tool with the aggregation type "time," the timezone of UTC is used. One way to ensure the aggregation time fits your time zone is to create a new condition with your appropriate time zone from Tools > Identify > Periodic Condition. Using this new condition, change the aggregation type in your histogram to Condition (instead of time). Let me know if this works for you. Best Regards, Rupesh
    1 point
  23. Seeq's .inverse() Formula function creates a new condition that is the inverse of the capsules of another condition. It is often used for identifying time periods between the capsules in an event related condition. For example, a user may create an "event" condition which represents equipment changes or maintenance events, and they then want to quantify the time duration in between the events, as well as the time elapsed from the last event to the current time. It may be important to statistically analyze the historical time between events, or they may want to be notified if the time since the most recent event exceeds some guideline value. A common and possibly confusing issue encountered by users is that the most recent "Time Since...." capsule (created with $EventCondition.inverse()) may extend indefinitely into the future, making it impossible to quantify the time elapsed from the last event to the current time. This issue is easily resolved with the approach shown in step 3 in the example below. 1. The user already has an event condition created named "Filter Changes", which represents maintenance events on a process filter which removes particulates. The user wants to monitor the time elapsed between filter changes and therefore creates a "Time Since Filter Change" condition using $FilterChanges.inverse(): 2. Signal from Condition is used to create the "Calculated Time Since Filter Change" signal, and the user chooses to place the result at the end of each capsule. Because the most recent capsule in the "Time Since Filter Change" condition extends past the current time and indefinitely into the future, the duration of that capsule can't be quantified (and it of course exceeds any specified maximum capsule duration). The user may be confused by the missing data point for the signal (see the trend screenshot below), and the missing value is an important result needed for monitoring. 3. The issue is easily resolved by clipping the most recent "Time Since Filter Change" capsule at the current time by adding .intersect(past()) to the Formula. This ensures the most recent "time since" capsule will not extend past the current time, by intersecting it with the "past" capsule. The "Calculated Time Since Filter Change" signal (lane 3 in screenshot, based on the clipped condition) updates dynamically as time moves forward, giving the user near real time information on the time elapsed.
    1 point
  24. Hi John. This works like a charm. Thanks alot. 🙂
    1 point
  25. Hi Brandon Vincent, May I clarify the question? the issue arises when the drums loading occur in the middle of the day which causes your daily condition to split? If this is the case, one way to calculate the total material flow out of the tank daily could be as follows: Step 1. Use Composite Condition with intersection logic between "not loading condition" and "daily condition". This will result in splitting the daily condition based on the not loading condition Step 2. Use Signal from Condition to calculate the tank level delta (End value - Start value) bounded to the condition (1) and place the statistic at the start of the capsule. The result below shows 2 value for Sep 2023 capsules, one at 00:00 and another one at 13:32 timestamp. Step 3. The delta value will be negative as the End value is smaller than Start value. Use Formula, abs() function to take the absolute value (positive value) $end_start.abs() Step 4. Find the Sum of delta (End value - Start value) using Signal from Condition and bound the calculation to the "daily condition". This should be the total amount of material flow out from the tank daily despite having drum loading in the middle of the day. If the steps mentioned earlier don't apply to the problem you're facing, here's how you can calculate how many drums have been loaded. Step 1. Use Signal from Condition to calculate the tank level delta (End value - Start value) during "drum loading" condition. Step 2. Use formula to calculate the number of drums loaded by dividing it with 21% (as mentioned in the question, 21% level increased per drum loaded). The floor() function is used to round a numeric value down to the next smallest integer. For example if the tank level during drum loading is 68.69%, 68.69%/21% = 3.27. The floor function round the value down to 3 drum. ($level/21).floor().setunits('') Let me know if this helps.
    1 point
  26. If I'm understanding you correctly, you want to find the last N batches that were ran that are of the same product as the most recent batch, correct? We can do that with some more creative condition logic and filtering 🙂 Assuming that our batch capsules have a property called 'Product' that tells us what product that batch ran, we can add some filtering to our above formula to only pass in capsules to toCapsulesByCount() that match the last product produced: // return X most recent batches in the past Y amount of time $numBatches = 3 // X $lookback = 3mo // Y $currentLookback = capsule(now()-$lookback, now()) // find the product of the most recent batch and assign that // as a capsule property to the lookback condition // this will let us filter the batches condition dynamically $lookbackCond = condition($lookback, $currentLookback) .setProperty( 'Product', $batchCondition.toSignal('Product', endKey()), endValue(true) ) // filter the batch condition to only find batches of the active product $filteredBatches = $batchCondition.removeLongerThan($lookback) .touches($lookbackCond, 'Product') // create a rolling condition with capsules that contain X adjacent capsules $rollingBatches = $filteredBatches.toCapsulesByCount($numBatches, $lookback) // find the last capsule in the rolling condition that's within the lookback period $batchWindow = condition( $lookback, $rollingBatches.toGroup($currentLookback, CAPSULEBOUNDARY.ENDSIN).last() ) // find all the batches within the capsule identified // ensure all the batches are within the lookback period $filteredBatches.inside($batchWindow) .touches(condition($lookback, $currentLookback)) The key here is to pass a capsule property name to touches() to allow filtering with this dynamic property value (the keep() function requires the input comparison scalar to be certain). The output will look something like this -- note I'm only getting the 3 more recent batches so you can actually read the property labels 🙂
    1 point
  27. Hi KOKI, Generally for both #2 and #3, we often will bring the data in as capsules with capsule properties. It ultimately depends on how your system reports it, but typically the data is often reported as a time series of when the test is run (e.g. when the material property is recorded or quality data is run) and/or when the material is used (e.g. when you add a material to the batch, it will record what lot number you used and how much). Obviously it depends a bit on what and when your data is recorded, but Seeq has a number of tools to move the timeseries to align with when you want them as long as there is some logic that we can attach to (such as X batch ID had Y material number, which then had Z property value. There's some examples of this time alignment available at the article below - you specifically may want to look at #4:
    1 point
  28. Hi Marcelo, I am not certain I understand but I think you want to count the number of regenerations in between each "cleanup". If this is the case then you likely need to define a condition for when "cleanups" actually occur, and then you can create a condition for "time in between cleanups" using the .inverse() Formula function (there are several relatively easy you might find the "time in between cleanups"). Once you have the condition for "time in between cleanups", then a runningSum() Formula function would be one way to count the current number of regenerations. I'm happy to try to help iterate with your solution here on the forum, but because I'm not certain about the calculations you want to do, the most efficient way to get a solution for your questions would be to sign up for an upcoming Seeq Office Hours where you can share your screen and get help: https://info.seeq.com/office-hours John
    1 point
  29. Hello, yes this is possible and you can do it in different ways: You can use the Union Operator to unify all these condition into a single one then calculate the average for each capsule. The Formula tool is useful here and forumula should look like: $signal.aggregate(average(), $condition1.union($condition2).union($condition3).union($condition4).union($condition5), startKey()) You can also use formula and the splice operator to achieve similar result: $signal.aggregate(average(), $condition1, startKey()).splice($signal.aggregate(average(), $condition2, startKey()), $condition2).splice($signal.aggregate(average(), $condition3, startKey()), $condition3).... With one of the above methods, you will get the average of your signal for each capsule of the conditions 1 to 5. Finally, as you want to calculate the cumulative average, you can apply the runningsum() formula to the result of the previous step. Let me know if this is helpful.
    1 point
  30. Data Lab allows for multiple jobs to be scheduled in the same project. Each job should create a separate executor.0.html file in the _Job Results folder after the job has run. Are you sure that your first job was scheduled correctly? The attached screenshots don't show the full schedule confirmation & detail so it is hard for me to tell. If you are still having issues, feel free to create a support ticket at https://support.seeq.com/ so we can troubleshoot further!
    1 point
  31. Sorry, try this: # Pull the workbook workbooks = spy.workbooks.pull(workbook_id) workbook = workbooks[0] # Add the template worksheet if template_worksheet.name in workbook.worksheets: workbook.worksheets[template_worksheet.name] = template_worksheet else: workbook.worksheets.append(template_worksheet) # Push back up spy.workbooks.push(workbook)
    1 point
  32. OK Pat here is the fixed-up ipynb file. Do a search for "MarkD says" and you'll see how I've left comments to explain what changes I made. topic_document_deprecated (1).ipynb
    1 point
  33. When attempting to install Seeq version R60.1.1, I received the error depicted in the screenshot when executing the "from seeq import spy" command. I successfully attempted to uninstall the currently installed version of Seeq to match the server version but couldn't import spy. When switching back to Seeq version R58.1.1, I was able to run this line without error. Is this a bug related to R60? Thanks, Bao
    1 point
  34. Looking back, you're correct that R57+ does not have the ability to configure a scheduled update every X months. Could you submit a support ticket and include the link to this post: https://seeq.atlassian.net/servicedesk/customer/portal/3? That way we can log the change and see if we can get you a workaround in the meantime.
    1 point
  35. Hi Emre, you could try to sum up the running deltas for each months: $signal.setMaxInterpolation(20d).runningdelta().runningSum(months('CET')) Regards, Thorsten
    1 point
  36. An oil and gas engineer would like to reproduce below Batch monitoring table for vessels in Seeq. Green color indicates that the valve for the respective batch step is open, while red color indicates the valve status is close. 1. Use Condition with Properties to create conditions with regen steps as property name. 2. Define the respective valves position at each regen step. $Running = ($Prod_Valve ~= "Open").intersect($Step.keep('Batch Step',isEqualTo('RUNNING'))) $Isopropyl = ($Prod_Valve ~= "Open").intersect($Step.keep('Batch Step',isEqualTo('ISOPROPYL FLUSH'))) $Startup = ($Prod_Valve ~= "Open").intersect($Step.keep('Batch Step',isEqualTo('STARTUP'))) $Down = ($Prod_Valve ~= "Close").intersect($Step.keep('Batch Step',isEqualTo('DOWN'))) combineWith ($Running,$Isopropyl,$Startup,$Down) 3. Create a scorecard metric with colour threshold for each valve: Count =1; Green – good with single valve opening Count >1 or <1; Red – alarm as potential multiple valves opening or issue with signal status 4. In condition Table view, select the Capsule Property at the Headers. Optionally, the count value can be hidden by editing "" in the Number Format in the Item Properties as shown below. With this method, if all the valves indicates green means the valve Open/Close correctly. Thus user just need to focus on the red indicator.
    1 point
  37. One way of achieving this would be to compute the aggregate value (maxValue in your case) outside the call to setProperty() and then round the result before you use setProperty: // condition you want the aggregate of $condition = ($t > 100).removeLongerThan(1d) // output the maxValue as a signal that we can then round easily $maxPerCapsule = $t.aggregate(maxValue(), $condition, startKey()) .round(2) // round to two decimal places // use the rounded maxValue to set the property in the condition $condition.setProperty('Max Value', $maxPerCapsule, startValue()) This results in the aggregates getting computed with the original data and then rounded:
    1 point
  38. To add what Kin How mentioned above, I would encourage you to submit a feature request for this capability as I could see this being very helpful when iterating analyses over time. To do so, go to support.seeq.com, click on "Support Portal" in the lower right, and submit an "Analytics Help" request referencing this post. A one-liner will be sufficient. We will make sure it gets linked to a developer feature request.
    1 point
  39. yes, i'd rather do on capsule view only, because then i can also overlay with other asset runs in one visualization, like the figure i posted, but with the scorecard visualization, which is more powerful (very nice to have it in a control room for instance)
    1 point
  40. In version R60 and up Seeq supports native notifications for SaaS customers. Below is a link to the detailed documentation. https://support.seeq.com/space/KB/2594111502/Notifications+on+Conditions
    1 point
  41. Seeq is able to do many time periods, but what if you want to compare a time period from now, with similar periods in the past? An example of this could be comparing the 24 hour period before now, with the previous 24 hour period, and the one before etc. Add in that you want this period to be rolling and it becomes more complex. Example: The time now is 3PM, you want to look at the period of yesterday 3PM till today 3PM, and compare with similar 24 hour periods over the last 30 days. Then you want to do it again every 5 minutes with up to date time periods, so 3:05PM with the pervious 24 hours and similar periods for the last 30 days. The way to do this involves several steps, but basically the solution is to look at the time now from Midnight past, then move a daily periodic condition by this time, which is ever changing. Once you have executed this formula you can use the resulting capsules for aggregations on the rolling time periods extending into the past. You can paste the following formula directly into Seeq Formula tool, no need for any signals to be added. $days = days("Europe/London") $daynow = condition(1d, capsule(now()-24h, now())) //isolate the current daily capsule $timesince = timeSince($days, 1min) //time since midnight in minutes $timesincediscrete = $timesince.aggregate(endValue(true), $daynow, endKey(), 40h) //make discrete //Now going to use the discrete data point for time since midnight to make that time go into the past to cover // every daily capsule that is going to be moved in time, I am using 30 days, but change this to cover your own use case $movetimesincetopast = $timesincediscrete.move(-30 days) //now to join the current timesince midnight to the one moved into the past //we also have to resample the result to ensure data points are available for every capsule. The maxinterpolation should be greater //than your timescale, in my case greater than the 30 days used above $combined = $timesincediscrete.combineWith($movetimesincetopast).setMaxInterpolation(40 days).setunits('mins') $combinedresampled = $combined.resample(1min) //Now to transform our daily capsules to move the start of the capsules by the current time from midnight, note we can //only move the start as if we also move the end it will result in an error as the resulting capsules would no longer //be contained by the original capsules $transform = $days .transform($capsule -> { $newStartKey = $capsule.startKey() + $combinedresampled.valueAt($capsule.startKey()) capsule($newStartKey, $capsule.endKey()) }) //Finally we can use the start of the transformed capsules to create new 24 hour capsules, these will continually update on a refresh, or update frequency assigned in workbench //Use the 'step to current time' button in the display pane to see the start/end times of the capsules update in the capsule pane $finalcapsules = $transform.afterstart(24 hours) Return $finalcapsules Notes: If you extend this over a large time period/very high frequency refresh, you will cause significant calculations to be performed, slowing your Seeq system The resulting capsules will always be uncertain, as the time period is ever changing. Whilst I have used 'Days' & '24h', you can alter the formula for any time periods, but bear in mind that very short periods will cause significant calculations if over a relatively long time period into the past.
    1 point
  42. Hi Kate, The easiest way is to rename the tag by making a Formula that is just a formula of: $signal That way it will just reference your original tag, but you can give it whatever name you would like.
    1 point
  43. When I use the spy.push() command to insert some capsules into a worksheet it overwrites the existing worksheet rather than just inserting into it. The capsules are made correctly but all other data in the worksheet disappears. I've read through the docs and can't see anywhere where I need to specify to insert and not overwrite. Hoping someone could point me in the right direction. spy.push(data=CapsuleData, metadata=pd.DataFrame([{ 'Name': 'Capsules Times', 'Type': 'Condition', 'Maximum Duration': '1d' }]), workbook='Practice', worksheet='Practice Worksheet')
    1 point
  44. Hello, haven't been able to find this anywhere, Does Seeq currently have the capability to call a stored procedure from a SQL db? Thank you, -Alex
    1 point
  45. I know this is an old thread, but I am including what I did in case posterity finds it useful. I am more or less working the the same issue, but with a somewhat noisier and less reliable signal. I found the above a helpful starting point, but had to do a bit of tweaking to get something reliable that didn't require tuning. The top lane is the raw signal, from which I remove all the drop outs, filled in any gaps with a persisted value, and did some smoothing with agile to get the cleansed level on the next lane. For the value decreasing condition I used a small positive threshold (since there were some small periods of the levels fluctuating and the tank being refilled was a very large positive slope) and a merge to eliminate any gaps in the condition shorter than 2 hours (since all the true fills were several hours). For the mins and maxes I did not use the grow function on the condition like was done above, instead just used relatively wide max durations and trusted that the cleansing I did on value decreasing was good enough. I was then able to use the combinewith and running delta function on the mins and maxes, and filter to get the deliveries and the usage. One additional set of calculations I added was to filter out all the periods of deliveries by converting the Delta function to a condition and removing all the data points in conditions that started positive from the cleansed signal. I then subtracted a running sum of the delta function over a quarter, yielding a signal that without the effect of an of the deliveries over each quarter. I could then aggregate the delta for days and quarters of that signal to get the daily and quarterly consumption figures. Chart showing all the calculated signals for this example. Top lane is the raw signals. Next lane shows the cleansed signal with the nexus of the mins and maxes between deliveries. Middle lane combines the mins and maxes and takes the running deltas, and then filters them into delivery and usage numbers. The next lane removes the deliveries from the cleansed signal and does a running sum of the consumption over the quarter. The last two lanes are daily and quarterly deltas in those consumption figures. Calculation for identifying the periods in which the chemical level is decreasing. I used a small positive threshold and removed two hour gaps, and that allowed it to span the whole time between deliveries. Aggregate the cleansed signal over those decreasing time periods to find the min and max values. Used the combinewith and running delta functions to get the next deltas of consumption and deliveries. Filtered based on positive and negative value to separate into deliveries and consumption numbers. Removed the delivery numbers from the cleansed signal in order to get a running sum of consumption over a quarter. aggregated the deltas in the consumption history over days and quarters to calculate daily and quarterly consumption.
    1 point
  46. Question: I've already got a Seeq Workbench with a mix of raw signals and calculated items for my process. Is there a way to grab all signals and conditions on the display rather than having to find all of those through a spy.search()? Answer: Yes, you can absolutely specify the workbook ID and the worksheet number to grab the display items from the Seeq Workbench. Here's an example of how to do that: The first step is to pull the desired workbook by specifying the Workbook ID: desired_workbook = spy.workbooks.pull(spy.workbooks.search({ 'ID': '741F06AE-62D6-4729-A4C3-8C9CC701A2A1' }),include_referenced_workbooks=False) If you are not aware, the Workbook ID can be found in the URL by finding the identifier following the .../workbook/... part of the URL. (e.g. https://explore.seeq.com/workbook/741F06AE-62D6-4729-A4C3-8C9CC701A2A1/worksheet/DFAC6933-A68F-4EEB-8C57-C34956F3F238). In this case, I added an extra function to not include referenced workbooks so that we only get the workbook with that specific ID. Once you have the desired workbook, you will want to grab the index 0 of the desired_workbook since there should only be one workbook with that ID. You will then want to specify which worksheet you want to grab the display items from. In order to see your options for the worksheet index, run the following command: desired_workbook[0].worksheets This command should return a list of the possible worksheets within the Workbook ID you specified. For example, an output might look like this: [Worksheet "1" (EDAA0608-29EA-4EA6-96FA-B6A59D8AE003), Worksheet "From Data Lab" (34AC07F9-F2FF-4C9E-A923-B636D6642B32)] Depending on which worksheet in the list you want to grab the display items from, you will then specify that index. Please note that the indexes start at 0, not 1 so the first worksheet will be index 0. Therefore, if I wanted the first worksheet in the list, I can specify that I want to know the display items as: displayed_items = desired_workbook[0].worksheets[0].display_items If you show what the item "displayed_items" looks like, you should get a Pandas DataFrame of the Workbench items as if you had done a spy.search for all of them. For example: You can then use displayed_items as your DataFrame to perform the spy.pull() command to grab the data from a specific time range.
    1 point
  47. To better understand their process, users often want to compare time-series signals in a dimension other than time. For example, seeing how the temperature within a reactor changes as a function of distance. Seeq is built to compare data against time but this method highlights how we can use time to mimic an alternate dimension. Step 1: Sample Alignment In order to accurately mimic the alternate dimension, the samples to be included in each profile must occur at the same time. This can be achieved through a couple methods in Seeq if the samples don't already align. Option 1: Re-sampling Re-sampling selects points along a signal at select intervals. You can also re-sample based on another signal's keys. Since its possible for there not to be a sample at that select interval, the interpolated value is chosen. An example Formula demonstrating how to use the function is shown below. //Function to resample a signal $signal.resample(5sec) Option 2: Average Aggregation Aggregating allows users to determine the average of a signal over a given period of time and then place this average at a specific point within that period. Signal From condition can be used to find the average over a period and place this average at a specific timestamp within the period. In the example below, the sample is placed at the start but alignment will occur if the samples are placed at the middle or end as well. Step 2: Delay Samples In Formula, apply a delay to the samples of the signal that represents their value in the alternative dimension. For example, if a signal occurs at 6 feet from the start of a reactor, delay it by 6. If there is not a signal with a 0 value in the alternate dimension, the final graph will be offset by the smallest value in the alternate dimension. To fix this, in Formula create a placeholder signal such as 0 and ensure its samples align with the other samples using the code listed below. This placeholder would serve as a signal delayed by 0, meaning it would have a value of 0 in the alternate dimension. //Substitute Period_of_Time_for_Alignment with the period used above for aligning your samples 0.toSignal(Period_of_Time_for_Alignment) Note: Choosing the unit of the delay depends upon the new sampling frequency of your aligned signals as well as the largest value you will have in the alternative dimension. For example, if your samples occur every 5 minutes, you should choose a unit where your maximum delay is not greater than 5 minutes. Please refer to the table below for selecting units Largest Value in Alternate Dimension Highest Possible Delay Unit 23 Hour, Hour (24 Hour Clock) 59 Minute 99 Centisecond 999 Millisecond Step 3: Develop Sample Profiles Use the Formula listed below to create a new signal that joins the samples from your separate signals into a new signal. Replace "Max_Interpolation" with a number large enough to connect the samples within a profile, but small enough to not connect the separate profiles. For example, if the signals were re-sampled every 5 minutes but the largest delay applied was 60 seconds, any value below 4 minutes would work for the Max_Interpolation. This is meant to ensure the last sample within a profile does not interpolate to the first sample of the next profile. //Make signals into discrete to only get raw samples, and then use combineWith and toLinear to combine the signals while maintaining their uniqueness combineWith($signal1.toDiscrete() , $signal2.toDiscrete() , $signal3.toDiscrete()).toLinear(Max_Interpolation) Step 4: Condition Highlighting Profiles Create a condition in Formula for each instance of this new signal using the formula below. The isValid() function was introduced in Seeq version 44. For versions 41 to 43, you can use .valueSearch(isValid()). Versions prior to 41 can use .validityCapsules() //Develop capsule highlighting the profile to leverage other views based on capsules to compare profiles $sample_profiles.isValid() Step 5: Comparing Profiles Now with a condition highlighting each profile, Seeq views built around conditions can be used. Chain View can be used to compare the profiles side by side while Capsule View can overlay these profiles. Since we delayed our samples before, we are able to look at their relative times and use that to represent the alternate dimension. Further Applications With these profiles now available in Seeq, all of the tools available in Seeq can be used to gain more insight from these examples. Below are a few examples. Comparing profiles against a golden profile Determine at what value in the alternate dimension does each profile reach a threshold Developing a soft sensor based on another sensor and a calibration curve profile Example Use Cases Assess rotating equipment performance based on OEM curve regressions that vary based on equipment speed due to a VFD (alternate dimension = speed) Monitor distillation cut points based on distillation lab data (alternate dimension = lab standard, boil % in this case) Observe temperature profile along a reactor or well (alternate dimension = distance, length and depth in these cases)
    1 point
  48. For those like me who keep getting this error: Error getting data: condition must have a maximum duration. Consider using removeLongerThan() to apply a maximum duration. Click the wrench icon to add a Maximum Capsule duration. The resolution seemed to make sense to apply removeLongerThan() or setMaximumDuration() to the signal, but the correct answer is to set it to the capsule. For example, this is the incorrect formula I attempted. $series.aggregate(maxValue(), $capsules, endKey(), 0s) Here is the resolution: $series.aggregate(maxValue(), $capsules.setMaximumDuration(40h), endKey(), 0s) or $series.aggregate(maxValue(), $capsules.removeLongerThan(40h), endKey(), 0s) Hope this helps others who didn't have luck searching this specific alarm previously.
    1 point
  49. Background In this Use Case, a user created a condition to identify when the compressor is running. During each Compressor Running capsule, the compressor operates in a variety of modes. The user would like a summary of the modes of operation for each capsule in the form of a new signal that reports all modes for each capsule (i.e. Transition;Stage 1;Transition;Stage 2;Transition, Stage 1;Transition). Method 1. The first step is to resample the string value to only have data points at the value changes. It's possible the signal is already sampled this way, but if it is not, use the following Formula syntax to create a "compressed" signal: $stringSignal.tocondition().setMaximumDuration(3d).transformToSamples($capsule -> sample($capsule.getStart(), $capsule.getProperty('Value')), 4d) 2. Now, you can create a signal that concatenates the string values during each capsule. This is achieved using the following Formula syntax: $compressorRunning.setmaximumduration(10d).transformToSamples($cap-> sample( $cap.getStart(), $compressedStringSignal.toGroup($cap).reduce("", ($s, $capsule) -> $s + $capsule.getvalue())), 7d).toStep()
    1 point
This leaderboard is set to Los Angeles/GMT-07:00
×
×
  • Create New...