John Brezovec

The path doesn't need to already exist in the tree -- what's happening here is that SPy is truncating the path to the highest common asset in the Path. Since all of the items being inserted at Level 1 >> Level 2 >> Level 3, Level 3 is highest common asset, so the Level 1 and Level 2 are stripped out before inserting. In practice with your actual tags this shouldn't be an issue, unless you're intending to have a levels at the beginning of your tree that only contain a single asset as a child.

Got it! When working with large trees with spy.assets.Tree, you want to call insert as few times as possible. The way to do that is to insert using DataFrames. The first workflow that I would try is: Get the existing hierarchy tree using spy.assets.Tree(...) Use spy.search(...) to get all of the signals and conditions that I'm interested in Manipulate the results of spy.search to construct a DataFrame to add columns 'Path' and 'Friendly Name', which represent where in the tree you want to place the item, and what you want its name to be in the tree. Insert the entire DataFrame into your tree, don't worry about inserting items that already exist or not (they'll just get overwritten) When pushing the tree, specify a metadata_state_file. This file enables 'incremental pushing', meaning SPy will only push items that were not previously pushed. This should dramatically decrease how long it takes to repeatedly push large trees with small changes. An example of this on example data (anyone should be able to run it on their Seeq instance): import pandas as pd from seeq import spy tree = spy.assets.Tree('Insert with DataFrame', workbook='Example of DataFrame Insert') tags_to_insert = spy.search({'Name': 'Area ?_Temperature', 'Datasource Name': 'Example Data'}) tags_to_insert['Path'] = tags_to_insert['Name'].str.extract(r'(Area \w+)_\w+') tags_to_insert['Friendly Name'] = 'Temperature' tree.insert(tags_to_insert) tree.push(metadata_state_file='insert_with_dataframe.pkl')

When using spy.push you can use the workbook argument to push the data and metadata to a specific workbook. This can be specified as a path (Folder >> Path >> Workbook Name) or as an ID. My preference is to use an ID in case the workbook is moved or changes name. When building an asset tree with spy.assets.Tree, you specify the workbook at creation time, rather than at push time (e.g. spy.assets.Tree('My Tree', workbook='My Workbook'))

Just to double check -- it sounds you're trying to run this script on a schedule in order to keep the tree updated as new signals come in from the datasource, is that correct?

Seeq Data Lab provides a controlled environment to ensure stability and security for your data analysis projects. This means users do not have sudo privileges within the Data Lab project itself, and therefore cannot use typical package managers like apt-get or yum to install OS packages.

Is it possible to install additional system packages using apt in a Seeq Data Lab project?

You can specify NAN for the levels you don't want to generate. Using a csv like the following will let you generate a tree with uneven branch lengths: Level 1 Level 2 Level 3 Name Friendly Name My Equipment Cooling Tower Area A Area A_Temperature Temperature Area A_Relative Humidity Relative Humidity Area B Area B_Temperature Temperature Area B_Relative Humidity Relative Humidity Furnace NAN Furnace_Temperature Temperature Note the usage of NAN for Level 3 of my Furnace Temperature -- this will cause SPy to insert the Furnace Temperature at the path 'My Equipment >> Furnace' instead of 'My Equipment >> Furnace >> Area A'

I want to build an asset tree from a csv using spy.assets.Tree(), but I need branches of different lengths. For instance I want to build the following tree: My Equipment ├── Cooling Tower │ ├── Area A │ │ ├── Temperature │ │ └── Relative Humidity │ └── Area B │ ├── Temperature │ └── Relative Humidity └── Furnace └── Temperature Notice how the Temperature signal is at a different level for my Furnace compared to my Cooling Tower areas. I've tried to do this with the following csv: Level 1 Level 2 Level 3 Name Friendly Name My Equipment Cooling Tower Area A Area A_Temperature Temperature Area A_Relative Humidity Relative Humidity Area B Area B_Temperature Temperature Area B_Relative Humidity Relative Humidity Furnace Furnace_Temperature Temperature But it creates an additional 'Area B' under my Furnace asset currently.

Files deleted through the Data Lab UI are put into a Trash folder specific to the current project. Most of the time you can find and restore the deleted file using the Terminal: Open a new terminal session Find the location of the deleted file with the find command: find . -name "Important Notebook.ipynb" Once you know where your file was moved, use the copy command to copy the file back to the desired location. The following example copies the file back to the home directory of the project. Replace the path given in quotes with your result from the find command. cp "./.local/share/Trash/files/Important Notebook.ipynb" ~ Considerations: Files deleted through the terminal with rm are permanently deleted and unrecoverable. If you delete a file, create a new one with the same name, and then delete the new file, a number will be appended to the file when it gets placed in the trash (e.g. Important File 1.ipynb)

I accidentally deleted a file in a Data Lab project using the Data Lab UI. Is there a way to recover it?

Could you download the notebook and attach it to the thread? We're not able to access your internal link.

spy.user returns an object that contains various user information. For instance you can do things like spy.user.email or spy.user.name to get the information of the currently logged in user.

When using join, the useEarliestStart argument can be used to control the behavior when there are multiple start capsules before you see an end capsule. Setting to true will result in the longest possible capsule, while setting to false will result in the shortest possible capsule. This option is present in the Composite Condition tool in newer versions of Seeq: This can also be specified when using formula: $conditionA.join($conditionB, 10s, true)

Here's some examples of archiving entire trees as well as sections of trees -- let me know if this answers your questions! :

Data Lab and SPy allows the quick generation of Asset Trees within Seeq. Once you get proficient in building trees with SPy, the next question is: how do I get rid of these things? Deleting an Entire Tree The easiest way to do this is to use spy.search() to find all the items contained within your tree, set the Archived column to True, and then push that change as metadata using spy.push(). If I want to delete my asset tree whose root asset has the ID 0EF34B5A-D265-6470-8957-AE797AD00EF5, I can do the following: tree_root_id = '0EF34B5A-D265-6470-8957-AE797AD00EF5' # can be found in the item properties for the root asset items_in_tree = spy.search( [{'Asset': tree_root_id}, {'ID': tree_root_id}], workbook=spy.GLOBALS_AND_ALL_WORKBOOKS, # so I don't have to specify workbook all_properties=True, ) items_in_tree['Archived'] = True # set the items to archived spy.push(metadata=items_in_tree) # push the change back into Seeq Deleting a Section of a Tree Option 1: spy.assets.Tree If you are already using spy.assets.Tree() to create and push your asset trees, you can also use it to remove sections of your trees. If you pull an existing tree, you can then use tree.remove() to remove items by Name, path, wildcard, and spy.search() results (Refer to the documentation for more details). Once your tree is in the state you're happy with, re-push the tree with tree.push() and SPy will archive the removed items: tree = spy.assets.Tree('Cooling Tower 2', workbook='delete example') # pull in an existing tree called 'Cooling Tower 1' from workbook 'delete example' tree.remove('Area F') # remove Area F and its children tree.push() # push the change back to Seeq Option 2: spy.search & spy.push Just like with entire trees, you can search for a collection of items to remove from a tree, set the Archived column to True, and then push the metadata into Seeq. To perform the equivalent removal as Option 1: items_to_archive = spy.search( {"Path": "Cooling Tower 2", "Asset": "Area F"}, workbook="delete section", all_properties=True ) # search for the items we want to remove from our tree items_to_archive['Archived'] = True # set items to archived spy.push(metadata=items_to_archive) # push the change back into Seeq Considerations When using spy.search() and spy.push() to archive items in a tree, make sure to archive both the assets and child items. For instance, in the following tree, if you want to remove Area F from the tree, make sure to also archive the child Compressor Power item. Cooling Tower 2 |-- Area E | |-- Compressor Power | |-- Compressor Stage | |-- Irregular Data | |-- Optimizer | |-- Relative Humidity | |-- Temperature | |-- Wet Bulb |-- Area F <---- if I want to archive this branch... |-- Compressor Power <------ ALSO ARCHIVE THIS ITEM Not following this can leave your trees in inconsistent states and will lead to errors when working with this tree in SPy in the future, such as: - Item's position in tree does not match its path.

Please submit an IT support ticket on https://support.seeq.com/kb/latest/cloud/ -- we can best support you there.

If an expression isn't assigned to a variable in formula, Seeq expects it to be the formula output, and that output is expected to be the last expression in the formula. That's where your error is coming from -- your line 1 isn't assigned to a variable so Seeq expects it to be the formula output.

Passing the Totalized statistic behaves the same as if you called the Totalized statistic in formula without specifying a unit of measure (i.e. $condition.setProperty('Total Quantity', $signal, totalized()) : If you pass a signal with a rate unit like kg/min, totalize() will coerce the resultant value to a quantity (kg), since a unit like kg/min * s isn't useful If you pass a signal that is not a rate, totalize() will default to use seconds -- things like kg * s I'd be curious if you could recreate it giving you seconds one instance and minutes another. I'm also going to create a feature request in our system to be able to specify the units when using totalize with shape='capsules'

You're correct that it looks like this signal has been assigned the improper unit. These errors get raised outside of SPy, so there isn't a way to ignore them in Python. If you try trending this same signal in workbench you should get the same error. This is either a datasource configuration issue or a historian misconfiguration -- I'd suggest raising a support ticket so we can help you track down the source of the issue: https://support.seeq.com/kb/latest/cloud/

The max duration always has to be specified as a scalar (fixed number) -- you should try to pick the shortest max duration you're comfortable with for performance reasons, but if your use case requires a super long max duration then that's what your use case requires!

Your last formula is pretty close! You also need to include where you want to place the result of the aggregation - do you want it to display for the duration of the capsule, or maybe put it at the start or the end? $signal.aggregate(maxValue(), $condition, startKey()) // could replace startKey() with endKey(), middleKey(), durationKey(), maxKey() or minKey() This aggregation could also be performed using the Signal from Condition tool, which will let you pick your inputs in a point-and-click UI

Check out inserting with references, which will do what you're looking for: search_results = spy.search({'Name': '/Area [D,E,F]_Compressor Power/', 'Datasource Name': 'Example Data'}, order_by='Name') tree = spy.assets.Tree('My Tree') tree.insert(children=['Area D', 'Area E', 'Area F']) tree.insert( children=search_results, friendly_name='{{Name}}', parent='{{Name}(Area ?)_*}' ) Results in a tree that looks like: My Tree |-- Area D | |-- Area D_Compressor Power |-- Area E | |-- Area E_Compressor Power |-- Area F |-- Area F_Compressor Power In most cases though you're going to want the leaf node to have a 'generic' name (i.e. Compressor Power), and use the context of the tree to tell you what area it belongs to. You can also accomplish this using references: search_results = spy.search({'Name': '/Area [D,E,F]_Compressor Power/', 'Datasource Name': 'Example Data'}, order_by='Name') tree = spy.assets.Tree('My Tree') tree.insert(children=['Area D', 'Area E', 'Area F']) tree.insert( children=search_results, friendly_name='{{Name}Area ?_(*)}', # note the new inclusion of the capture group parent='{{Name}(Area ?)_*}' )

This page from our SPy documentation walks through a very similar example - again, the key is to specify the mustache ({{}}) notation on the image alt text, rather than just in the body of the document.

Thanks for the additional context. I agree I'm not sold on the utility of stacking the capsules end-to-end, but I figured I'd give an example of how it could be done since it may help move you in the right direction. In much the same way we calculated when the capsule should end based on the start_value + reset_quantity, we can find when the capsule should start based on the reset quantities of all the capsules before it in the same year (since we reset the totalizer every year). In practice this adds a bit of complexity to the formula 😉 $grow_amount = 3wk $condition_with_reset_quantity .move(-$grow_amount, $grow_amount) // grow the condition since a transform can only output capsules if they are within the originals .transform($c -> { $actual_start_key = $c.startKey() + $grow_amount // un-grown capsule start $actual_end_key = $c.endKey() - $grow_amount // un-grown capsule end $actual_input_capsule = capsule($actual_start_key, $actual_end_key) $current_year_capsule = $years.toGroup($actual_input_capsule, CAPSULEBOUNDARY.overlap) .first() $current_year_start = $current_year_capsule.startKey() // find all the reset quantity capsules that overlap the year // then sum up all the 'Reset Quantities' for those capsules. $start_value = $condition_with_reset_quantity.toGroup(capsule($current_year_start, $actual_start_key+1ns), CAPSULEBOUNDARY.INTERSECT) .forEach($cap -> $cap.property('Reset Quantity')) .sum() - $c.property('Reset Quantity') // what value should we stop the capsule at? $end_value = $start_value + $c.property('Reset Quantity') // generate a condition internal to the transform that tells us when we reach this end value $cond_volume_reached = $yearly_integral == $end_value // turn this condition back into a capsule so we can grab its start key $cap_volume_reached = $cond_volume_reached.removeLongerThan(1s) .toGroup($c) .last() // do the same thing with the start value $start_volume_reached = $yearly_integral == $start_value $cap_start_volume_reached = $start_volume_reached.removelongerthan(1s) .toGroup($c) .last() // create the output condition. Manually reassign the Reset Quantity property capsule($cap_start_volume_reached.startkey(), $cap_volume_reached.startKey()) .setProperty('Reset Quantity', $c.property('Reset Quantity')) .setProperty('Start Quantity', $start_value) .setProperty('End Quantity', $end_value) }) Notice how the resultant purple capsules stack end-to-end. The first capsule property on the purple condition is the reset quantity for that capsule, the second is the volume we should start that capsule at, and the third is the volume we want to end the capsule at. As a noted before, notice how the capsules are starting to creep forward over time.

If you do that, you're likely going to keep pushing your capsules further and further out into the future. Can you share more about the use case so I can make sure we're going down the right path?