patjdixon

The attached illustrates what I am trying to do. I want to have an Organizer template document that I can use for replacing the signals and appending to an existing Organizer topic. How do I do this in DataLab? seeq_organizer_templates.pdf

Setting include_inventory and refresh false dramatically reduced the time, Emilio helped me with this: spy.workbooks.push(WorkbookInstance, include_inventory=False, refresh = False)

Any update to the ticket?

Incident filed,

I am getting kernel crashes in a DataLab cell in which a loop is appending worksheets from a template. If the loop is a small number of iterations no crash occurs. If the loop a specific number of iterations, the kernel dies. It looks like the kernel is running out of memory when the crash occurs. Is there a way to allocate more memory, or do so programmatically? Are there diagnostics that would expose the problem?

Then how is it the metadata for NewTest_PRED shows 'true', the item is deleted each time the script is run, and UIConfig is not used in the script, but UIConfig shows 'FALSE'?

Mark YUDAMAN!!! That fixed it. What I really don't understand is the original code did not set UIConfig, it only set a formula. It looks like something gets stuck anytime you write to UIConfig for any item that carries over to all items? I don't get it, but this fixes the problem and I can get this working. This took a couple of weeks for me to figure out.

Is it possible that a prior script set UIConfig and that item is stuck with a bad UIConfig until I archive it in the API?

If you look at the image I originally posted, you see the code with 'true'. How does that end up being 'FALSE'?

I don't understand. The formula string is: '$target.regressionModelOLS(group(capsule("2021-01-03T02:12:48.021Z","2021-03-04T22:12:48.021Z")), true,$a, $a^2).variableSelection(0.05).predict($a, $a^2)'

For UIConfig alone: { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/UIConfig", "name": "UIConfig", "value": "{\"type\":\"prediction\",\"option\":\"linear\",\"polynomialValue\":2,\"mustGoThroughZero\":FALSE,\"variableSelectionEnabled\":FALSE,\"variableSelection\":0.05,\"windowStart\":1609365600000,\"windowEnd\":1616306430000,\"regressionMethod\":\"OLS\"}", "unitOfMeasure": "string" },

{ "id": "0EE6557D-882B-F970-9854-502353F5A143", "name": "NewTest_PRED", "type": "CalculatedSignal", "description": "Prediction tag", "effectivePermissions": { "read": true, "write": true, "manage": true }, "properties": [ { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Archived", "name": "Archived", "value": "false" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Cache+Enabled", "name": "Cache Enabled", "value": "true" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Cache+ID", "name": "Cache ID", "value": "0ee66176-8ce5-ee20-b559-9dcc6c36eb73", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Data+ID", "name": "Data ID", "value": "[0EE473B9-95A2-FB50-9DE6-29EBF1F67009] {Signal} NewTest_PRED", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Datasource+Class", "name": "Datasource Class", "value": "Seeq Data Lab", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Datasource+ID", "name": "Datasource ID", "value": "Seeq Data Lab", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Datasource+Name", "name": "Datasource Name", "value": "Seeq Data Lab", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Description", "name": "Description", "value": "Prediction tag", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Enabled", "name": "Enabled", "value": "true" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Formula", "name": "Formula", "value": "$target.regressionModelOLS(group(capsule(\"2021-01-03T02:12:48.021Z\",\"2021-03-04T22:12:48.021Z\")), true,$a, $a^2).variableSelection(0.05).predict($a, $a^2)", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/ID", "name": "ID", "value": "0EE6557D-882B-F970-9854-502353F5A143", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Interpolation+Method", "name": "Interpolation Method", "value": "Linear", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Key+Unit+Of+Measure", "name": "Key Unit Of Measure", "value": "ns", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Maximum+Interpolation", "name": "Maximum Interpolation", "value": "40", "unitOfMeasure": "h" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Scoped+To", "name": "Scoped To", "value": "0EE473B9-95A2-FB50-9DE6-29EBF1F67009", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Sync+Token", "name": "Sync Token", "value": "2023-10-08T20:16:06.907564", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/UIConfig", "name": "UIConfig", "value": "{\"type\":\"prediction\",\"option\":\"linear\",\"polynomialValue\":2,\"mustGoThroughZero\":FALSE,\"variableSelectionEnabled\":FALSE,\"variableSelection\":0.05,\"windowStart\":1609365600000,\"windowEnd\":1616306430000,\"regressionMethod\":\"OLS\"}", "unitOfMeasure": "string" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Unsearchable", "name": "Unsearchable", "value": "false" }, { "href": "/items/0EE6557D-882B-F970-9854-502353F5A143/properties/Value+Unit+Of+Measure", "name": "Value Unit Of Measure", "value": "", "unitOfMeasure": "string" } ], "cached": true, "scopedTo": "0EE473B9-95A2-FB50-9DE6-29EBF1F67009", "datasource": { "id": "1E25E639-5FFF-450A-BEC7-C91ED0080D01", "name": "Seeq Data Lab", "type": "Datasource", "datasourceClass": "Seeq Data Lab", "datasourceId": "Seeq Data Lab", "isArchived": false, "isRedacted": false }, "fingerprint": "0e798e88-9fdd-4e48-acd1-5bc073641a55", "isArchived": false, "isRedacted": false }

I think I now understand, you were talking about developer tools in Chrome, not Seeq. When I open the worksheet and go to Console I see: When I select item properties on NewTest_PRED I get:

Is this what you mean? You get to it in advanced mode, Commands, New Console. When I select the item properties in workbench and get a crash, nothing shows up here.

I am on a Mac, not sure how to do Ctrl+Shift+I and where to do it (worksheet, datalab script, etc)?

I have a DataLab script that creates a regression formula and pushes the item. The problem is that when I see the item in Workbench, it has no user interface (there is no edit capability, no UIConfig). When I try to open the item properties, the worksheet seems to crash and reverts to the home folder. I am sure I used to have this working about a month ago, but now I can't get it to work. For an example, the item "Prediction 1" that was created in the Workbench user interface (Tools, Model & Predict, Prediction), shows what a regression formula should look like: If I create a formula to remove data, the code below: produces the desired results, an item (TestRunning_pred) in WorkBench that I can edit. However, the code below to use a regression formula: produces an item (NewTest_PRED) that I can add to a worksheet but I cannot edit and if I select the item properties the worksheet disappears, as if it crashes: I don't see any difference between the formula created from the Tools (Prediction 1) and the item I create in DataLab (NewTest_PRED). I also don't see any difference in the way I create the metadata between "TestRunning_Pred" and "NewTest_PRED". However, it looks like a regression formula does not work when pushed from DataLab. I am on R61.1.3-v202307120420 Any ideas to fix this?

I am pushing 8 worksheets to a workbook and it is taking nearly 3 hours. I notice the count is 321. Is that what it thinks the number of worksheets is, or is it items, or something else? Regardless, this is a much higher number than I think it should be, which might explain why it is taking so long.

Thorsten, yup, that's it

I notice the UIConfig does not show the advanced setting "Limit to condition within training window". In the user interface you can select a condition for limiting the training window. This is different from the hard coded training window dates. If I look for this parameter in UIConfig it does now show up. The WindowStart and WindowEnd are the hard coded dates for Training Window, not related to the condition configured for "Limit to condition ..." The limit to condition does not appear in the formula parameter either: Is there any way to access this parameter from DataLab?

Is there any documentation on the property names or parameters? Perhaps if I knew them I could use the API to set the type as well as the scale, order, regression method, etc so that I could get what I need? Does anyone know where to find such documentation?

Thorsten, unfortunately your right. When I convert to "prediction\" I always get the default configuration of the model, which is linear. My formula has a 3rd order polynomial model, but the conversion makes it linear. That does not work for me. I guess I am really stuck.

This is very helpful. I found that you can convert a formula to a tool interface this way. In my case I had prediction models built in DataLab using formulas, but formulas do not show model coefficients. This technique allows me to specify UIConfig as: {"value": "{\"type\":\"prediction\"}”} This converts the formula to a tool interface It should be noted that if you do a "Get" on the property UIConfig for an item created in DataLab, it won't find UIConfig: However, you can still use the API to create UIConfig and it works. When using: {"value": "{\"type\":\"prediction\"}”} it produces to Tool user interface: When using: {"value": "{\"type\":\"formula\"}”} it produces:

I can confirm everything works

This is not what I want. I am not looking for a manual process or a script that inserts whole worksheets into an Organizer document. I want a programmatic way to insert objects in a Worksheet (trend, XY plot, table, or other item) into an Organizer document I would like to have a DataLab script that can take a trend, XY plot, table, or other item in a Workbench worksheet and insert the linked object in an Organizer page. I presume this is impossible?

188.0