Mark Derbecker

Pat if you're only pushing a workbook (and not accompanying that workbook push with calculations as well), you'll use `spy.workbooks.push()` instead of `spy.push()`.

Take a look at the "Using Pre-existing Signals/Conditions/Scalars etc" section of the Workbook Templates.ipynb documentation file:

Hi Pat, here's the documentation. There's not much there, but this is it:

https://python-docs.seeq.com/reference/submodules/workbooks.html#seeq.spy.workbooks.TopicDocument.report

OK Pat here is the fixed-up ipynb file. Do a search for "MarkD says" and you'll see how I've left comments to explain what changes I made.

topic_document_deprecated (1).ipynb

Can you post your ipynb file Pat, I think that it will be fastest to show you what I mean if I can just rewrite your code.

It looks like you're working with a TopicDocument. As per the error message, you would just change `document` to `report`.

OK great-- it'll end up being someone else to answer that question because I'm not sure myself.

In the code I gave you, you'll see I use the `FormulaParameters` variable like so:

'Formula Parameters': FormulaParameters

The only thing wrong with the code I gave you (so far) is that you're not handling the case where spy.search() is returning zero rows. You'll need to debug that and put in error handling code.

> Has anyone ever done this successfully? 

Yes. However, I'd say that you have to understand some Python basics: What is a dictionary and how does it work, what is its syntax; what is a set and what is its syntax. I supplied some references above for learning.

This error is coming from Pandas and indicates that your search result returned zero rows. You'll need to check to see that your spy.search() result includes at least one row.

I think it's important that you step through your code and really try to understand what each line is doing, it'll be too hard for us to have a back and forth for every error you encounter.

There are a bunch of online resources for understanding Python and Pandas basics:

https://www.google.com/search?q=best+resource+for+learning+python+and+pandas

PM_SIM_SetModels_230416 - MarkD Edits.ipynb

Pat, here's an updated version. I'm not able to test it because I don't have your data sets. But the primary difference is that I replaced this line that dynamically assigns values to local variables:

# locals()[InputVar + '_ID'] = spy.search({

with code that just stores the IDs in a dictionary:

            SearchResults = spy.search({
                'Name': InputVar + '_ID',
                'Scoped To':scopedtoid })
            InputVar_ID = SearchResults.iloc[0]['ID']
            TagIDs[InputVar] = InputVar_ID

You can see that instead of constructing a string, we construct a `FormulaParameters` dictionary where the keys are variable names and the values are IDs.

If it doesn't work right away, it won't surprise me because I couldn't test it. Please take a little time debugging it by adding print statements and walking through it so you have a feel for what it's doing..

Any chance you could just upload your entire .ipynb file? That way I can propose a rewrite that doesn't involve constructing a string.

You'll want to click Show Stack Trace. I believe that error is likely coming from Pandas, indicating that one of your DataFrames is empty.

The reason I say this is an unorthodox method is that it's very hard to debug. It would be better to deal with the variables names in string form rather than assigning them to Python variables and then constructing code via string concatenation. I.e., instead of creating a variable called PulpEye_BlendFreeness_ID, just keep a DataFrame with row a row whose Name is PulpEye_BlendFreeness and has an ID column.

ChatGPT may be able to help you understand your code and suggest a better method overall. You can paste it in and ask it to help you structure it in a more supportable way.

Hmmm, you're constructing Python code within a string... so I think you could add

FormVarListDict = eval('{' + FormVarList + '}')

and then use it as

'Formula Parameters': FormVarListDict

It's unorthodox but I think it would work.

Can you post a good chunk of your code as text so it's easier to see what you're doing in the lines above your screenshot?

Formula Parameters needs to be a Python dictionary, where the keys are the variable names and the values are the IDs, or a Pandas DataFrame row containing the ID.

So instead of constructing a string on line 60, you'll need to create a dictionary and assign rows from a spy.search() call.

You can see how to do it near the end of the SPy Tutorial:

https://python-docs.seeq.com/user_guide/Tutorial.html#pushing-a-new-condition-to-seeq

I'm not sure what's happening, but I'd recommend uninstalling all "private" versions of the seeq module. "Private" versions are those that are found in /home/datalab/.local. That way you will be using the version of seeq module that is supplied with Data Lab.

Hi Mohamed, do you have a question about what the error message is trying to tell you?

Have you tried the command it recommends that you execute at the end of the message?

Hi Wen,

You should be able to supply the "errors" argument to skip over the error:

spy.pull(items, errors='catalog')

Hi Wen-- Maximum Interpolation is special, it has to be "overridden". Just change property_name to 'Override Maximum Interpolation'.

It is better to correlate the calculation variables with the stored signals by using the ID -- the name of the variable can be anything, so it is not a reliable link between the two.

Hi Mohammad,

When you include the "all_properties=True" argument in spy.search(), you will receive a "Formula Parameters" column that has the variables as a list of strings in the form "variable_name=item_id" like so:

To get exactly the DataFrame you're asking for in your original post, you can then use the following DataFrame manipulation code and another call to spy.search():

variables_df = pd.DataFrame([{'Variable Name': f"${fp.split('=')[0]}", 'ID': fp.split('=')[1]}
                             for fp in search_df.iloc[0]['Formula Parameters']])
merged_df = spy.search(variables_df)
merged_df = merged_df[['ID', 'Name', 'Type']]
merged_df = merged_df.merge(variables_df, on='ID')
merged_df

It'll look like this:

There is a new capability in SPy version 184.12 (released today to PyPI) called Workbook Templates that I believe fits the bill for what @hanskanns wants to do.

To see it in action, you'll need to install the new version of SPy using the instructions at https://pypi.org/project/seeq and then get the updated documentation by doing

spy.docs.copy()

Then you'll see a new documentation notebook called Workbook Templates.ipynb that will walk you through it. You can put some "placeholders" for text fields that you want to replace using syntax like

{{MyTextField}}

in your Organizer Topic.

If you end up trying it, let us know if you have any problems getting it to work.

OK thanks Ivan. I think this is a bug on our side. We're looking into it.

But in the meantime, I think you should be able to drop the `Permission Inheritance Disabled` column from the DataFrame before pushing it. That should solve the issue. Can you try that?

Ah, sorry, maybe my first instinct is wrong. "May not modify the property directly" seems to indicate that we're sending that property incorrectly. But still can you tell me what version of SPy you are using? You can do `spy.__version__` in a notebook cell to see it.

Hey Ivan-- that looks to me like a SPy version mismatched to the server. What is the version of SPy and the version of the server you are using?

Hi Nate,

There are currently no storage retention limitations on such data. Seeq stores this data in its main data storage area (along with workbook definitions, calculation definitions etc). Seeq is not meant to be a "system of record" for pushed data -- we recommend that you always retain the ability to recreate this data from original sources if necessary. But to be clear, Seeq will currently not pare down this data using any sort of retention policy.

Mark