Martin Boden

Main Issue: generating an organizer with png graphs from a custom template

I am trying to programmatically setup an Organizer from Python using spy. The pipeline I am trying to have is as follows:

1. Manually build a Organizer and add curly braces variables, save this organizer as Organizer_Template

2. From python, pull the organizer and save it as a zip template

reload_templates = True
if reload_templates:
    organizer_ids = {
        "Template_Organizer": "0EEDAD9D-B7EC-6220-8BF9-1BED3C3ED8B3",
    }
    for key in organizer_ids.keys():
        topic_search = spy.workbooks.search({"ID": organizer_ids[key]})
        topic = spy.workbooks.pull(topic_search)
        print(type(topic))
        spy.workbooks.save(topic, template_save_folder + key + ".zip")

3. Load the zip as template

# Load template
workbooklist = spy.workbooks.load(
    template_save_folder + "Template_Organizer.zip",
    as_template_with_label="Template Organizer",
)
organizer = workbooklist["Organizer_Template"] 
print(organizer.documents)

Output:

Success
[Worksheet "Template" (0EEDAD9D-BBB0-77D0-9C47-E014E0E3DB83 Template Organizer)]

4. Rename organizer and document name (the goal here is to keep the template as is in Seeq and generate a new populated organizer )

# Change the name of the organizer 
organizer.name = "Production Organizer"
# Change the name of the organizer document
organizer.documents["Template"].name = "Prod"

print(organizer.documents["Prod"].code)

Output:

document.parameters = {
    "Title": None,
    "[Image] cell1": None,
    "[AltText] cell1": None,
    "[Image] cell2": None,
    "[AltText] cell2": None
}

5. Populate the parameters

organizer.documents["Prod"].parameters = {
    "Title": "Prod 1",
    "[Image] cell1": os.path.join(
            plot_path, "G1", "long_term", "z_score_max.png"
        ),
    "[AltText] cell1": "G1 long_term z_score_max",
    "[Image] cell2": os.path.join(
            plot_path, "G2", "long_term", "z_score_max.png"
        ),
    "[AltText] cell2": "G1 long_term z_score_max"
}
print(organizer.documents["Prod"].parameters)

Output:

{'Title': 'Prod 1',
 '[Image] cell1': '/home/datalab/pdm/vibration_models/dashboard/docs/_contents/G1/long_term/z_score_max.png',
 '[AltText] cell1': 'G1 long_term z_score_max',
 '[Image] cell2': '/home/datalab/pdm/vibration_models/dashboard/docs/_contents/G2/long_term/z_score_max.png',
 '[AltText] cell2': 'G1 long_term z_score_max'}

6. Push the workbook back to Seeq

spy.workbooks.push(organizer,path=path)

Output:

ID	Name	Type	Workbook Type	Count	Time	Result	Pushed Workbook ID	URL
0	0EEDAD9D-B7EC-6220-8BF9-1BED3C3ED8B3 Template ...	Production Organizer	Workbook	Topic	1	0:00:00.151992	Success	... ... 

All seems good but then when I go to the organizer that I see:

Note:

- The path the the png is correct, if not an error appears when trying to push the notebook

So Seeq does find the path, processes the png but they do not appear correctly on the Organizer. Anyone knows what the issue is?

Secondary Issue: Generating multiple Organizers from a single template

The main reason for defining manually a template is so I can generate multiple organizers from the same template but showing different data.

In this case if I rerun exactly the same steps but in step 4 I write

organizer.name = "Production Organizer 2"

And for consistency I change the title of the document and paths to the plots in step 5 (G1-G2 to G3-G4)

organizer.documents["Prod"].parameters = {
    "Title": "Prod 2",
    "[Image] cell1": os.path.join(
            plot_path, "G3", "long_term", "z_screore_max.png"
        ),
    "[AltText] cell1": "G3 long_term z_score_max",
    "[Image] cell2": os.path.join(
            plot_path, "G4", "long_term", "z_score_max.png"
        ),
    "[AltText] cell2": "G4 long_term z_score_max"
}
organizer.documents["Prod"].parameters

Then, when I push back the organizer to Seeq, the changes have been taken into account Production Organizer 2 is in the folder defined by the path but it overwrote my first organizer. Note that when I push Production Organizer it does not overwrite Organizer_Template even though they are both in the same folder. How can I circumvent this issue?

Thank you in advance for your help!

When generating an asset tree using spy.assets.build() and specifying workbook=None as below I have no problem pushing the tree globally.

from my_asset_definition import MyPlant

# Assign a blank Build Path because we are creating a top-level node
metadata_df["Build Path"] = None

# There will only be one Build Asset -- all other sub-assets will be created by Components in our class definitions
metadata_df["Build Asset"] = "Test Plant"

build_df = spy.assets.build(MyPlant, metadata_df)
spy.push(metadata=build_df, workbook=None)

Pushed successfully to datasource Seeq Data Lab [Datasource ID: Seeq Data Lab] and globally scoped.

However, when I try to push trees globally using the spy.assets.Tree() approach, with the tree defined in a csv, the tree gets pushed to Data Lab Analysis - From Data Lab

my_csv_tree = spy.assets.Tree("asset_tree.csv", workbook=None)
my_csv_tree.push()

Pushed successfully to datasource Seeq Data Lab [Datasource ID: Seeq Data Lab] and scoped to workbook ID DDB8CEA5-6B25-46F0-9C56-237B40338FB6

I gathered that workbook=None  is overwritten in this case by the default workbook path as defined in _common.py

How can I ensure that workbook=None properly pushes the tree globally when using the CSV approach to generating asset trees?

Thank you in advance