Push plot from DataLab to Organizer?

[patjdixon]

I have DataLab code that generates a plot:

I thought there was a way in DataLab to push "ax" to an Organizer topic/sheet, but I can't find any information on how to do this.  The result I am looking for would be like:

Is there a way to do this, or do I have to export an image from DataLab and then manually import it into Organizer?

[Patrick]

Hi patjdixon - You do have the ability to push images directly into an Organizer Topic.  To do this, first save the image using plt's savefig method:

Lab_Pred_Fig.savefig('Awesome_Chart.png')

Create an Organizer Topic Object:

topic = spy.workbooks.Topic({'Name': "Charts"})

Add a page to the topic object:

page = topic.document('Visualizations Using Data Lab')

Add the image to the page.  Note the semicolon on the end suppresses the output from the cell.

page.document.add_image(filename='Awesome_Chart.png', placement='end');

Publish the Topic:

spy.workbooks.push(topic)

 

If you have an existing Topic, you can use spy.workbooks.pull() to pull in the object and follow the same methodology as above.  Note I included a [0] index to reference the Topic document in the pull results.

topic_search=spy.workbooks.search({'ID':'DFFC7BB8-9EE3-42DD-937A-2CE2FAAAB0E8'})
topic=spy.workbooks.pull(topic_search)

page = topic[0].document('Visualizations Using Data Lab')

 

[patjdixon]

This is great, but in my case I already have a topic and worksheets created.  What I would like to do is append the end of each Organizer worksheet with the appropriate plot.  For example, my worksheets could be 'Tensile' , 'Burst', 'Tear', and 'Fold'.  On each one, I want to append the plot of the property to the worksheet, so 'Tensile' would have a plot of Tensile, 'Burst' a plot of Burst, etc.

I cannot figure out a way to do pull/push to make this work.  Any ideas?

[Patrick]

You can append to existing topics/documents (pages) by first pulling the Organizer topic, and then adding to the respective page:

 

#Search for topic (I used the Topic ID, but there are other options as well):
topic_search=spy.workbooks.search({'ID':'DFFC7BB8-9EE3-42DD-937A-2CE2FAAAB0E8'})

#Pull the topic associated with that ID.  This creates an object of the Organizer that you can modify.
topic=spy.workbooks.pull(topic_search)

#Extract the "Tensile" page so you can modify it
tensile = topic[0].document('Tensile')

#Add image to the "Tensile" page
tensile.document.add_image(filename='Awesome_Chart.png', placement='end');

#Push modified Organizer back to Seeq:
spy.workbooks.push(topic)

Let me know if this works!

 

[patjdixon]

Yudaman!  Thanks, works great

[Kato]

Hi Patrick!

Is there a way to replace the image instead of adding it to the end? Or can you maybe delete the old one and add the new one? 

#Add image to the "Tensile" page
tensile.document.add_image(filename='Awesome_Chart.png', placement='end');

I'm having my code run with a scheduling and now it's pushing all of the images underneath eachother.

[Kato]

Also, I've realised that if I've made a Seeq calculation within a worksheet, and I paste this worksheet's link in the same Organizer Topic as where I want to push the png image, I get the following errors and it stops working.

How do I solve this without having to make a separate Organizer Topic for my calculations?

[Kin How]

Starting from version R63, we have introduced a new method for inserting data lab content in the organizer topic using IOTA content. It's important to note that this feature is currently in Preview mode, and the process for inserting and configuring this type of content is subject to change in the future.

[Kato]

This new method isn't working for me. Could you let me know how I need to change my code?

[patjdixon]

Kato,

"Is there a way to replace the image instead of adding it to the end? Or can you maybe delete the old one and add the new one?"

Wish I knew.  I have resorted to just deleting organizer documents and creating them new each time.  In my usage, each time you run the addon it re-creates everything. 

[patjdixon]

Kato,

Regarding the error, I don't think it can be related to the PNG image in the document.  My guess is you would have the same error with a blank organizer document.  The error shows you have a formula that references a signal, but it can't find the signal.  Hard to diagnose without looking at the signals and code.

[Patrick]

On 8/5/2024 at 1:17 AM, Kato said:

This new method isn't working for me. Could you let me know how I need to change my code?

  

For the Datalab Function approach, you need two things:
1) A Seeq Datalab API endpoint that can be called by Organizer
2) A html payload l so it can be returned when the endpoint is called

For 1), refer to https://support.seeq.com/kb/R63/cloud/invoking-a-data-lab-functions-rest-api-endpoint.  tl;dr, for the cell you want to execute when the endpoint is called, you will want to add `# GET /endpoint_name` to the top of the cell.  Make sure to include the hash (#).

For 2), assuming you have a png file, you will want to convert it to html.  The code block to achieve this is:

# GET /send_png_to_organizer

from io import BytesIO
import base64

#Open image and encode it as as a string
with open('my_image.png', 'rb') as image_file:
    # Read the binary data
    image_data = image_file.read()
    
    # Encode the binary data to base64
    base64_encoded_data = base64.b64encode(image_data)
    
    # Convert the base64 bytes to a string
    base64_string = base64_encoded_data.decode('utf-8')

html='<img src=\'data:image/png;base64,{}\'>'.format(base64_string)

{'headers': {'Content-Type': 'text/html'}, 'data': html}

 When the "/send_png_to_organizer" is called, this block will return the image encoded in an html string.

Once that is done, add the endpoint in Organizer via the "Seeq and IOTA Content" button located in the upper right corner of your Organizer Topic

The format for the URL is:

data-lab/{project_id}/sdl/api/notebooks/{notebook}/endpoints/{endpoint}

You don't need to include the ".ipynb" extension for the notebook.  So, for my example, the url looks like this:

"data-lab/55CEFBDC-8DE0-4F46-9129-2CC1FBD1235A/sdl/api/notebooks/image_in_organizer/endpoints/send_png_to_organizer"

When Organizer calls the endpoint, SDL will send back the image in the proper encoding:

 

Note this is not just limited to static images.  Some libraries, such as Bokeh, allow you to embed interactive content in html via file_html:

# GET /send_bokeh_to_organizer

# https://docs.bokeh.org/en/2.4.3/docs/gallery/color_scatter.html
# Install bokeh via `pip install bokeh` and restart the kernel.  Recommend installing in both 3.8 and 3.11 kernels as Seeq is transitioning to 3.11

import numpy as np
from bokeh.plotting import figure, show
from bokeh.embed import autoload_static, file_html
from bokeh.resources import CDN, INLINE


#Create a bokeh plot
N = 4000
x = np.random.random(size=N) * 100
y = np.random.random(size=N) * 100
radii = np.random.random(size=N) * 1.5
colors = np.array([ [r, g, 150] for r, g in zip(50 + 2*x, 30 + 2*y) ], dtype="uint8")

TOOLS="hover,crosshair,pan,wheel_zoom,zoom_in,zoom_out,box_zoom,undo,redo,reset,tap,save,box_select,poly_select,lasso_select,"

p = figure(tools=TOOLS)

p.circle(x, y, radius=radii,
          fill_color=colors, fill_alpha=0.6,
          line_color=None)

#Convert the bokeh plot to html
html = file_html(p, INLINE, "bokeh")

#Return paylaod as html
{'headers': {'Content-Type': 'text/html'}, 'data': html}

Attached are som e example code for png, Bokeh, and Matplotlib figures.  Let me know if this helps.

image_in_organizer.ipynb

 

 

Edited August 9 by Patrick
typos

[Kato]

Thanks so much! I was able to make it work indeed by using an html approach.