Ability to restrict Data Sources to business hours

[nilesh.p]

Hi All,

We are using Azure Databricks clusters as a data source (will potentially move to Serverless SQL) for Seeq.

However, we would prefer to reduce cluster uptime to business hours to save costs.

It seems that Seeq continues to ping the cluster and brings it up every 30 minutes.

Is there a setting or schedule that can be applied to the data sources so that it only remains active during the day?

[James DeMarco]

Indeed, there is a parameter "ShouldMonitor" for Databricks connections that makes it possible to disable connection monitoring - see https://support.seeq.com/kb/latest/cloud/databricks-lakehouse#DatabricksLakehouse-Databricks-SpecificSQLAdditionalConfiguration

[nilesh.p]

Thanks for the response.

This parameter is already being used and is set to false, but the cluster is still being brought up every half hour.

[James DeMarco]

Hi Nilesh,

In that case, it's likely that some user-initiated automated activity is inducing the connection, e.g., a scheduled notebook querying for data from Databricks.  You may be able to identify this by looking at the Requests, Jobs, or Schedules tabs in the Administration UI, or in the logs - if you aren't able to figure out the reason for the activity from there, you can file a support request at support.seeq.com for assistance from our team.

Cheers,
James