spy.pull() calculation with parameters

[Seth Gilchrist]

I recently received a question about passing formula parameters to the spy.pull, calculation argument, such as:

spy.pull(signals_dataframe, start='2019-01-01', end='2019-02-01', grid='15min', calculation='$signal.aggregate(average(), $capsules, endKey(), 0s)')

where $capsules is a condition parameter of the calculation.

The calculation argument in pull only performs calculations with a single parameter of the signal or condition being pulled, such as $signal.removeOutliers(), or $condition.removeLongerThan(10min).

To pull signals or conditions and apply calculations that require parameters, we can iterate over the DataFrame containing the items we want and create calculated signals on the server, which we then pull using the output from spy.push().

# find the condition that is the parameter of our calculation
condition_parameter = spy.search({'Name': 'Parameter Condition', 'Asset': 'Parameter Condition Asset'})

# make a list of dictionaries that we will turn into a dataframe for spy.push()
metadata_for_calcs = list()
for signal in signals_dataframe.itertuples():
    calc_dict = dict()
    calc_dict['Name'] = signal.Name + '_calced'
    calc_dict['Type'] = 'Signal'
    calc_dict['Formula'] = '$signal.aggregate(average(), $capsules, endKey(), 0s)'
    # Formula Parameters takes a dictionary with keys of the parameter name, and values of
    # a dictionary or DataFrame that has an ‘ID’ attribute identifying the item. 
    # We'll specify the $signal by ID, and pass the condition_parameter dataframe for $capsules.
    calc_dict['Formula Parameters'] = {
        '$signal': {'ID': signal.ID},
        '$capsules': condition_parameter
    }
    metadata_for_calcs.append(calc_dict)
# convert the list of dicts to a dataframe
metadata_for_calculated_signals = pd.DataFrame(metadata_for_calcs)

# Push the calculated signal definitions.
# I would recommend scoping them to a workbook to prevent clutter
pushed_caled_signals = spy.push(metadata=metadata_for_calculated_signals,
                                workbook='Scratch Folder >> DataLab Calcs',
                                worksheet='Calculated Signals')

calculated_signals = spy.pull(pushed_calced_signals, start='2019-01-01', end='2019-02-01', grid='15min')