Chemical Usage Report

[IsaacNZ]

Hi all, 

Trying to set up a new Organizer Topic which tracks chemical usage which I can then use to form a report. 

I have a bulk hopper which decreases in weight as product is used but will also increase in weight when a delivery is taking place.

So what would be the most simple way to only sum the decreases in weight over a certain period? 

Thanks!

[Thorsten Vogt]

Hi Isaac,

I created an example based on this demo data:

First I recommend smoothing the signal to remove noise from it. 

Next step is calculating the derivative of the signal, which will be used to identify the periods of decreasing value and for determining the total quantity used:

Periods with decreasing value can be identified by using Value Search:

As you mentioned to calculate the amount only for a certain time period I created a time period for a specific shift. In this example the shift is 8h in duration and starts at 8 a.m.:

By using within() you can extract that portion of a signal existing during a specified condition. In this case I want to keep the derivative only during the periods of decreasing value:

Last step is calculating the amount for each shift:

As this is just an example you may have to make adjustments to it regarding your needs.

Hope this helps.

Regards,

Thorsten

 

[IsaacNZ]

Thanks so much for this Thorsten! Apologies it has taken me so long to get back to this.

I've been trying various filters on the original signal (as well as no filter) to try and get the final result to be as accurate as possible but I usually end up being up to 10% out for daily results, which can accumulate quite quickly when looking at monthly usage etc. I can usually filter the original signal to the point where one day's result will be exactly right but this doesn't carry over to other days. 

Do you know if this error is normal for this method or if there are any other approaches I could try which might be more accurate? 

Please let me know if this doesn't make sense..

Thanks!

[Andrew Fontenot]

I've faced similar issues. You may find it more accurate to take the opposite approach. If the error you are seeing is due to the totalization of all the decreases, you could instead totalize the increase and add that to the difference between the starting/ending amount. Example: Calculate amount used by subtracting the ending amount from the starting amount over whatever time span you'd like. Let's say it's daily and at the start you were at 1000lbs and the end you were at 500lbs plus you totalized an increase period (delivery) during that day of 250lbs. Your daily usage would then be (start - end + delivered) = 750lbs. 

Andrew

[IsaacNZ]

Thanks Andrew, just tried this and looks great!

Much appreciated

[Thorsten Vogt]

Hi Isaac, 

sorry for my late reply. I found another solution you might want to try. 

Starting point is identifying the decreasing periods by creating a smoothed signal, creating the derivative and using Value Search like I did in in my first post:

Next I use the following formulas to create two signals that creates samples at the timestamps on the turning points of the original signal, which can then by combined:

MinValues:

$tankLevel.aggregate(minvalue(), $valueDecreasing.growEnd(1mo), minkey()).setMaxInterpolation(1wk)

MaxValues:

$tankLevel.aggregate(maxvalue(), $valueDecreasing.inverse().growEnd(1mo), maxkey()).setMaxInterpolation(1wk)

Combined Signal:

$minValues.combineWith($maxValues)

By using within() and runningDelta() I filter for the portions of the signal during the decreasing periods:
 

$combinedSignal.within($combinedSignal.runningdelta().tostep() > 0)

Then I create capsules for each of that periods and intersect them with the target range:

$signalWhileDecreasing.toCapsules().intersect($days)

You can see that the new created capsules are more accurate then the old ones created by the derivative. I think this deviation results in the error you described.

Last steps is to calculate the amount for each of these capsules and aggregate them for the target range:

$tankLevel.aggregate(delta(), $intersectingCapsules, durationkey()).aggregate(sum(), $days, durationkey())

If I also display the values calculated from the derivative as described in my first post you can see the deviation between these calculations.

Regards,

Thorsten

 

[David Tiffany]

I know this is an old thread, but I am including what I did in case posterity finds it useful. I am more or less working the the same issue, but with a somewhat noisier and less reliable signal. I found the above a helpful starting point, but had to do a bit of tweaking to get something reliable that didn't require tuning. The top lane is the raw signal, from which I remove all the drop outs, filled in any gaps with a persisted value, and did some smoothing with agile to get the cleansed level on the next lane. For the value decreasing condition I used a small positive threshold (since there were some small periods of the levels fluctuating and the tank being refilled was a very large positive slope) and a merge to eliminate any gaps in the condition shorter than 2 hours (since all the true fills were several hours). For the mins and maxes I did not use the grow function on the condition like was done above, instead just used relatively wide max durations and trusted that the cleansing I did on value decreasing was good enough. I was then able to use the combinewith and running delta function on the mins and maxes, and filter to get the deliveries and the usage.

One additional set of calculations I added was to filter out all the periods of deliveries by converting the Delta function to a condition and removing all the data points in conditions that started positive from the cleansed signal. I then subtracted a running sum of the delta function over a quarter, yielding a signal that without the effect of an of the deliveries over each quarter. I could then aggregate the delta for days and quarters of that signal to get the daily and quarterly consumption figures.

Chart showing all the calculated signals for this example. Top lane is the raw signals. Next lane shows the cleansed signal with the nexus of the mins and maxes between deliveries. Middle lane combines the mins and maxes and takes the running deltas, and then filters them into delivery and usage numbers. The next lane removes the deliveries from the cleansed signal and does a running sum of the consumption over the quarter. The last two lanes are daily and quarterly deltas in those consumption figures.

Calculation for identifying the periods in which the chemical level is decreasing. I used a small positive threshold and removed two hour gaps, and that allowed it to span the whole time between deliveries.

Aggregate the cleansed signal over those decreasing time periods to find the min and max values.

Used the combinewith and running delta functions to get the next deltas of consumption and deliveries.

Filtered based on positive and negative value to separate into deliveries and consumption numbers.

Removed the delivery numbers from the cleansed signal in order to get a running sum of consumption over a quarter.

aggregated the deltas in the consumption history over days and quarters to calculate daily and quarterly consumption.

Edited November 17, 2021 by David Tiffany