Unable to capture real-time intra-bar values with varip variable [pinescript]

Question

I'm trying to use varip variables to capture intrabar data in real time. A simplified example would be capturing the high at the 2 minute point of a 5 minute bar. The simplified code below illustrates what works, and what doesn't.

//@version=5
indicator("My script", overlay=true)

varip float h0 = na
varip float h2 = na
varip float h5 = na

d   = 8
hr  = 0
min = 10
capture_time_0  = time == timestamp(year, month, d, hr, min + 0)   
capture_time_2  = time == timestamp(year, month, d, hr, min + 2)   
capture_time_5  = time == timestamp(year, month, d, hr, min + 5)    

h0  := capture_time_0 ? high + 0 : h0
h2  := capture_time_2 ? high + 2 : h2
h5  := capture_time_5 ? high + 5 : h5

plot(h0,   color=color.white,   linewidth = 3)
plot(h2,   color=color.fuchsia, linewidth = 3)
plot(h5,   color=color.yellow,  linewidth = 3)
plot(high, color=color.blue)

Varip variables h0, h2, h5 were initiated to na. "Capture time" is defined as 0, 2 & 5 minutes after a particular starting point (Jul 8, 00:10 exchange time). When exchange time reaches each of the 3 designated capture times, h0, h2, h5 are re-assigned to the high value that exists at that time. For graphical clarity in this example, 0,2,5 are added to the highs associated with h0, h2, h5, respectively. As time passes and exchange time is no longer equal to capture time, h0, h2, h5 are maintained at the value that existed at capture time. All 3 values are plotted (h0=white, h2=fuchsia, h5=yellow) as is the high itself (blue line).

The top chart shows the results on a live 1 min chart. Everything works as expected. The h0=white, h2=fuchsia, h5=yellow lines show that all 3 were captured at the designated times, assigned the correct values and maintained as constant values for the remainder of the chart. The problem arises in the bottom chart when one switches to the 5 minute time frame (or when viewing the 5 min chart in real-time). Everything works as above EXCEPT the h2=fuchsia line is missing. In this case the missing h2 is the only value that is captured intrabar (2 minutes into the 5 minutes bar). Troubleshooting shows that this problem is repeatable on every time frame, e.g. you can capture values at 0 and 5 minutes on a 5 minute chart, but not at 1-4 minutes (intrabar). Similarly, you can capture values at the beginning an end of a 15 minute bar, but not at 1-14 minutes, etc.

Bottom line: although varip is designed to retain intrabar values, in this use case it is effectively not doing so. Any ideas how to get around this?

UPDATE: With @Mario's input (see comment below) the following changes improved the situation: replacing time with timenow and collecting over a 5 second range instead of the precise collection time:

capture_time_0  = timenow >= timestamp(year, month, d, hr, min + 0) and timenow <= timestamp(year, month, d, hr, min + 0) + 5000
capture_time_2  = timenow >= timestamp(year, month, d, hr, min + 2) and timenow <= timestamp(year, month, d, hr, min + 2) + 5000
capture_time_5  = timenow >= timestamp(year, month, d, hr, min + 5) and timenow <= timestamp(year, month, d, hr, min + 5) + 5000

While this allows you to collect data at all times, you are now collecting within a range of time, so the data is not the actual high of the range, but the high at the time of collection. I've tried to cut the range down, but below 5 seconds you go back to the problem of not collecting data. Moreover, the optimal range depends on how actively traded the current ticker is so whatever time you use is not uniformly applicable. Basically, for each ticker you are balancing the opposing forces of increased likelihood of data collection vs decreasing data accuracy. A second issue is that the collected data can only be used on the current chart. If you switch time frames the collected data does not "transfer". This is presumably an issue inherent to the use of varip and pinescript's recalculation upon loading new charts/time frames.

SOLUTION: I eventually found the following workaround. Unlike most indicators which are based on the current chart time frame, the capture time collection above is independent of the active chart's time frame. As a result, the code must have separate calculations to correctly capture on each chart's time frame.

• when capture time (e.g. 2, 5 or 30 min) is above the chart's time frame use ta.highest or ta.lowest to identify the highs/lows of the opening range

• when capture time (e.g. 2, 5 or 30 min) is below the chart's time frame use request.security_lower_tf to capture the lower time frame values into an array. Then identify the highs/lows of the capture range using a var variable (not varip) that extracts the high/low from the array using array.max and array.min with array.slice.

This approach works for all time frames. HOWEVER, there is a minor problem that only arises with thinly traded tickers. Specifically, there may not be a price point, or they may be less price points than expected if no trade has occurred when the array is being filled. One would think that pinescript would introduce na into the array in such situations but instead it makes no entry. This has unfortunate implications for extractions based on array an element's position in an array. I previously described that issue in a separate post: Handling arrays when request.security_lower_tf for 1 minute does not return values for each 1 min bar

Answer 1

First off, there is no way to use historical data to test varip.

The documentation says...

Because varip only affects the behavior of your code in the realtime bar, it follows that backtest results on strategies designed using logic based on varip variables will not be able to reproduce that behavior on historical bars, which will invalidate test results on them. This also entails that plots on historical bars will not be able to reproduce the script’s behavior in realtime.

Beside that you have still a flaw in your code example.
You using time instead of timenow.
time will only give you the Unix time of the open from that bar.
It is the opposite of time_close, which will give you the Unix time of the close.
That is the reason you wont catch any value in between.
You can read about time and time_close HERE.

Timenow on the other hand will give you the real time,
and it doesn't matter when using replay or historical data,
it will always return real time.
HERE you can read about timenow.

You should use the following in your code example...

capture_time_0  = timenow == timestamp(year, month, d, hr, min + 0)   
capture_time_2  = timenow == timestamp(year, month, d, hr, min + 2)   
capture_time_5  = timenow == timestamp(year, month, d, hr, min + 5) 

This should solve your problem, as long you work with live data.
Meaning no historical data nor replay mode.

---

It may be the case that you wont catch all the values of that specific time, cause you compare milliseconds. There might be lags or delays of some kind. To solve that problem you could do the following ...

capture_time_0  = math.ceil(timenow / 1000) == math.ceil(timestamp(year, month, d, hr, min + 0) / 1000)  
capture_time_2  = math.ceil(timenow / 1000) == math.ceil(timestamp(year, month, d, hr, min + 2) / 1000) 
capture_time_5  = math.ceil(timenow / 1000) == math.ceil(timestamp(year, month, d, hr, min + 5) / 1000)

This will compare seconds instead of milliseconds.
When this isn't enough either then you have to divide all by 60 additionally,
which will result in comparing minutes.

math.ceil() is simply rounding up all numbers to be integer values.

---

Addition

The problem seems to be that the time interval used by timenow is more or less unregular. To account for this you need to create a range.

This is what I did to create a range...

bool capture_time_0  = math.ceil(timenow / 1000) >= math.ceil(timestamp(year, month, d, hr, min + 0) / 1000) and math.ceil(timenow / 1000) <= math.ceil(timestamp(year, month, d, hr, min + 0) / 1000 + 15)
bool capture_time_2  = math.ceil(timenow / 1000) >= math.ceil(timestamp(year, month, d, hr, min + 2) / 1000) and math.ceil(timenow / 1000) <= math.ceil(timestamp(year, month, d, hr, min + 2) / 1000 + 15)
bool capture_time_5  = math.ceil(timenow / 1000) >= math.ceil(timestamp(year, month, d, hr, min + 5) / 1000) and math.ceil(timenow / 1000) <= math.ceil(timestamp(year, month, d, hr, min + 5) / 1000 + 15)

I added a 15 sec range which did the trick for my test. It seems timenow updates on each tick, and scripts only run on price updates in realtime.

Answer 2

I understand your use-case and makes absolutely sense, however pine script simply seems to work the way that it does not support your idea.

According to the official varip reference it cannot and should not work on historical bars, but as you formulated your question you're not interested in those anyway, so far so good.
Let's take the dummy example from the official docs and modify it slightly and execute it on a 5min chart:

//@version=5
indicator("")
varip int t = na
varip float foo = na
if barstate.isnew
    t := 1
    foo := close
else
    t := timenow
    foo := minute

plot(t, display = display.data_window)
plot(foo, display = display.data_window)

As you can see in the data window (right side panel) your tgets incremented, however not really second/millisecondwise. There seems to be a delay, though I'm not sure if in the script execution or in the local data update and rendering. This may ruin your timestamp idea already. Anyways, instead of minute - you could use time as well-, both of these values are going to be a value divisible by 5 instead of the real underlying time data. So in your example your intermediate condition simply never gets to be true. The other two get executed because these %5 values get stored in varip. Seemingly...

As an ugly workaround you could use something like this:

...
exp()  =>
    float t = na
    if minute == 2
        t := close
    
    t

min2 = request.security_lower_tf("", "1", exp()) // you retrieve your data from the lower timeframe instead of varip on 5min

float val = na
for value in min2
    if not na(value)
        val := value
        break
plot(val)

Since varip seems to have this "behavior" that it handles time specific data unexpectedly, this could be your only solution right now.
You may open a support ticket and address the issue. I would have expected varip to behave the way you do.