CODEWITHSUNDEEP
rpromisefuture
r2evans
r2evans

Reputation: 160437

adapting 'future' to 'future_promise'

I am trying to convert this to a future_promise for a few reasons (some of which are just curiosity, tbh), but I can't seem to do everything I'd like to do. Perhaps some of it is just misreading the docs, but I think there's something else I'm doing incorrectly than just misunderstanding the purpose/flow.

Using future alone, I can work with the future process fairly easily.

library(future)
# library(logger)
plan(multicore, workers = 10)
(ppid <- Sys.getpid())
# [1] 2704731
quux <- future({
  logger::log_info(msg="in", ppid=ppid)
  Sys.sleep(3)
  out <- mtcars[sample(32, size=3),]
  logger::log_info(msg="out", ppid=ppid); out; }, seed=TRUE)
# {"time":"2022-11-12 14:13:43","level":"INFO","pid":3189217,"user":"r2","msg":"in","ppid":2704731}
quux$job$pid
# [1] 3189217
1+3   ### demo that the repl is still interactive
# [1] 4
# {"time":"2022-11-12 14:13:46","level":"INFO","pid":3189217,"user":"r2","msg":"out","ppid":2704731}
value(quux)
#                 mpg cyl  disp  hp drat    wt qsec vs am gear carb
# Merc 240D      24.4   4 146.7  62 3.69 3.190 20.0  1  0    4    2
# Toyota Corolla 33.9   4  71.1  65 4.22 1.835 19.9  1  1    4    1
# Lotus Europa   30.4   4  95.1 113 3.77 1.513 16.9  1  1    5    2

However, I do not know how to:

  1. Know the forked PID, formerly determined with quux$job$pid (useful for parent/current process tracking);
  2. Perform a then on the promised results, and have the then expression operate in the parent/current (not a child/forked) process (solved with %>% instead of %...>%); and
  3. I am failing to get to the value itself.
library(promises)
did_something <- FALSE
quux <- future_promise({
  logger::log_info(msg="in", ppid=ppid)
  Sys.sleep(3)
  out <- mtcars[sample(32, size=3),]
  out
}, seed=TRUE) %>%
  then(~ {
    logger::log_info(msg="out", ppid=ppid)
    did_something <<- TRUE
    .
  })
# >
# {"time":"2022-11-12 14:40:53","level":"INFO","pid":2704731,"user":"r2","msg":"out","ppid":2704731}
# >
quux$job$pid
# NULL
did_something
# [1] FALSE
quux
# <Promise [pending]>
# >
# {"time":"2022-11-12 14:40:53","level":"INFO","pid":2704731,"user":"r2","msg":"out","ppid":2704731}
quux
# <Promise [fulfilled: data.frame]>
did_something
# [1] TRUE
value(quux)
#  Error in UseMethod("value") :
#    no applicable method for 'value' applied to an object of class "promise"

Is that the canonical use of future_promise with then-clauses?

It's clear I'm missing something rather fundamental to future+promises.

(This is not intended to run in shiny, in case anyone thinks to bring in observers or reactivity.)

sessionInfo()
# R version 4.1.3 (2022-03-10)
# Platform: x86_64-pc-linux-gnu (64-bit)
# Running under: Ubuntu 22.04.1 LTS
# Matrix products: default
# BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
# LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so
# locale:
#  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8
#  [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
#  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C
# [10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
# attached base packages:
# [1] stats     graphics  grDevices utils     datasets  methods   base
# other attached packages:
# [1] promises_1.2.0.1  future_1.29.0     data.table_1.14.2 dplyr_1.0.10      optparse_1.7.3
# loaded via a namespace (and not attached):
#  [1] Rcpp_1.0.9        later_1.3.0       pillar_1.8.1      compiler_4.1.3    tools_4.1.3
#  [6] odbc_1.3.3        digest_0.6.29     bit_4.0.4         jsonlite_1.8.0    lifecycle_1.0.1
# [11] tibble_3.1.8      pkgconfig_2.0.3   rlang_1.0.5       bench_1.1.2       DBI_1.1.3
# [16] cli_3.3.0         parallel_4.1.3    curl_4.3.2        yaml_2.3.5        fastmap_1.1.0
# [21] globals_0.16.1    generics_0.1.3    vctrs_0.4.1       fs_1.5.2          hms_1.1.2
# [26] RPushbullet_0.3.4 bit64_4.0.5       getopt_1.20.3     tidyselect_1.1.2  glue_1.6.2
# [31] listenv_0.8.0     R6_2.5.1          redux_1.1.3       processx_3.7.0    parallelly_1.32.1
# [36] fansi_1.0.3       profmem_0.6.0     callr_3.7.2       purrr_0.3.4       logger_0.2.2
# [41] blob_1.2.3        magrittr_2.0.3    codetools_0.2-18  ps_1.7.1          ellipsis_0.3.2
# [46] assertthat_0.2.1  config_0.3.1      arrow_10.0.0      utf8_1.2.2        crayon_1.5.1

Upvotes: 3

Views: 525

Answers (1)

r2evans
r2evans

Reputation: 160437

  1. I believe getting the in-proc PID is not supported.

  2. (Already resolved, use %>% instead of %...>%.)

  3. As long as then(.) does not return the value (nor make it available), I think the only way to get the value from a future_promise is side-effect from the then block. For instance,

    quux <- NULL
ign <- future_promise({
  logger::log_info(msg="in")
  Sys.sleep(3)
  out <- mtcars[sample(32, size=3),]
  out
}, seed=TRUE) %>%
  then(~ {
    logger::log_info(msg="out")
    quux <<- .
  })
quux
# NULL
#### pause
quux
# INFO [2023-01-16 15:09:50] {"msg":"out"}
#                 mpg cyl  disp  hp drat    wt qsec vs am gear carb
# Toyota Corolla 33.9   4  71.1  65 4.22 1.835 19.9  1  1    4    1
# Merc 230       22.8   4 140.8  95 3.92 3.150 22.9  1  0    4    2
# Ferrari Dino   19.7   6 145.0 175 3.62 2.770 15.5  0  1    5    6

Upvotes: 1

Related Questions