How to control number of CPUs used by greta?

gosselinf · January 31, 2023, 9:27am

Hello every one,

I was wondering how the user can control the number of cores/CPUs that greta will use. I have tried using n_cores in great::mcmc function or setting mc.cores to 1 (see below), but it did not work. I am sorry, this might be a silly question since I am not an expert in parallelization but I would like to give greta a try on only one CPU, and does not know yet how to do this.

## load required packages and set basic options
################################################################################

library("tidyverse"); theme_set(theme_minimal())
library("parallel"); options(mc.cores = 1)
library("greta")
library("bayesplot")
library("bench")


## generate/specify data
################################################################################

theta <- 5 # poisson theta

set.seed(1)

(y <- rpois(1, theta))
y <- as_data(y)



## specify greta model
################################################################################

theta <- gamma(3,1)
distribution(y) <- poisson(theta)

greta_model <- model(theta)

plot(greta_model)

## configure model settings
################################################################################

n_chains <- 4
n_iter <- 1e4L
n_warmup <- 1e3L


## fit model
################################################################################

  
  
  greta_fit <- greta::mcmc(
    "model" = greta_model, "n_samples" = n_iter,
    "warmup" = n_warmup, "chains" = n_chains, n_cores=1 )
  
  
  
  ## assess fit
  ################################################################################
  
  summary(greta_fit)

All the best,

Frédéric

njtierney · February 6, 2023, 4:55am

Hi there @gosselinf!

I was able to get this to work - the code states that it is running 1 core, however have you found that it isn’t running 1 core?

I changed two things about your code:

I turned off library("parallel"); options(mc.cores = 1) - as greta is able to manage its cores, this might conflate/confuse things
removed the quotes around arguments:

reta::mcmc(
    "model" = greta_model, "n_samples" = n_iter,
    "warmup" = n_warmup, "chains" = n_chains, n_cores=1 )

While this technically works I would recommend against quoting the argument names in quotes as you lose autocomplete functionality in places like RStudio.

``` r
## load required packages and set basic options
################################################################################

library("tidyverse"); theme_set(theme_minimal())
# library("parallel"); options(mc.cores = 1)
library("greta")
#> 
#> Attaching package: 'greta'
#> The following object is masked from 'package:dplyr':
#> 
#>     slice
#> The following objects are masked from 'package:stats':
#> 
#>     binomial, cov2cor, poisson
#> The following objects are masked from 'package:base':
#> 
#>     %*%, apply, backsolve, beta, chol2inv, colMeans, colSums, diag,
#>     eigen, forwardsolve, gamma, identity, rowMeans, rowSums, sweep,
#>     tapply
library("bayesplot")
#> This is bayesplot version 1.10.0
#> - Online documentation and vignettes at mc-stan.org/bayesplot
#> - bayesplot theme set to bayesplot::theme_default()
#>    * Does _not_ affect other ggplot2 plots
#>    * See ?bayesplot_theme_set for details on theme setting
library("bench")


## generate/specify data
################################################################################

theta <- 5 # poisson theta

set.seed(1)

(y <- rpois(1, theta))
#> [1] 4
y <- as_data(y)
#> ℹ Initialising python and checking dependencies, this may take a moment.
#> ✔ Initialising python and checking dependencies ... done!
#> 



## specify greta model
################################################################################

theta <- gamma(3,1)
distribution(y) <- poisson(theta)

greta_model <- model(theta)

# plot(greta_model)

## configure model settings
################################################################################

n_chains <- 4
n_iter <- 1e4L
n_warmup <- 1e3L


## fit model
################################################################################



greta_fit <- greta::mcmc(
  model = greta_model, 
  n_samples = n_iter,
  warmup = n_warmup,
  chains = n_chains, 
  n_cores = 1
  )
#> running 4 chains simultaneously each on 1 core
#> 
#>     warmup                                           0/1000 | eta:  ?s              warmup ==                                       50/1000 | eta: 14s | <1% bad    warmup ====                                    100/1000 | eta:  8s | <1% bad    warmup =====                                   150/1000 | eta:  6s | 1% bad     warmup =======                                 200/1000 | eta:  5s | 1% bad     warmup =========                               250/1000 | eta:  4s | <1% bad    warmup ===========                             300/1000 | eta:  3s | <1% bad    warmup =============                           350/1000 | eta:  3s | <1% bad    warmup ==============                          400/1000 | eta:  3s | <1% bad    warmup ================                        450/1000 | eta:  2s | <1% bad    warmup ==================                      500/1000 | eta:  2s | <1% bad    warmup ====================                    550/1000 | eta:  2s | <1% bad    warmup ======================                  600/1000 | eta:  1s | <1% bad    warmup =======================                 650/1000 | eta:  1s | <1% bad    warmup =========================               700/1000 | eta:  1s | <1% bad    warmup ===========================             750/1000 | eta:  1s | <1% bad    warmup =============================           800/1000 | eta:  1s | <1% bad    warmup ===============================         850/1000 | eta:  1s | <1% bad    warmup ================================        900/1000 | eta:  0s | <1% bad    warmup ==================================      950/1000 | eta:  0s | <1% bad    warmup ====================================   1000/1000 | eta:  0s | <1% bad
#>   sampling                                          0/10000 | eta:  ?s            sampling                                         50/10000 | eta: 14s            sampling                                        100/10000 | eta: 13s            sampling =                                      150/10000 | eta: 12s            sampling =                                      200/10000 | eta: 12s            sampling =                                      250/10000 | eta: 13s            sampling =                                      300/10000 | eta: 13s            sampling =                                      350/10000 | eta: 13s            sampling =                                      400/10000 | eta: 13s            sampling ==                                     450/10000 | eta: 12s            sampling ==                                     500/10000 | eta: 12s            sampling ==                                     550/10000 | eta: 12s            sampling ==                                     600/10000 | eta: 11s            sampling ==                                     650/10000 | eta: 11s            sampling ===                                    700/10000 | eta: 12s            sampling ===                                    750/10000 | eta: 12s            sampling ===                                    800/10000 | eta: 11s            sampling ===                                    850/10000 | eta: 12s            sampling ===                                    900/10000 | eta: 11s            sampling ===                                    950/10000 | eta: 11s            sampling ====                                  1000/10000 | eta: 11s            sampling ====                                  1050/10000 | eta: 11s            sampling ====                                  1100/10000 | eta: 11s            sampling ====                                  1150/10000 | eta: 11s            sampling ====                                  1200/10000 | eta: 11s            sampling ====                                  1250/10000 | eta: 11s            sampling =====                                 1300/10000 | eta: 11s            sampling =====                                 1350/10000 | eta: 11s            sampling =====                                 1400/10000 | eta: 11s            sampling =====                                 1450/10000 | eta: 11s            sampling =====                                 1500/10000 | eta: 11s            sampling ======                                1550/10000 | eta: 11s            sampling ======                                1600/10000 | eta: 10s            sampling ======                                1650/10000 | eta: 11s | <1% bad  sampling ======                                1700/10000 | eta: 10s | <1% bad  sampling ======                                1750/10000 | eta: 10s | <1% bad  sampling ======                                1800/10000 | eta: 10s | <1% bad  sampling =======                               1850/10000 | eta: 10s | <1% bad  sampling =======                               1900/10000 | eta: 10s | <1% bad  sampling =======                               1950/10000 | eta: 10s | <1% bad  sampling =======                               2000/10000 | eta: 10s | <1% bad  sampling =======                               2050/10000 | eta: 10s | <1% bad  sampling ========                              2100/10000 | eta: 10s | <1% bad  sampling ========                              2150/10000 | eta: 10s | <1% bad  sampling ========                              2200/10000 | eta: 10s | <1% bad  sampling ========                              2250/10000 | eta: 10s | <1% bad  sampling ========                              2300/10000 | eta: 10s | <1% bad  sampling ========                              2350/10000 | eta: 10s | <1% bad  sampling =========                             2400/10000 | eta: 10s | <1% bad  sampling =========                             2450/10000 | eta: 10s | <1% bad  sampling =========                             2500/10000 | eta: 10s | <1% bad  sampling =========                             2550/10000 | eta: 10s | <1% bad  sampling =========                             2600/10000 | eta:  9s | <1% bad  sampling ==========                            2650/10000 | eta:  9s | <1% bad  sampling ==========                            2700/10000 | eta:  9s | <1% bad  sampling ==========                            2750/10000 | eta:  9s | <1% bad  sampling ==========                            2800/10000 | eta:  9s | <1% bad  sampling ==========                            2850/10000 | eta:  9s | <1% bad  sampling ==========                            2900/10000 | eta:  9s | <1% bad  sampling ===========                           2950/10000 | eta:  9s | <1% bad  sampling ===========                           3000/10000 | eta:  9s | <1% bad  sampling ===========                           3050/10000 | eta:  9s | <1% bad  sampling ===========                           3100/10000 | eta:  9s | <1% bad  sampling ===========                           3150/10000 | eta:  9s | <1% bad  sampling ============                          3200/10000 | eta:  9s | <1% bad  sampling ============                          3250/10000 | eta:  9s | <1% bad  sampling ============                          3300/10000 | eta:  9s | <1% bad  sampling ============                          3350/10000 | eta:  9s | <1% bad  sampling ============                          3400/10000 | eta:  8s | <1% bad  sampling ============                          3450/10000 | eta:  8s | <1% bad  sampling =============                         3500/10000 | eta:  8s | <1% bad  sampling =============                         3550/10000 | eta:  8s | <1% bad  sampling =============                         3600/10000 | eta:  8s | <1% bad  sampling =============                         3650/10000 | eta:  8s | <1% bad  sampling =============                         3700/10000 | eta:  8s | <1% bad  sampling ==============                        3750/10000 | eta:  8s | <1% bad  sampling ==============                        3800/10000 | eta:  8s | <1% bad  sampling ==============                        3850/10000 | eta:  8s | <1% bad  sampling ==============                        3900/10000 | eta:  8s | <1% bad  sampling ==============                        3950/10000 | eta:  8s | <1% bad  sampling ==============                        4000/10000 | eta:  8s | <1% bad  sampling ===============                       4050/10000 | eta:  8s | <1% bad  sampling ===============                       4100/10000 | eta:  8s | <1% bad  sampling ===============                       4150/10000 | eta:  8s | <1% bad  sampling ===============                       4200/10000 | eta:  8s | <1% bad  sampling ===============                       4250/10000 | eta:  7s | <1% bad  sampling ===============                       4300/10000 | eta:  7s | <1% bad  sampling ================                      4350/10000 | eta:  7s | <1% bad  sampling ================                      4400/10000 | eta:  7s | <1% bad  sampling ================                      4450/10000 | eta:  7s | <1% bad  sampling ================                      4500/10000 | eta:  7s | <1% bad  sampling ================                      4550/10000 | eta:  7s | <1% bad  sampling =================                     4600/10000 | eta:  7s | <1% bad  sampling =================                     4650/10000 | eta:  7s | <1% bad  sampling =================                     4700/10000 | eta:  7s | <1% bad  sampling =================                     4750/10000 | eta:  7s | <1% bad  sampling =================                     4800/10000 | eta:  7s | <1% bad  sampling =================                     4850/10000 | eta:  7s | <1% bad  sampling ==================                    4900/10000 | eta:  7s | <1% bad  sampling ==================                    4950/10000 | eta:  7s | <1% bad  sampling ==================                    5000/10000 | eta:  7s | <1% bad  sampling ==================                    5050/10000 | eta:  6s | <1% bad  sampling ==================                    5100/10000 | eta:  6s | <1% bad  sampling ===================                   5150/10000 | eta:  6s | <1% bad  sampling ===================                   5200/10000 | eta:  6s | <1% bad  sampling ===================                   5250/10000 | eta:  6s | <1% bad  sampling ===================                   5300/10000 | eta:  6s | <1% bad  sampling ===================                   5350/10000 | eta:  6s | <1% bad  sampling ===================                   5400/10000 | eta:  6s | <1% bad  sampling ====================                  5450/10000 | eta:  6s | <1% bad  sampling ====================                  5500/10000 | eta:  6s | <1% bad  sampling ====================                  5550/10000 | eta:  6s | <1% bad  sampling ====================                  5600/10000 | eta:  6s | <1% bad  sampling ====================                  5650/10000 | eta:  6s | <1% bad  sampling =====================                 5700/10000 | eta:  6s | <1% bad  sampling =====================                 5750/10000 | eta:  6s | <1% bad  sampling =====================                 5800/10000 | eta:  6s | <1% bad  sampling =====================                 5850/10000 | eta:  5s | <1% bad  sampling =====================                 5900/10000 | eta:  5s | <1% bad  sampling =====================                 5950/10000 | eta:  5s | <1% bad  sampling ======================                6000/10000 | eta:  5s | <1% bad  sampling ======================                6050/10000 | eta:  5s | <1% bad  sampling ======================                6100/10000 | eta:  5s | <1% bad  sampling ======================                6150/10000 | eta:  5s | <1% bad  sampling ======================                6200/10000 | eta:  5s | <1% bad  sampling ======================                6250/10000 | eta:  5s | <1% bad  sampling =======================               6300/10000 | eta:  5s | <1% bad  sampling =======================               6350/10000 | eta:  5s | <1% bad  sampling =======================               6400/10000 | eta:  5s | <1% bad  sampling =======================               6450/10000 | eta:  5s | <1% bad  sampling =======================               6500/10000 | eta:  5s | <1% bad  sampling ========================              6550/10000 | eta:  5s | <1% bad  sampling ========================              6600/10000 | eta:  4s | <1% bad  sampling ========================              6650/10000 | eta:  4s | <1% bad  sampling ========================              6700/10000 | eta:  4s | <1% bad  sampling ========================              6750/10000 | eta:  4s | <1% bad  sampling ========================              6800/10000 | eta:  4s | <1% bad  sampling =========================             6850/10000 | eta:  4s | <1% bad  sampling =========================             6900/10000 | eta:  4s | <1% bad  sampling =========================             6950/10000 | eta:  4s | <1% bad  sampling =========================             7000/10000 | eta:  4s | <1% bad  sampling =========================             7050/10000 | eta:  4s | <1% bad  sampling ==========================            7100/10000 | eta:  4s | <1% bad  sampling ==========================            7150/10000 | eta:  4s | <1% bad  sampling ==========================            7200/10000 | eta:  4s | <1% bad  sampling ==========================            7250/10000 | eta:  4s | <1% bad  sampling ==========================            7300/10000 | eta:  4s | <1% bad  sampling ==========================            7350/10000 | eta:  3s | <1% bad  sampling ===========================           7400/10000 | eta:  3s | <1% bad  sampling ===========================           7450/10000 | eta:  3s | <1% bad  sampling ===========================           7500/10000 | eta:  3s | <1% bad  sampling ===========================           7550/10000 | eta:  3s | <1% bad  sampling ===========================           7600/10000 | eta:  3s | <1% bad  sampling ============================          7650/10000 | eta:  3s | <1% bad  sampling ============================          7700/10000 | eta:  3s | <1% bad  sampling ============================          7750/10000 | eta:  3s | <1% bad  sampling ============================          7800/10000 | eta:  3s | <1% bad  sampling ============================          7850/10000 | eta:  3s | <1% bad  sampling ============================          7900/10000 | eta:  3s | <1% bad  sampling =============================         7950/10000 | eta:  3s | <1% bad  sampling =============================         8000/10000 | eta:  3s | <1% bad  sampling =============================         8050/10000 | eta:  3s | <1% bad  sampling =============================         8100/10000 | eta:  2s | <1% bad  sampling =============================         8150/10000 | eta:  2s | <1% bad  sampling ==============================        8200/10000 | eta:  2s | <1% bad  sampling ==============================        8250/10000 | eta:  2s | <1% bad  sampling ==============================        8300/10000 | eta:  2s | <1% bad  sampling ==============================        8350/10000 | eta:  2s | <1% bad  sampling ==============================        8400/10000 | eta:  2s | <1% bad  sampling ==============================        8450/10000 | eta:  2s | <1% bad  sampling ===============================       8500/10000 | eta:  2s | <1% bad  sampling ===============================       8550/10000 | eta:  2s | <1% bad  sampling ===============================       8600/10000 | eta:  2s | <1% bad  sampling ===============================       8650/10000 | eta:  2s | <1% bad  sampling ===============================       8700/10000 | eta:  2s | <1% bad  sampling ================================      8750/10000 | eta:  2s | <1% bad  sampling ================================      8800/10000 | eta:  2s | <1% bad  sampling ================================      8850/10000 | eta:  1s | <1% bad  sampling ================================      8900/10000 | eta:  1s | <1% bad  sampling ================================      8950/10000 | eta:  1s | <1% bad  sampling ================================      9000/10000 | eta:  1s | <1% bad  sampling =================================     9050/10000 | eta:  1s | <1% bad  sampling =================================     9100/10000 | eta:  1s | <1% bad  sampling =================================     9150/10000 | eta:  1s | <1% bad  sampling =================================     9200/10000 | eta:  1s | <1% bad  sampling =================================     9250/10000 | eta:  1s | <1% bad  sampling =================================     9300/10000 | eta:  1s | <1% bad  sampling ==================================    9350/10000 | eta:  1s | <1% bad  sampling ==================================    9400/10000 | eta:  1s | <1% bad  sampling ==================================    9450/10000 | eta:  1s | <1% bad  sampling ==================================    9500/10000 | eta:  1s | <1% bad  sampling ==================================    9550/10000 | eta:  1s | <1% bad  sampling ===================================   9600/10000 | eta:  1s | <1% bad  sampling ===================================   9650/10000 | eta:  0s | <1% bad  sampling ===================================   9700/10000 | eta:  0s | <1% bad  sampling ===================================   9750/10000 | eta:  0s | <1% bad  sampling ===================================   9800/10000 | eta:  0s | <1% bad  sampling ===================================   9850/10000 | eta:  0s | <1% bad  sampling ====================================  9900/10000 | eta:  0s | <1% bad  sampling ====================================  9950/10000 | eta:  0s | <1% bad  sampling ==================================== 10000/10000 | eta:  0s | <1% bad



## assess fit
################################################################################

summary(greta_fit)
#> 
#> Iterations = 1:10000
#> Thinning interval = 1 
#> Number of chains = 4 
#> Sample size per chain = 10000 
#> 
#> 1. Empirical mean and standard deviation for each variable,
#>    plus standard error of the mean:
#> 
#>           Mean             SD       Naive SE Time-series SE 
#>       3.511833       1.316709       0.006584       0.013042 
#> 
#> 2. Quantiles for each variable:
#> 
#>  2.5%   25%   50%   75% 97.5% 
#> 1.419 2.568 3.362 4.282 6.450

^{Created on 2023-02-06 with reprex v2.0.2}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.2 (2022-10-31)
#>  os       macOS Ventura 13.2
#>  system   aarch64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       Australia/Hobart
#>  date     2023-02-06
#>  pandoc   2.19.2 @ /Applications/RStudio.app/Contents/Resources/app/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package       * version    date (UTC) lib source
#>  abind           1.4-5      2016-07-21 [1] CRAN (R 4.2.0)
#>  assertthat      0.2.1      2019-03-21 [1] CRAN (R 4.2.0)
#>  backports       1.4.1      2021-12-13 [1] CRAN (R 4.2.0)
#>  base64enc       0.1-3      2015-07-28 [1] CRAN (R 4.2.0)
#>  bayesplot     * 1.10.0     2022-11-16 [1] CRAN (R 4.2.0)
#>  bench         * 1.1.2      2021-11-30 [1] CRAN (R 4.2.0)
#>  broom           1.0.3      2023-01-25 [1] CRAN (R 4.2.0)
#>  callr           3.7.3      2022-11-02 [1] CRAN (R 4.2.0)
#>  cellranger      1.1.0      2016-07-27 [1] CRAN (R 4.2.0)
#>  cli             3.6.0      2023-01-09 [1] CRAN (R 4.2.0)
#>  coda            0.19-4     2020-09-30 [1] CRAN (R 4.2.0)
#>  codetools       0.2-19     2023-02-01 [1] CRAN (R 4.2.0)
#>  colorspace      2.1-0      2023-01-23 [1] CRAN (R 4.2.0)
#>  crayon          1.5.2      2022-09-29 [1] CRAN (R 4.2.0)
#>  DBI             1.1.3      2022-06-18 [1] CRAN (R 4.2.0)
#>  dbplyr          2.3.0      2023-01-16 [1] CRAN (R 4.2.0)
#>  digest          0.6.31     2022-12-11 [1] CRAN (R 4.2.0)
#>  dplyr         * 1.1.0      2023-01-29 [1] CRAN (R 4.2.1)
#>  ellipsis        0.3.2      2021-04-29 [1] CRAN (R 4.2.0)
#>  evaluate        0.20       2023-01-17 [1] CRAN (R 4.2.0)
#>  fansi           1.0.4      2023-01-22 [1] CRAN (R 4.2.0)
#>  fastmap         1.1.0      2021-01-25 [1] CRAN (R 4.2.0)
#>  forcats       * 1.0.0      2023-01-29 [1] CRAN (R 4.2.0)
#>  fs              1.6.0      2023-01-23 [1] CRAN (R 4.2.0)
#>  future          1.31.0     2023-02-01 [1] CRAN (R 4.2.0)
#>  gargle          1.3.0      2023-01-30 [1] CRAN (R 4.2.0)
#>  generics        0.1.3      2022-07-05 [1] CRAN (R 4.2.0)
#>  ggplot2       * 3.4.0      2022-11-04 [1] CRAN (R 4.2.0)
#>  globals         0.16.2     2022-11-21 [1] CRAN (R 4.2.1)
#>  glue            1.6.2      2022-02-24 [1] CRAN (R 4.2.0)
#>  googledrive     2.0.0      2021-07-08 [1] CRAN (R 4.2.0)
#>  googlesheets4   1.0.1      2022-08-13 [1] CRAN (R 4.2.0)
#>  greta         * 0.4.3.9000 2023-02-06 [1] local
#>  gtable          0.3.1      2022-09-01 [1] CRAN (R 4.2.0)
#>  haven           2.5.1      2022-08-22 [1] CRAN (R 4.2.0)
#>  here            1.0.1      2020-12-13 [1] CRAN (R 4.2.0)
#>  hms             1.1.2      2022-08-19 [1] CRAN (R 4.2.0)
#>  htmltools       0.5.4      2022-12-07 [1] CRAN (R 4.2.0)
#>  httr            1.4.4      2022-08-17 [1] CRAN (R 4.2.0)
#>  jsonlite        1.8.4      2022-12-06 [1] CRAN (R 4.2.0)
#>  knitr           1.42       2023-01-25 [1] CRAN (R 4.2.0)
#>  lattice         0.20-45    2021-09-22 [1] CRAN (R 4.2.2)
#>  lifecycle       1.0.3      2022-10-07 [1] CRAN (R 4.2.0)
#>  listenv         0.9.0      2022-12-16 [1] CRAN (R 4.2.0)
#>  lubridate       1.9.1      2023-01-24 [1] CRAN (R 4.2.0)
#>  magrittr        2.0.3      2022-03-30 [1] CRAN (R 4.2.0)
#>  Matrix          1.5-3      2022-11-11 [1] CRAN (R 4.2.0)
#>  modelr          0.1.10     2022-11-11 [1] CRAN (R 4.2.0)
#>  munsell         0.5.0      2018-06-12 [1] CRAN (R 4.2.0)
#>  parallelly      1.34.0     2023-01-13 [1] CRAN (R 4.2.0)
#>  pillar          1.8.1      2022-08-19 [1] CRAN (R 4.2.0)
#>  pkgconfig       2.0.3      2019-09-22 [1] CRAN (R 4.2.0)
#>  png             0.1-8      2022-11-29 [1] CRAN (R 4.2.0)
#>  prettyunits     1.1.1      2020-01-24 [1] CRAN (R 4.2.0)
#>  processx        3.8.0      2022-10-26 [1] CRAN (R 4.2.0)
#>  progress        1.2.2      2019-05-16 [1] CRAN (R 4.2.0)
#>  ps              1.7.2      2022-10-26 [1] CRAN (R 4.2.0)
#>  purrr         * 1.0.1      2023-01-10 [1] CRAN (R 4.2.0)
#>  R.cache         0.16.0     2022-07-21 [1] CRAN (R 4.2.0)
#>  R.methodsS3     1.8.2      2022-06-13 [1] CRAN (R 4.2.0)
#>  R.oo            1.25.0     2022-06-12 [1] CRAN (R 4.2.0)
#>  R.utils         2.12.2     2022-11-11 [1] CRAN (R 4.2.0)
#>  R6              2.5.1      2021-08-19 [1] CRAN (R 4.2.0)
#>  Rcpp            1.0.10     2023-01-22 [1] CRAN (R 4.2.0)
#>  readr         * 2.1.3      2022-10-01 [1] CRAN (R 4.2.0)
#>  readxl          1.4.1      2022-08-17 [1] CRAN (R 4.2.0)
#>  reprex          2.0.2      2022-08-17 [1] CRAN (R 4.2.0)
#>  reticulate      1.28       2023-01-27 [1] CRAN (R 4.2.0)
#>  rlang           1.0.6      2022-09-24 [1] CRAN (R 4.2.0)
#>  rmarkdown       2.20       2023-01-19 [1] CRAN (R 4.2.0)
#>  rprojroot       2.0.3      2022-04-02 [1] CRAN (R 4.2.0)
#>  rstudioapi      0.14       2022-08-22 [1] CRAN (R 4.2.0)
#>  rvest           1.0.3      2022-08-19 [1] CRAN (R 4.2.0)
#>  scales          1.2.1      2022-08-20 [1] CRAN (R 4.2.0)
#>  sessioninfo     1.2.2      2021-12-06 [1] CRAN (R 4.2.0)
#>  stringi         1.7.12     2023-01-11 [1] CRAN (R 4.2.0)
#>  stringr       * 1.5.0      2022-12-02 [1] CRAN (R 4.2.0)
#>  styler          1.9.0      2023-01-15 [1] CRAN (R 4.2.0)
#>  tensorflow      2.11.0     2022-12-19 [1] CRAN (R 4.2.0)
#>  tfautograph     0.3.2      2021-09-17 [1] CRAN (R 4.2.0)
#>  tfruns          1.5.1      2022-09-05 [1] CRAN (R 4.2.0)
#>  tibble        * 3.1.8      2022-07-22 [1] CRAN (R 4.2.0)
#>  tidyr         * 1.3.0      2023-01-24 [1] CRAN (R 4.2.0)
#>  tidyselect      1.2.0      2022-10-10 [1] CRAN (R 4.2.0)
#>  tidyverse     * 1.3.2      2022-07-18 [1] CRAN (R 4.2.0)
#>  timechange      0.2.0      2023-01-11 [1] CRAN (R 4.2.0)
#>  tzdb            0.3.0      2022-03-28 [1] CRAN (R 4.2.0)
#>  utf8            1.2.3      2023-01-31 [1] CRAN (R 4.2.0)
#>  vctrs           0.5.2      2023-01-23 [1] CRAN (R 4.2.0)
#>  whisker         0.4.1      2022-12-05 [1] CRAN (R 4.2.0)
#>  withr           2.5.0      2022-03-03 [1] CRAN (R 4.2.0)
#>  xfun            0.37       2023-01-31 [1] CRAN (R 4.2.0)
#>  xml2            1.3.3      2021-11-30 [1] CRAN (R 4.2.0)
#>  yaml            2.3.7      2023-01-23 [1] CRAN (R 4.2.0)
#> 
#>  [1] /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library
#> 
#> ─ Python configuration ───────────────────────────────────────────────────────
#>  python:         /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/bin/python
#>  libpython:      /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/libpython3.8.dylib
#>  pythonhome:     /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2:/Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2
#>  version:        3.8.15 | packaged by conda-forge | (default, Nov 22 2022, 08:49:06)  [Clang 14.0.6 ]
#>  numpy:          /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/python3.8/site-packages/numpy
#>  numpy_version:  1.23.2
#>  tensorflow:     /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/python3.8/site-packages/tensorflow
#>  
#>  NOTE: Python version was forced by use_python function
#> 
#> ──────────────────────────────────────────────────────────────────────────────

```

Let me know if you’ve got any questions!

gosselinf · February 6, 2023, 10:00am

Thank you very much, Nick. When I run the new code, the R session on windows uses a little more than one core (11.5% of CPU wheras one core would be 8.5% - since I have 12 cores). It appears that for more complicated models, even with the argument n_cores=1, I get more.

My question is therefore what this argument should do: if I understand you (and the printing on R) correctly, it will run 4 chains on 1 core each. Therfeore, it would be OK that a mcmc with 4 chains uses less than 33% of CPU. Is there a solution for running them simultanoausly on one core? Does it make sense? Should I actually launch 4 separate MCMCs (possibly sequentially), each on one core?

Sincerely,

Frédéric

njtierney · February 7, 2023, 6:02am

Hi Frédéric,

My understanding is that the default option in greta is to use all cores, when n_cores is not specified.

What you describe about CPU usage makes sense to me, in that it is likely R is running on another core, and then the greta code is utilising another core.

Is there a solution for running them simultanoausly on one core? Does it make sense? Should I actually launch 4 separate MCMCs (possibly sequentially), each on one core?

If you want to run many chains on 1 core, then you can set the arguments to reflect that:

mcmc(model, n_cores = 1, chains = 4)

If you want to have 4 chains, one on each core, we have written greta to work with the future package, so you can write:

library(future)
plan(multisession, workers = 4)
mcmc(model, n_cores = 1, chains = 4)

(However, note that this only works on the current CRAN version of greta, if you are using the latest version from an experimental TF2 branch of greta (which I don’t think you are?) then it will not work.)

Does this help you do what you are trying to achieve in this instance? Or are you in a situation where you only have one core that you can run on a remote machine, or are you looking to maximise the number of cores/the efficiency of the operation?

gosselinf · February 8, 2023, 5:49am

Dear Nick,

Thank you for your answer. I am just wishing to control the number of cores being used in order to launch the code in a remote computing server, in a sequential way. At present, the code I launch is mismanaged by the server since it requests more than 1 core.
I still do not manage to have greta use only one core - maybe as you say because 1 core is working for R and 1 fore greta. Actually the metric indicates that between 1 and 2 cores are used. I also get the same result if I only launch 1 chain (which might have been a solution to my problem, by launching the MCMC chain by chain or restricting the fit to a one chain MCMC).
Sincerely,
Frédéric

njtierney · February 8, 2023, 6:18am

Hello Frédéric,

OK it sounds like that’s solved the issue for the time being, to just run 1 core and 1 chain?

Let me know if I can help any further,

Cheers,

Nick

gosselinf · February 8, 2023, 10:41am

Sorry, Nick, I was not clear. The issue is not solved for me as running 1 chain of greta in R uses more than 1 core. If I run the same model with e.g. Nimble or Jags in R (even actually with 4 chains), I do not use more than 1 core. This is the behavior I would like to be able to reproduce with greta as well.
Frédéric

njtierney · February 9, 2023, 4:19am

Hi Frédéric - OK I see, I’ll see what I can do to take a look at this and get back to you.

gosselinf · February 9, 2023, 4:39am

Thank you very much, Nick. BTW I use version 0.4.3.9000 of greta, in Windows. To be more precise:

when I use ‘plan(multisession, workers = 4)’, the behavior is OK: four chains are run in seperate R processes that each use at most 1 core (but in total we use something like 4 cores)
when I use ‘plan(multisession, workers = 1)’, the chains are run sequentially by one R process that uses more than 1 core (between 1 & 2). This behavior is strange to me and I would prefer it uses strictly only one core (which is what I expect with ‘plan(multisession, workers = 1)’).
Sincerely,
Frédéric

njtierney · February 9, 2023, 5:06am

Thank you, Frédéric - I’ll have a look and a think about this. My first thought is that this could be some interface/issue with how TF sets itself up, or potentially an interaction with how future sets up its parallel workers. In those cases, we are inherently limited by what they can do. However what you said about JAGS/Nimble makes me think that perhaps there is something going on with greta itself.

gosselinf · February 9, 2023, 8:00am

Thank you Nick. Note that for JAGS/Nimble I did not try to parallelize (and did not use the future package).