How to compile ruby in debug mode?

Question

There are plenty of ruby debug log calls in the source code that help me better understand internals.

How do you debug ruby vm? Do you have a documentation for that?

I haven't found any information in the GitHub documentation. What I tried but didn't get any result are:

1. Compile ruby in kind of debug mode:

./autogen.sh
mkdir build && cd build # it's good practice to build outside of source dir
mkdir ~/.rubies # we will install to .rubies/ruby-master in our home dir
../configure cppflags='-DUSE_RUBY_DEBUG_LOG=1 -DRUBY_DEBUG' --prefix="${HOME}/.rubies/ruby-master"
make install

Tried to run ruby with -d flag

export RUBY_DEBUG_LOG='/home/root/ruby.log'
export USE_RUBY_DEBUG_LOG=1
export DEBUG=1

bin/ruby -d ~/test.rb

Answer 1

To add to @richard-michael's excellent answer and confirm some testing on older Ruby version 3.1.2...

For older Ruby versions prior to the mentioned patch, you need to compile with: -DUSE_RUBY_DEBUG_LOG=1 -DRUBY_DEBUG=1 -DRUBY_DEVEL=1.

When running the Autoconf-generated ./configure script, set cppflags with those C pre-processor definitions:

env cppflags='-DUSE_RUBY_DEBUG_LOG=1 -DRUBY_DEBUG=1 -DRUBY_DEVEL=1' ./configure  --enable-debug-env  [...]
make

Then, for options to pass to RUBY_DEBUG environment variable take a look at the code in debug.c:

int ruby_env_debug_option(const char *str, int len, void *arg)
{
    // ...SNIP...
    SET_WHEN("gc_stress", *ruby_initial_gc_stress_ptr, Qtrue);
    SET_WHEN("core", ruby_enable_coredump, 1);
    SET_WHEN("ci", ruby_on_ci, 1);
    if (NAME_MATCH_VALUE("rgengc")) {
  if (!len) ruby_rgengc_debug = 1;
  else SET_UINT_LIST("rgengc", &ruby_rgengc_debug, 1);
  return 1;
    }
#if defined _WIN32
# if RUBY_MSVCRT_VERSION >= 80
    SET_WHEN("rtc_error", ruby_w32_rtc_error, 1);
# endif
#endif
#if defined _WIN32 || defined __CYGWIN__
    if (NAME_MATCH_VALUE("codepage")) {
  if (!len) fprintf(stderr, "missing codepage argument");
  else SET_UINT_LIST("codepage", ruby_w32_codepage, numberof(ruby_w32_codepage));
  return 1;
    }
#endif
    return 0;
}

So, the valid values are: gc_stress, core, ci, rgengc, rtc_error (Windows & RUBY_MSVCRT_VERSION >= 80 Only), and codepage (Windows & Cygwin Only).

Multiple values can be combined also. For example:

RUBY_DEBUG_LOG=stderr; export  RUBY_DEBUG='ci core rgengc' ./ruby -e 'puts "do something that runs functions with debug output enabled"' 2>&1

On those versions of ruby, this flag doesn't seem to output much unless you run functions which have debugging output. For Ruby 3.1.2 this includes functions such as: vm_lock_enter, vm_lock_leave, heap_add_freepage, heap_next_freepage, gc_sweep_step, gc_exit, rb_transient_heap_start_marking, transient_heap_update_status, rb_transient_heap_finish_marking, rb_ractor_blocking_threads_inc, ractor_check_blocking, ractor_status_set, vm_ractor_blocking_cnt_inc, rb_ractor_blocking_threads_dec, rb_vm_ractor_blocking_cnt_dec, ractor_status_set, etc...

Answer 2

I built Ruby today with debugging, and here's what I did. I experimented with several configure options and flags; this was the minimum I needed for VM logging.

I have a guess about why your build didn't work, which I'll describe at the end.

1. Clone ruby, it's at: git rev-parse master --short # => 191e4ae33f
2. Follow the instructions in doc/contributing/building_ruby.md.
3. Configure as:
   • Enable the debug CPP macro
   • An install prefix
   • Disable documentation (particular to this build, I didn't need docs)
   • Configure OpenSSL (particular to this build, on macOS)
4. Build, I didn't bother to make install

Just to be clear, there were no Ruby or debug-related environment variables influencing the build.

$ env | egrep -i 'RUBY|DEBUG'
$ git clone https://github.com/ruby/ruby.git
$ cd ruby
$ ./autogen.sh
$ mkdir build && cd build
$ ../configure cppflags="-DUSE_RUBY_DEBUG_LOG=1" --prefix=${HOME}/.rubies/ruby-master --disable-install-doc --with-openssl-dir=$( 
$ make -j8

There is no need to pass -d or all those environment variables for Ruby to emit the VM debug information, RUBY_DEBUG_LOG=[dest] is sufficient.

Below is Ruby emitting debug output, and I also verified filtering works.

1. Verify there is no output unless requested, this is expected:

$ ./ruby -e ''    # No output, ok

2. Ask for debug output on stderr:

$ RUBY_DEBUG_LOG=stderr ./ruby -e '' 2>&1 | head -5
RUBY_DEBUG_LOG=stderr [stderr]
   0: rb_ractor_set_current_ec_ ec:0x0000000000000000->0x000000013a704740       vm.c:4046               th:0
   1: heap_add_freepage page:0x000000013b80b000 freelist:0x000000010491ffc8     gc.c:2026               th:0
   2: heap_add_freepage page:0x000000013b80b600 freelist:0x000000010493ffd0     gc.c:2026               th:0
   3: heap_add_freepage page:0x000000013b80bc00 freelist:0x000000010495ffd8     gc.c:2026               th:0

3. Ask for specific debug output on stderr:

$ RUBY_DEBUG_LOG=stderr RUBY_DEBUG_LOG_FILTER='vm_lock_enter' ./ruby -e '' 2>&1 | head -5
RUBY_DEBUG_LOG=stderr [stderr]
RUBY_DEBUG_LOG_FILTER[0]=vm_lock_enter (all)
   0: vm_lock_enter     start locked:0  gc.c:2799               th:0
   1: vm_lock_enter     rec:1 owner:1   gc.c:2799               th:0
   2: vm_lock_enter     start locked:0  gc.c:2799               th:0

You don't mention what revision (or commit) of Ruby you were building. Since the date of your post, there is this patch on master:

commit 0d415a322f5dd7158efcbc6c3226266e312620c7
Author: Koichi Sasada <ko1@atdot.net>
Date:   Wed Mar 1 17:07:50 2023 +0900

    Enable DEBUG_LOG feature on USE_RUBY_DEBUG_LOG
    
    `ruby_set_debug_option` is needed for `RUBY_DEBUG_LOG` feature
    so it should be called when `USE_RUBY_DEBUG_LOG` is true.

diff --git a/main.c b/main.c
index 0d0ec147cd..072dc56dd5 100644
--- a/main.c
+++ b/main.c
@@ -23,6 +23,7 @@
 #ifdef HAVE_LOCALE_H
 #include <locale.h>
 #endif
+
 #if defined RUBY_DEVEL && !defined RUBY_DEBUG_ENV
 # define RUBY_DEBUG_ENV 1
 #endif
@@ -46,7 +47,7 @@ int rb_wasm_rt_start(int (main)(int argc, char **argv), int argc, char **argv);
 int
 main(int argc, char **argv)
 {
-#ifdef RUBY_DEBUG_ENV
+#if defined(RUBY_DEBUG_ENV) || USE_RUBY_DEBUG_LOG
     ruby_set_debug_option(getenv("RUBY_DEBUG"));
 #endif
 #ifdef HAVE_LOCALE_H

Reading the diff and the context lines, and looking at your ./configure, I think (back in October) you would have needed to pass --enable-devel or cppflags="-DRUBY_DEBUG_ENV", and not "-DRUBY_DEBUG". I didn't checkout a commit from that time and to test. :-)

After this patch, if only USE_RUBY_DEBUG_LOG is set, then RUBY_DEBUG will be enabled in the VM, which is presumably happening in ruby_set_debug_option().

Aside, you may also be interested in the --enable-devel option, I did subsequently build with it, but I didn't want to confuse this answer, nor have I investigated details yet.