Detect whether tracee is in a signal handler when using ptrace

Question

I test that on Linux and it seems that when the tracee is in a signal handler, the tracer can use ptrace() to attach to it, as usual. But since tracee is in a signal handler, some functions might not be OK to invoke because of the asyn-signal-safe problem. So, is there any methods to detect that situation after calling ptrace()?

Answer 1

It turns out that determining whether or not you are currently in a signal handler is trivial with libunwind:

Assume that you have build libunwind properly:

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <signal.h>
#include <errno.h>
#include <string.h>

/* assume that you have build libunwind properly */
#include "./libunwindout/include/libunwind.h"

/* Simple error handling functions */
#define handle_error_en(en, msg) \
        do { errno = en; perror(msg); exit(EXIT_FAILURE); } while (0)

void show_backtrace(void) {
    unw_cursor_t cursor; unw_context_t uc;
    unw_word_t ip, sp;

    unw_getcontext(&uc);
    unw_init_local(&cursor, &uc);
    while(unw_step(&cursor) > 0) {
        unw_get_reg(&cursor, UNW_REG_IP, &ip);
        unw_get_reg(&cursor, UNW_REG_SP, &sp);

        /* Upon unwinding to a signal handler, you get a "1" */
        printf("Is in a signal handler [%d]\n",
                          unw_is_signal_frame(&cursor));

        printf("ip = %lx, sp = %lx \n", (long)ip, (long)sp);
    }
}  

struct sigaction act;

/* Upon receiving a SIGQUIT, this signal handler will be invoked */
void sighandler(int signum, siginfo_t *info, void *ptr) {
    printf("Received signal: %d\n", signum);
    printf("signal originate from pid[%d]\n", info->si_pid);
    printf("Inside a signal handler...\n");
    show_backtrace();
    while(1)
        ;
    printf("[FATAL] quiting the signal handler\n");
}

int main(int argc, char *argv[]) {
{
    printf("Pid of the current process: %d\n", getpid());

    memset(&act, 0, sizeof(act));

    act.sa_sigaction = sighandler;
    act.sa_flags = SA_SIGINFO;

    sigaction(SIGQUIT, &act, NULL);

    while(1)
       ;

    return 0;
}

So you should run this program, and then send it a SIGQUIT. Upon receiving a SIGQUIT, the signal handler will be invoked and the show_backtrace() function will be called, which will unwind the stack and eventually find the signal handler frame, reporting 1.

More interesting, libunwind allows you to detect "remotely" with its libunwind-ptrace module. By "remotely", it simply means that you can use ptrace(2) to attach to a process and then you can use libunwind-ptrace to detect the remote process is running in a signal handler.

For more info, please refer to libunwind's doc

Answer 2

This recent discussion may interest you.

The short answer is that you can tell whether inferior (tracee) is in a signal handler by unwinding its stack, and looking for rt_sigreturn entry.

That is the entry that GDB prints as <signal handler called>.

However, the question is: why do you care?

Presumably it is to prevent your debugger from calling into the tracee when your end user asks you to perform equivalent of (gdb) call malloc(10).

Note that:

1. GDB does not prevent end-user from doing so. If the process corrupts its heap or deadlocks as the result, it's the end user's problem, not GDB's.

2. It is impossible for the debugger to know what functions should be allowed or disallowed, and this determination depends on whether the signal is synchronous and where it originates. For example:

   void handler(int signo)
   {
      while (1)
      {
        char *p = malloc(20);  // perfectly safe (but only in this program)
        free(p);
      }
   }
   
   int main()
   {
     signal(SIGINT, handler);
     kill(getpid(), SIGINT);
     return 0; // control never reaches here
   }