Function to find column names dynamically

Question

In the function find_value_in_table() provided below, I am trying to find the name of any columns which have a record where the column value matches the String "255.255.255.255".

In short, something like this:

SELECT column_name
  FROM dynamic_table
  WHERE column_value = '255.255.255.255';

Note:

• the name of the table is passed in as a parameter to the function, hence the "dynamic_table"
• I am trying to determine the name of the column, not the data value.

This is a first step. Later I will parameterize the column_value as well. I know there is a table storing the value "255.255.255.255" somewhere, and want to prove functionality of this function by finding the table and column name storing this value.

The aim of all this: I have big database, that I am retro-engineering. I know it contains, somewhere, some configuration parameters of an application. In the current configuration I know the precise value of some of these parameters (seen in the application config GUI: e.g. a computer name, ip addresses). I want to browse the entire database in order to determine which tables store this configuration information.

I have been building the function find_value() to return these clues.

How can this be done?

create or replace function find_all_columns(tablename in text) 
    return setof record as
$func$
    declare r record;
    begin 
        return select   a.attname as "Column",
            pg_catalog.format_type(a.atttypid, a.atttypmod) as "Datatype"
            from
            pg_catalog.pg_attribute a
            where
            a.attnum > 0
            and not a.attisdropped
            and a.attrelid = (  select c.oid from pg_catalog.pg_class c left join pg_catalog.pg_namespace n on n.oid = c.relnamespace where c.relname ~ '^('  ||   quote_ident(tablename)    ||   ')$' and pg_catalog.pg_table_is_visible(c.oid);
        end loop;
    end;
$func$ language 'plpgsql';  

create or replace function find_value_in_table(tablename text) 
    returns setof record as 
$func$
    declare r record;
        return select 
    begin 

        for r in (select find_all_columns(tablename)) loop
            return select * from tablename t where t... = "255.255.255.255" /*  here column would be the value in the record: r.Column*/ 
        end loop;
    end;
$func$ language 'plpgsql';

create or replace function find_tables_name(_username text)
  returns setof record as
$func$
declare
   tbl text;
begin
   for tbl in 
      select t.tablename from pg_tables t
      where  t.tableowner = _username and t.schemaname = 'public'
   loop
      return quote_ident(tbl);
   end loop;
end;
$func$ language 'plpgsql';

create or replace function find_value(_username text, valuetofind text) 
    returns setof record as 
$func$
    declare r record;
    begin 
        for r in (select find_tables_name(_username)) loop
            return find_value_in_table( r.tablename );
        end loop;
    end;
$func$ language 'plpgsql';

Answer 1

One primitive way to achieve this would be to make a plain-text dump and use an editor of your choice (vim in my case) to search for the string.

But this function does a better job. :)

CREATE OR REPLACE FUNCTION find_columns(_owner text
                                       ,_valuetofind text
                                       ,_part bool = FALSE)
  RETURNS TABLE (tbl text, col text, typ text) LANGUAGE plpgsql STRICT AS
$func$
DECLARE
   _go         bool;
   _search_row text := '%' || _search || '%';  -- Search row for part of string
BEGIN
   IF _part THEN   -- search col for part of string?
      valuetofind := '%' || valuetofind || '%';
   END IF;

FOR tbl IN 
   SELECT quote_ident(t.schemaname) || '.' || quote_ident(t.tablename)
   FROM pg_tables t
   WHERE  t.tableowner = _owner
-- AND t.schemaname = 'public' -- uncomment to only search one schema
LOOP
   EXECUTE '
   SELECT EXISTS (
      SELECT 1 FROM ' || tbl || ' t WHERE t::text ~~ $1)' -- check whole row
   INTO  _go
   USING _search_row;

   IF _go THEN
      FOR col, typ IN 
         SELECT quote_ident(a.attname) -- AS col
               ,pg_catalog.format_type(a.atttypid, a.atttypmod) -- AS typ
         FROM   pg_catalog.pg_attribute a
         WHERE  a.attnum > 0
         AND    NOT a.attisdropped
         AND    a.attrelid = tbl::regclass
      LOOP
         EXECUTE '
         SELECT EXISTS (
            SELECT 1
            FROM ' || tbl || ' WHERE ' || col || '::text ~~ $1)' -- check col
         INTO   _go
         USING  valuetofind;

         IF _go THEN
            RETURN NEXT;
         END IF;
      END LOOP;
   END IF;

END LOOP;
END;
$func$;
COMMENT ON FUNCTION x.find_columns(text, text, boolean) IS 'Search all tables
 owned by "_owner" user for a value "_search" (text representation).
 Match full or partial (_part)';

Call:

SELECT * FROM find_columns('postgres', '255.255.255.255');
SELECT * FROM find_columns('fadmin', '255.255.255.255', TRUE);

Returns:

       tbl       |     col     | typ
-----------------+-------------+------
 event.eventkat  | eventkat    | text
 public.foo      | description | text
 public.bar      | filter      | text

Tested with PostgreSQL 9.1

Major points

• The function is a one-stop-shop.

• I built an option to search for part of the value (_part). The default is to search for whole columns.

• I built in a quick test on the whole row to eliminate tables, that don't have the valuetofind in them at all. I use PostgreSQL's ability to convert whole rows to text quickly for this. This should make the function a lot faster - except when all or almost all tables qualify or when tables only have one columns.

• I define the return type as RETURNS TABLE (tbl text, col text, typ text) and assign the implicitly defined variables tbl, col and typ right away. So I don't need additional variables and can RETURN NEXT right away when a column qualifies.

• Make heavy use of EXISTS here! That's the fastest option, as you are only interested whether the column has the value at all.

• Use LIKE (or ~~ for short) instead of regular expressions. Simpler, faster.

• I quote_ident() all identifiers right away.

• EXECUTE *command* INTO USING is instrumental.