Spark alternative for Redshift REGEXP_SUBSTR function

Question

Trying to convert part of Redshift query into SparkSQL or some combo of SQL and UDF:

REGEXP_SUBSTR(referrer, '[^/]+\\.[^/:]+') as referrer_domain,

Tried using regexp_extract(referrer, '[^/]+\\.[^/:]+', 1), but that doesn’t seem to work the same way and returns results inconsistently.

Any pointers appreciated!

Answer 1

You should be able to use regexp_extract in spark sql, something like this -

regexp_extract(columnName, '(YourRegex)', 1) as aliasName

Note the () around regex to capture the group. Hope it helps!