CODEWITHSUNDEEP
scalareplaceall
Pranjal
Pranjal

Reputation: 73

Replacing special characters in scala

I am working on a scala and want to replace special characters from my dataframe replaceAll doesn't seem to work, is there any other way ?

My code is this:

val specialchar = dataframe.select(column).replaceAll("[^A-za-z]+","")

Upvotes: 0

Views: 1476

Answers (1)

Ehsan Ullah Nazir
Ehsan Ullah Nazir

Reputation: 1917

You can provide the allowed characters in regex .

Try following 

 val badDF = Seq(("7369", "SMI_)(TH" , "2010-12-17", "800.00"), ("7499", "AL@;__#$LEN","2011-02-20", "1600.00")).toDF("empno", "ename","hire_date", "sal")
 val cleanedDF = badDF.select(badDF.columns.map(c => regexp_replace(badDF(c), """[^A-Z a-z 0-9]""", "").alias(c)): _*)
 cleanedDF.show

ename contains special characters. above regex will only allow Capital/Small a-z characters and 0-9 digits. All other characters will be removed.

enter image description here

Upvotes: 1

Related Questions