False positive mod_security Cross Site Scripting Attack due to content of submitted form

Question

This is somewhat similar to 403 Forbidden Error, probably ModSecurity

I have a form on a webpage, this is for submitting maths equations and content. When used legitimatly the fields will contain the unsafe characters ", <, >, &, ' and ` I was escaping these with

 ", <, >, &, ' and `

I was encoding/escaping the unsafe chars with the above HTML entities before submitting the form from the web page to the PHP script. So even though the fields legitimally contained the unsafe characters the submitted form did not. In my PHP script I first checked the $_POST['formFieldData']; and if it contained any of the unsafe characters then I immediatly and silently aborted processing for that entire post. My thinking was that a legitimate use of the form on my webpage would not contain any of unsafe characters. So if anyone set up their own page with a form aimed at my PHP script they would not be able to send it unsafe characters. I only ever write the data to the browser screen with the unsafe characters HTML entity encoded.

Originally (a few years ago) when I set the form up the raw data (legitimatly containing unsafe characters) was posted to the PHP script and what was echoed back to the browser after encoding the unsafe characters, then the data was emailed to me for further processing and storage. I was recieving a substantial number of scamming/phishing/junk emails that originated from a robot or some other form. To cut down on the volume of these and still allow legitimate ones through I implemented the encoding on my webpage and rejected incomming posts containing unsafe characters. Over night the volume of these went to zero. I must add that I also considered the . : ; / \ @ unsafe and encoded/escaped them and dropped the post if it contained them in unencoded form.

Recently even with the above cleaning/escaping I started recieving false positive mod-security trips for XSS filtering that blocked the submissions from reaching the php file for processing. The message was "Potential Cross Site Scripting Attack'"

Unfortunatly the hosting company do not have these trips logged in the users log file, all the told me was that there are trips and they sent me the rule

I have pasted the rule at the end of this post as there is a huge amount of text in the rule.

The hosting company had two suggestions: White list the rule so that it does not apply to my site. Turning mod-security off. I would rather not do one of these if possible.

Towards the end of the rule is the code t:htmlEntityDecode, This gave me the idea that the rule is unescaping the content therefore defeating my html escaping.

I set about jumbling up the content that I had escaped using the Javascript replace .replace(/&#x/g, "x#&"); for example the " becomes x#&22; on the server in the PHP form processing file then I undid this jumbling up with $form_Data = str_replace("x#&", "&#x", $form_Data);

This worked perfectly, that is the mod-security filter did not trip for the content that previously caused false positives.

This is a hack but I don't think I have any choice but to do some kind of hack.

The rule as detailed below looks like one giant regular expression plus some directives to unesape HTMLentities, compressWhitespace etc.

I would be most obliged if someone that knows about these filters could offer some suggestions on any methods that I can use to preprocess the form content before posting it from the webpage to make it more resilient to false positives.

Thanks in advance.

This rule is:

Rule 340147: Generic XSS filter
SecRule ARGS|ARGS_NAMES|!ARGS:js_includes|!ARGS:/m1_source/|!ARGS:/geodir/|!ARGS:/banner_block/
|!ARGS:/introcopy/|!ARGS:eingabe|!ARGS:ausgabe|!ARGS:/previewdata/|!ARGS:/tracking_extra/
|!ARGS:/^groups/|!ARGS:video|!ARGS:/google_map/|!ARGS:/field_map/|!ARGS:/gacode/|!ARGS:code1
|!ARGS:ga_code|!ARGS:customized|!ARGS:code_analytics|!ARGS:uvod|!ARGS:/^field_video/|!ARGS:q
|!ARGS:/^textarea-video/|!ARGS:leirro|!ARGS:lomake|!ARGS:vastaus|!ARGS:/^texte$/|!ARGS:vraag
|!ARGS:qti_data|!ARGS:tracklist|!ARGS:i_google|!ARGS:code_area_text|!ARGS:/log_code/
|!ARGS:/^ADVERT_/|!ARGS:UserData|!ARGS:areas|!ARGS:templatecode|!ARGS:/prevObject/
|!ARGS:/replaceAll/|!ARGS:/insertBefore/|!ARGS:/insertAfter/|!ARGS:/prependTo/|!ARGS:/appendTo/
|!ARGS:/mapcode/|!ARGS:googleCode|!ARGS:/sidebar/|!ARGS:/ad_code/|!ARGS:/^recipient/
|!ARGS:optional_head|!ARGS:/^form/|!ARGS:/^var_value/|!ARGS:variable_data|!ARGS:/^instance/
|!ARGS:/customfield/|!ARGS:notice|!ARGS:/formcode/|!ARGS:/ajax/|!ARGS:all|!ARGS:allowedTags
|!ARGS:/google_analytics/|!ARGS:/widget/|!ARGS:ad_code|!ARGS:/keycaptcha_code/|!ARGS:/jscode/
|!ARGS:postcontents|!ARGS:/adsense/|!ARGS:video1|!ARGS:/updateAds/|!ARGS:map|!ARGS:gmapcode
|!ARGS:/^Sidebar/|!ARGS:/^wpTextbox/|!ARGS:paragrafo|!ARGS:/question/|!ARGS:/style/
|!ARGS:tracking_code|!ARGS:whats-new|!ARGS:analyticscode|!ARGS:top_news|!ARGS:data[config]
|!ARGS:fulltext|!ARGS:introtext|!ARGS:offertext|!ARGS:block|!ARGS:livezillacode|!ARGS:/embed/
|!ARGS:/desc/|!ARGS:/script/|!ARGS:/^p_process_chats/|!ARGS:obj_itop|!ARGS:/cms/
|!ARGS:eventDescription|!ARGS:/^product/|!ARGS:match_report|!ARGS:eip_value|!ARGS:/^usergroup/
|!ARGS:sendDescription|!ARGS:email_id|!ARGS:obj_itop|!ARGS:sml_prt_1|!ARGS:pay_inst_1
|!ARGS:/^jform/|!ARGS:phpcode|!ARGS:intro|!ARGS:Snippet|!ARGS:oid|!ARGS:Submit2|!ARGS:/^obj_/
|!ARGS:layout|!ARGS:pageset|!ARGS:contact_form_information|!ARGS:/^site_/|!ARGS:/^translations/
|!ARGS:create_tables|!ARGS:insertfile|!ARGS:video_credits|!ARGS:input[Desarrollo]|!ARGS:move2
|!ARGS:hoperation|!ARGS:login_form|!ARGS:/product_benefits/|!ARGS:/custom_code/|!ARGS:arg2
|!ARGS:resumoDetalhe|!ARGS:bbcode_tpl|!ARGS:Right_photo_1|!ARGS:embedVideo|!ARGS:/^K2ExtraField/
|!ARGS:mentorhelp|!ARGS:/submitcode/|!ARGS:beschrijving|!ARGS:custombannercode|!ARGS:bannercode
|!ARGS:privatecapacity|!ARGS:diz|!ARGS:FormLayout|!ARGS:/^fck/|!ARGS:parent_name
|!ARGS:/^code_tscript/|!ARGS:_qf_Group_next|!ARGS:project_company|!ARGS:categories_title
|!ARGS:antwoord|!ARGS:project_company|!ARGS:signature|!ARGS:paepdc|!ARGS:tpl_source
|!ARGS:teaser_js|!ARGS:/^autoDS/|!ARGS:FrmSide|!ARGS:mainKeywords|!ARGS:/VB_announce/
|!ARGS:guardar|!ARGS:/serendipity/|!ARGS:omschrijving|!ARGS:resolution|!ARGS:newyddionc
|!ARGS:bericht|!ARGS:property_copy|!ARGS:/^outpay/|!ARGS:bedrijfsprofiel|!ARGS:s_query
|!ARGS:finish_survey|!ARGS:photolater|!ARGS:ticket_response|!ARGS:/element/
|!ARGS:option[vbpclosedreason]|!ARGS:/introduction/|!ARGS:/contenido/|!ARGS:/sql/
|!ARGS:prefix|!ARGS:query|!ARGS:c_features|!ARGS:/tekst/|!ARGS:embeddump|!ARGS:other_clubs
|!ARGS:/^elm/|!ARGS:/^saes/|!ARGS:dlv_instructions|!ARGS:/^cymr/|!ARGS:_qf_Register_upload
|!ARGS:/^elm/|!ARGS:verbiage|!ARGS:news|!ARGS:/^wz/|!ARGS:tiny_vals|!ARGS:sSave|!ARGS:/article/
|!ARGS:/about/|!ARGS:/Summarize/|!ARGS:/^product_options/|!ARGS:/SiteStructure/|!ARGS:/anmerkung/
|!ARGS:/summary/|!ARGS:/edit/|!ARGS:reply|!ARGS:/story/|!ARGS:resource_box|!ARGS:navig
|!ARGS:preview__hidden|!ARGS:/page/|!ARGS:order|!ARGS:/post/|!ARGS:youtube|!ARGS:reply
|!ARGS:business|!ARGS:/homePage/|!ARGS:pagimenu_inhoud|!ARGS:/note/|!ARGS:Post|!ARGS:/^field_id/
|!ARGS:area|!ARGS:/detail/|!ARGS:/comment/|!ARGS:LongDesc|!ARGS:ta|!ARGS:Returnid|!ARGS:busymess
|!ARGS_NAMES:/^V\*/|!ARGS_NAMES:/^S\*/|!ARGS:/^quickrise_advertise/|!ARGS:rt_xformat
|!ARGS:/wysiwyg/|!ARGS:contingut|!ARGS:/^werg/|!ARGS:/body/|!ARGS:/css/|!ARGS:/^section/
|!ARGS:/msg/|!ARGS:t_cont|!ARGS:/^doc/|!ARGS:/xml/|!ARGS:tekst|!ARGS:formsubmit
|!ARGS:invoice_snapshot|!ARGS:submit|!ARGS:/html/|!ARGS:/content/|!ARGS:/footer/|!ARGS:/header/
|!ARGS:/footer/|!ARGS:/link/|!ARGS:text|!ARGS:txt|!ARGS:/refer/|!ARGS:/referrer/|!ARGS:/template/
|!ARGS:/ajax/ "(?:< ?script|(?:<|< ?/)(?:(?:java|vb)script|about|applet|activex|chrome)|
< ?/?i?frame|\%env)" \Potential Cross Site Scripting Attack'
"phase:2,deny,status:406,t:none,t:removeNulls,t:utf8toUnicode,t:urlDecodeUni,t:htmlEntityDecode,
t:jsdecode,t:cssdecode,t:replaceComments,t:compressWhitespace,t:lowercase,capture,id:340147,
rev:137,severity:2,msg:'Atomicorp.com WAF Rules: ,logdata:'%{TX.0}'"

Answer 1

I would be most obliged if someone that knows about these filters could offer some suggestions on any methods that I can use to preprocess the form content before posting it from the webpage to make it more resilient to false positives.

base64?

When used legitimatly the fields will contain the unsafe characters ", <, >, &, ' and `

Then mod_security isn't a good choice of tool for you. (Personally my view is that input filtering rules like these are rarely a good idea for anyone.)

This is a hack but I don't think I have any choice but to do some kind of hack.

Is someone imposing mod_security on you? By tunnelling through it you are making it ineffective in any case, so it is (even more than normal) a hopeless waste of time.