The Web Application Firewall Evaluation Criteria Version 2

WAFEC 2

Outline

Contents

1Introduction

1.1WAFEC Mission Statement

1.2Using WAFEC

1.2.1Structure

1.2.2Evaluation Process

1.3What’s New

1.4License

2What is a WAF

2.1Definition

2.2Use Cases

2.2.1Logging and Troubleshooting

2.2.2Attack Detection

2.2.3Attack Mitigation

2.2.4Virtual Patching

2.3Minimum Requirements

3Security

3.1Threat landscape coverage

3.2Mitigation

3.2.1Blocking

3.2.2Response

3.2.3Throttling

3.3Protection techniques

3.3.1Positive Security

3.3.2Virtual Patching

3.3.3Signatures

3.3.4Protocol Validation

3.3.5Session Tracking

4Deployment Options

4.1Method of Delivery

4.2Network Integration

4.2.1Deployment Architecture

4.2.2Inline Modes

4.2.3High Availability and Clustering

4.3Application Support

4.3.1HTTP and HTTPS Support

4.3.2Authentication Support

5Supporting Features

5.1Management

5.2Reporting and Analytics

5.3System Security

5.4Performance

5.5Reliability

5.6Extensibility and Integration

5.7Physical Characteristics

6Appendices

6.1Integrated Related Features

6.2None Technical Criteria

6.3Alternative and Complementary Solutions

6.4Testing Framework

6.5Suggested Weighting Schemes

6.6Unaddressed Threats

6.7V1 to V2 translation table

6.8Response Table

1Introduction

1.1WAFEC Mission Statement

WAFEC provides interested partied, including users, vendors and 3rd party evaluators with a tool to learn about WAFs and evaluate the suitability of different WAFs for specific use cases and environments.

1.2Using WAFEC

1.2.1Structure

WAFEC has two inherent goals: education and evaluation. Section ### “What is a WAF?” focuses on the educational goal and set the framework for the following discussion.

When evaluating a WAF, one has to address several high level functional and technical criteria groups which affect selection in different ways. A chapter of the document is dedicated to each one of them:

Provide sufficient security - WAFEC security criteria focus on results rather than methods used for protection as those may vary between WAFs while still providing the same benefit. To that end the key criteria is what threats, based on the Web Application Security Consortium Threat Classification (WASC-TC) a WAF protects from. Not all the WASC-TC threats can be protected from by a WAF, and those which cannot are omitted from the chapter and listed in appendix ###.

Protection methods and their implementation do affect the quality of the protection provided. To that end, when evaluating based on WAFEC one needs to determine which methods supported by the WAF are used for protecting from each threat. In addition a section of the security criteria chapter is dedicated to Evaluating the quality of implementation of the protection methods. A WAF does not have to implement all protection methods, however those implemented should be of quality and address the listed threats

Suit the environment it is used at - To protect web applications a WAF has to be implemented in the organization’s IT environment and be compatible with it. Compatibility focuses on two areas:
Deployment compatibility -
Protected applications compatibility – ensuring both that they continue to run and that they are well protected.

When weighting those criteria, one must decide whether to require all or just the options required for the specific use case at hand. While the former ensures more future flexibility it may lead to a higher cost and complexity. One way around it would be to give a higher weight to the currently needed requirements while giving lower weights to potential future requirements.

Include supporting features - Lastly, once deployed and protecting applications, a WAF needs a large number of supporting features in order to be efficient and provide value. Those include management, reporting, integration with other systems as well as self-security and performance. While no WAF can exist without addressing many of those criteria, the list is long and a solution never can or should implement them all. One should focus on the criteria which are relevant to the environment in which the WAF operates.

While this document focuses on the intrinsic functional and technical criteria of WAFs, selecting a solution may involve other criteria such as price, vendor reputation, quality of support and related featuresintegratedinto the WAF. When analyzing the results a user must add such criteria. A list of criteria to consider in this respect is included in Appendix ### but should not be taken as comprehensive.

1.2.2Evaluation Process

The table at appendix ### provides a convenient tool for actually performing an evaluation. Note that it is not enough to evaluate a criterion for yes or no. In addition an explanation of the implementation methods is required.

Lastly, WAFEC does not provide weights for specific criteria as this would be too organization specific. This does not imply that all criteria are equal in importance. As a guideline we suggest applying weights hierarchically, first weighting chapters and then sections within chapters and so on to avoid weighting by numbers or criteria rather than by importance. Appendix ### provides suggested weighting schemes.

1.3What’s New

WAFEC v1 was release in 2006. Naturally the long time since justifies a new version of the document to address the changing threat landscape as well as newer protection and deployment technologies.

Changes in the new version include:

To the extent possible, WAFEC now focus on feature and benefits rather than implementation details. Implementation details are at times important to assess the quality of a solution and are included in those cases.
WAFEC now clearly identifies what is a requirement and how to use it to evaluate WAFs.
Descriptive text is added to all sections and to individual requirements to help explain their context and importance.

Add on features that do not directly support WAF core functionality (as defined by WAFEC) are excluded from body of the document and listed in appendix ###.

Refer to appendix ### for a translation table between WAFEC v1 and WAFEC v2

1.4License

This work is licensed under the Creative Commons Attribution License. To view a copy of this license, visit send a letter to: Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.

2What is a WAF

Note: the information in this section is descriptive in nature and should help the user to identify the specific use cases that are required from a WAF in her organization. This selection affects the relevance of criteria in subsequence sections.

2.1Definition

2.2Use Cases

2.2.1Logging and Troubleshooting

2.2.2Attack Detection

2.2.3Attack Mitigation

2.2.4Virtual Patching

2.3Minimum Requirements

List of requirements (from other sections) that a WAF must have

3Security

3.1Threat landscape coverage

Evaluation guidelines: For each one of the threats listed below evaluate the WAF detection and mitigation capabilities using the following methods:

Test the WAF for detection and mitigation of each threat.
Establish whether a trusted 3rd party evaluator has tested the WAF detection and mitigation capabilities for the threat.
List the detection and mitigation techniques (from those listed in section ### or others) used by the WAF to detect and mitigate each theat[AS1].

Since many of the threats below address a broad array of attacks, note that the WAFs coverage for a specific threat may be partial, for example limited to specific types of applications.

A WAF main function is to protect web applications from attacks. Todo so, it has to detect and potentially mitigate attacks. Therefore a key criterion for WAFs is their ability to detect and mitigate attacks.

WAFEC uses the Web Application Security Consortium Threat Classification[1] (WASC-TC) as the base list of threats that a WAF should detect and mitigate. Some threats cannot be effectively detected or mitigated by a WAF and therefore are omitted (see appendix ### for a list of those). Note that WAFEC does not provide descriptive text of threats and the reader is encourages using the original WASC TC description.

You may find the WASC TC Taxonomy cross reference[2] useful if you would like to establish a WAF threat coverage with regard to other taxonomies such as CW, the OWASP top 10 and SANS top 25.

The threatsagainst which the WAF should be evaluated are:[AS2]

Weakness Name[AS3] / Attack Name(s) / WAF Mitigation Techniques
WASC-20: Improper Input Handling[AS4] / WASC-03: Integer Overflows
WASC-05: Remote File Inclusion
WASC-06: Format String
WASC-07: Buffer Overflow
WASC-08: Cross-site Scripting
WASC-19: SQL Injection
WASC-23: XML Injection
WASC-25: HTTP Response Splitting
WASC-26: HTTP Request Smuggling
WASC-27: HTTP Response Smuggling
WASC-28: Null Byte Injection
WASC-29: LDAP Injection
WASC-30: Mail Command Injection
WASC-31: OS Commanding
WASC-33: Path Traversal
WASC-36: SSI Injection /

Positive Security Model Input Validation
Negative Security Model Input Validation(Signatures)[AS5]
Virtual Patching[AS6]

WASC-21: Insufficient Anti-Automation[AS7] / WASC-10: Denial of Service
WASC-11: Brute Force /

Access Rate Thresholding[AS8]

WASC-22: Improper Output Handling / WASC-08: Cross-site Scripting /

Response Page Profile Deviations[AS9]

WASC-40: Insufficient Process Validation[AS10] / WASC-09: Cross-site Request Forgery
WASC-24: HTTP Request Splitting[AS11]
WASC-34: Predictable Resource Location /

Application Flow Control
Cryptographic Hash Validation
URL Encryption

WASC-15: Application Misconfiguration / WASC-37: Session Fixation /

Client Meta-data Mismatch (IP, User-Agent, Fingerprint)

WASC-13: Information Leakage /

Negative Security Model Output Validation (Signatures)[AS12]

WASC-47: Insufficient Session Expiration / WASC-37: Session Fixation /

Session Timeouts

3.2Response[AS13]

3.2.1Passive Response

Passive response actions are ones that a WAF may take in response to detected attacks or application errors that do not disrupt the transaction in an perceivable way.

A. / Increasing Client Anomaly Scores / WAF may track anomaly scoring for the client over multiple transactions and then apply an active response action at a later time.[AS14]
B. / Increase Logging for Client / WAF may mark the client as suspicious and log full transactions for a period of time regardless of whether the client triggers any alerts.
C. / Email Alerts / WAF may send out email alerts to notify security personnel.[AS15]
D. / Request Header Tagging / WAF may export alert meta-data and alter the transaction to add new request headers. This data may be used to share data with the back-end web application or fraud detection systems.[AS16]
E. / Notify Network Infrastructure / WAF may notify networking infrastructure components such as a Firewall and make policy changes to temporarily block further access from the client source IP address.[AS17]

3.2.2Active Response

Active response actions are ones that a WAF may take that apply disruptive actions to either the current or future transactions. Your security policy may require specific responses once an attack is detected. Those responses depend on the deployment mode used and attack detected as described in the table.

A. / TCP Reset / WAF will send spoofed TCP RST/FIN packets in an attempt to forcefully terminate the connection. / Network placement is critical for effectiveness.[AS18]
B. / Block the Client Source Address / WAF may temporarily block the client IP address from accessing the application. / Blocking based on client IP address may deny legitimate users who are using the same IP address. Attackers may also easily use a different IP address.
C. / GeoLocation Restrictions / WAF may utilize GeoLocation data to either deny or allow access to the application under certain situations. / May decide to block access from locations with high fraud rates or to only allow certain locations when under heavy attack.[AS19]
D. / Redirection to Error Pages / WAF may issue HTTP Redirection response that will send the user to an informative web page describing the reason for the blocking. / Error pages could provide detailed data for resolution by contacting support or helpdesk.[AS20]
E. / Forcing Transaction Delays / WAF may apply transaction delays to slow down attacker traffic. / May allow more time for incident response personnel to respond.
F. / Spoofing Successful Attack Responses / WAF may alter response data to simulate a successful attack. / May allow more time for incident response personnel to respond.
G. / Proxying Traffic to Honeypots / WAF may selectively proxy attacker traffic to an isolated honeypot system. / May allow more time for incident response personnel to respond.[AS21]
H. / Forcing an Application Logout / WAF may use the active SessionID to spoof an application logout request. / May not work well if the application does not properly invalidate the SessionIDwitin the application.[AS22]
I. / Temporarily Blocking User Access / WAF may temporarily deny access to the application for a specifc user. / WAF must be configured to track the proper user logon parameter value.

3.2.3Intrusive Response[AS23]

Under certain circumstances, it may be necessary for the WAF to interact directly with the end user or their browser rather than the HTTP connection. Here are a few options:

A. / Forcing CAPTCHA Test / WAF will redirect user to a CAPTCHA testing page. / Helps to identify automated tools.
B. / Javascript Cookie Testing / WAF may send Javascript to the client that will create a new Cookie. / Helps to identify automated tools.

3.3Protection techniques[AS24]

Evaluation guidelines:

Specify for each threat in section ### what protection methods are used to protect from it.
In this section specify the attributes of the specific protection technique.[AS25]

While the key to the effectiveness of WAF is the end result: which threats it protects from, protection techniques and their implementation do affect the quality of the protection provided. Use this section to evaluate the quality of the implementation of the different protection methods.

3.3.1Protocol Validation[AS26]

3.3.2Positive Security

Positive security model is a comprehensive security mechanism that provides an independent input validation envelope to an application. By defining rules of allowed user input for every parameter in every page in the application the application is protected by an additional security envelop independent from its code.

3.3.3Learning

The limitation of this a positive security model is that it requires deep knowledge of the application and a considerable on going effort to maintain the rule set. The maintainer needs to define such rules for each parameter and page in the application. Essentially the rules have to follow closely the application and every change in the application requires a modification to the rule set as well.

In order to reduce the effort required, different learning mechanisms have been implemented. In a session based learning approach rules are dynamically created based on previous transactions in the session. Specifically, based on forms returned by the web server the WAF generates validation rules for input submitted using this form by a user, validating field existence, length and special attributes such as "hidden" or "read-only". Other dynamic validation rules include rules limiting the allowed URLs only to those appearing in links in previous pages and rules limiting cookie values. However, this approach is limited since the information sent by the server does not convey all the information required to generate a rule. For example, the type of a parameter is not available. Furthermore, this method became nearly obsolete with the major shift to client side scripting, interactive web front ends such as AJAX [find] and web services. In all these technologies the client sends requests that are not based on previous server responses or are hard to determine from the responses.

More recently anomaly based learning approach was suggested. In anomaly based approach an input validation profile is created for an application based on observing real usage traffic and determining normal usage patterns. As with most anomaly based detection techniques, the key challenges are differentiating between attacks and non malicious abnormal traffic, not including in the normal usage profile information derived from attacks and compensating for time based variability in the usage profile.

Differentiating attack traffic from non malicious abnormal traffic is a major challenge for monitoring systems, but is less severe for protection only systems as many non attack abnormal requests can be blocked as they would not generate useful results. The problem can be further reduced by using a detected anomaly only as an indicator and determining an attack only based on multiple indicators, both anomaly based and other.

3.3.4Virtual Patching[AS27]

Example Criteria:

URL rewrite and REST interface support

3.3.5Signatures[AS28]

Example Criteria:

Used defined signatures[AS29]
Granularity of signatures[AS30]
Frequency of vendor update
Are updates automated?[AS31]
Complexity supported by signatures[AS32]
Exceptions supported[AS33]
Learning supported[AS34]
Anti-Evasion techniques[AS35]
URL rewrite and REST interface support[AS36]

3.3.6Session Tracking[AS37]

Example Criteria:

Automated session identification
User defined session identification

4Deployment Options

The WAF has to work well in your own environment. The following sections describe aspects of a WAF required to support your environment.

Evaluation guidelines: unless described otherwise, for each criteria group below select and evaluate only the criteria that are relevant to your organization. Keep in mind that requiring unneeded criteria may lead to higher cost and complexity in the solution selected.

4.1Method of Delivery

Packaging is often a matter of choice and corporate culture and do not have significant effect on the quality of the solution. Verify that the WAF supports the packaging method you prefer. Note that method of delivery will limit the network connectivity options (see ###) available as described below.

A. / Appliance / Software bundled with hardware. Appliances are usually highly optimized and may contain specialized hardware components to increase performance. Are there other optional hardware components that may further increase performance (except for SSL, which was already covered in 1.2)? / Relevant for Inline & Sniffer deployment.
B. / Virtual Appliance / Transfer control from a faulty system to a backup system in case of a failure. Fail over systems should ensure minimal disruption of service during fail over. Relevant areas include latency of switch and granularity of maintained state when switching.
For performance related aspects of high availability, refer to the “performance” section below. / Relevant for Inline, Sniffer and Cloud deployments.
C. / Software / Relevant for Inline, Sniffer, Cloud and Embedded deployments.
D. / SaaS / Relevant for Cloud Deployment.

4.2Network Integration