怎样写一个RefererFilter

缘起

首先,用检查Referer的方式来防披御CSRF并不是很好的方法。因项目临时有需要,所以做为过渡方案。

为什么判断referer不是很好的办法?

  • referer 可能为空
    • https跳转http没有referer
    • https跳转不同的域名的https没有referer
    • 通过特殊构造的POST请求没有referer
    • 一些的proxy会把referer去掉
    • 用户直接在浏览器里访问(GET请求)
  • 判断的逻辑复杂(用正则匹配?)
  • 友站中招,殃及池鱼
  • 可以作为过渡方案,非长久之计

构造空referer请求的一些参考资料

防御CSRF目前比较好的办法是CSRF Token,参考另一篇blog:Cookie & Session & CSRF

收集资料

先搜索下前人有没有这类相关的工作。
搜索到的关于RefererFilter的信息并不多。

不过这里学到了一些东东:
https://svn.apache.org/repos/asf/sling/tags/org.apache.sling.security-1.0.0/src/main/java/org/apache/sling/security/impl/ReferrerFilter.java

  • 是否允许localhost, 127.0.0.1这样referer的请求?
  • 是否允许本地的IP/host的请求?

再搜索下java里提取request的referer的方法,还有filter里重定向请求的方法。

再仔细看了下OWASP的文档:

https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet

确定方案

  • 默认拦截“POST|PUT|DELETE|CONNECT|PATCH”的请求
  • HttpServletRequest里提取到referer
  • 用java.net.URL来提取referer里的host
  • 判断host是否符合要求,支持完全匹配的域名和子域名
  • 不符合要求的请求回应403或者重定向到指定的页面

为什么不用正则的方式来处理referer?

  • 正则表达式通常比较慢
  • 很难判断一个复杂的正则表达式是否真的正确
  • URL是很复杂的,不要手动处理URL,参考URL的语法

思考需要提供的配置项

实际最终提供了这些配置项,考虑到像host这样的配置不是经常变动的,所以没有提供从外部配置文件加载配置的功能。

1
2
3
4
5
6
7
8
9
10
11
12
matchMethods   即拦截的方法,默认值"POST|PUT|DELETE|CONNECT|PATCH",通常不用配置
allowSubDomainHosts 匹配子域名,以"|"分隔,如"test.com|abc.com",
则http://test.com, http://xxx.test.com这样的请求都会匹配到,推荐优先使用这个配置
completeMatchHosts 完全匹配的域名,以"|"分隔,如"test.com|abc.com",则只有http://test.com 这样的请求会匹配
像http://www.test.com 这样的请求不会被匹配

responseError 被拦截的请求的response的返回值,默认是403
redirectPath 被拦截的请求重定向到的url,如果配置了这个值,则会忽略responseError的配置。
比如可以配置重定向到自己定义的错误页: /referer_error.html
bAllowEmptyReferer 是否允许空referer,默认是false,除非很清楚,否则不要改动这个
bAllowLocalhost 是否允许localhost, 127.0.0.1 这样的referer的请求,默认是true,便于调试
bAllowAllIPAndHost 是否允许本机的所有IP和host的referer请求,默认是false

编码的细节

  • 重定向时,注意加上contextPath

    1
    response.sendRedirect(request.getContextPath() + redirectPath);
    • 构造URL时,非法的URL会抛出RuntimeException,需要处理

正确地处理URL

感觉这个有必要再次说明下:

http://docs.oracle.com/javase/tutorial/networking/urls/urlInfo.html

用contain, indexOf, endWitch这些函数时都要小心。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
public static void main(String[] args) throws Exception {

URL aURL = new URL("http://example.com:80/docs/books/tutorial"
+ "/index.html?name=networking#DOWNLOADING");

System.out.println("protocol = " + aURL.getProtocol());
System.out.println("authority = " + aURL.getAuthority());
System.out.println("host = " + aURL.getHost());
System.out.println("port = " + aURL.getPort());
System.out.println("path = " + aURL.getPath());
System.out.println("query = " + aURL.getQuery());
System.out.println("filename = " + aURL.getFile());
System.out.println("ref = " + aURL.getRef());
}

用curl来测试

最后用curl来做了一些测试:

1
2
3
4
5
6
curl  --header "Referer:http://test.com" http://localhost:8080/filter-test/referer
curl -X POST --header "Referer:http://test.com" http://localhost:8080/filter-test/referer
curl -X POST --header "Referer:xxxxx" http://localhost:8080/filter-test/referer
curl -X POST http://localhost:8080/filter-test/referer
curl -X POST --header "Referer:http://abc.test.com" http://localhost:8080/filter-test/referer
curl -X POST --header "Referer:http://abc.hello.com.test.com" http://localhost:8080/filter-test/referer

实现的代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
* <pre>
* 支持的配置项:
* matchMethods 即拦截的方法,默认值"POST|PUT|DELETE|CONNECT|PATCH",通常不用配置
* allowSubDomainHosts 匹配子域名,以"|"分隔,如"test.com|abc.com",
* 则http://test.com, http://xxx.test.com这样的请求都会匹配到,推荐优先使用这个配置
* completeMatchHosts 完全匹配的域名,以"|"分隔,如"test.com|abc.com",则只有http://test.com 这样的请求会匹配
* 像http://www.test.com 这样的请求不会被匹配
*
* responseError 被拦截的请求的response的返回值,默认是403
* redirectPath 被拦截的请求重定向到的url,如果配置了这个值,则会忽略responseError的配置。
* 比如可以配置重定向到自己定义的错误页: /referer_error.html
* bAllowEmptyReferer 是否允许空referer,默认是false,除非很清楚,否则不要改动这个
* bAllowLocalhost 是否允许localhost, 127.0.0.1 这样的referer的请求,默认是true,便于调试
* bAllowAllIPAndHost 是否允许本机的所有IP和host的referer请求,默认是false
*
* {@code
* <filter>
* <filter-name>refererFilter</filter-name>
* <filter-class>com.test.RefererFilter</filter-class>
* <init-param>
* <param-name>completeMatchHosts</param-name>
* <param-value>test.com|abc.com</param-value>
* </init-param>
* <init-param>
* <param-name>allowSubDomainHosts</param-name>
* <param-value>hello.com|xxx.yyy.com</param-value>
* </init-param>
* </filter>
*
* <filter-mapping>
* <filter-name>refererFilter</filter-name>
* <url-pattern>/*</url-pattern>
* </filter-mapping>
* }
* </pre>
*
* @author hengyunabc
*
*/
public class RefererFilter implements Filter {
static final Logger logger = LoggerFactory.getLogger(RefererFilter.class);
public static final String DEFAULT_MATHMETHODS = "POST|PUT|DELETE|CONNECT|PATCH";

List<String> mathMethods = new ArrayList<>();

boolean bAllowEmptyReferer = false;

boolean bAllowLocalhost = true;
boolean bAllowAllIPAndHost = false;

/**
* when bAllowSubDomain is true, allowHosts is "test.com", then
* "www.test.com", "xxx.test.com" will be allow.
*/
boolean bAllowSubDomain = false;

String redirectPath = null;
int responseError = HttpServletResponse.SC_FORBIDDEN;

HashSet<String> completeMatchHosts = new HashSet<String>();

List<String> allowSubDomainHostList = new ArrayList<String>();

@Override
public void init(FilterConfig filterConfig) throws ServletException {
mathMethods.addAll(getSplitStringList(filterConfig, "matchMethods", "\\|", DEFAULT_MATHMETHODS));

completeMatchHosts.addAll(getSplitStringList(filterConfig, "completeMatchHosts", "\\|", ""));

List<String> allowSubDomainHosts = getSplitStringList(filterConfig, "allowSubDomainHosts", "\\|", "");
completeMatchHosts.addAll(allowSubDomainHosts);
for (String host : allowSubDomainHosts) {
// check the first char if is '.'
if (!host.isEmpty() && host.charAt(0) != '.') {
allowSubDomainHostList.add("." + host);
} else {
allowSubDomainHostList.add(host);
}
}

responseError = getInt(filterConfig, "responseError", responseError);
redirectPath = filterConfig.getInitParameter("redirectPath");

bAllowEmptyReferer = getBoolean(filterConfig, "bAllowEmptyReferer", bAllowEmptyReferer);

bAllowLocalhost = getBoolean(filterConfig, "bAllowLocalhost", bAllowLocalhost);
if (bAllowLocalhost) {
completeMatchHosts.add("localhost");
completeMatchHosts.add("127.0.0.1");
completeMatchHosts.add("[::1]");
}

bAllowAllIPAndHost = getBoolean(filterConfig, "bAllowAllIPAndHost", bAllowAllIPAndHost);
if (bAllowAllIPAndHost) {
completeMatchHosts.addAll(getAllIPAndHost());
}
}

@Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException,
ServletException {
if (servletRequest instanceof HttpServletRequest && servletResponse instanceof HttpServletResponse) {
HttpServletRequest request = (HttpServletRequest) servletRequest;
HttpServletResponse response = (HttpServletResponse) servletResponse;

String method = request.getMethod();
/**
* if method not in POST|PUT|DELETE|CONNECT|PATCH, don't check
* referrer.
*/
if (!mathMethods.contains(method.trim().toUpperCase())) {
filterChain.doFilter(request, response);
return;
}

String referrer = request.getHeader("referer");

boolean bAllow = false;
if (isBlank(referrer)) {
bAllow = bAllowEmptyReferer;
} else {
URL url = null;
try {
url = new URL(referrer);
String host = url.getHost();
if (completeMatchHosts.contains(host)) {
bAllow = true;
} else {
for (String domain : allowSubDomainHostList) {
if (host.endsWith(domain)) {
bAllow = true;
break;
}
}
}
} catch (RuntimeException e) {
logger.error("illegal referrer! referrer: " + referrer, e);
bAllow = false;
}
}

if (bAllow) {
filterChain.doFilter(request, response);
return;
} else {
if (isBlank(redirectPath)) {
response.sendError(HttpServletResponse.SC_FORBIDDEN);
} else {
response.sendRedirect(request.getContextPath() + redirectPath);
}
}
} else {
filterChain.doFilter(servletRequest, servletResponse);
}
}

@Override
public void destroy() {

}

private static boolean isBlank(CharSequence cs) {
int strLen;
if (cs == null || (strLen = cs.length()) == 0) {
return true;
}
for (int i = 0; i < strLen; i++) {
if (Character.isWhitespace(cs.charAt(i)) == false) {
return false;
}
}
return true;
}

private static boolean getBoolean(FilterConfig filterConfig, String parameter, boolean defaultParameterValue) {
String parameterString = filterConfig.getInitParameter(parameter);
if (parameterString == null) {
return defaultParameterValue;
}
return Boolean.parseBoolean(parameterString.trim());
}

private static int getInt(FilterConfig filterConfig, String parameter, int defaultParameterValue) {
String parameterString = filterConfig.getInitParameter(parameter);
if (parameterString == null) {
return defaultParameterValue;
}
return Integer.parseInt(parameterString.trim());
}

/**
* <pre>
* getSplitStringList(filterConfig, "hosts", "\\|", "test.com|abc.com");
*
* if hosts is "hello.com|google.com", will return {"hello.com", google.com"}.
* if hosts is null, will return {"test.com", "abc.com"}
* </pre>
*
* @param filterConfig
* @param parameter
* @param regex
* @param defaultParameterValue
* @return
*/
private static List<String> getSplitStringList(FilterConfig filterConfig, String parameter, String regex, String defaultParameterValue) {
String parameterString = filterConfig.getInitParameter(parameter);
if (parameterString == null) {
parameterString = defaultParameterValue;
}

String[] split = parameterString.split("\\|");
if (split != null) {
List<String> resultList = new LinkedList<String>();
for (String method : split) {
resultList.add(method.trim());
}
return resultList;
}
return Collections.emptyList();
}

public static Set<String> getAllIPAndHost() {
HashSet<String> resultSet = new HashSet<String>();

Enumeration<NetworkInterface> interfaces;
try {
interfaces = NetworkInterface.getNetworkInterfaces();
while (interfaces.hasMoreElements()) {
NetworkInterface nic = interfaces.nextElement();
Enumeration<InetAddress> addresses = nic.getInetAddresses();
while (addresses.hasMoreElements()) {
InetAddress address = addresses.nextElement();
if (address instanceof Inet4Address) {
resultSet.add(address.getHostAddress());
resultSet.add(address.getHostName());
} else if (address instanceof Inet6Address) {
// TODO how to process Inet6Address?
// resultSet.add("[" + address.getHostAddress() + "]");
// resultSet.add(address.getHostName());
}
}
}
} catch (SocketException e) {
logger.error("getAllIPAndHost error!", e);
}
return resultSet;
}
}

其它的一些东东

在浏览器里如何访问IPV6的地址?

用”[]”把IPV6地址包围起来,比如localhost的:

1
http://[::1]

参考

http://superuser.com/questions/367780/how-to-connect-a-website-has-only-ipv6-address-without-domain-name

https://msdn.microsoft.com/en-us/library/windows/desktop/ms740593(v=vs.85).aspx

横云断岭/hengyunabc wechat
欢迎您扫一扫上面的微信公众号,订阅横云断岭的专栏