<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Feb 28, 2014 at 2:16 PM, John Miller <span dir="ltr"><<a href="mailto:johnmill@brandeis.edu" target="_blank">johnmill@brandeis.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Feb 28, 2014 at 12:14 PM, John P. Rouillard <span dir="ltr"><<a href="mailto:rouilj@cs.umb.edu" target="_blank">rouilj@cs.umb.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
In message <<a href="mailto:5310B66A.5090101@brandeis.edu" target="_blank">5310B66A.5090101@brandeis.edu</a>>,<br>
John Miller writes:<br>
>We're running into an issue with our LDAP servers where legitimate<br>
>packets are being intermittently (say 1/10000) rejected by iptables.<br>
>They match one of our ACCEPT rules, yet still get rejected (we're seeing<br>
>the return ICMP traffic).<br>
<br>
Have you captured the inbound and outbound packets? Are you sure that<br>
the first ldap packet is being rejected and not a packet that should<br>
be allowed through by a:<br>
<br>
--state established -j ACCEPT<br>
<br>
rule somewhere in your rule stack? This would also point to a state<br>
table issue, but might provide some more insight.<br>
<br></blockquote><div><br></div><div>Yep--have done captures on the outbound packets (when sending from a test host); have not approached from the incoming point of view. The packets are being logged by the next-to-last rule (last is a reject-all rule), so I don't think there's anything interfering in the routing path.<br>
</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
>I don't yet have any evidence to support this, but I'm thinking that<br>
>we're bumping up against some sort of internal netfilter limit--perhaps<br>
>with connection tracking or stateful matching.<br>
><br>
>The problem seems to have cropped up in moving from RHEL 5 to RHEL 6.<br>
><br>
>The rules in question:<br>
><br>
>What should be hit:<br>
>-A RH-Firewall-1-INPUT -s <a href="http://129.64.0.0/255.255.0.0" target="_blank">129.64.0.0/255.255.0.0</a> -p tcp -m state --state<br>
>NEW -m tcp --dport 636 -j ACCEPT<br>
<br>
I have not seen this, but you could simplify the rule and remove<br>
<br>
"-m state --state NEW"<br>
<br>
for testing to see if the problem goes away. That should eliminate any<br>
issues with the state setup and allow all ldap traffic to pass<br>
through.<br>
<br></blockquote><div><br></div><div>Have tried this; results were inconclusive, so going to test it out again this afternoon.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Or what you could do is put a rule:<br>
<br>
-A RH-Firewall-1-INPUT -s <a href="http://129.64.0.0/255.255.0.0" target="_blank">129.64.0.0/255.255.0.0</a> -p tcp -m state -m tcp --dport 636 -j RECORD_LDAP<br>
<br>
after the rule you expect to get hit. If any packet hits this new<br>
RECORD_LDAP rule, it should be an error and in the RECORD_LDAP chain<br>
you can get a count or possibly full packet dump for a better idea of<br>
what's happening.<br>
<br>
Also there was a bash script that automtically instrumented an<br>
existing iptbles rule. Try googling for Tony Clayton iptables-trace.<br>
<br></blockquote><div><br></div><div>Thanks--was not aware of <br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Also you can try looking for:<br>
<br>
conntrack-tools<br>
<br>
to see if that provides any info.<br>
<br></blockquote><div><br></div><div>Also saw that in my Googling; if the problem turns out to be connection tracking, I'll certainly post to that effect. I'd have expected, however, to see something along the lines of<br>
<h1 style="color:rgb(102,102,102);font:22px/17px Georgia,'Times New Roman',serif"><span style="border-collapse:collapse;color:black;font-family:verdana,geneva,helvetica,arial,sans-serif;font-size:13px;line-height:normal"><b style="background-color:rgb(255,255,102);background-image:initial">nf_conntrack</b>: <b style="background-color:rgb(255,255,102);background-image:initial">table</b> <b style="background-color:rgb(255,255,102);background-image:initial">full</b>, <b style="background-color:rgb(255,255,102);background-image:initial">dropping</b> <b style="background-color:rgb(255,255,102);background-image:initial">packet</b>.</span></h1>
<p>rather than the packet traversing the entire ruleset. If I understand correctly, packets hit the conntrack module fairly early on in the netfilter process, so you'd think they would be dropped at that step.<br></p>
</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
>What is actually being hit:<br>
>-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited<br>
<br>
I assume this is the last rule in RH-Firewall-1-INPUT after the ldap rule.<br>
<br></blockquote><div><br></div><div>Correct--we allow a limited set of traffic, then reject everything else.<br></div><div> </div>The help is much appreciated, John. Always helps to get a fresh perspective on something.<span class="HOEnZb"><font color="#888888"><br>
<br></font></span></div><span class="HOEnZb"><font color="#888888"><div class="gmail_quote">John<br></div></font></span></div></div>
</blockquote></div><br><br clear="all"></div><div class="gmail_extra">I've just disabled connection tracking for all of the incoming LDAP traffic on one of our servers. Will see what sort of difference, if any, this makes to our packet reject logs.</div>
</div>