logs archiveBotHelp.net / Freenode / #389 / 2015 / September / 22 / 1
jkalchik
good morning, all.
I've got an LDAP search call (Perl, using Net::LDAP calls to 389-DS services) that's failing with an "I/O interrupted system call". I don't see any other tracks. The failure occurs somewhat randomly. at the moment, it's happening after some 80 LDAP searches.
a Dumper output from $result does reveal some garbage.
any suggestions?
hey.... Mr. Reynolds.... got a sec?
mreynolds
jkalchik: how can I help you
jkalchik
good morning... actually, I'm starting to suspect I've got other issues.
In short, I'm using perl-LDAP (Net::LDAP) to query a 389-DS service
I'm having a problem with "I/O interrupted system call" errors.
it's smelling more like a bug outside of 389-ds
I can abstract out the failing call into a test perl script, and that runs just fine.
printing out Dumper(\$result) returns some garbage values.
the problem isn't in 389-ds
any of that make sense?
mreynolds
jkalchik: you can always check the DS access & errors log to see how/if the server is even receiving your connection
it might provide more insight
jkalchik
nothing in the errors log, and a number of calls do succeed before the failing call.
mreynolds
jkalchik: can you fpaste the perldap code, showing where things go wrong?
jkalchik
yeah, just a sec.
mreynolds
and the access log output as well - thanks
jkalchik
<scratches head> that'll be a bit.... interesting. this is running against a production host with ~300 clients
initial object creation: my $LDAP = Net::LDAP->new (\@{$CFG{"ldap_server"}});
LDAP->search call:
my $result = $LDAP->search (
"base" => $CFG{"base_dn"},
"scope" => "sub",
"filter" => $ldapsearch,
"attrs" => ['uid', 'uidnumber', 'gidnumber', 'gecos', 'homedirectory', 'loginshell' ]
);
<scratches head>
interesting.....
looking further into the access log, I do see the queries, with base descriptors, filters & attribute requests. this particular request doesn't seem to be making it there at all.
mreynolds
jkalchik: can you fpaste.org the access log output?
jkalchik
just a sec..... I'm gonna set up something else, a wireshark capture. I'm beginning to wonder if this failed call is making it to the directory service at all.
mreynolds
there is access log buffering, takes up to 30 seconds for output to show up in the log
jkalchik
oh, that I know....... but I've been swearing at this since yesterday afternoon, and I'm not finding any trace of any of the failed queries.
okay.... wireshark capture on an unencrypted port does show that the connection is being made, I can see the search request.
sunnuvagun.... there's a connection reset. only partial results are being returned.
but the reset is being thrown by the host running the perl script, not the directory server.
I'll apologize for seeming to ramble.
mreynolds
jkalchik: if it would reach the server, you would at least see a "connection from" line in the access log. Sounds like you aren't hitting the server, or the connection is getting closed. Are you going through a load balancer or firewall?
jkalchik
no load balancer or firewalls. I can see the connection in wireshark, which means I need to look at the access log closer.
normally, this runs local to the directory server, I'm executing it remotely at the moment to ensure I can capture network traffic.
so, I do see the searchRequest packet, with the expected contents.
and I see a number of packets with returned results, but not all of the expected results
but why would I get a connection reset packet out of the host running the Perl script? (yes, that's a semi-rhetorical question...)
richm
what version of 389-ds-base?
jkalchik
ahpcos249:<jkalc>:(/home/jkalc/389-ds)
$ rpm -qa | grep 389
389-admin-console-1.1.8-1.el6.noarch
389-ds-1.2.2-1.el6.noarch
389-adminutil-1.1.19-1.el6.x86_64
389-ds-console-doc-1.2.6-1.el6.noarch
389-ds-base-1.2.11.15-46.el6.x86_64
389-console-1.1.7-1.el6.noarch
389-admin-console-doc-1.1.8-1.el6.noarch
389-admin-1.1.35-1.el6.x86_64
389-dsgw-1.1.11-1.el6.x86_64
389-ds-base-libs-1.2.11.15-46.el6.x86_64
I rather strongly suspect that the issue is not in 389-DS, it's in perl-LDAP, or an underlying module.
I appreciate your ears (in spite of the fact that I'm apparently not fulfilling your requests.)
it is sort of odd that I don't see any SRCH records in the access log that correspond to the remote LDAP queries I'm sending it.
and now I do find them. coffee. must have coffee.
I appreciate being pushed to do the obvious.
[22/Sep/2015:10:50:42 -0500] conn=166108 op=2 SRCH base="ou=Unix,dc=cnxlol,dc=co
m" scope=2 filter="(&(objectClass=posixaccount)(!(nsRole=cn=nsmanageddisabledrol
e,ou=Unix,dc=cnxlol,dc=com))(|(host=ahphp214)(nsRole=cn=syseng,ou=unix,dc=cnxlol
,dc=com)(nsRole=cn=operations,ou=unix,dc=cnxlol,dc=com)(nsRole=cn=prodsupp,ou=un
ix,dc=cnxlol,dc=com)(nsRole=cn=dba,ou=Unix,dc=cnxlol,dc=com)(nsRole=cn=hrms,ou=U
nix,dc=cnxlol,dc=com)(nsRole=cn=hrms-prod,ou=Unix,dc=cnxlol,dc=com)(nsRole=cn=hr
ms-prod-app,ou=Unix,dc=cnxlol,dc=com)(nsRole=cn=oem12c_agt,ou=Unix,dc=cnxlol,dc=
com)))" attrs="uid uidNumber gidNumber gecos homeDirectory loginShell"
[22/Sep/2015:10:50:42 -0500] conn=166108 op=-1 fd=70 closed - B1
oops....
mreynolds
jkalchik: can you do ldapsearches from the same system?
jkalchik
yes.
using the spec'ed filter in ldapsearch works just fine.
mreynolds
sorry I just don't use perl-ldap enough to really help out more. But this definitely looks like a client issue. Maybe enabling "connection logging" might also provide more insight
jkalchik
same behavior if I direct the perl script with an array of hosts (LDAPS,) ldaps://localhost, ldap://localhost, ldap://some.remote.host, etc.
mreynolds
https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/9.0/html/Configuration_Command_and_File_Reference/error-logs.html#error-logs-levels
jkalchik
mreynolds: agreed. this isn't a 389-ds issue, it's definitely in the perl stuff.
mreynolds
have a meeting now...
jkalchik
thanks for the use of your ears. I do appreciate it.
next »