Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux-HA: Users

socket of lrmd

 

 

Linux-HA users RSS feed   Index | Next | Previous | View Threaded


ikedaj at intellilink

Jul 2, 2009, 12:09 AM

Post #1 of 14 (676 views)
Permalink
socket of lrmd

Hi,

When I run LVM RA with Heartbeat 2.1.4,
I could see the following message.

lrmd[14994]: 2009/07/02_15:29:20 info: RA output: (lvm:monitor:stderr) File descriptor 3 (socket:[46374]) leaked on vgck invocation. Parent PID 15446: /bin/sh

It seems that some LVM commands (vgdisplay or vgck) try to close the file descriptor which lrmd opened.
lsof command showed that lrmd had socket like this;

# lsof | grep lrmd | grep socket
lrmd 14994 nobody 3u unix 0xffff81011b05f740 46374 socket

(for more details, see the attached hb_report and lsof.txt)

lvm commands tried to close this socket and failed. :(
What is this socket for?
For the communication with othre processes like crmd or ha_logd?

This problem depends on the specification of lvm,
but I just want to confirm whether the above log message could have an impact on the cluster behavior or not.

This list might be the same issue.
http://www.gossamer-threads.com/lists/linuxha/users/49687

Thanks,
Junko
Attachments: LVM.tar.gz (46.9 KB)
  lsof.txt (4.11 KB)


ikedaj at intellilink

Jul 2, 2009, 2:40 AM

Post #2 of 14 (650 views)
Permalink
Re: socket of lrmd [In reply to]

Hi,

I try to strace of lrmd.

(1) running syslog, and strace
# service syslog start
# strace -o /tmp/strace-syslog-start.txt /usr/lib64/heartbeat/lrmd -rv

socket(PF_FILE, SOCK_DGRAM, 0) = 3
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102

(2) stop syslog and strace
# service syslog stop
# strace -o /tmp/strace-syslog-stop.txt /usr/lib64/heartbeat/lrmd -rv

socket(PF_FILE, SOCK_DGRAM, 0) = 3
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
open("/dev/console", O_WRONLY|O_NOCTTY) = 3

(for more details, see the attached files)

Is the socket of lrmd for syslog?
wild guess?


Thanks,
Junko


On Thu, 02 Jul 2009 16:09:16 +0900, Junko IKEDA <ikedaj[at]intellilink.co.jp> wrote:

>
> Hi,
>
> When I run LVM RA with Heartbeat 2.1.4,
> I could see the following message.
>
> lrmd[14994]: 2009/07/02_15:29:20 info: RA output: (lvm:monitor:stderr) File descriptor 3 (socket:[46374]) leaked on vgck invocation. Parent PID 15446: /bin/sh
>
> It seems that some LVM commands (vgdisplay or vgck) try to close the file descriptor which lrmd opened.
> lsof command showed that lrmd had socket like this;
>
> # lsof | grep lrmd | grep socket
> lrmd 14994 nobody 3u unix 0xffff81011b05f740 46374 socket
>
> (for more details, see the attached hb_report and lsof.txt)
>
> lvm commands tried to close this socket and failed. :(
> What is this socket for?
> For the communication with othre processes like crmd or ha_logd?
>
> This problem depends on the specification of lvm,
> but I just want to confirm whether the above log message could have an impact on the cluster behavior or not.
>
> This list might be the same issue.
> http://www.gossamer-threads.com/lists/linuxha/users/49687
>
> Thanks,
> Junko
Attachments: strace-syslog-start.txt (29.2 KB)
  strace-syslog-stop.txt (32.9 KB)


dejanmm at fastmail

Jul 2, 2009, 6:04 AM

Post #3 of 14 (647 views)
Permalink
Re: socket of lrmd [In reply to]

Hi,

On Thu, Jul 02, 2009 at 06:40:26PM +0900, Junko IKEDA wrote:
> Hi,
>
> I try to strace of lrmd.
>
> (1) running syslog, and strace
> # service syslog start
> # strace -o /tmp/strace-syslog-start.txt /usr/lib64/heartbeat/lrmd -rv
>
> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
>
> (2) stop syslog and strace
> # service syslog stop
> # strace -o /tmp/strace-syslog-stop.txt /usr/lib64/heartbeat/lrmd -rv
>
> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> close(3) = 0
> open("/dev/console", O_WRONLY|O_NOCTTY) = 3
>
> (for more details, see the attached files)
>
> Is the socket of lrmd for syslog?
> wild guess?

Right. See lib/lrm/racommon.c:closefiles(). I must say I can't
remember why exactly was the connection to syslogd/ha_logd
needed. There has been a problem with stonithd, but now this code
is not in that execution path anymore. Perhaps you can test this
patch:

diff -r 731f8f7b5450 lib/lrm/racommon.c
--- a/lib/lrm/racommon.c Tue Jun 30 12:02:16 2009 +0200
+++ b/lib/lrm/racommon.c Thu Jul 02 15:02:55 2009 +0200
@@ -145,7 +145,7 @@

/* close all descriptors except stdin/out/err and
* channels to logd */
for (fd = getdtablesize(); fd > STDERR_FILENO; fd--) {
- if (!cl_log_is_logd_fd(fd))
+ /*if (!cl_log_is_logd_fd(fd))*/
close(fd);
}
}

Thanks,

Dejan

>
> Thanks,
> Junko
>
>
> On Thu, 02 Jul 2009 16:09:16 +0900, Junko IKEDA <ikedaj[at]intellilink.co.jp> wrote:
>
> >
> > Hi,
> >
> > When I run LVM RA with Heartbeat 2.1.4,
> > I could see the following message.
> >
> > lrmd[14994]: 2009/07/02_15:29:20 info: RA output: (lvm:monitor:stderr) File descriptor 3 (socket:[46374]) leaked on vgck invocation. Parent PID 15446: /bin/sh
> >
> > It seems that some LVM commands (vgdisplay or vgck) try to close the file descriptor which lrmd opened.
> > lsof command showed that lrmd had socket like this;
> >
> > # lsof | grep lrmd | grep socket
> > lrmd 14994 nobody 3u unix 0xffff81011b05f740 46374 socket
> >
> > (for more details, see the attached hb_report and lsof.txt)
> >
> > lvm commands tried to close this socket and failed. :(
> > What is this socket for?
> > For the communication with othre processes like crmd or ha_logd?
> >
> > This problem depends on the specification of lvm,
> > but I just want to confirm whether the above log message could have an impact on the cluster behavior or not.
> >
> > This list might be the same issue.
> > http://www.gossamer-threads.com/lists/linuxha/users/49687
> >
> > Thanks,
> > Junko
>
>

> execve("/usr/lib64/heartbeat/lrmd", ["/usr/lib64/heartbeat/lrmd", "-rv"], [/* 24 vars */]) = 0
> brk(0) = 0x13652000
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96315000
> uname({sys="Linux", node="dl380g5d", ...}) = 0
> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
> open("/etc/ld.so.cache", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2abc96316000
> close(3) = 0
> open("/usr/lib64/liblrm.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\27\240\304;\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=317571, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc9632e000
> mmap(0x3bc4a00000, 2123232, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4a00000
> mprotect(0x3bc4a07000, 2093056, PROT_NONE) = 0
> mmap(0x3bc4c06000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x3bc4c06000
> close(3) = 0
> open("/usr/lib64/libplumb.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\254\340\302;\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=3143917, ...}) = 0
> mmap(0x3bc2e00000, 2281208, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc2e00000
> mprotect(0x3bc2e2b000, 2093056, PROT_NONE) = 0
> mmap(0x3bc302a000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2a000) = 0x3bc302a000
> close(3) = 0
> open("/usr/lib64/libpils.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@) \303;\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=309004, ...}) = 0
> mmap(0x3bc3200000, 2148672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc3200000
> mprotect(0x3bc320c000, 2097152, PROT_NONE) = 0
> mmap(0x3bc340c000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc000) = 0x3bc340c000
> close(3) = 0
> open("/usr/lib64/libapphb.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\t \304;\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=103494, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc9632f000
> mmap(0x3bc4200000, 2102920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4200000
> mprotect(0x3bc4202000, 2093056, PROT_NONE) = 0
> mmap(0x3bc4401000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x3bc4401000
> close(3) = 0
> open("/lib64/libglib-2.0.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\377\240\3161\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=647864, ...}) = 0
> mmap(0x31cea00000, 2742472, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cea00000
> mprotect(0x31cea9d000, 2093056, PROT_NONE) = 0
> mmap(0x31cec9c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9c000) = 0x31cec9c000
> close(3) = 0
> open("/usr/lib64/libbz2.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\26\0\3371\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=67664, ...}) = 0
> mmap(0x31df000000, 2160656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31df000000
> mprotect(0x31df00f000, 2093056, PROT_NONE) = 0
> mmap(0x31df20e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe000) = 0x31df20e000
> close(3) = 0
> open("/usr/lib64/libz.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\37\240\3151\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=85928, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96330000
> mmap(0x31cda00000, 2178920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cda00000
> mprotect(0x31cda14000, 2093056, PROT_NONE) = 0
> mmap(0x31cdc13000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13000) = 0x31cdc13000
> close(3) = 0
> open("/usr/lib64/libxml2.so.2", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\262\"\3231\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=1297104, ...}) = 0
> mmap(0x31d3200000, 3395256, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d3200000
> mprotect(0x31d3333000, 2097152, PROT_NONE) = 0
> mmap(0x31d3533000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x133000) = 0x31d3533000
> mmap(0x31d353c000, 3768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31d353c000
> close(3) = 0
> open("/lib64/libc.so.6", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\332\241\3141\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=1713088, ...}) = 0
> mmap(0x31cca00000, 3494168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cca00000
> mprotect(0x31ccb4c000, 2097152, PROT_NONE) = 0
> mmap(0x31ccd4c000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14c000) = 0x31ccd4c000
> mmap(0x31ccd51000, 16664, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31ccd51000
> close(3) = 0
> open("/lib64/libuuid.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\25\300\3321\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=17728, ...}) = 0
> mmap(0x31dac00000, 2110416, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31dac00000
> mprotect(0x31dac03000, 2097152, PROT_NONE) = 0
> mmap(0x31dae03000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x31dae03000
> close(3) = 0
> open("/lib64/libpam.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300$\200\3321\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=46800, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96331000
> mmap(0x31da800000, 2140040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31da800000
> mprotect(0x31da80b000, 2093056, PROT_NONE) = 0
> mmap(0x31daa0a000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa000) = 0x31daa0a000
> close(3) = 0
> open("/lib64/librt.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \"\340\3151\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=53448, ...}) = 0
> mmap(0x31cde00000, 2132936, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cde00000
> mprotect(0x31cde07000, 2097152, PROT_NONE) = 0
> mmap(0x31ce007000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x31ce007000
> close(3) = 0
> open("/lib64/libdl.so.2", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\16 \3151\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0
> mmap(0x31cd200000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd200000
> mprotect(0x31cd202000, 2097152, PROT_NONE) = 0
> mmap(0x31cd402000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x31cd402000
> close(3) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96332000
> open("/lib64/libm.so.6", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`>\340\3141\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=615136, ...}) = 0
> mmap(0x31cce00000, 2629848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cce00000
> mprotect(0x31cce82000, 2093056, PROT_NONE) = 0
> mmap(0x31cd081000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x81000) = 0x31cd081000
> close(3) = 0
> open("/lib64/libaudit.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340& \3221\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=2200168, ...}) = 0
> mmap(0x31d2200000, 2195984, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d2200000
> mprotect(0x31d2218000, 2093056, PROT_NONE) = 0
> mmap(0x31d2417000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x217000) = 0x31d2417000
> close(3) = 0
> open("/lib64/libpthread.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200W`\3151\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=145592, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96333000
> mmap(0x31cd600000, 2204528, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd600000
> mprotect(0x31cd616000, 2093056, PROT_NONE) = 0
> mmap(0x31cd815000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x31cd815000
> mmap(0x31cd817000, 13168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31cd817000
> close(3) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96334000
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96335000
> arch_prctl(ARCH_SET_FS, 0x2abc963349d0) = 0
> mprotect(0x31ccd4c000, 16384, PROT_READ) = 0
> mprotect(0x31ce007000, 4096, PROT_READ) = 0
> mprotect(0x31cd402000, 4096, PROT_READ) = 0
> mprotect(0x31cd081000, 4096, PROT_READ) = 0
> mprotect(0x31cc81b000, 4096, PROT_READ) = 0
> mprotect(0x31cd815000, 4096, PROT_READ) = 0
> munmap(0x2abc96316000, 94877) = 0
> set_tid_address(0x2abc96334a60) = 16211
> set_robust_list(0x2abc96334a70, 0x18) = 0
> futex(0x7fff1479207c, FUTEX_WAKE_PRIVATE, 1) = 0
> rt_sigaction(SIGRTMIN, {0x31cd605370, [], SA_RESTORER|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
> rt_sigaction(SIGRT_1, {0x31cd6052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
> getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
> open("/var/run/lrmd.pid", O_RDONLY) = 3
> read(3, " 15838\n", 12) = 11
> close(3) = 0
> kill(15838, SIG_0) = -1 ESRCH (No such process)
> open("/var/run/lrmd.pid", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
> read(3, " 15838\n", 12) = 11
> kill(15838, SIG_0) = -1 ESRCH (No such process)
> unlink("/var/run/lrmd.pid") = 0
> close(3) = 0
> open("/var/run/lrmd.pid.16211", O_WRONLY|O_CREAT|O_EXCL, 0644) = 3
> write(3, " 16211\n", 11) = 11
> close(3) = 0
> link("/var/run/lrmd.pid.16211", "/var/run/lrmd.pid") = 0
> stat("/var/run/lrmd.pid.16211", {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
> unlink("/var/run/lrmd.pid.16211") = 0
> umask(022) = 022
> close(0) = 0
> open("/dev/null", O_RDONLY) = 0
> close(1) = 0
> open("/dev/null", O_WRONLY) = 1
> close(2) = 0
> open("/dev/null", O_WRONLY) = 2
> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGPIPE, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> brk(0) = 0x13652000
> brk(0x13673000) = 0x13673000
> open("/etc/localtime", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96316000
> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\3\0\0\0\0"..., 4096) = 331
> lseek(3, -206, SEEK_CUR) = 125
> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0\5\0\0\0\0"..., 4096) = 206
> close(3) = 0
> munmap(0x2abc96316000, 4096) = 0
> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
> rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGINT, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGHUP, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
> access("/usr/lib64/heartbeat/plugins/InterfaceMgr/generic.so", R_OK) = -1 ENOENT (No such file or directory)
> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
> futex(0x31cd4030ec, FUTEX_WAKE_PRIVATE, 2147483647) = 0
> open("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", O_RDONLY) = 4
> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\t\0\0\0\0\0\0"..., 832) = 832
> fstat(4, {st_mode=S_IFREG|0755, st_size=85667, ...}) = 0
> mmap(NULL, 2104816, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2abc96336000
> mprotect(0x2abc96338000, 2093056, PROT_NONE) = 0
> mmap(0x2abc96537000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x1000) = 0x2abc96537000
> close(4) = 0
> open("/usr/lib64/heartbeat/plugins/RAExec", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 4
> fstat(4, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
> fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> getdents(4, /* 14 entries */, 4096) = 432
> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
> open("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", O_RDONLY) = 5
> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\23\0\0\0\0\0\0"..., 832) = 832
> fstat(5, {st_mode=S_IFREG|0755, st_size=224961, ...}) = 0
> mmap(NULL, 2110632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc96538000
> mprotect(0x2abc9653b000, 2097152, PROT_NONE) = 0
> mmap(0x2abc9673b000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x3000) = 0x2abc9673b000
> close(5) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
> open("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", O_RDONLY) = 5
> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\25\0\0\0\0\0\0"..., 832) = 832
> fstat(5, {st_mode=S_IFREG|0755, st_size=223932, ...}) = 0
> mmap(NULL, 2110696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc9673c000
> mprotect(0x2abc9673f000, 2097152, PROT_NONE) = 0
> mmap(0x2abc9693f000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x3000) = 0x2abc9693f000
> close(5) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
> open("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", O_RDONLY) = 5
> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \22\0\0\0\0\0\0"..., 832) = 832
> fstat(5, {st_mode=S_IFREG|0755, st_size=220859, ...}) = 0
> mmap(NULL, 2107944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc96940000
> mprotect(0x2abc96943000, 2093056, PROT_NONE) = 0
> mmap(0x2abc96b42000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x2abc96b42000
> close(5) = 0
> open("/etc/ld.so.cache", O_RDONLY) = 5
> fstat(5, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 5, 0) = 0x2abc96b43000
> close(5) = 0
> open("/usr/lib64/libstonithd.so.0", O_RDONLY) = 5
> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\25`\304;\0\0\0"..., 832) = 832
> fstat(5, {st_mode=S_IFREG|0755, st_size=227569, ...}) = 0
> mmap(0x3bc4600000, 2118768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x3bc4600000
> mprotect(0x3bc4605000, 2097152, PROT_NONE) = 0
> mmap(0x3bc4805000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x5000) = 0x3bc4805000
> close(5) = 0
> open("/usr/lib64/libstonith.so.1", O_RDONLY) = 5
> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\35\240\303;\0\0\0"..., 832) = 832
> fstat(5, {st_mode=S_IFREG|0755, st_size=391447, ...}) = 0
> mmap(0x3bc3a00000, 2115776, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x3bc3a00000
> mprotect(0x3bc3a05000, 2093056, PROT_NONE) = 0
> mmap(0x3bc3c04000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x4000) = 0x3bc3c04000
> close(5) = 0
> open("/usr/lib64/libhbclient.so.1", O_RDONLY) = 5
> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\35`\303;\0\0\0"..., 832) = 832
> fstat(5, {st_mode=S_IFREG|0755, st_size=308671, ...}) = 0
> mmap(0x3bc3600000, 2141672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x3bc3600000
> mprotect(0x3bc3609000, 2097152, PROT_NONE) = 0
> mmap(0x3bc3809000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x9000) = 0x3bc3809000
> close(5) = 0
> munmap(0x2abc96b43000, 94877) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
> open("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", O_RDONLY) = 5
> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\21\0\0\0\0\0\0"..., 832) = 832
> fstat(5, {st_mode=S_IFREG|0755, st_size=215232, ...}) = 0
> mmap(NULL, 2109032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc96b43000
> mprotect(0x2abc96b46000, 2093056, PROT_NONE) = 0
> mmap(0x2abc96d45000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x2abc96d45000
> close(5) = 0
> getdents(4, /* 0 entries */, 4096) = 0
> close(4) = 0
> socket(PF_FILE, SOCK_STREAM, 0) = 4
> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
> close(4) = 0
> socket(PF_FILE, SOCK_STREAM, 0) = 4
> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
> close(4) = 0
> open("/etc/nsswitch.conf", O_RDONLY) = 4
> fstat(4, {st_mode=S_IFREG|0644, st_size=1692, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
> read(4, "#\n# /etc/nsswitch.conf\n#\n# An ex"..., 4096) = 1692
> read(4, "", 4096) = 0
> close(4) = 0
> munmap(0x2abc96d46000, 4096) = 0
> open("/etc/ld.so.cache", O_RDONLY) = 4
> fstat(4, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 4, 0) = 0x2abc96d46000
> close(4) = 0
> open("/lib64/libnss_files.so.2", O_RDONLY) = 4
> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\37\0\0\0\0\0\0"..., 832) = 832
> fstat(4, {st_mode=S_IFREG|0755, st_size=53880, ...}) = 0
> mmap(NULL, 2139432, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2abc96d5e000
> mprotect(0x2abc96d68000, 2093056, PROT_NONE) = 0
> mmap(0x2abc96f67000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x9000) = 0x2abc96f67000
> close(4) = 0
> mprotect(0x2abc96f67000, 4096, PROT_READ) = 0
> munmap(0x2abc96d46000, 94877) = 0
> open("/etc/passwd", O_RDONLY) = 4
> fcntl(4, F_GETFD) = 0
> fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> fstat(4, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
> read(4, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> close(4) = 0
> munmap(0x2abc96d46000, 4096) = 0
> socket(PF_FILE, SOCK_STREAM, 0) = 4
> unlink("/var/run/heartbeat/lrm_cmd_sock") = 0
> bind(4, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_cmd_sock"...}, 110) = 0
> chmod("/var/run/heartbeat/lrm_cmd_sock", 0777) = 0
> listen(4, 10) = 0
> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> socket(PF_FILE, SOCK_STREAM, 0) = 5
> unlink("/var/run/heartbeat/lrm_callback_sock") = 0
> bind(5, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_callback_sock"...}, 110) = 0
> chmod("/var/run/heartbeat/lrm_callback_sock", 0777) = 0
> listen(5, 10) = 0
> fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGCHLD, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> write(2, "debug: Enabling coredumps\n", 26) = 26
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> sendto(3, "<31>Jul 2 17:02:46 lrmd: [.16211"..., 61, MSG_NOSIGNAL, NULL, 0) = 61
> chdir("/var/lib/heartbeat/cores") = 0
> getuid() = 0
> open("/etc/passwd", O_RDONLY) = 6
> fcntl(6, F_GETFD) = 0
> fcntl(6, F_SETFD, FD_CLOEXEC) = 0
> fstat(6, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
> read(6, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> close(6) = 0
> munmap(0x2abc96d46000, 4096) = 0
> chdir("root") = 0
> getrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=RLIM_INFINITY}) = 0
> setrlimit(RLIMIT_CORE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> open("/proc/sys/kernel/core_pattern", O_RDONLY) = 6
> read(6, "core\n", 256) = 5
> close(6) = 0
> open("/proc/sys/kernel/core_uses_pid", O_RDONLY) = 6
> read(6, "1\n", 2) = 2
> close(6) = 0
> rt_sigaction(SIGQUIT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGILL, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGABRT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGFPE, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGSEGV, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGBUS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGSYS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGTRAP, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGXCPU, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGXFSZ, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> getgid() = 0
> getuid() = 0
> open("/etc/passwd", O_RDONLY) = 6
> fcntl(6, F_GETFD) = 0
> fcntl(6, F_SETFD, FD_CLOEXEC) = 0
> fstat(6, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
> read(6, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> close(6) = 0
> munmap(0x2abc96d46000, 4096) = 0
> setresgid(-1, 99, -1) = 0
> setresuid(-1, 99, -1) = 0
> prctl(0x4, 0x1, 0, 0, 0) = 0
> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> setresuid(-1, 0, -1) = 0
> setresgid(-1, 0, -1) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> setresgid(-1, 99, -1) = 0
> setresuid(-1, 99, -1) = 0
> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGUSR1, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> setresuid(-1, 0, -1) = 0
> setresgid(-1, 0, -1) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> setresgid(-1, 99, -1) = 0
> setresuid(-1, 99, -1) = 0
> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGUSR2, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> write(2, "debug: main: run the loop...\n", 29) = 29
> setresuid(-1, 0, -1) = 0
> setresgid(-1, 0, -1) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> sendto(3, "<31>Jul 2 17:02:46 lrmd: [.16211"..., 64, MSG_NOSIGNAL, NULL, 0) = 64
> setresgid(-1, 99, -1) = 0
> setresuid(-1, 99, -1) = 0
> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> write(2, "info: Started.\n", 15) = 15
> setresuid(-1, 0, -1) = 0
> setresgid(-1, 0, -1) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 50, MSG_NOSIGNAL, NULL, 0) = 50
> setresgid(-1, 99, -1) = 0
> setresuid(-1, 99, -1) = 0
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = -1 EINTR (Interrupted system call)
> --- SIGINT (Interrupt) @ 0 (0) ---
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000

> execve("/usr/lib64/heartbeat/lrmd", ["/usr/lib64/heartbeat/lrmd", "-rv"], [/* 24 vars */]) = 0
> brk(0) = 0x199cf000
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d73000
> uname({sys="Linux", node="dl380g5d", ...}) = 0
> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
> open("/etc/ld.so.cache", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b13f0d74000
> close(3) = 0
> open("/usr/lib64/liblrm.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\27\240\304;\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=317571, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8c000
> mmap(0x3bc4a00000, 2123232, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4a00000
> mprotect(0x3bc4a07000, 2093056, PROT_NONE) = 0
> mmap(0x3bc4c06000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x3bc4c06000
> close(3) = 0
> open("/usr/lib64/libplumb.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\254\340\302;\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=3143917, ...}) = 0
> mmap(0x3bc2e00000, 2281208, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc2e00000
> mprotect(0x3bc2e2b000, 2093056, PROT_NONE) = 0
> mmap(0x3bc302a000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2a000) = 0x3bc302a000
> close(3) = 0
> open("/usr/lib64/libpils.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@) \303;\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=309004, ...}) = 0
> mmap(0x3bc3200000, 2148672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc3200000
> mprotect(0x3bc320c000, 2097152, PROT_NONE) = 0
> mmap(0x3bc340c000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc000) = 0x3bc340c000
> close(3) = 0
> open("/usr/lib64/libapphb.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\t \304;\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=103494, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8d000
> mmap(0x3bc4200000, 2102920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4200000
> mprotect(0x3bc4202000, 2093056, PROT_NONE) = 0
> mmap(0x3bc4401000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x3bc4401000
> close(3) = 0
> open("/lib64/libglib-2.0.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\377\240\3161\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=647864, ...}) = 0
> mmap(0x31cea00000, 2742472, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cea00000
> mprotect(0x31cea9d000, 2093056, PROT_NONE) = 0
> mmap(0x31cec9c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9c000) = 0x31cec9c000
> close(3) = 0
> open("/usr/lib64/libbz2.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\26\0\3371\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=67664, ...}) = 0
> mmap(0x31df000000, 2160656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31df000000
> mprotect(0x31df00f000, 2093056, PROT_NONE) = 0
> mmap(0x31df20e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe000) = 0x31df20e000
> close(3) = 0
> open("/usr/lib64/libz.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\37\240\3151\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=85928, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8e000
> mmap(0x31cda00000, 2178920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cda00000
> mprotect(0x31cda14000, 2093056, PROT_NONE) = 0
> mmap(0x31cdc13000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13000) = 0x31cdc13000
> close(3) = 0
> open("/usr/lib64/libxml2.so.2", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\262\"\3231\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=1297104, ...}) = 0
> mmap(0x31d3200000, 3395256, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d3200000
> mprotect(0x31d3333000, 2097152, PROT_NONE) = 0
> mmap(0x31d3533000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x133000) = 0x31d3533000
> mmap(0x31d353c000, 3768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31d353c000
> close(3) = 0
> open("/lib64/libc.so.6", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\332\241\3141\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=1713088, ...}) = 0
> mmap(0x31cca00000, 3494168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cca00000
> mprotect(0x31ccb4c000, 2097152, PROT_NONE) = 0
> mmap(0x31ccd4c000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14c000) = 0x31ccd4c000
> mmap(0x31ccd51000, 16664, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31ccd51000
> close(3) = 0
> open("/lib64/libuuid.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\25\300\3321\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=17728, ...}) = 0
> mmap(0x31dac00000, 2110416, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31dac00000
> mprotect(0x31dac03000, 2097152, PROT_NONE) = 0
> mmap(0x31dae03000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x31dae03000
> close(3) = 0
> open("/lib64/libpam.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300$\200\3321\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=46800, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8f000
> mmap(0x31da800000, 2140040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31da800000
> mprotect(0x31da80b000, 2093056, PROT_NONE) = 0
> mmap(0x31daa0a000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa000) = 0x31daa0a000
> close(3) = 0
> open("/lib64/librt.so.1", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \"\340\3151\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=53448, ...}) = 0
> mmap(0x31cde00000, 2132936, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cde00000
> mprotect(0x31cde07000, 2097152, PROT_NONE) = 0
> mmap(0x31ce007000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x31ce007000
> close(3) = 0
> open("/lib64/libdl.so.2", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\16 \3151\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0
> mmap(0x31cd200000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd200000
> mprotect(0x31cd202000, 2097152, PROT_NONE) = 0
> mmap(0x31cd402000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x31cd402000
> close(3) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d90000
> open("/lib64/libm.so.6", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`>\340\3141\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=615136, ...}) = 0
> mmap(0x31cce00000, 2629848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cce00000
> mprotect(0x31cce82000, 2093056, PROT_NONE) = 0
> mmap(0x31cd081000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x81000) = 0x31cd081000
> close(3) = 0
> open("/lib64/libaudit.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340& \3221\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=2200168, ...}) = 0
> mmap(0x31d2200000, 2195984, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d2200000
> mprotect(0x31d2218000, 2093056, PROT_NONE) = 0
> mmap(0x31d2417000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x217000) = 0x31d2417000
> close(3) = 0
> open("/lib64/libpthread.so.0", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200W`\3151\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=145592, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d91000
> mmap(0x31cd600000, 2204528, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd600000
> mprotect(0x31cd616000, 2093056, PROT_NONE) = 0
> mmap(0x31cd815000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x31cd815000
> mmap(0x31cd817000, 13168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31cd817000
> close(3) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d92000
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d93000
> arch_prctl(ARCH_SET_FS, 0x2b13f0d929d0) = 0
> mprotect(0x31ccd4c000, 16384, PROT_READ) = 0
> mprotect(0x31ce007000, 4096, PROT_READ) = 0
> mprotect(0x31cd402000, 4096, PROT_READ) = 0
> mprotect(0x31cd081000, 4096, PROT_READ) = 0
> mprotect(0x31cc81b000, 4096, PROT_READ) = 0
> mprotect(0x31cd815000, 4096, PROT_READ) = 0
> munmap(0x2b13f0d74000, 94877) = 0
> set_tid_address(0x2b13f0d92a60) = 16907
> set_robust_list(0x2b13f0d92a70, 0x18) = 0
> futex(0x7fffb9d3661c, FUTEX_WAKE_PRIVATE, 1) = 0
> rt_sigaction(SIGRTMIN, {0x31cd605370, [], SA_RESTORER|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
> rt_sigaction(SIGRT_1, {0x31cd6052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
> getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
> open("/var/run/lrmd.pid", O_RDONLY) = 3
> read(3, " 16211\n", 12) = 11
> close(3) = 0
> kill(16211, SIG_0) = -1 ESRCH (No such process)
> open("/var/run/lrmd.pid", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
> read(3, " 16211\n", 12) = 11
> kill(16211, SIG_0) = -1 ESRCH (No such process)
> unlink("/var/run/lrmd.pid") = 0
> close(3) = 0
> open("/var/run/lrmd.pid.16907", O_WRONLY|O_CREAT|O_EXCL, 0644) = 3
> write(3, " 16907\n", 11) = 11
> close(3) = 0
> link("/var/run/lrmd.pid.16907", "/var/run/lrmd.pid") = 0
> stat("/var/run/lrmd.pid.16907", {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
> unlink("/var/run/lrmd.pid.16907") = 0
> umask(022) = 022
> close(0) = 0
> open("/dev/null", O_RDONLY) = 0
> close(1) = 0
> open("/dev/null", O_WRONLY) = 1
> close(2) = 0
> open("/dev/null", O_WRONLY) = 2
> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGPIPE, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> brk(0) = 0x199cf000
> brk(0x199f0000) = 0x199f0000
> open("/etc/localtime", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d74000
> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\3\0\0\0\0"..., 4096) = 331
> lseek(3, -206, SEEK_CUR) = 125
> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0\5\0\0\0\0"..., 4096) = 206
> close(3) = 0
> munmap(0x2b13f0d74000, 4096) = 0
> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> close(3) = 0
> open("/dev/console", O_WRONLY|O_NOCTTY) = 3
> fstat(3, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d74000
> lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> write(3, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
> munmap(0x2b13f0d74000, 4096) = 0
> close(3) = 0
> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
> rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGINT, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGHUP, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
> access("/usr/lib64/heartbeat/plugins/InterfaceMgr/generic.so", R_OK) = -1 ENOENT (No such file or directory)
> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
> futex(0x31cd4030ec, FUTEX_WAKE_PRIVATE, 2147483647) = 0
> open("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\t\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=85667, ...}) = 0
> mmap(NULL, 2104816, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x2b13f0d94000
> mprotect(0x2b13f0d96000, 2093056, PROT_NONE) = 0
> mmap(0x2b13f0f95000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x2b13f0f95000
> close(3) = 0
> open("/usr/lib64/heartbeat/plugins/RAExec", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 3
> fstat(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> getdents(3, /* 14 entries */, 4096) = 432
> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
> open("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", O_RDONLY) = 4
> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\23\0\0\0\0\0\0"..., 832) = 832
> fstat(4, {st_mode=S_IFREG|0755, st_size=224961, ...}) = 0
> mmap(NULL, 2110632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f0f96000
> mprotect(0x2b13f0f99000, 2097152, PROT_NONE) = 0
> mmap(0x2b13f1199000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x3000) = 0x2b13f1199000
> close(4) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
> open("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", O_RDONLY) = 4
> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\25\0\0\0\0\0\0"..., 832) = 832
> fstat(4, {st_mode=S_IFREG|0755, st_size=223932, ...}) = 0
> mmap(NULL, 2110696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f119a000
> mprotect(0x2b13f119d000, 2097152, PROT_NONE) = 0
> mmap(0x2b13f139d000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x3000) = 0x2b13f139d000
> close(4) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
> open("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", O_RDONLY) = 4
> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \22\0\0\0\0\0\0"..., 832) = 832
> fstat(4, {st_mode=S_IFREG|0755, st_size=220859, ...}) = 0
> mmap(NULL, 2107944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f139e000
> mprotect(0x2b13f13a1000, 2093056, PROT_NONE) = 0
> mmap(0x2b13f15a0000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x2000) = 0x2b13f15a0000
> close(4) = 0
> open("/etc/ld.so.cache", O_RDONLY) = 4
> fstat(4, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 4, 0) = 0x2b13f15a1000
> close(4) = 0
> open("/usr/lib64/libstonithd.so.0", O_RDONLY) = 4
> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\25`\304;\0\0\0"..., 832) = 832
> fstat(4, {st_mode=S_IFREG|0755, st_size=227569, ...}) = 0
> mmap(0x3bc4600000, 2118768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x3bc4600000
> mprotect(0x3bc4605000, 2097152, PROT_NONE) = 0
> mmap(0x3bc4805000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x5000) = 0x3bc4805000
> close(4) = 0
> open("/usr/lib64/libstonith.so.1", O_RDONLY) = 4
> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\35\240\303;\0\0\0"..., 832) = 832
> fstat(4, {st_mode=S_IFREG|0755, st_size=391447, ...}) = 0
> mmap(0x3bc3a00000, 2115776, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x3bc3a00000
> mprotect(0x3bc3a05000, 2093056, PROT_NONE) = 0
> mmap(0x3bc3c04000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x4000) = 0x3bc3c04000
> close(4) = 0
> open("/usr/lib64/libhbclient.so.1", O_RDONLY) = 4
> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\35`\303;\0\0\0"..., 832) = 832
> fstat(4, {st_mode=S_IFREG|0755, st_size=308671, ...}) = 0
> mmap(0x3bc3600000, 2141672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x3bc3600000
> mprotect(0x3bc3609000, 2097152, PROT_NONE) = 0
> mmap(0x3bc3809000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x9000) = 0x3bc3809000
> close(4) = 0
> munmap(0x2b13f15a1000, 94877) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
> open("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", O_RDONLY) = 4
> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\21\0\0\0\0\0\0"..., 832) = 832
> fstat(4, {st_mode=S_IFREG|0755, st_size=215232, ...}) = 0
> mmap(NULL, 2109032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f15a1000
> mprotect(0x2b13f15a4000, 2093056, PROT_NONE) = 0
> mmap(0x2b13f17a3000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x2000) = 0x2b13f17a3000
> close(4) = 0
> getdents(3, /* 0 entries */, 4096) = 0
> close(3) = 0
> socket(PF_FILE, SOCK_STREAM, 0) = 3
> fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
> close(3) = 0
> socket(PF_FILE, SOCK_STREAM, 0) = 3
> fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
> close(3) = 0
> open("/etc/nsswitch.conf", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=1692, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> read(3, "#\n# /etc/nsswitch.conf\n#\n# An ex"..., 4096) = 1692
> read(3, "", 4096) = 0
> close(3) = 0
> munmap(0x2b13f17a4000, 4096) = 0
> open("/etc/ld.so.cache", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b13f17a4000
> close(3) = 0
> open("/lib64/libnss_files.so.2", O_RDONLY) = 3
> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\37\0\0\0\0\0\0"..., 832) = 832
> fstat(3, {st_mode=S_IFREG|0755, st_size=53880, ...}) = 0
> mmap(NULL, 2139432, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x2b13f17bc000
> mprotect(0x2b13f17c6000, 2093056, PROT_NONE) = 0
> mmap(0x2b13f19c5000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9000) = 0x2b13f19c5000
> close(3) = 0
> mprotect(0x2b13f19c5000, 4096, PROT_READ) = 0
> munmap(0x2b13f17a4000, 94877) = 0
> open("/etc/passwd", O_RDONLY) = 3
> fcntl(3, F_GETFD) = 0
> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> fstat(3, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> read(3, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> close(3) = 0
> munmap(0x2b13f17a4000, 4096) = 0
> socket(PF_FILE, SOCK_STREAM, 0) = 3
> unlink("/var/run/heartbeat/lrm_cmd_sock") = 0
> bind(3, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_cmd_sock"...}, 110) = 0
> chmod("/var/run/heartbeat/lrm_cmd_sock", 0777) = 0
> listen(3, 10) = 0
> fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> socket(PF_FILE, SOCK_STREAM, 0) = 4
> unlink("/var/run/heartbeat/lrm_callback_sock") = 0
> bind(4, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_callback_sock"...}, 110) = 0
> chmod("/var/run/heartbeat/lrm_callback_sock", 0777) = 0
> listen(4, 10) = 0
> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> close(5) = 0
> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> write(5, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
> munmap(0x2b13f17a4000, 4096) = 0
> close(5) = 0
> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGCHLD, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> write(2, "debug: Enabling coredumps\n", 26) = 26
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> close(5) = 0
> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> write(5, "lrmd: [16907]: debug: Enabling c"..., 42) = 42
> munmap(0x2b13f17a4000, 4096) = 0
> close(5) = 0
> chdir("/var/lib/heartbeat/cores") = 0
> getuid() = 0
> open("/etc/passwd", O_RDONLY) = 5
> fcntl(5, F_GETFD) = 0
> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> fstat(5, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> read(5, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> close(5) = 0
> munmap(0x2b13f17a4000, 4096) = 0
> chdir("root") = 0
> getrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=RLIM_INFINITY}) = 0
> setrlimit(RLIMIT_CORE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> open("/proc/sys/kernel/core_pattern", O_RDONLY) = 5
> read(5, "core\n", 256) = 5
> close(5) = 0
> open("/proc/sys/kernel/core_uses_pid", O_RDONLY) = 5
> read(5, "1\n", 2) = 2
> close(5) = 0
> rt_sigaction(SIGQUIT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGILL, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGABRT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGFPE, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGSEGV, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGBUS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGSYS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGTRAP, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGXCPU, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGXFSZ, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> getgid() = 0
> getuid() = 0
> open("/etc/passwd", O_RDONLY) = 5
> fcntl(5, F_GETFD) = 0
> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> fstat(5, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> read(5, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> close(5) = 0
> munmap(0x2b13f17a4000, 4096) = 0
> setresgid(-1, 99, -1) = 0
> setresuid(-1, 99, -1) = 0
> prctl(0x4, 0x1, 0, 0, 0) = 0
> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> setresuid(-1, 0, -1) = 0
> setresgid(-1, 0, -1) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> close(5) = 0
> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> write(5, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
> munmap(0x2b13f17a4000, 4096) = 0
> close(5) = 0
> setresgid(-1, 99, -1) = 0
> setresuid(-1, 99, -1) = 0
> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGUSR1, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> setresuid(-1, 0, -1) = 0
> setresgid(-1, 0, -1) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> close(5) = 0
> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> write(5, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
> munmap(0x2b13f17a4000, 4096) = 0
> close(5) = 0
> setresgid(-1, 99, -1) = 0
> setresuid(-1, 99, -1) = 0
> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> rt_sigaction(SIGUSR2, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> write(2, "debug: main: run the loop...\n", 29) = 29
> setresuid(-1, 0, -1) = 0
> setresgid(-1, 0, -1) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> close(5) = 0
> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> write(5, "lrmd: [16907]: debug: main: run "..., 45) = 45
> munmap(0x2b13f17a4000, 4096) = 0
> close(5) = 0
> setresgid(-1, 99, -1) = 0
> setresuid(-1, 99, -1) = 0
> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> write(2, "info: Started.\n", 15) = 15
> setresuid(-1, 0, -1) = 0
> setresgid(-1, 0, -1) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> close(5) = 0
> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> write(5, "lrmd: [16907]: info: Started.\r\n", 31) = 31
> munmap(0x2b13f17a4000, 4096) = 0
> close(5) = 0
> setresgid(-1, 99, -1) = 0
> setresuid(-1, 99, -1) = 0
> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = -1 EINTR (Interrupted system call)
> --- SIGINT (Interrupt) @ 0 (0) ---
> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000
> _______________________________________________
> Linux-HA mailing list
> Linux-HA[at]lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


ikedaj at intellilink

Jul 2, 2009, 6:59 PM

Post #4 of 14 (640 views)
Permalink
Re: socket of lrmd [In reply to]

Hi Dejan,

Your patch could stop the error message from LVM RA.
Many thanks!
But I run Heartbeat 2.1.4 so I worry about whether 2.1.4 still have a problem about stonithd that you pointed.

By the way, does closefiles() close both of the connections for syslogd and ha_logd ?

Thanks,
Junko


On Thu, 02 Jul 2009 22:04:06 +0900, Dejan Muhamedagic <dejanmm[at]fastmail.fm> wrote:

> Hi,
>
> On Thu, Jul 02, 2009 at 06:40:26PM +0900, Junko IKEDA wrote:
>> Hi,
>>
>> I try to strace of lrmd.
>>
>> (1) running syslog, and strace
>> # service syslog start
>> # strace -o /tmp/strace-syslog-start.txt /usr/lib64/heartbeat/lrmd -rv
>>
>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
>> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
>>
>> (2) stop syslog and strace
>> # service syslog stop
>> # strace -o /tmp/strace-syslog-stop.txt /usr/lib64/heartbeat/lrmd -rv
>>
>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
>> close(3) = 0
>> open("/dev/console", O_WRONLY|O_NOCTTY) = 3
>>
>> (for more details, see the attached files)
>>
>> Is the socket of lrmd for syslog?
>> wild guess?
>
> Right. See lib/lrm/racommon.c:closefiles(). I must say I can't
> remember why exactly was the connection to syslogd/ha_logd
> needed. There has been a problem with stonithd, but now this code
> is not in that execution path anymore. Perhaps you can test this
> patch:
>
> diff -r 731f8f7b5450 lib/lrm/racommon.c
> --- a/lib/lrm/racommon.c Tue Jun 30 12:02:16 2009 +0200
> +++ b/lib/lrm/racommon.c Thu Jul 02 15:02:55 2009 +0200
> @@ -145,7 +145,7 @@
>
> /* close all descriptors except stdin/out/err and
> * channels to logd */
> for (fd = getdtablesize(); fd > STDERR_FILENO; fd--) {
> - if (!cl_log_is_logd_fd(fd))
> + /*if (!cl_log_is_logd_fd(fd))*/
> close(fd);
> }
> }
>
> Thanks,
>
> Dejan
>
>>
>> Thanks,
>> Junko
>>
>>
>> On Thu, 02 Jul 2009 16:09:16 +0900, Junko IKEDA <ikedaj[at]intellilink.co.jp> wrote:
>>
>> >
>> > Hi,
>> >
>> > When I run LVM RA with Heartbeat 2.1.4,
>> > I could see the following message.
>> >
>> > lrmd[14994]: 2009/07/02_15:29:20 info: RA output: (lvm:monitor:stderr) File descriptor 3 (socket:[46374]) leaked on vgck invocation. Parent PID 15446: /bin/sh
>> >
>> > It seems that some LVM commands (vgdisplay or vgck) try to close the file descriptor which lrmd opened.
>> > lsof command showed that lrmd had socket like this;
>> >
>> > # lsof | grep lrmd | grep socket
>> > lrmd 14994 nobody 3u unix 0xffff81011b05f740 46374 socket
>> >
>> > (for more details, see the attached hb_report and lsof.txt)
>> >
>> > lvm commands tried to close this socket and failed. :(
>> > What is this socket for?
>> > For the communication with othre processes like crmd or ha_logd?
>> >
>> > This problem depends on the specification of lvm,
>> > but I just want to confirm whether the above log message could have an impact on the cluster behavior or not.
>> >
>> > This list might be the same issue.
>> > http://www.gossamer-threads.com/lists/linuxha/users/49687
>> >
>> > Thanks,
>> > Junko
>>
>>
>
>> execve("/usr/lib64/heartbeat/lrmd", ["/usr/lib64/heartbeat/lrmd", "-rv"], [/* 24 vars */]) = 0
>> brk(0) = 0x13652000
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96315000
>> uname({sys="Linux", node="dl380g5d", ...}) = 0
>> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
>> open("/etc/ld.so.cache", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
>> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2abc96316000
>> close(3) = 0
>> open("/usr/lib64/liblrm.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\27\240\304;\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=317571, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc9632e000
>> mmap(0x3bc4a00000, 2123232, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4a00000
>> mprotect(0x3bc4a07000, 2093056, PROT_NONE) = 0
>> mmap(0x3bc4c06000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x3bc4c06000
>> close(3) = 0
>> open("/usr/lib64/libplumb.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\254\340\302;\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=3143917, ...}) = 0
>> mmap(0x3bc2e00000, 2281208, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc2e00000
>> mprotect(0x3bc2e2b000, 2093056, PROT_NONE) = 0
>> mmap(0x3bc302a000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2a000) = 0x3bc302a000
>> close(3) = 0
>> open("/usr/lib64/libpils.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@) \303;\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=309004, ...}) = 0
>> mmap(0x3bc3200000, 2148672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc3200000
>> mprotect(0x3bc320c000, 2097152, PROT_NONE) = 0
>> mmap(0x3bc340c000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc000) = 0x3bc340c000
>> close(3) = 0
>> open("/usr/lib64/libapphb.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\t \304;\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=103494, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc9632f000
>> mmap(0x3bc4200000, 2102920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4200000
>> mprotect(0x3bc4202000, 2093056, PROT_NONE) = 0
>> mmap(0x3bc4401000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x3bc4401000
>> close(3) = 0
>> open("/lib64/libglib-2.0.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\377\240\3161\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=647864, ...}) = 0
>> mmap(0x31cea00000, 2742472, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cea00000
>> mprotect(0x31cea9d000, 2093056, PROT_NONE) = 0
>> mmap(0x31cec9c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9c000) = 0x31cec9c000
>> close(3) = 0
>> open("/usr/lib64/libbz2.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\26\0\3371\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=67664, ...}) = 0
>> mmap(0x31df000000, 2160656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31df000000
>> mprotect(0x31df00f000, 2093056, PROT_NONE) = 0
>> mmap(0x31df20e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe000) = 0x31df20e000
>> close(3) = 0
>> open("/usr/lib64/libz.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\37\240\3151\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=85928, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96330000
>> mmap(0x31cda00000, 2178920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cda00000
>> mprotect(0x31cda14000, 2093056, PROT_NONE) = 0
>> mmap(0x31cdc13000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13000) = 0x31cdc13000
>> close(3) = 0
>> open("/usr/lib64/libxml2.so.2", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\262\"\3231\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=1297104, ...}) = 0
>> mmap(0x31d3200000, 3395256, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d3200000
>> mprotect(0x31d3333000, 2097152, PROT_NONE) = 0
>> mmap(0x31d3533000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x133000) = 0x31d3533000
>> mmap(0x31d353c000, 3768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31d353c000
>> close(3) = 0
>> open("/lib64/libc.so.6", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\332\241\3141\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=1713088, ...}) = 0
>> mmap(0x31cca00000, 3494168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cca00000
>> mprotect(0x31ccb4c000, 2097152, PROT_NONE) = 0
>> mmap(0x31ccd4c000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14c000) = 0x31ccd4c000
>> mmap(0x31ccd51000, 16664, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31ccd51000
>> close(3) = 0
>> open("/lib64/libuuid.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\25\300\3321\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=17728, ...}) = 0
>> mmap(0x31dac00000, 2110416, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31dac00000
>> mprotect(0x31dac03000, 2097152, PROT_NONE) = 0
>> mmap(0x31dae03000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x31dae03000
>> close(3) = 0
>> open("/lib64/libpam.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300$\200\3321\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=46800, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96331000
>> mmap(0x31da800000, 2140040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31da800000
>> mprotect(0x31da80b000, 2093056, PROT_NONE) = 0
>> mmap(0x31daa0a000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa000) = 0x31daa0a000
>> close(3) = 0
>> open("/lib64/librt.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \"\340\3151\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=53448, ...}) = 0
>> mmap(0x31cde00000, 2132936, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cde00000
>> mprotect(0x31cde07000, 2097152, PROT_NONE) = 0
>> mmap(0x31ce007000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x31ce007000
>> close(3) = 0
>> open("/lib64/libdl.so.2", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\16 \3151\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0
>> mmap(0x31cd200000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd200000
>> mprotect(0x31cd202000, 2097152, PROT_NONE) = 0
>> mmap(0x31cd402000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x31cd402000
>> close(3) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96332000
>> open("/lib64/libm.so.6", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`>\340\3141\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=615136, ...}) = 0
>> mmap(0x31cce00000, 2629848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cce00000
>> mprotect(0x31cce82000, 2093056, PROT_NONE) = 0
>> mmap(0x31cd081000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x81000) = 0x31cd081000
>> close(3) = 0
>> open("/lib64/libaudit.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340& \3221\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=2200168, ...}) = 0
>> mmap(0x31d2200000, 2195984, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d2200000
>> mprotect(0x31d2218000, 2093056, PROT_NONE) = 0
>> mmap(0x31d2417000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x217000) = 0x31d2417000
>> close(3) = 0
>> open("/lib64/libpthread.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200W`\3151\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=145592, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96333000
>> mmap(0x31cd600000, 2204528, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd600000
>> mprotect(0x31cd616000, 2093056, PROT_NONE) = 0
>> mmap(0x31cd815000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x31cd815000
>> mmap(0x31cd817000, 13168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31cd817000
>> close(3) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96334000
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96335000
>> arch_prctl(ARCH_SET_FS, 0x2abc963349d0) = 0
>> mprotect(0x31ccd4c000, 16384, PROT_READ) = 0
>> mprotect(0x31ce007000, 4096, PROT_READ) = 0
>> mprotect(0x31cd402000, 4096, PROT_READ) = 0
>> mprotect(0x31cd081000, 4096, PROT_READ) = 0
>> mprotect(0x31cc81b000, 4096, PROT_READ) = 0
>> mprotect(0x31cd815000, 4096, PROT_READ) = 0
>> munmap(0x2abc96316000, 94877) = 0
>> set_tid_address(0x2abc96334a60) = 16211
>> set_robust_list(0x2abc96334a70, 0x18) = 0
>> futex(0x7fff1479207c, FUTEX_WAKE_PRIVATE, 1) = 0
>> rt_sigaction(SIGRTMIN, {0x31cd605370, [], SA_RESTORER|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
>> rt_sigaction(SIGRT_1, {0x31cd6052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
>> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
>> getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
>> open("/var/run/lrmd.pid", O_RDONLY) = 3
>> read(3, " 15838\n", 12) = 11
>> close(3) = 0
>> kill(15838, SIG_0) = -1 ESRCH (No such process)
>> open("/var/run/lrmd.pid", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
>> read(3, " 15838\n", 12) = 11
>> kill(15838, SIG_0) = -1 ESRCH (No such process)
>> unlink("/var/run/lrmd.pid") = 0
>> close(3) = 0
>> open("/var/run/lrmd.pid.16211", O_WRONLY|O_CREAT|O_EXCL, 0644) = 3
>> write(3, " 16211\n", 11) = 11
>> close(3) = 0
>> link("/var/run/lrmd.pid.16211", "/var/run/lrmd.pid") = 0
>> stat("/var/run/lrmd.pid.16211", {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
>> unlink("/var/run/lrmd.pid.16211") = 0
>> umask(022) = 022
>> close(0) = 0
>> open("/dev/null", O_RDONLY) = 0
>> close(1) = 0
>> open("/dev/null", O_WRONLY) = 1
>> close(2) = 0
>> open("/dev/null", O_WRONLY) = 2
>> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGPIPE, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> brk(0) = 0x13652000
>> brk(0x13673000) = 0x13673000
>> open("/etc/localtime", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96316000
>> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\3\0\0\0\0"..., 4096) = 331
>> lseek(3, -206, SEEK_CUR) = 125
>> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0\5\0\0\0\0"..., 4096) = 206
>> close(3) = 0
>> munmap(0x2abc96316000, 4096) = 0
>> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
>> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
>> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
>> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
>> rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGINT, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGHUP, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
>> access("/usr/lib64/heartbeat/plugins/InterfaceMgr/generic.so", R_OK) = -1 ENOENT (No such file or directory)
>> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
>> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
>> futex(0x31cd4030ec, FUTEX_WAKE_PRIVATE, 2147483647) = 0
>> open("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", O_RDONLY) = 4
>> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\t\0\0\0\0\0\0"..., 832) = 832
>> fstat(4, {st_mode=S_IFREG|0755, st_size=85667, ...}) = 0
>> mmap(NULL, 2104816, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2abc96336000
>> mprotect(0x2abc96338000, 2093056, PROT_NONE) = 0
>> mmap(0x2abc96537000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x1000) = 0x2abc96537000
>> close(4) = 0
>> open("/usr/lib64/heartbeat/plugins/RAExec", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 4
>> fstat(4, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
>> fcntl(4, F_SETFD, FD_CLOEXEC) = 0
>> getdents(4, /* 14 entries */, 4096) = 432
>> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
>> open("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", O_RDONLY) = 5
>> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\23\0\0\0\0\0\0"..., 832) = 832
>> fstat(5, {st_mode=S_IFREG|0755, st_size=224961, ...}) = 0
>> mmap(NULL, 2110632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc96538000
>> mprotect(0x2abc9653b000, 2097152, PROT_NONE) = 0
>> mmap(0x2abc9673b000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x3000) = 0x2abc9673b000
>> close(5) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
>> open("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", O_RDONLY) = 5
>> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\25\0\0\0\0\0\0"..., 832) = 832
>> fstat(5, {st_mode=S_IFREG|0755, st_size=223932, ...}) = 0
>> mmap(NULL, 2110696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc9673c000
>> mprotect(0x2abc9673f000, 2097152, PROT_NONE) = 0
>> mmap(0x2abc9693f000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x3000) = 0x2abc9693f000
>> close(5) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
>> open("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", O_RDONLY) = 5
>> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \22\0\0\0\0\0\0"..., 832) = 832
>> fstat(5, {st_mode=S_IFREG|0755, st_size=220859, ...}) = 0
>> mmap(NULL, 2107944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc96940000
>> mprotect(0x2abc96943000, 2093056, PROT_NONE) = 0
>> mmap(0x2abc96b42000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x2abc96b42000
>> close(5) = 0
>> open("/etc/ld.so.cache", O_RDONLY) = 5
>> fstat(5, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
>> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 5, 0) = 0x2abc96b43000
>> close(5) = 0
>> open("/usr/lib64/libstonithd.so.0", O_RDONLY) = 5
>> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\25`\304;\0\0\0"..., 832) = 832
>> fstat(5, {st_mode=S_IFREG|0755, st_size=227569, ...}) = 0
>> mmap(0x3bc4600000, 2118768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x3bc4600000
>> mprotect(0x3bc4605000, 2097152, PROT_NONE) = 0
>> mmap(0x3bc4805000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x5000) = 0x3bc4805000
>> close(5) = 0
>> open("/usr/lib64/libstonith.so.1", O_RDONLY) = 5
>> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\35\240\303;\0\0\0"..., 832) = 832
>> fstat(5, {st_mode=S_IFREG|0755, st_size=391447, ...}) = 0
>> mmap(0x3bc3a00000, 2115776, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x3bc3a00000
>> mprotect(0x3bc3a05000, 2093056, PROT_NONE) = 0
>> mmap(0x3bc3c04000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x4000) = 0x3bc3c04000
>> close(5) = 0
>> open("/usr/lib64/libhbclient.so.1", O_RDONLY) = 5
>> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\35`\303;\0\0\0"..., 832) = 832
>> fstat(5, {st_mode=S_IFREG|0755, st_size=308671, ...}) = 0
>> mmap(0x3bc3600000, 2141672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x3bc3600000
>> mprotect(0x3bc3609000, 2097152, PROT_NONE) = 0
>> mmap(0x3bc3809000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x9000) = 0x3bc3809000
>> close(5) = 0
>> munmap(0x2abc96b43000, 94877) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
>> open("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", O_RDONLY) = 5
>> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\21\0\0\0\0\0\0"..., 832) = 832
>> fstat(5, {st_mode=S_IFREG|0755, st_size=215232, ...}) = 0
>> mmap(NULL, 2109032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc96b43000
>> mprotect(0x2abc96b46000, 2093056, PROT_NONE) = 0
>> mmap(0x2abc96d45000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x2abc96d45000
>> close(5) = 0
>> getdents(4, /* 0 entries */, 4096) = 0
>> close(4) = 0
>> socket(PF_FILE, SOCK_STREAM, 0) = 4
>> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
>> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
>> connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
>> close(4) = 0
>> socket(PF_FILE, SOCK_STREAM, 0) = 4
>> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
>> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
>> connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
>> close(4) = 0
>> open("/etc/nsswitch.conf", O_RDONLY) = 4
>> fstat(4, {st_mode=S_IFREG|0644, st_size=1692, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
>> read(4, "#\n# /etc/nsswitch.conf\n#\n# An ex"..., 4096) = 1692
>> read(4, "", 4096) = 0
>> close(4) = 0
>> munmap(0x2abc96d46000, 4096) = 0
>> open("/etc/ld.so.cache", O_RDONLY) = 4
>> fstat(4, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
>> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 4, 0) = 0x2abc96d46000
>> close(4) = 0
>> open("/lib64/libnss_files.so.2", O_RDONLY) = 4
>> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\37\0\0\0\0\0\0"..., 832) = 832
>> fstat(4, {st_mode=S_IFREG|0755, st_size=53880, ...}) = 0
>> mmap(NULL, 2139432, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2abc96d5e000
>> mprotect(0x2abc96d68000, 2093056, PROT_NONE) = 0
>> mmap(0x2abc96f67000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x9000) = 0x2abc96f67000
>> close(4) = 0
>> mprotect(0x2abc96f67000, 4096, PROT_READ) = 0
>> munmap(0x2abc96d46000, 94877) = 0
>> open("/etc/passwd", O_RDONLY) = 4
>> fcntl(4, F_GETFD) = 0
>> fcntl(4, F_SETFD, FD_CLOEXEC) = 0
>> fstat(4, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
>> read(4, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
>> close(4) = 0
>> munmap(0x2abc96d46000, 4096) = 0
>> socket(PF_FILE, SOCK_STREAM, 0) = 4
>> unlink("/var/run/heartbeat/lrm_cmd_sock") = 0
>> bind(4, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_cmd_sock"...}, 110) = 0
>> chmod("/var/run/heartbeat/lrm_cmd_sock", 0777) = 0
>> listen(4, 10) = 0
>> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
>> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
>> socket(PF_FILE, SOCK_STREAM, 0) = 5
>> unlink("/var/run/heartbeat/lrm_callback_sock") = 0
>> bind(5, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_callback_sock"...}, 110) = 0
>> chmod("/var/run/heartbeat/lrm_callback_sock", 0777) = 0
>> listen(5, 10) = 0
>> fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
>> fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
>> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
>> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
>> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGCHLD, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
>> write(2, "debug: Enabling coredumps\n", 26) = 26
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> sendto(3, "<31>Jul 2 17:02:46 lrmd: [.16211"..., 61, MSG_NOSIGNAL, NULL, 0) = 61
>> chdir("/var/lib/heartbeat/cores") = 0
>> getuid() = 0
>> open("/etc/passwd", O_RDONLY) = 6
>> fcntl(6, F_GETFD) = 0
>> fcntl(6, F_SETFD, FD_CLOEXEC) = 0
>> fstat(6, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
>> read(6, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
>> close(6) = 0
>> munmap(0x2abc96d46000, 4096) = 0
>> chdir("root") = 0
>> getrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=RLIM_INFINITY}) = 0
>> setrlimit(RLIMIT_CORE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
>> open("/proc/sys/kernel/core_pattern", O_RDONLY) = 6
>> read(6, "core\n", 256) = 5
>> close(6) = 0
>> open("/proc/sys/kernel/core_uses_pid", O_RDONLY) = 6
>> read(6, "1\n", 2) = 2
>> close(6) = 0
>> rt_sigaction(SIGQUIT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGILL, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGABRT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGFPE, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGSEGV, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGBUS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGSYS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGTRAP, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGXCPU, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGXFSZ, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> getgid() = 0
>> getuid() = 0
>> open("/etc/passwd", O_RDONLY) = 6
>> fcntl(6, F_GETFD) = 0
>> fcntl(6, F_SETFD, FD_CLOEXEC) = 0
>> fstat(6, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
>> read(6, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
>> close(6) = 0
>> munmap(0x2abc96d46000, 4096) = 0
>> setresgid(-1, 99, -1) = 0
>> setresuid(-1, 99, -1) = 0
>> prctl(0x4, 0x1, 0, 0, 0) = 0
>> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
>> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
>> setresuid(-1, 0, -1) = 0
>> setresgid(-1, 0, -1) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
>> setresgid(-1, 99, -1) = 0
>> setresuid(-1, 99, -1) = 0
>> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGUSR1, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
>> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
>> setresuid(-1, 0, -1) = 0
>> setresgid(-1, 0, -1) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
>> setresgid(-1, 99, -1) = 0
>> setresuid(-1, 99, -1) = 0
>> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGUSR2, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
>> write(2, "debug: main: run the loop...\n", 29) = 29
>> setresuid(-1, 0, -1) = 0
>> setresgid(-1, 0, -1) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> sendto(3, "<31>Jul 2 17:02:46 lrmd: [.16211"..., 64, MSG_NOSIGNAL, NULL, 0) = 64
>> setresgid(-1, 99, -1) = 0
>> setresuid(-1, 99, -1) = 0
>> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
>> write(2, "info: Started.\n", 15) = 15
>> setresuid(-1, 0, -1) = 0
>> setresgid(-1, 0, -1) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 50, MSG_NOSIGNAL, NULL, 0) = 50
>> setresgid(-1, 99, -1) = 0
>> setresuid(-1, 99, -1) = 0
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = -1 EINTR (Interrupted system call)
>> --- SIGINT (Interrupt) @ 0 (0) ---
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000
>
>> execve("/usr/lib64/heartbeat/lrmd", ["/usr/lib64/heartbeat/lrmd", "-rv"], [/* 24 vars */]) = 0
>> brk(0) = 0x199cf000
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d73000
>> uname({sys="Linux", node="dl380g5d", ...}) = 0
>> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
>> open("/etc/ld.so.cache", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
>> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b13f0d74000
>> close(3) = 0
>> open("/usr/lib64/liblrm.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\27\240\304;\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=317571, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8c000
>> mmap(0x3bc4a00000, 2123232, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4a00000
>> mprotect(0x3bc4a07000, 2093056, PROT_NONE) = 0
>> mmap(0x3bc4c06000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x3bc4c06000
>> close(3) = 0
>> open("/usr/lib64/libplumb.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\254\340\302;\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=3143917, ...}) = 0
>> mmap(0x3bc2e00000, 2281208, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc2e00000
>> mprotect(0x3bc2e2b000, 2093056, PROT_NONE) = 0
>> mmap(0x3bc302a000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2a000) = 0x3bc302a000
>> close(3) = 0
>> open("/usr/lib64/libpils.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@) \303;\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=309004, ...}) = 0
>> mmap(0x3bc3200000, 2148672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc3200000
>> mprotect(0x3bc320c000, 2097152, PROT_NONE) = 0
>> mmap(0x3bc340c000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc000) = 0x3bc340c000
>> close(3) = 0
>> open("/usr/lib64/libapphb.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\t \304;\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=103494, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8d000
>> mmap(0x3bc4200000, 2102920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4200000
>> mprotect(0x3bc4202000, 2093056, PROT_NONE) = 0
>> mmap(0x3bc4401000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x3bc4401000
>> close(3) = 0
>> open("/lib64/libglib-2.0.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\377\240\3161\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=647864, ...}) = 0
>> mmap(0x31cea00000, 2742472, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cea00000
>> mprotect(0x31cea9d000, 2093056, PROT_NONE) = 0
>> mmap(0x31cec9c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9c000) = 0x31cec9c000
>> close(3) = 0
>> open("/usr/lib64/libbz2.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\26\0\3371\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=67664, ...}) = 0
>> mmap(0x31df000000, 2160656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31df000000
>> mprotect(0x31df00f000, 2093056, PROT_NONE) = 0
>> mmap(0x31df20e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe000) = 0x31df20e000
>> close(3) = 0
>> open("/usr/lib64/libz.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\37\240\3151\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=85928, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8e000
>> mmap(0x31cda00000, 2178920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cda00000
>> mprotect(0x31cda14000, 2093056, PROT_NONE) = 0
>> mmap(0x31cdc13000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13000) = 0x31cdc13000
>> close(3) = 0
>> open("/usr/lib64/libxml2.so.2", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\262\"\3231\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=1297104, ...}) = 0
>> mmap(0x31d3200000, 3395256, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d3200000
>> mprotect(0x31d3333000, 2097152, PROT_NONE) = 0
>> mmap(0x31d3533000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x133000) = 0x31d3533000
>> mmap(0x31d353c000, 3768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31d353c000
>> close(3) = 0
>> open("/lib64/libc.so.6", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\332\241\3141\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=1713088, ...}) = 0
>> mmap(0x31cca00000, 3494168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cca00000
>> mprotect(0x31ccb4c000, 2097152, PROT_NONE) = 0
>> mmap(0x31ccd4c000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14c000) = 0x31ccd4c000
>> mmap(0x31ccd51000, 16664, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31ccd51000
>> close(3) = 0
>> open("/lib64/libuuid.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\25\300\3321\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=17728, ...}) = 0
>> mmap(0x31dac00000, 2110416, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31dac00000
>> mprotect(0x31dac03000, 2097152, PROT_NONE) = 0
>> mmap(0x31dae03000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x31dae03000
>> close(3) = 0
>> open("/lib64/libpam.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300$\200\3321\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=46800, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8f000
>> mmap(0x31da800000, 2140040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31da800000
>> mprotect(0x31da80b000, 2093056, PROT_NONE) = 0
>> mmap(0x31daa0a000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa000) = 0x31daa0a000
>> close(3) = 0
>> open("/lib64/librt.so.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \"\340\3151\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=53448, ...}) = 0
>> mmap(0x31cde00000, 2132936, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cde00000
>> mprotect(0x31cde07000, 2097152, PROT_NONE) = 0
>> mmap(0x31ce007000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x31ce007000
>> close(3) = 0
>> open("/lib64/libdl.so.2", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\16 \3151\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0
>> mmap(0x31cd200000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd200000
>> mprotect(0x31cd202000, 2097152, PROT_NONE) = 0
>> mmap(0x31cd402000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x31cd402000
>> close(3) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d90000
>> open("/lib64/libm.so.6", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`>\340\3141\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=615136, ...}) = 0
>> mmap(0x31cce00000, 2629848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cce00000
>> mprotect(0x31cce82000, 2093056, PROT_NONE) = 0
>> mmap(0x31cd081000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x81000) = 0x31cd081000
>> close(3) = 0
>> open("/lib64/libaudit.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340& \3221\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=2200168, ...}) = 0
>> mmap(0x31d2200000, 2195984, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d2200000
>> mprotect(0x31d2218000, 2093056, PROT_NONE) = 0
>> mmap(0x31d2417000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x217000) = 0x31d2417000
>> close(3) = 0
>> open("/lib64/libpthread.so.0", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200W`\3151\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=145592, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d91000
>> mmap(0x31cd600000, 2204528, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd600000
>> mprotect(0x31cd616000, 2093056, PROT_NONE) = 0
>> mmap(0x31cd815000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x31cd815000
>> mmap(0x31cd817000, 13168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31cd817000
>> close(3) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d92000
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d93000
>> arch_prctl(ARCH_SET_FS, 0x2b13f0d929d0) = 0
>> mprotect(0x31ccd4c000, 16384, PROT_READ) = 0
>> mprotect(0x31ce007000, 4096, PROT_READ) = 0
>> mprotect(0x31cd402000, 4096, PROT_READ) = 0
>> mprotect(0x31cd081000, 4096, PROT_READ) = 0
>> mprotect(0x31cc81b000, 4096, PROT_READ) = 0
>> mprotect(0x31cd815000, 4096, PROT_READ) = 0
>> munmap(0x2b13f0d74000, 94877) = 0
>> set_tid_address(0x2b13f0d92a60) = 16907
>> set_robust_list(0x2b13f0d92a70, 0x18) = 0
>> futex(0x7fffb9d3661c, FUTEX_WAKE_PRIVATE, 1) = 0
>> rt_sigaction(SIGRTMIN, {0x31cd605370, [], SA_RESTORER|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
>> rt_sigaction(SIGRT_1, {0x31cd6052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
>> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
>> getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
>> open("/var/run/lrmd.pid", O_RDONLY) = 3
>> read(3, " 16211\n", 12) = 11
>> close(3) = 0
>> kill(16211, SIG_0) = -1 ESRCH (No such process)
>> open("/var/run/lrmd.pid", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
>> read(3, " 16211\n", 12) = 11
>> kill(16211, SIG_0) = -1 ESRCH (No such process)
>> unlink("/var/run/lrmd.pid") = 0
>> close(3) = 0
>> open("/var/run/lrmd.pid.16907", O_WRONLY|O_CREAT|O_EXCL, 0644) = 3
>> write(3, " 16907\n", 11) = 11
>> close(3) = 0
>> link("/var/run/lrmd.pid.16907", "/var/run/lrmd.pid") = 0
>> stat("/var/run/lrmd.pid.16907", {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
>> unlink("/var/run/lrmd.pid.16907") = 0
>> umask(022) = 022
>> close(0) = 0
>> open("/dev/null", O_RDONLY) = 0
>> close(1) = 0
>> open("/dev/null", O_WRONLY) = 1
>> close(2) = 0
>> open("/dev/null", O_WRONLY) = 2
>> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGPIPE, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> brk(0) = 0x199cf000
>> brk(0x199f0000) = 0x199f0000
>> open("/etc/localtime", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d74000
>> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\3\0\0\0\0"..., 4096) = 331
>> lseek(3, -206, SEEK_CUR) = 125
>> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0\5\0\0\0\0"..., 4096) = 206
>> close(3) = 0
>> munmap(0x2b13f0d74000, 4096) = 0
>> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
>> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
>> close(3) = 0
>> open("/dev/console", O_WRONLY|O_NOCTTY) = 3
>> fstat(3, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
>> ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d74000
>> lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
>> write(3, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
>> munmap(0x2b13f0d74000, 4096) = 0
>> close(3) = 0
>> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
>> rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGINT, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGHUP, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
>> access("/usr/lib64/heartbeat/plugins/InterfaceMgr/generic.so", R_OK) = -1 ENOENT (No such file or directory)
>> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
>> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
>> futex(0x31cd4030ec, FUTEX_WAKE_PRIVATE, 2147483647) = 0
>> open("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\t\0\0\0\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=85667, ...}) = 0
>> mmap(NULL, 2104816, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x2b13f0d94000
>> mprotect(0x2b13f0d96000, 2093056, PROT_NONE) = 0
>> mmap(0x2b13f0f95000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x2b13f0f95000
>> close(3) = 0
>> open("/usr/lib64/heartbeat/plugins/RAExec", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 3
>> fstat(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>> getdents(3, /* 14 entries */, 4096) = 432
>> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
>> open("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", O_RDONLY) = 4
>> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\23\0\0\0\0\0\0"..., 832) = 832
>> fstat(4, {st_mode=S_IFREG|0755, st_size=224961, ...}) = 0
>> mmap(NULL, 2110632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f0f96000
>> mprotect(0x2b13f0f99000, 2097152, PROT_NONE) = 0
>> mmap(0x2b13f1199000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x3000) = 0x2b13f1199000
>> close(4) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
>> open("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", O_RDONLY) = 4
>> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\25\0\0\0\0\0\0"..., 832) = 832
>> fstat(4, {st_mode=S_IFREG|0755, st_size=223932, ...}) = 0
>> mmap(NULL, 2110696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f119a000
>> mprotect(0x2b13f119d000, 2097152, PROT_NONE) = 0
>> mmap(0x2b13f139d000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x3000) = 0x2b13f139d000
>> close(4) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
>> open("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", O_RDONLY) = 4
>> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \22\0\0\0\0\0\0"..., 832) = 832
>> fstat(4, {st_mode=S_IFREG|0755, st_size=220859, ...}) = 0
>> mmap(NULL, 2107944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f139e000
>> mprotect(0x2b13f13a1000, 2093056, PROT_NONE) = 0
>> mmap(0x2b13f15a0000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x2000) = 0x2b13f15a0000
>> close(4) = 0
>> open("/etc/ld.so.cache", O_RDONLY) = 4
>> fstat(4, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
>> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 4, 0) = 0x2b13f15a1000
>> close(4) = 0
>> open("/usr/lib64/libstonithd.so.0", O_RDONLY) = 4
>> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\25`\304;\0\0\0"..., 832) = 832
>> fstat(4, {st_mode=S_IFREG|0755, st_size=227569, ...}) = 0
>> mmap(0x3bc4600000, 2118768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x3bc4600000
>> mprotect(0x3bc4605000, 2097152, PROT_NONE) = 0
>> mmap(0x3bc4805000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x5000) = 0x3bc4805000
>> close(4) = 0
>> open("/usr/lib64/libstonith.so.1", O_RDONLY) = 4
>> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\35\240\303;\0\0\0"..., 832) = 832
>> fstat(4, {st_mode=S_IFREG|0755, st_size=391447, ...}) = 0
>> mmap(0x3bc3a00000, 2115776, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x3bc3a00000
>> mprotect(0x3bc3a05000, 2093056, PROT_NONE) = 0
>> mmap(0x3bc3c04000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x4000) = 0x3bc3c04000
>> close(4) = 0
>> open("/usr/lib64/libhbclient.so.1", O_RDONLY) = 4
>> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\35`\303;\0\0\0"..., 832) = 832
>> fstat(4, {st_mode=S_IFREG|0755, st_size=308671, ...}) = 0
>> mmap(0x3bc3600000, 2141672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x3bc3600000
>> mprotect(0x3bc3609000, 2097152, PROT_NONE) = 0
>> mmap(0x3bc3809000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x9000) = 0x3bc3809000
>> close(4) = 0
>> munmap(0x2b13f15a1000, 94877) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
>> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
>> open("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", O_RDONLY) = 4
>> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\21\0\0\0\0\0\0"..., 832) = 832
>> fstat(4, {st_mode=S_IFREG|0755, st_size=215232, ...}) = 0
>> mmap(NULL, 2109032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f15a1000
>> mprotect(0x2b13f15a4000, 2093056, PROT_NONE) = 0
>> mmap(0x2b13f17a3000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x2000) = 0x2b13f17a3000
>> close(4) = 0
>> getdents(3, /* 0 entries */, 4096) = 0
>> close(3) = 0
>> socket(PF_FILE, SOCK_STREAM, 0) = 3
>> fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
>> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
>> connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
>> close(3) = 0
>> socket(PF_FILE, SOCK_STREAM, 0) = 3
>> fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
>> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
>> connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
>> close(3) = 0
>> open("/etc/nsswitch.conf", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=1692, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
>> read(3, "#\n# /etc/nsswitch.conf\n#\n# An ex"..., 4096) = 1692
>> read(3, "", 4096) = 0
>> close(3) = 0
>> munmap(0x2b13f17a4000, 4096) = 0
>> open("/etc/ld.so.cache", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
>> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b13f17a4000
>> close(3) = 0
>> open("/lib64/libnss_files.so.2", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\37\0\0\0\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=53880, ...}) = 0
>> mmap(NULL, 2139432, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x2b13f17bc000
>> mprotect(0x2b13f17c6000, 2093056, PROT_NONE) = 0
>> mmap(0x2b13f19c5000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9000) = 0x2b13f19c5000
>> close(3) = 0
>> mprotect(0x2b13f19c5000, 4096, PROT_READ) = 0
>> munmap(0x2b13f17a4000, 94877) = 0
>> open("/etc/passwd", O_RDONLY) = 3
>> fcntl(3, F_GETFD) = 0
>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>> fstat(3, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
>> read(3, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
>> close(3) = 0
>> munmap(0x2b13f17a4000, 4096) = 0
>> socket(PF_FILE, SOCK_STREAM, 0) = 3
>> unlink("/var/run/heartbeat/lrm_cmd_sock") = 0
>> bind(3, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_cmd_sock"...}, 110) = 0
>> chmod("/var/run/heartbeat/lrm_cmd_sock", 0777) = 0
>> listen(3, 10) = 0
>> fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
>> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
>> socket(PF_FILE, SOCK_STREAM, 0) = 4
>> unlink("/var/run/heartbeat/lrm_callback_sock") = 0
>> bind(4, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_callback_sock"...}, 110) = 0
>> chmod("/var/run/heartbeat/lrm_callback_sock", 0777) = 0
>> listen(4, 10) = 0
>> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
>> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
>> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
>> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> socket(PF_FILE, SOCK_DGRAM, 0) = 5
>> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
>> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
>> close(5) = 0
>> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
>> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
>> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
>> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
>> write(5, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
>> munmap(0x2b13f17a4000, 4096) = 0
>> close(5) = 0
>> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGCHLD, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
>> write(2, "debug: Enabling coredumps\n", 26) = 26
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> socket(PF_FILE, SOCK_DGRAM, 0) = 5
>> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
>> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
>> close(5) = 0
>> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
>> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
>> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
>> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
>> write(5, "lrmd: [16907]: debug: Enabling c"..., 42) = 42
>> munmap(0x2b13f17a4000, 4096) = 0
>> close(5) = 0
>> chdir("/var/lib/heartbeat/cores") = 0
>> getuid() = 0
>> open("/etc/passwd", O_RDONLY) = 5
>> fcntl(5, F_GETFD) = 0
>> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
>> fstat(5, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
>> read(5, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
>> close(5) = 0
>> munmap(0x2b13f17a4000, 4096) = 0
>> chdir("root") = 0
>> getrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=RLIM_INFINITY}) = 0
>> setrlimit(RLIMIT_CORE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
>> open("/proc/sys/kernel/core_pattern", O_RDONLY) = 5
>> read(5, "core\n", 256) = 5
>> close(5) = 0
>> open("/proc/sys/kernel/core_uses_pid", O_RDONLY) = 5
>> read(5, "1\n", 2) = 2
>> close(5) = 0
>> rt_sigaction(SIGQUIT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGILL, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGABRT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGFPE, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGSEGV, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGBUS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGSYS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGTRAP, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGXCPU, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGXFSZ, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> getgid() = 0
>> getuid() = 0
>> open("/etc/passwd", O_RDONLY) = 5
>> fcntl(5, F_GETFD) = 0
>> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
>> fstat(5, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
>> read(5, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
>> close(5) = 0
>> munmap(0x2b13f17a4000, 4096) = 0
>> setresgid(-1, 99, -1) = 0
>> setresuid(-1, 99, -1) = 0
>> prctl(0x4, 0x1, 0, 0, 0) = 0
>> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
>> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
>> setresuid(-1, 0, -1) = 0
>> setresgid(-1, 0, -1) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> socket(PF_FILE, SOCK_DGRAM, 0) = 5
>> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
>> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
>> close(5) = 0
>> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
>> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
>> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
>> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
>> write(5, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
>> munmap(0x2b13f17a4000, 4096) = 0
>> close(5) = 0
>> setresgid(-1, 99, -1) = 0
>> setresuid(-1, 99, -1) = 0
>> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGUSR1, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
>> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
>> setresuid(-1, 0, -1) = 0
>> setresgid(-1, 0, -1) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> socket(PF_FILE, SOCK_DGRAM, 0) = 5
>> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
>> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
>> close(5) = 0
>> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
>> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
>> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
>> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
>> write(5, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
>> munmap(0x2b13f17a4000, 4096) = 0
>> close(5) = 0
>> setresgid(-1, 99, -1) = 0
>> setresuid(-1, 99, -1) = 0
>> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> rt_sigaction(SIGUSR2, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
>> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
>> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
>> write(2, "debug: main: run the loop...\n", 29) = 29
>> setresuid(-1, 0, -1) = 0
>> setresgid(-1, 0, -1) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> socket(PF_FILE, SOCK_DGRAM, 0) = 5
>> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
>> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
>> close(5) = 0
>> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
>> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
>> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
>> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
>> write(5, "lrmd: [16907]: debug: main: run "..., 45) = 45
>> munmap(0x2b13f17a4000, 4096) = 0
>> close(5) = 0
>> setresgid(-1, 99, -1) = 0
>> setresuid(-1, 99, -1) = 0
>> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
>> write(2, "info: Started.\n", 15) = 15
>> setresuid(-1, 0, -1) = 0
>> setresgid(-1, 0, -1) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
>> socket(PF_FILE, SOCK_DGRAM, 0) = 5
>> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
>> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
>> close(5) = 0
>> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
>> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
>> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
>> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
>> write(5, "lrmd: [16907]: info: Started.\r\n", 31) = 31
>> munmap(0x2b13f17a4000, 4096) = 0
>> close(5) = 0
>> setresgid(-1, 99, -1) = 0
>> setresuid(-1, 99, -1) = 0
>> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = -1 EINTR (Interrupted system call)
>> --- SIGINT (Interrupt) @ 0 (0) ---
>> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
>> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000
>> _______________________________________________
>> Linux-HA mailing list
>> Linux-HA[at]lists.linux-ha.org
>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> See also: http://linux-ha.org/ReportingProblems
> _______________________________________________
> Linux-HA mailing list
> Linux-HA[at]lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>



_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


ikedaj at intellilink

Jul 3, 2009, 12:15 AM

Post #5 of 14 (632 views)
Permalink
Re: socket of lrmd [In reply to]

Hi,

On Fri, 03 Jul 2009 10:59:21 +0900, Junko IKEDA <ikedaj[at]intellilink.co.jp> wrote:

> Hi Dejan,
>
> Your patch could stop the error message from LVM RA.
> Many thanks!
> But I run Heartbeat 2.1.4 so I worry about whether 2.1.4 still have a problem about stonithd that you pointed.
>
> By the way, does closefiles() close both of the connections for syslogd and ha_logd ?

The patch works well.
but, what will happen if we don't use it?
It seems that lrmd keeps the file descriptor (3) for ha_logd/syslogd while running.
Will the number of the file descriptor reach its max limit without this patch?

Thanks,
Junko

>
> On Thu, 02 Jul 2009 22:04:06 +0900, Dejan Muhamedagic <dejanmm[at]fastmail.fm> wrote:
>
>> Hi,
>>
>> On Thu, Jul 02, 2009 at 06:40:26PM +0900, Junko IKEDA wrote:
>>> Hi,
>>>
>>> I try to strace of lrmd.
>>>
>>> (1) running syslog, and strace
>>> # service syslog start
>>> # strace -o /tmp/strace-syslog-start.txt /usr/lib64/heartbeat/lrmd -rv
>>>
>>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
>>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
>>> sendto(3, "<30>Jul 2 17:02:46 lrmd: [16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
>>>
>>> (2) stop syslog and strace
>>> # service syslog stop
>>> # strace -o /tmp/strace-syslog-stop.txt /usr/lib64/heartbeat/lrmd -rv
>>>
>>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
>>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
>>> close(3) = 0
>>> open("/dev/console", O_WRONLY|O_NOCTTY) = 3
>>>
>>> (for more details, see the attached files)
>>>
>>> Is the socket of lrmd for syslog?
>>> wild guess?
>>
>> Right. See lib/lrm/racommon.c:closefiles(). I must say I can't
>> remember why exactly was the connection to syslogd/ha_logd
>> needed. There has been a problem with stonithd, but now this code
>> is not in that execution path anymore. Perhaps you can test this
>> patch:
>>
>> diff -r 731f8f7b5450 lib/lrm/racommon.c
>> --- a/lib/lrm/racommon.c Tue Jun 30 12:02:16 2009 +0200
>> +++ b/lib/lrm/racommon.c Thu Jul 02 15:02:55 2009 +0200
>> @@ -145,7 +145,7 @@
>>
>> /* close all descriptors except stdin/out/err and
>> * channels to logd */
>> for (fd = getdtablesize(); fd > STDERR_FILENO; fd--) {
>> - if (!cl_log_is_logd_fd(fd))
>> + /*if (!cl_log_is_logd_fd(fd))*/
>> close(fd);
>> }
>> }
>>
>> Thanks,
>>
>> Dejan

_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


dejanmm at fastmail

Jul 3, 2009, 12:25 AM

Post #6 of 14 (630 views)
Permalink
Re: socket of lrmd [In reply to]

Hi Junko-san,

On Fri, Jul 03, 2009 at 10:59:21AM +0900, Junko IKEDA wrote:
> Hi Dejan,
>
> Your patch could stop the error message from LVM RA.
> Many thanks!
> But I run Heartbeat 2.1.4 so I worry about whether 2.1.4 still
> have a problem about stonithd that you pointed.

No, that was found early before any heartbeat release.

> By the way, does closefiles() close both of the connections for
> syslogd and ha_logd ?

There's always only one, depending on the configuration.

Thanks,

Dejan

> Thanks,
> Junko
>
>
> On Thu, 02 Jul 2009 22:04:06 +0900, Dejan Muhamedagic <dejanmm[at]fastmail.fm> wrote:
>
> > Hi,
> >
> > On Thu, Jul 02, 2009 at 06:40:26PM +0900, Junko IKEDA wrote:
> >> Hi,
> >>
> >> I try to strace of lrmd.
> >>
> >> (1) running syslog, and strace
> >> # service syslog start
> >> # strace -o /tmp/strace-syslog-start.txt /usr/lib64/heartbeat/lrmd -rv
> >>
> >> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> >> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> >> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
> >> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> >>
> >> (2) stop syslog and strace
> >> # service syslog stop
> >> # strace -o /tmp/strace-syslog-stop.txt /usr/lib64/heartbeat/lrmd -rv
> >>
> >> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> >> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> >> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(3) = 0
> >> open("/dev/console", O_WRONLY|O_NOCTTY) = 3
> >>
> >> (for more details, see the attached files)
> >>
> >> Is the socket of lrmd for syslog?
> >> wild guess?
> >
> > Right. See lib/lrm/racommon.c:closefiles(). I must say I can't
> > remember why exactly was the connection to syslogd/ha_logd
> > needed. There has been a problem with stonithd, but now this code
> > is not in that execution path anymore. Perhaps you can test this
> > patch:
> >
> > diff -r 731f8f7b5450 lib/lrm/racommon.c
> > --- a/lib/lrm/racommon.c Tue Jun 30 12:02:16 2009 +0200
> > +++ b/lib/lrm/racommon.c Thu Jul 02 15:02:55 2009 +0200
> > @@ -145,7 +145,7 @@
> >
> > /* close all descriptors except stdin/out/err and
> > * channels to logd */
> > for (fd = getdtablesize(); fd > STDERR_FILENO; fd--) {
> > - if (!cl_log_is_logd_fd(fd))
> > + /*if (!cl_log_is_logd_fd(fd))*/
> > close(fd);
> > }
> > }
> >
> > Thanks,
> >
> > Dejan
> >
> >>
> >> Thanks,
> >> Junko
> >>
> >>
> >> On Thu, 02 Jul 2009 16:09:16 +0900, Junko IKEDA <ikedaj[at]intellilink.co.jp> wrote:
> >>
> >> >
> >> > Hi,
> >> >
> >> > When I run LVM RA with Heartbeat 2.1.4,
> >> > I could see the following message.
> >> >
> >> > lrmd[14994]: 2009/07/02_15:29:20 info: RA output: (lvm:monitor:stderr) File descriptor 3 (socket:[46374]) leaked on vgck invocation. Parent PID 15446: /bin/sh
> >> >
> >> > It seems that some LVM commands (vgdisplay or vgck) try to close the file descriptor which lrmd opened.
> >> > lsof command showed that lrmd had socket like this;
> >> >
> >> > # lsof | grep lrmd | grep socket
> >> > lrmd 14994 nobody 3u unix 0xffff81011b05f740 46374 socket
> >> >
> >> > (for more details, see the attached hb_report and lsof.txt)
> >> >
> >> > lvm commands tried to close this socket and failed. :(
> >> > What is this socket for?
> >> > For the communication with othre processes like crmd or ha_logd?
> >> >
> >> > This problem depends on the specification of lvm,
> >> > but I just want to confirm whether the above log message could have an impact on the cluster behavior or not.
> >> >
> >> > This list might be the same issue.
> >> > http://www.gossamer-threads.com/lists/linuxha/users/49687
> >> >
> >> > Thanks,
> >> > Junko
> >>
> >>
> >
> >> execve("/usr/lib64/heartbeat/lrmd", ["/usr/lib64/heartbeat/lrmd", "-rv"], [/* 24 vars */]) = 0
> >> brk(0) = 0x13652000
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96315000
> >> uname({sys="Linux", node="dl380g5d", ...}) = 0
> >> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
> >> open("/etc/ld.so.cache", O_RDONLY) = 3
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> >> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2abc96316000
> >> close(3) = 0
> >> open("/usr/lib64/liblrm.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\27\240\304;\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=317571, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc9632e000
> >> mmap(0x3bc4a00000, 2123232, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4a00000
> >> mprotect(0x3bc4a07000, 2093056, PROT_NONE) = 0
> >> mmap(0x3bc4c06000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x3bc4c06000
> >> close(3) = 0
> >> open("/usr/lib64/libplumb.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\254\340\302;\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=3143917, ...}) = 0
> >> mmap(0x3bc2e00000, 2281208, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc2e00000
> >> mprotect(0x3bc2e2b000, 2093056, PROT_NONE) = 0
> >> mmap(0x3bc302a000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2a000) = 0x3bc302a000
> >> close(3) = 0
> >> open("/usr/lib64/libpils.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@) \303;\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=309004, ...}) = 0
> >> mmap(0x3bc3200000, 2148672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc3200000
> >> mprotect(0x3bc320c000, 2097152, PROT_NONE) = 0
> >> mmap(0x3bc340c000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc000) = 0x3bc340c000
> >> close(3) = 0
> >> open("/usr/lib64/libapphb.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\t \304;\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=103494, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc9632f000
> >> mmap(0x3bc4200000, 2102920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4200000
> >> mprotect(0x3bc4202000, 2093056, PROT_NONE) = 0
> >> mmap(0x3bc4401000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x3bc4401000
> >> close(3) = 0
> >> open("/lib64/libglib-2.0.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\377\240\3161\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=647864, ...}) = 0
> >> mmap(0x31cea00000, 2742472, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cea00000
> >> mprotect(0x31cea9d000, 2093056, PROT_NONE) = 0
> >> mmap(0x31cec9c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9c000) = 0x31cec9c000
> >> close(3) = 0
> >> open("/usr/lib64/libbz2.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\26\0\3371\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=67664, ...}) = 0
> >> mmap(0x31df000000, 2160656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31df000000
> >> mprotect(0x31df00f000, 2093056, PROT_NONE) = 0
> >> mmap(0x31df20e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe000) = 0x31df20e000
> >> close(3) = 0
> >> open("/usr/lib64/libz.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\37\240\3151\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=85928, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96330000
> >> mmap(0x31cda00000, 2178920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cda00000
> >> mprotect(0x31cda14000, 2093056, PROT_NONE) = 0
> >> mmap(0x31cdc13000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13000) = 0x31cdc13000
> >> close(3) = 0
> >> open("/usr/lib64/libxml2.so.2", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\262\"\3231\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=1297104, ...}) = 0
> >> mmap(0x31d3200000, 3395256, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d3200000
> >> mprotect(0x31d3333000, 2097152, PROT_NONE) = 0
> >> mmap(0x31d3533000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x133000) = 0x31d3533000
> >> mmap(0x31d353c000, 3768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31d353c000
> >> close(3) = 0
> >> open("/lib64/libc.so.6", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\332\241\3141\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=1713088, ...}) = 0
> >> mmap(0x31cca00000, 3494168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cca00000
> >> mprotect(0x31ccb4c000, 2097152, PROT_NONE) = 0
> >> mmap(0x31ccd4c000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14c000) = 0x31ccd4c000
> >> mmap(0x31ccd51000, 16664, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31ccd51000
> >> close(3) = 0
> >> open("/lib64/libuuid.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\25\300\3321\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=17728, ...}) = 0
> >> mmap(0x31dac00000, 2110416, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31dac00000
> >> mprotect(0x31dac03000, 2097152, PROT_NONE) = 0
> >> mmap(0x31dae03000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x31dae03000
> >> close(3) = 0
> >> open("/lib64/libpam.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300$\200\3321\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=46800, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96331000
> >> mmap(0x31da800000, 2140040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31da800000
> >> mprotect(0x31da80b000, 2093056, PROT_NONE) = 0
> >> mmap(0x31daa0a000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa000) = 0x31daa0a000
> >> close(3) = 0
> >> open("/lib64/librt.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \"\340\3151\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=53448, ...}) = 0
> >> mmap(0x31cde00000, 2132936, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cde00000
> >> mprotect(0x31cde07000, 2097152, PROT_NONE) = 0
> >> mmap(0x31ce007000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x31ce007000
> >> close(3) = 0
> >> open("/lib64/libdl.so.2", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\16 \3151\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0
> >> mmap(0x31cd200000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd200000
> >> mprotect(0x31cd202000, 2097152, PROT_NONE) = 0
> >> mmap(0x31cd402000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x31cd402000
> >> close(3) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96332000
> >> open("/lib64/libm.so.6", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`>\340\3141\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=615136, ...}) = 0
> >> mmap(0x31cce00000, 2629848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cce00000
> >> mprotect(0x31cce82000, 2093056, PROT_NONE) = 0
> >> mmap(0x31cd081000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x81000) = 0x31cd081000
> >> close(3) = 0
> >> open("/lib64/libaudit.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340& \3221\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=2200168, ...}) = 0
> >> mmap(0x31d2200000, 2195984, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d2200000
> >> mprotect(0x31d2218000, 2093056, PROT_NONE) = 0
> >> mmap(0x31d2417000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x217000) = 0x31d2417000
> >> close(3) = 0
> >> open("/lib64/libpthread.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200W`\3151\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=145592, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96333000
> >> mmap(0x31cd600000, 2204528, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd600000
> >> mprotect(0x31cd616000, 2093056, PROT_NONE) = 0
> >> mmap(0x31cd815000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x31cd815000
> >> mmap(0x31cd817000, 13168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31cd817000
> >> close(3) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96334000
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96335000
> >> arch_prctl(ARCH_SET_FS, 0x2abc963349d0) = 0
> >> mprotect(0x31ccd4c000, 16384, PROT_READ) = 0
> >> mprotect(0x31ce007000, 4096, PROT_READ) = 0
> >> mprotect(0x31cd402000, 4096, PROT_READ) = 0
> >> mprotect(0x31cd081000, 4096, PROT_READ) = 0
> >> mprotect(0x31cc81b000, 4096, PROT_READ) = 0
> >> mprotect(0x31cd815000, 4096, PROT_READ) = 0
> >> munmap(0x2abc96316000, 94877) = 0
> >> set_tid_address(0x2abc96334a60) = 16211
> >> set_robust_list(0x2abc96334a70, 0x18) = 0
> >> futex(0x7fff1479207c, FUTEX_WAKE_PRIVATE, 1) = 0
> >> rt_sigaction(SIGRTMIN, {0x31cd605370, [], SA_RESTORER|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
> >> rt_sigaction(SIGRT_1, {0x31cd6052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
> >> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
> >> getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
> >> open("/var/run/lrmd.pid", O_RDONLY) = 3
> >> read(3, " 15838\n", 12) = 11
> >> close(3) = 0
> >> kill(15838, SIG_0) = -1 ESRCH (No such process)
> >> open("/var/run/lrmd.pid", O_RDONLY) = 3
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
> >> read(3, " 15838\n", 12) = 11
> >> kill(15838, SIG_0) = -1 ESRCH (No such process)
> >> unlink("/var/run/lrmd.pid") = 0
> >> close(3) = 0
> >> open("/var/run/lrmd.pid.16211", O_WRONLY|O_CREAT|O_EXCL, 0644) = 3
> >> write(3, " 16211\n", 11) = 11
> >> close(3) = 0
> >> link("/var/run/lrmd.pid.16211", "/var/run/lrmd.pid") = 0
> >> stat("/var/run/lrmd.pid.16211", {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
> >> unlink("/var/run/lrmd.pid.16211") = 0
> >> umask(022) = 022
> >> close(0) = 0
> >> open("/dev/null", O_RDONLY) = 0
> >> close(1) = 0
> >> open("/dev/null", O_WRONLY) = 1
> >> close(2) = 0
> >> open("/dev/null", O_WRONLY) = 2
> >> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGPIPE, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> brk(0) = 0x13652000
> >> brk(0x13673000) = 0x13673000
> >> open("/etc/localtime", O_RDONLY) = 3
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96316000
> >> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\3\0\0\0\0"..., 4096) = 331
> >> lseek(3, -206, SEEK_CUR) = 125
> >> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0\5\0\0\0\0"..., 4096) = 206
> >> close(3) = 0
> >> munmap(0x2abc96316000, 4096) = 0
> >> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> >> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> >> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> >> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
> >> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> >> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
> >> rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGINT, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGHUP, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
> >> access("/usr/lib64/heartbeat/plugins/InterfaceMgr/generic.so", R_OK) = -1 ENOENT (No such file or directory)
> >> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
> >> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
> >> futex(0x31cd4030ec, FUTEX_WAKE_PRIVATE, 2147483647) = 0
> >> open("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", O_RDONLY) = 4
> >> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\t\0\0\0\0\0\0"..., 832) = 832
> >> fstat(4, {st_mode=S_IFREG|0755, st_size=85667, ...}) = 0
> >> mmap(NULL, 2104816, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2abc96336000
> >> mprotect(0x2abc96338000, 2093056, PROT_NONE) = 0
> >> mmap(0x2abc96537000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x1000) = 0x2abc96537000
> >> close(4) = 0
> >> open("/usr/lib64/heartbeat/plugins/RAExec", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 4
> >> fstat(4, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
> >> fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> >> getdents(4, /* 14 entries */, 4096) = 432
> >> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
> >> open("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", O_RDONLY) = 5
> >> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\23\0\0\0\0\0\0"..., 832) = 832
> >> fstat(5, {st_mode=S_IFREG|0755, st_size=224961, ...}) = 0
> >> mmap(NULL, 2110632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc96538000
> >> mprotect(0x2abc9653b000, 2097152, PROT_NONE) = 0
> >> mmap(0x2abc9673b000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x3000) = 0x2abc9673b000
> >> close(5) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
> >> open("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", O_RDONLY) = 5
> >> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\25\0\0\0\0\0\0"..., 832) = 832
> >> fstat(5, {st_mode=S_IFREG|0755, st_size=223932, ...}) = 0
> >> mmap(NULL, 2110696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc9673c000
> >> mprotect(0x2abc9673f000, 2097152, PROT_NONE) = 0
> >> mmap(0x2abc9693f000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x3000) = 0x2abc9693f000
> >> close(5) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
> >> open("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", O_RDONLY) = 5
> >> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \22\0\0\0\0\0\0"..., 832) = 832
> >> fstat(5, {st_mode=S_IFREG|0755, st_size=220859, ...}) = 0
> >> mmap(NULL, 2107944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc96940000
> >> mprotect(0x2abc96943000, 2093056, PROT_NONE) = 0
> >> mmap(0x2abc96b42000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x2abc96b42000
> >> close(5) = 0
> >> open("/etc/ld.so.cache", O_RDONLY) = 5
> >> fstat(5, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> >> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 5, 0) = 0x2abc96b43000
> >> close(5) = 0
> >> open("/usr/lib64/libstonithd.so.0", O_RDONLY) = 5
> >> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\25`\304;\0\0\0"..., 832) = 832
> >> fstat(5, {st_mode=S_IFREG|0755, st_size=227569, ...}) = 0
> >> mmap(0x3bc4600000, 2118768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x3bc4600000
> >> mprotect(0x3bc4605000, 2097152, PROT_NONE) = 0
> >> mmap(0x3bc4805000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x5000) = 0x3bc4805000
> >> close(5) = 0
> >> open("/usr/lib64/libstonith.so.1", O_RDONLY) = 5
> >> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\35\240\303;\0\0\0"..., 832) = 832
> >> fstat(5, {st_mode=S_IFREG|0755, st_size=391447, ...}) = 0
> >> mmap(0x3bc3a00000, 2115776, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x3bc3a00000
> >> mprotect(0x3bc3a05000, 2093056, PROT_NONE) = 0
> >> mmap(0x3bc3c04000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x4000) = 0x3bc3c04000
> >> close(5) = 0
> >> open("/usr/lib64/libhbclient.so.1", O_RDONLY) = 5
> >> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\35`\303;\0\0\0"..., 832) = 832
> >> fstat(5, {st_mode=S_IFREG|0755, st_size=308671, ...}) = 0
> >> mmap(0x3bc3600000, 2141672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x3bc3600000
> >> mprotect(0x3bc3609000, 2097152, PROT_NONE) = 0
> >> mmap(0x3bc3809000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x9000) = 0x3bc3809000
> >> close(5) = 0
> >> munmap(0x2abc96b43000, 94877) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
> >> open("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", O_RDONLY) = 5
> >> read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\21\0\0\0\0\0\0"..., 832) = 832
> >> fstat(5, {st_mode=S_IFREG|0755, st_size=215232, ...}) = 0
> >> mmap(NULL, 2109032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x2abc96b43000
> >> mprotect(0x2abc96b46000, 2093056, PROT_NONE) = 0
> >> mmap(0x2abc96d45000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x2abc96d45000
> >> close(5) = 0
> >> getdents(4, /* 0 entries */, 4096) = 0
> >> close(4) = 0
> >> socket(PF_FILE, SOCK_STREAM, 0) = 4
> >> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
> >> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> >> connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(4) = 0
> >> socket(PF_FILE, SOCK_STREAM, 0) = 4
> >> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
> >> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> >> connect(4, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(4) = 0
> >> open("/etc/nsswitch.conf", O_RDONLY) = 4
> >> fstat(4, {st_mode=S_IFREG|0644, st_size=1692, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
> >> read(4, "#\n# /etc/nsswitch.conf\n#\n# An ex"..., 4096) = 1692
> >> read(4, "", 4096) = 0
> >> close(4) = 0
> >> munmap(0x2abc96d46000, 4096) = 0
> >> open("/etc/ld.so.cache", O_RDONLY) = 4
> >> fstat(4, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> >> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 4, 0) = 0x2abc96d46000
> >> close(4) = 0
> >> open("/lib64/libnss_files.so.2", O_RDONLY) = 4
> >> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\37\0\0\0\0\0\0"..., 832) = 832
> >> fstat(4, {st_mode=S_IFREG|0755, st_size=53880, ...}) = 0
> >> mmap(NULL, 2139432, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2abc96d5e000
> >> mprotect(0x2abc96d68000, 2093056, PROT_NONE) = 0
> >> mmap(0x2abc96f67000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x9000) = 0x2abc96f67000
> >> close(4) = 0
> >> mprotect(0x2abc96f67000, 4096, PROT_READ) = 0
> >> munmap(0x2abc96d46000, 94877) = 0
> >> open("/etc/passwd", O_RDONLY) = 4
> >> fcntl(4, F_GETFD) = 0
> >> fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> >> fstat(4, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
> >> read(4, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> >> close(4) = 0
> >> munmap(0x2abc96d46000, 4096) = 0
> >> socket(PF_FILE, SOCK_STREAM, 0) = 4
> >> unlink("/var/run/heartbeat/lrm_cmd_sock") = 0
> >> bind(4, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_cmd_sock"...}, 110) = 0
> >> chmod("/var/run/heartbeat/lrm_cmd_sock", 0777) = 0
> >> listen(4, 10) = 0
> >> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
> >> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> >> socket(PF_FILE, SOCK_STREAM, 0) = 5
> >> unlink("/var/run/heartbeat/lrm_callback_sock") = 0
> >> bind(5, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_callback_sock"...}, 110) = 0
> >> chmod("/var/run/heartbeat/lrm_callback_sock", 0777) = 0
> >> listen(5, 10) = 0
> >> fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
> >> fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> >> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> >> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> >> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGCHLD, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> >> write(2, "debug: Enabling coredumps\n", 26) = 26
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> sendto(3, "<31>Jul 2 17:02:46 lrmd: [.16211"..., 61, MSG_NOSIGNAL, NULL, 0) = 61
> >> chdir("/var/lib/heartbeat/cores") = 0
> >> getuid() = 0
> >> open("/etc/passwd", O_RDONLY) = 6
> >> fcntl(6, F_GETFD) = 0
> >> fcntl(6, F_SETFD, FD_CLOEXEC) = 0
> >> fstat(6, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
> >> read(6, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> >> close(6) = 0
> >> munmap(0x2abc96d46000, 4096) = 0
> >> chdir("root") = 0
> >> getrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=RLIM_INFINITY}) = 0
> >> setrlimit(RLIMIT_CORE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> >> open("/proc/sys/kernel/core_pattern", O_RDONLY) = 6
> >> read(6, "core\n", 256) = 5
> >> close(6) = 0
> >> open("/proc/sys/kernel/core_uses_pid", O_RDONLY) = 6
> >> read(6, "1\n", 2) = 2
> >> close(6) = 0
> >> rt_sigaction(SIGQUIT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGILL, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGABRT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGFPE, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGSEGV, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGBUS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGSYS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGTRAP, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGXCPU, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGXFSZ, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> getgid() = 0
> >> getuid() = 0
> >> open("/etc/passwd", O_RDONLY) = 6
> >> fcntl(6, F_GETFD) = 0
> >> fcntl(6, F_SETFD, FD_CLOEXEC) = 0
> >> fstat(6, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abc96d46000
> >> read(6, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> >> close(6) = 0
> >> munmap(0x2abc96d46000, 4096) = 0
> >> setresgid(-1, 99, -1) = 0
> >> setresuid(-1, 99, -1) = 0
> >> prctl(0x4, 0x1, 0, 0, 0) = 0
> >> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> >> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> >> setresuid(-1, 0, -1) = 0
> >> setresgid(-1, 0, -1) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> >> setresgid(-1, 99, -1) = 0
> >> setresuid(-1, 99, -1) = 0
> >> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGUSR1, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> >> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> >> setresuid(-1, 0, -1) = 0
> >> setresgid(-1, 0, -1) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> >> setresgid(-1, 99, -1) = 0
> >> setresuid(-1, 99, -1) = 0
> >> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGUSR2, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> >> write(2, "debug: main: run the loop...\n", 29) = 29
> >> setresuid(-1, 0, -1) = 0
> >> setresgid(-1, 0, -1) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> sendto(3, "<31>Jul 2 17:02:46 lrmd: [.16211"..., 64, MSG_NOSIGNAL, NULL, 0) = 64
> >> setresgid(-1, 99, -1) = 0
> >> setresuid(-1, 99, -1) = 0
> >> write(2, "lrmd[16211]: 2009/07/02_17:02:46"..., 33) = 33
> >> write(2, "info: Started.\n", 15) = 15
> >> setresuid(-1, 0, -1) = 0
> >> setresgid(-1, 0, -1) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> sendto(3, "<30>Jul 2 17:02:46 lrmd: [.16211"..., 50, MSG_NOSIGNAL, NULL, 0) = 50
> >> setresgid(-1, 99, -1) = 0
> >> setresuid(-1, 99, -1) = 0
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = -1 EINTR (Interrupted system call)
> >> --- SIGINT (Interrupt) @ 0 (0) ---
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=4, events=POLLIN|POLLPRI}, {fd=5, events=POLLIN|POLLPRI}], 2, 1000
> >
> >> execve("/usr/lib64/heartbeat/lrmd", ["/usr/lib64/heartbeat/lrmd", "-rv"], [/* 24 vars */]) = 0
> >> brk(0) = 0x199cf000
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d73000
> >> uname({sys="Linux", node="dl380g5d", ...}) = 0
> >> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
> >> open("/etc/ld.so.cache", O_RDONLY) = 3
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> >> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b13f0d74000
> >> close(3) = 0
> >> open("/usr/lib64/liblrm.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\27\240\304;\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=317571, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8c000
> >> mmap(0x3bc4a00000, 2123232, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4a00000
> >> mprotect(0x3bc4a07000, 2093056, PROT_NONE) = 0
> >> mmap(0x3bc4c06000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x3bc4c06000
> >> close(3) = 0
> >> open("/usr/lib64/libplumb.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\254\340\302;\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=3143917, ...}) = 0
> >> mmap(0x3bc2e00000, 2281208, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc2e00000
> >> mprotect(0x3bc2e2b000, 2093056, PROT_NONE) = 0
> >> mmap(0x3bc302a000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2a000) = 0x3bc302a000
> >> close(3) = 0
> >> open("/usr/lib64/libpils.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@) \303;\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=309004, ...}) = 0
> >> mmap(0x3bc3200000, 2148672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc3200000
> >> mprotect(0x3bc320c000, 2097152, PROT_NONE) = 0
> >> mmap(0x3bc340c000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc000) = 0x3bc340c000
> >> close(3) = 0
> >> open("/usr/lib64/libapphb.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\t \304;\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=103494, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8d000
> >> mmap(0x3bc4200000, 2102920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3bc4200000
> >> mprotect(0x3bc4202000, 2093056, PROT_NONE) = 0
> >> mmap(0x3bc4401000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x3bc4401000
> >> close(3) = 0
> >> open("/lib64/libglib-2.0.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\377\240\3161\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=647864, ...}) = 0
> >> mmap(0x31cea00000, 2742472, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cea00000
> >> mprotect(0x31cea9d000, 2093056, PROT_NONE) = 0
> >> mmap(0x31cec9c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9c000) = 0x31cec9c000
> >> close(3) = 0
> >> open("/usr/lib64/libbz2.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\26\0\3371\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=67664, ...}) = 0
> >> mmap(0x31df000000, 2160656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31df000000
> >> mprotect(0x31df00f000, 2093056, PROT_NONE) = 0
> >> mmap(0x31df20e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe000) = 0x31df20e000
> >> close(3) = 0
> >> open("/usr/lib64/libz.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\37\240\3151\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=85928, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8e000
> >> mmap(0x31cda00000, 2178920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cda00000
> >> mprotect(0x31cda14000, 2093056, PROT_NONE) = 0
> >> mmap(0x31cdc13000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13000) = 0x31cdc13000
> >> close(3) = 0
> >> open("/usr/lib64/libxml2.so.2", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\262\"\3231\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=1297104, ...}) = 0
> >> mmap(0x31d3200000, 3395256, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d3200000
> >> mprotect(0x31d3333000, 2097152, PROT_NONE) = 0
> >> mmap(0x31d3533000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x133000) = 0x31d3533000
> >> mmap(0x31d353c000, 3768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31d353c000
> >> close(3) = 0
> >> open("/lib64/libc.so.6", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\332\241\3141\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=1713088, ...}) = 0
> >> mmap(0x31cca00000, 3494168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cca00000
> >> mprotect(0x31ccb4c000, 2097152, PROT_NONE) = 0
> >> mmap(0x31ccd4c000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14c000) = 0x31ccd4c000
> >> mmap(0x31ccd51000, 16664, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31ccd51000
> >> close(3) = 0
> >> open("/lib64/libuuid.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\25\300\3321\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=17728, ...}) = 0
> >> mmap(0x31dac00000, 2110416, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31dac00000
> >> mprotect(0x31dac03000, 2097152, PROT_NONE) = 0
> >> mmap(0x31dae03000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x31dae03000
> >> close(3) = 0
> >> open("/lib64/libpam.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300$\200\3321\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=46800, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d8f000
> >> mmap(0x31da800000, 2140040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31da800000
> >> mprotect(0x31da80b000, 2093056, PROT_NONE) = 0
> >> mmap(0x31daa0a000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa000) = 0x31daa0a000
> >> close(3) = 0
> >> open("/lib64/librt.so.1", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \"\340\3151\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=53448, ...}) = 0
> >> mmap(0x31cde00000, 2132936, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cde00000
> >> mprotect(0x31cde07000, 2097152, PROT_NONE) = 0
> >> mmap(0x31ce007000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x31ce007000
> >> close(3) = 0
> >> open("/lib64/libdl.so.2", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\16 \3151\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0
> >> mmap(0x31cd200000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd200000
> >> mprotect(0x31cd202000, 2097152, PROT_NONE) = 0
> >> mmap(0x31cd402000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x31cd402000
> >> close(3) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d90000
> >> open("/lib64/libm.so.6", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`>\340\3141\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=615136, ...}) = 0
> >> mmap(0x31cce00000, 2629848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cce00000
> >> mprotect(0x31cce82000, 2093056, PROT_NONE) = 0
> >> mmap(0x31cd081000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x81000) = 0x31cd081000
> >> close(3) = 0
> >> open("/lib64/libaudit.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340& \3221\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=2200168, ...}) = 0
> >> mmap(0x31d2200000, 2195984, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31d2200000
> >> mprotect(0x31d2218000, 2093056, PROT_NONE) = 0
> >> mmap(0x31d2417000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x217000) = 0x31d2417000
> >> close(3) = 0
> >> open("/lib64/libpthread.so.0", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200W`\3151\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=145592, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d91000
> >> mmap(0x31cd600000, 2204528, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x31cd600000
> >> mprotect(0x31cd616000, 2093056, PROT_NONE) = 0
> >> mmap(0x31cd815000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x31cd815000
> >> mmap(0x31cd817000, 13168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x31cd817000
> >> close(3) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d92000
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d93000
> >> arch_prctl(ARCH_SET_FS, 0x2b13f0d929d0) = 0
> >> mprotect(0x31ccd4c000, 16384, PROT_READ) = 0
> >> mprotect(0x31ce007000, 4096, PROT_READ) = 0
> >> mprotect(0x31cd402000, 4096, PROT_READ) = 0
> >> mprotect(0x31cd081000, 4096, PROT_READ) = 0
> >> mprotect(0x31cc81b000, 4096, PROT_READ) = 0
> >> mprotect(0x31cd815000, 4096, PROT_READ) = 0
> >> munmap(0x2b13f0d74000, 94877) = 0
> >> set_tid_address(0x2b13f0d92a60) = 16907
> >> set_robust_list(0x2b13f0d92a70, 0x18) = 0
> >> futex(0x7fffb9d3661c, FUTEX_WAKE_PRIVATE, 1) = 0
> >> rt_sigaction(SIGRTMIN, {0x31cd605370, [], SA_RESTORER|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
> >> rt_sigaction(SIGRT_1, {0x31cd6052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x31cd60e4c0}, NULL, 8) = 0
> >> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
> >> getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
> >> open("/var/run/lrmd.pid", O_RDONLY) = 3
> >> read(3, " 16211\n", 12) = 11
> >> close(3) = 0
> >> kill(16211, SIG_0) = -1 ESRCH (No such process)
> >> open("/var/run/lrmd.pid", O_RDONLY) = 3
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
> >> read(3, " 16211\n", 12) = 11
> >> kill(16211, SIG_0) = -1 ESRCH (No such process)
> >> unlink("/var/run/lrmd.pid") = 0
> >> close(3) = 0
> >> open("/var/run/lrmd.pid.16907", O_WRONLY|O_CREAT|O_EXCL, 0644) = 3
> >> write(3, " 16907\n", 11) = 11
> >> close(3) = 0
> >> link("/var/run/lrmd.pid.16907", "/var/run/lrmd.pid") = 0
> >> stat("/var/run/lrmd.pid.16907", {st_mode=S_IFREG|0644, st_size=11, ...}) = 0
> >> unlink("/var/run/lrmd.pid.16907") = 0
> >> umask(022) = 022
> >> close(0) = 0
> >> open("/dev/null", O_RDONLY) = 0
> >> close(1) = 0
> >> open("/dev/null", O_WRONLY) = 1
> >> close(2) = 0
> >> open("/dev/null", O_WRONLY) = 2
> >> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGPIPE, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> brk(0) = 0x199cf000
> >> brk(0x199f0000) = 0x199f0000
> >> open("/etc/localtime", O_RDONLY) = 3
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d74000
> >> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\3\0\0\0\0"..., 4096) = 331
> >> lseek(3, -206, SEEK_CUR) = 125
> >> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0\5\0\0\0\0"..., 4096) = 206
> >> close(3) = 0
> >> munmap(0x2b13f0d74000, 4096) = 0
> >> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> >> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> >> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> >> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(3) = 0
> >> open("/dev/console", O_WRONLY|O_NOCTTY) = 3
> >> fstat(3, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> >> ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f0d74000
> >> lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> >> write(3, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
> >> munmap(0x2b13f0d74000, 4096) = 0
> >> close(3) = 0
> >> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGTERM, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGTERM, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
> >> rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGINT, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGINT, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGHUP, NULL, {0x1, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGHUP, {0x1, [], SA_RESTORER|SA_RESTART, 0x31cca30280}, NULL, 8) = 0
> >> access("/usr/lib64/heartbeat/plugins/InterfaceMgr/generic.so", R_OK) = -1 ENOENT (No such file or directory)
> >> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
> >> access("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", R_OK) = 0
> >> futex(0x31cd4030ec, FUTEX_WAKE_PRIVATE, 2147483647) = 0
> >> open("/usr/lib64/pils/plugins/InterfaceMgr/generic.so", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\t\0\0\0\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=85667, ...}) = 0
> >> mmap(NULL, 2104816, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x2b13f0d94000
> >> mprotect(0x2b13f0d96000, 2093056, PROT_NONE) = 0
> >> mmap(0x2b13f0f95000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x2b13f0f95000
> >> close(3) = 0
> >> open("/usr/lib64/heartbeat/plugins/RAExec", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 3
> >> fstat(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
> >> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> >> getdents(3, /* 14 entries */, 4096) = 432
> >> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", R_OK) = 0
> >> open("/usr/lib64/heartbeat/plugins/RAExec/lsb.so", O_RDONLY) = 4
> >> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\23\0\0\0\0\0\0"..., 832) = 832
> >> fstat(4, {st_mode=S_IFREG|0755, st_size=224961, ...}) = 0
> >> mmap(NULL, 2110632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f0f96000
> >> mprotect(0x2b13f0f99000, 2097152, PROT_NONE) = 0
> >> mmap(0x2b13f1199000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x3000) = 0x2b13f1199000
> >> close(4) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", R_OK) = 0
> >> open("/usr/lib64/heartbeat/plugins/RAExec/ocf.so", O_RDONLY) = 4
> >> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\25\0\0\0\0\0\0"..., 832) = 832
> >> fstat(4, {st_mode=S_IFREG|0755, st_size=223932, ...}) = 0
> >> mmap(NULL, 2110696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f119a000
> >> mprotect(0x2b13f119d000, 2097152, PROT_NONE) = 0
> >> mmap(0x2b13f139d000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x3000) = 0x2b13f139d000
> >> close(4) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", R_OK) = 0
> >> open("/usr/lib64/heartbeat/plugins/RAExec/stonith.so", O_RDONLY) = 4
> >> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \22\0\0\0\0\0\0"..., 832) = 832
> >> fstat(4, {st_mode=S_IFREG|0755, st_size=220859, ...}) = 0
> >> mmap(NULL, 2107944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f139e000
> >> mprotect(0x2b13f13a1000, 2093056, PROT_NONE) = 0
> >> mmap(0x2b13f15a0000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x2000) = 0x2b13f15a0000
> >> close(4) = 0
> >> open("/etc/ld.so.cache", O_RDONLY) = 4
> >> fstat(4, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> >> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 4, 0) = 0x2b13f15a1000
> >> close(4) = 0
> >> open("/usr/lib64/libstonithd.so.0", O_RDONLY) = 4
> >> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\25`\304;\0\0\0"..., 832) = 832
> >> fstat(4, {st_mode=S_IFREG|0755, st_size=227569, ...}) = 0
> >> mmap(0x3bc4600000, 2118768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x3bc4600000
> >> mprotect(0x3bc4605000, 2097152, PROT_NONE) = 0
> >> mmap(0x3bc4805000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x5000) = 0x3bc4805000
> >> close(4) = 0
> >> open("/usr/lib64/libstonith.so.1", O_RDONLY) = 4
> >> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\35\240\303;\0\0\0"..., 832) = 832
> >> fstat(4, {st_mode=S_IFREG|0755, st_size=391447, ...}) = 0
> >> mmap(0x3bc3a00000, 2115776, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x3bc3a00000
> >> mprotect(0x3bc3a05000, 2093056, PROT_NONE) = 0
> >> mmap(0x3bc3c04000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x4000) = 0x3bc3c04000
> >> close(4) = 0
> >> open("/usr/lib64/libhbclient.so.1", O_RDONLY) = 4
> >> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\35`\303;\0\0\0"..., 832) = 832
> >> fstat(4, {st_mode=S_IFREG|0755, st_size=308671, ...}) = 0
> >> mmap(0x3bc3600000, 2141672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x3bc3600000
> >> mprotect(0x3bc3609000, 2097152, PROT_NONE) = 0
> >> mmap(0x3bc3809000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x9000) = 0x3bc3809000
> >> close(4) = 0
> >> munmap(0x2b13f15a1000, 94877) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
> >> access("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", R_OK) = 0
> >> open("/usr/lib64/heartbeat/plugins/RAExec/heartbeat.so", O_RDONLY) = 4
> >> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\21\0\0\0\0\0\0"..., 832) = 832
> >> fstat(4, {st_mode=S_IFREG|0755, st_size=215232, ...}) = 0
> >> mmap(NULL, 2109032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x2b13f15a1000
> >> mprotect(0x2b13f15a4000, 2093056, PROT_NONE) = 0
> >> mmap(0x2b13f17a3000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x2000) = 0x2b13f17a3000
> >> close(4) = 0
> >> getdents(3, /* 0 entries */, 4096) = 0
> >> close(3) = 0
> >> socket(PF_FILE, SOCK_STREAM, 0) = 3
> >> fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
> >> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> >> connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(3) = 0
> >> socket(PF_FILE, SOCK_STREAM, 0) = 3
> >> fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
> >> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> >> connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(3) = 0
> >> open("/etc/nsswitch.conf", O_RDONLY) = 3
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=1692, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> >> read(3, "#\n# /etc/nsswitch.conf\n#\n# An ex"..., 4096) = 1692
> >> read(3, "", 4096) = 0
> >> close(3) = 0
> >> munmap(0x2b13f17a4000, 4096) = 0
> >> open("/etc/ld.so.cache", O_RDONLY) = 3
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=94877, ...}) = 0
> >> mmap(NULL, 94877, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b13f17a4000
> >> close(3) = 0
> >> open("/lib64/libnss_files.so.2", O_RDONLY) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\37\0\0\0\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=53880, ...}) = 0
> >> mmap(NULL, 2139432, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x2b13f17bc000
> >> mprotect(0x2b13f17c6000, 2093056, PROT_NONE) = 0
> >> mmap(0x2b13f19c5000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9000) = 0x2b13f19c5000
> >> close(3) = 0
> >> mprotect(0x2b13f19c5000, 4096, PROT_READ) = 0
> >> munmap(0x2b13f17a4000, 94877) = 0
> >> open("/etc/passwd", O_RDONLY) = 3
> >> fcntl(3, F_GETFD) = 0
> >> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> >> read(3, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> >> close(3) = 0
> >> munmap(0x2b13f17a4000, 4096) = 0
> >> socket(PF_FILE, SOCK_STREAM, 0) = 3
> >> unlink("/var/run/heartbeat/lrm_cmd_sock") = 0
> >> bind(3, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_cmd_sock"...}, 110) = 0
> >> chmod("/var/run/heartbeat/lrm_cmd_sock", 0777) = 0
> >> listen(3, 10) = 0
> >> fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
> >> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> >> socket(PF_FILE, SOCK_STREAM, 0) = 4
> >> unlink("/var/run/heartbeat/lrm_callback_sock") = 0
> >> bind(4, {sa_family=AF_FILE, path="/var/run/heartbeat/lrm_callback_sock"...}, 110) = 0
> >> chmod("/var/run/heartbeat/lrm_callback_sock", 0777) = 0
> >> listen(4, 10) = 0
> >> fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
> >> fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> >> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> >> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> >> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> >> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(5) = 0
> >> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> >> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> >> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> >> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> >> write(5, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
> >> munmap(0x2b13f17a4000, 4096) = 0
> >> close(5) = 0
> >> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGCHLD, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGCHLD, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> >> write(2, "debug: Enabling coredumps\n", 26) = 26
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> >> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> >> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(5) = 0
> >> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> >> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> >> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> >> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> >> write(5, "lrmd: [16907]: debug: Enabling c"..., 42) = 42
> >> munmap(0x2b13f17a4000, 4096) = 0
> >> close(5) = 0
> >> chdir("/var/lib/heartbeat/cores") = 0
> >> getuid() = 0
> >> open("/etc/passwd", O_RDONLY) = 5
> >> fcntl(5, F_GETFD) = 0
> >> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> >> fstat(5, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> >> read(5, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> >> close(5) = 0
> >> munmap(0x2b13f17a4000, 4096) = 0
> >> chdir("root") = 0
> >> getrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=RLIM_INFINITY}) = 0
> >> setrlimit(RLIMIT_CORE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
> >> open("/proc/sys/kernel/core_pattern", O_RDONLY) = 5
> >> read(5, "core\n", 256) = 5
> >> close(5) = 0
> >> open("/proc/sys/kernel/core_uses_pid", O_RDONLY) = 5
> >> read(5, "1\n", 2) = 2
> >> close(5) = 0
> >> rt_sigaction(SIGQUIT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGILL, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGABRT, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGFPE, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGSEGV, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGBUS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGSYS, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGTRAP, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGXCPU, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGXFSZ, {0x3bc2e171a0, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> getgid() = 0
> >> getuid() = 0
> >> open("/etc/passwd", O_RDONLY) = 5
> >> fcntl(5, F_GETFD) = 0
> >> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> >> fstat(5, {st_mode=S_IFREG|0644, st_size=1866, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> >> read(5, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1866
> >> close(5) = 0
> >> munmap(0x2b13f17a4000, 4096) = 0
> >> setresgid(-1, 99, -1) = 0
> >> setresuid(-1, 99, -1) = 0
> >> prctl(0x4, 0x1, 0, 0, 0) = 0
> >> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> >> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> >> setresuid(-1, 0, -1) = 0
> >> setresgid(-1, 0, -1) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> >> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> >> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(5) = 0
> >> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> >> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> >> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> >> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> >> write(5, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
> >> munmap(0x2b13f17a4000, 4096) = 0
> >> close(5) = 0
> >> setresgid(-1, 99, -1) = 0
> >> setresuid(-1, 99, -1) = 0
> >> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGUSR1, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGUSR1, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> >> write(2, "info: G_main_add_SignalHandler: "..., 67) = 67
> >> setresuid(-1, 0, -1) = 0
> >> setresgid(-1, 0, -1) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> >> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> >> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(5) = 0
> >> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> >> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> >> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> >> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> >> write(5, "lrmd: [16907]: info: G_main_add_"..., 83) = 83
> >> munmap(0x2b13f17a4000, 4096) = 0
> >> close(5) = 0
> >> setresgid(-1, 99, -1) = 0
> >> setresuid(-1, 99, -1) = 0
> >> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> rt_sigaction(SIGUSR2, NULL, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, 8) = 0
> >> rt_sigaction(SIGUSR2, {0x3bc2e19640, [], SA_RESTORER, 0x31cca30280}, NULL, 8) = 0
> >> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> >> write(2, "debug: main: run the loop...\n", 29) = 29
> >> setresuid(-1, 0, -1) = 0
> >> setresgid(-1, 0, -1) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> >> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> >> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(5) = 0
> >> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> >> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> >> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> >> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> >> write(5, "lrmd: [16907]: debug: main: run "..., 45) = 45
> >> munmap(0x2b13f17a4000, 4096) = 0
> >> close(5) = 0
> >> setresgid(-1, 99, -1) = 0
> >> setresuid(-1, 99, -1) = 0
> >> write(2, "lrmd[16907]: 2009/07/02_17:48:44"..., 33) = 33
> >> write(2, "info: Started.\n", 15) = 15
> >> setresuid(-1, 0, -1) = 0
> >> setresgid(-1, 0, -1) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=331, ...}) = 0
> >> socket(PF_FILE, SOCK_DGRAM, 0) = 5
> >> fcntl(5, F_SETFD, FD_CLOEXEC) = 0
> >> connect(5, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> >> close(5) = 0
> >> open("/dev/console", O_WRONLY|O_NOCTTY) = 5
> >> fstat(5, {st_mode=S_IFCHR|0600, st_rdev=makedev(5, 1), ...}) = 0
> >> ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13f17a4000
> >> lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
> >> write(5, "lrmd: [16907]: info: Started.\r\n", 31) = 31
> >> munmap(0x2b13f17a4000, 4096) = 0
> >> close(5) = 0
> >> setresgid(-1, 99, -1) = 0
> >> setresuid(-1, 99, -1) = 0
> >> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = -1 EINTR (Interrupted system call)
> >> --- SIGINT (Interrupt) @ 0 (0) ---
> >> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000) = 0 (Timeout)
> >> poll([{fd=3, events=POLLIN|POLLPRI}, {fd=4, events=POLLIN|POLLPRI}], 2, 1000
> >> _______________________________________________
> >> Linux-HA mailing list
> >> Linux-HA[at]lists.linux-ha.org
> >> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> >> See also: http://linux-ha.org/ReportingProblems
> > _______________________________________________
> > Linux-HA mailing list
> > Linux-HA[at]lists.linux-ha.org
> > http://lists.linux-ha.org/mailman/listinfo/linux-ha
> > See also: http://linux-ha.org/ReportingProblems
> >
>
>
>
> _______________________________________________
> Linux-HA mailing list
> Linux-HA[at]lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


dejanmm at fastmail

Jul 3, 2009, 12:28 AM

Post #7 of 14 (631 views)
Permalink
Re: socket of lrmd [In reply to]

Hi again Junko-san,

On Fri, Jul 03, 2009 at 04:15:40PM +0900, Junko IKEDA wrote:
> Hi,
>
> On Fri, 03 Jul 2009 10:59:21 +0900, Junko IKEDA <ikedaj[at]intellilink.co.jp> wrote:
>
> > Hi Dejan,
> >
> > Your patch could stop the error message from LVM RA.
> > Many thanks!
> > But I run Heartbeat 2.1.4 so I worry about whether 2.1.4 still have a problem about stonithd that you pointed.
> >
> > By the way, does closefiles() close both of the connections for syslogd and ha_logd ?
>
> The patch works well.
> but, what will happen if we don't use it?

There will be an unused file open.

> It seems that lrmd keeps the file descriptor (3) for
> ha_logd/syslogd while running.

Right.

> Will the number of the file descriptor reach its max limit without this patch?

No, there's always only one connection. That's inherited from the
parent process.

Thanks,

Dejan

>
> Thanks,
> Junko
>
> >
> > On Thu, 02 Jul 2009 22:04:06 +0900, Dejan Muhamedagic <dejanmm[at]fastmail.fm> wrote:
> >
> >> Hi,
> >>
> >> On Thu, Jul 02, 2009 at 06:40:26PM +0900, Junko IKEDA wrote:
> >>> Hi,
> >>>
> >>> I try to strace of lrmd.
> >>>
> >>> (1) running syslog, and strace
> >>> # service syslog start
> >>> # strace -o /tmp/strace-syslog-start.txt /usr/lib64/heartbeat/lrmd -rv
> >>>
> >>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> >>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> >>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
> >>> sendto(3, "<30>Jul 2 17:02:46 lrmd: [16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> >>>
> >>> (2) stop syslog and strace
> >>> # service syslog stop
> >>> # strace -o /tmp/strace-syslog-stop.txt /usr/lib64/heartbeat/lrmd -rv
> >>>
> >>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> >>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> >>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> >>> close(3) = 0
> >>> open("/dev/console", O_WRONLY|O_NOCTTY) = 3
> >>>
> >>> (for more details, see the attached files)
> >>>
> >>> Is the socket of lrmd for syslog?
> >>> wild guess?
> >>
> >> Right. See lib/lrm/racommon.c:closefiles(). I must say I can't
> >> remember why exactly was the connection to syslogd/ha_logd
> >> needed. There has been a problem with stonithd, but now this code
> >> is not in that execution path anymore. Perhaps you can test this
> >> patch:
> >>
> >> diff -r 731f8f7b5450 lib/lrm/racommon.c
> >> --- a/lib/lrm/racommon.c Tue Jun 30 12:02:16 2009 +0200
> >> +++ b/lib/lrm/racommon.c Thu Jul 02 15:02:55 2009 +0200
> >> @@ -145,7 +145,7 @@
> >>
> >> /* close all descriptors except stdin/out/err and
> >> * channels to logd */
> >> for (fd = getdtablesize(); fd > STDERR_FILENO; fd--) {
> >> - if (!cl_log_is_logd_fd(fd))
> >> + /*if (!cl_log_is_logd_fd(fd))*/
> >> close(fd);
> >> }
> >> }
> >>
> >> Thanks,
> >>
> >> Dejan
>
> _______________________________________________
> Linux-HA mailing list
> Linux-HA[at]lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


ikedaj at intellilink

Jul 3, 2009, 12:37 AM

Post #8 of 14 (631 views)
Permalink
Re: socket of lrmd [In reply to]

Hi again, :)

Thank you for your quick reply.
Our customer might hesitate to apply the new patch for their running system at once.
(I don't know their upgrade plan unfortunately)
So I want to know whether Heartbeat can run safely without your patch for a while.
I know that we should recommend the latest code, of course. :)

Thanks,
Junko

On Fri, 03 Jul 2009 16:28:20 +0900, Dejan Muhamedagic <dejanmm[at]fastmail.fm> wrote:

> Hi again Junko-san,
>
> On Fri, Jul 03, 2009 at 04:15:40PM +0900, Junko IKEDA wrote:
>> Hi,
>>
>> On Fri, 03 Jul 2009 10:59:21 +0900, Junko IKEDA <ikedaj[at]intellilink.co.jp> wrote:
>>
>> > Hi Dejan,
>> >
>> > Your patch could stop the error message from LVM RA.
>> > Many thanks!
>> > But I run Heartbeat 2.1.4 so I worry about whether 2.1.4 still have a problem about stonithd that you pointed.
>> >
>> > By the way, does closefiles() close both of the connections for syslogd and ha_logd ?
>>
>> The patch works well.
>> but, what will happen if we don't use it?
>
> There will be an unused file open.
>
>> It seems that lrmd keeps the file descriptor (3) for
>> ha_logd/syslogd while running.
>
> Right.
>
>> Will the number of the file descriptor reach its max limit without this patch?
>
> No, there's always only one connection. That's inherited from the
> parent process.
>
> Thanks,
>
> Dejan
>
>>
>> Thanks,
>> Junko
>>
>> >
>> > On Thu, 02 Jul 2009 22:04:06 +0900, Dejan Muhamedagic <dejanmm[at]fastmail.fm> wrote:
>> >
>> >> Hi,
>> >>
>> >> On Thu, Jul 02, 2009 at 06:40:26PM +0900, Junko IKEDA wrote:
>> >>> Hi,
>> >>>
>> >>> I try to strace of lrmd.
>> >>>
>> >>> (1) running syslog, and strace
>> >>> # service syslog start
>> >>> # strace -o /tmp/strace-syslog-start.txt /usr/lib64/heartbeat/lrmd -rv
>> >>>
>> >>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
>> >>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>> >>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
>> >>> sendto(3, "<30>Jul 2 17:02:46 lrmd: [16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
>> >>>
>> >>> (2) stop syslog and strace
>> >>> # service syslog stop
>> >>> # strace -o /tmp/strace-syslog-stop.txt /usr/lib64/heartbeat/lrmd -rv
>> >>>
>> >>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
>> >>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>> >>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
>> >>> close(3) = 0
>> >>> open("/dev/console", O_WRONLY|O_NOCTTY) = 3
>> >>>
>> >>> (for more details, see the attached files)
>> >>>
>> >>> Is the socket of lrmd for syslog?
>> >>> wild guess?
>> >>
>> >> Right. See lib/lrm/racommon.c:closefiles(). I must say I can't
>> >> remember why exactly was the connection to syslogd/ha_logd
>> >> needed. There has been a problem with stonithd, but now this code
>> >> is not in that execution path anymore. Perhaps you can test this
>> >> patch:
>> >>
>> >> diff -r 731f8f7b5450 lib/lrm/racommon.c
>> >> --- a/lib/lrm/racommon.c Tue Jun 30 12:02:16 2009 +0200
>> >> +++ b/lib/lrm/racommon.c Thu Jul 02 15:02:55 2009 +0200
>> >> @@ -145,7 +145,7 @@
>> >>
>> >> /* close all descriptors except stdin/out/err and
>> >> * channels to logd */
>> >> for (fd = getdtablesize(); fd > STDERR_FILENO; fd--) {
>> >> - if (!cl_log_is_logd_fd(fd))
>> >> + /*if (!cl_log_is_logd_fd(fd))*/
>> >> close(fd);
>> >> }
>> >> }
>> >>
>> >> Thanks,
>> >>
>> >> Dejan
>>
>> _______________________________________________
>> Linux-HA mailing list
>> Linux-HA[at]lists.linux-ha.org
>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> See also: http://linux-ha.org/ReportingProblems
> _______________________________________________
> Linux-HA mailing list
> Linux-HA[at]lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>



_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


dejanmm at fastmail

Jul 3, 2009, 1:50 AM

Post #9 of 14 (631 views)
Permalink
Re: socket of lrmd [In reply to]

Hi Junko-san,

On Fri, Jul 03, 2009 at 04:37:44PM +0900, Junko IKEDA wrote:
> Hi again, :)
>
> Thank you for your quick reply.
> Our customer might hesitate to apply the new patch for their running system at once.

Of course.

> (I don't know their upgrade plan unfortunately)
> So I want to know whether Heartbeat can run safely without your patch for a while.

Definitely.

> I know that we should recommend the latest code, of course. :)

Yes.

Thanks,

Dejan

> Thanks,
> Junko
>
> On Fri, 03 Jul 2009 16:28:20 +0900, Dejan Muhamedagic <dejanmm[at]fastmail.fm> wrote:
>
> > Hi again Junko-san,
> >
> > On Fri, Jul 03, 2009 at 04:15:40PM +0900, Junko IKEDA wrote:
> >> Hi,
> >>
> >> On Fri, 03 Jul 2009 10:59:21 +0900, Junko IKEDA <ikedaj[at]intellilink.co.jp> wrote:
> >>
> >> > Hi Dejan,
> >> >
> >> > Your patch could stop the error message from LVM RA.
> >> > Many thanks!
> >> > But I run Heartbeat 2.1.4 so I worry about whether 2.1.4 still have a problem about stonithd that you pointed.
> >> >
> >> > By the way, does closefiles() close both of the connections for syslogd and ha_logd ?
> >>
> >> The patch works well.
> >> but, what will happen if we don't use it?
> >
> > There will be an unused file open.
> >
> >> It seems that lrmd keeps the file descriptor (3) for
> >> ha_logd/syslogd while running.
> >
> > Right.
> >
> >> Will the number of the file descriptor reach its max limit without this patch?
> >
> > No, there's always only one connection. That's inherited from the
> > parent process.
> >
> > Thanks,
> >
> > Dejan
> >
> >>
> >> Thanks,
> >> Junko
> >>
> >> >
> >> > On Thu, 02 Jul 2009 22:04:06 +0900, Dejan Muhamedagic <dejanmm[at]fastmail.fm> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> On Thu, Jul 02, 2009 at 06:40:26PM +0900, Junko IKEDA wrote:
> >> >>> Hi,
> >> >>>
> >> >>> I try to strace of lrmd.
> >> >>>
> >> >>> (1) running syslog, and strace
> >> >>> # service syslog start
> >> >>> # strace -o /tmp/strace-syslog-start.txt /usr/lib64/heartbeat/lrmd -rv
> >> >>>
> >> >>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> >> >>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> >> >>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
> >> >>> sendto(3, "<30>Jul 2 17:02:46 lrmd: [16211"..., 102, MSG_NOSIGNAL, NULL, 0) = 102
> >> >>>
> >> >>> (2) stop syslog and strace
> >> >>> # service syslog stop
> >> >>> # strace -o /tmp/strace-syslog-stop.txt /usr/lib64/heartbeat/lrmd -rv
> >> >>>
> >> >>> socket(PF_FILE, SOCK_DGRAM, 0) = 3
> >> >>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
> >> >>> connect(3, {sa_family=AF_FILE, path="/dev/log"...}, 110) = -1 ENOENT (No such file or directory)
> >> >>> close(3) = 0
> >> >>> open("/dev/console", O_WRONLY|O_NOCTTY) = 3
> >> >>>
> >> >>> (for more details, see the attached files)
> >> >>>
> >> >>> Is the socket of lrmd for syslog?
> >> >>> wild guess?
> >> >>
> >> >> Right. See lib/lrm/racommon.c:closefiles(). I must say I can't
> >> >> remember why exactly was the connection to syslogd/ha_logd
> >> >> needed. There has been a problem with stonithd, but now this code
> >> >> is not in that execution path anymore. Perhaps you can test this
> >> >> patch:
> >> >>
> >> >> diff -r 731f8f7b5450 lib/lrm/racommon.c
> >> >> --- a/lib/lrm/racommon.c Tue Jun 30 12:02:16 2009 +0200
> >> >> +++ b/lib/lrm/racommon.c Thu Jul 02 15:02:55 2009 +0200
> >> >> @@ -145,7 +145,7 @@
> >> >>
> >> >> /* close all descriptors except stdin/out/err and
> >> >> * channels to logd */
> >> >> for (fd = getdtablesize(); fd > STDERR_FILENO; fd--) {
> >> >> - if (!cl_log_is_logd_fd(fd))
> >> >> + /*if (!cl_log_is_logd_fd(fd))*/
> >> >> close(fd);
> >> >> }
> >> >> }
> >> >>
> >> >> Thanks,
> >> >>
> >> >> Dejan
> >>
> >> _______________________________________________
> >> Linux-HA mailing list
> >> Linux-HA[at]lists.linux-ha.org
> >> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> >> See also: http://linux-ha.org/ReportingProblems
> > _______________________________________________
> > Linux-HA mailing list
> > Linux-HA[at]lists.linux-ha.org
> > http://lists.linux-ha.org/mailman/listinfo/linux-ha
> > See also: http://linux-ha.org/ReportingProblems
> >
>
>
>
> _______________________________________________
> Linux-HA mailing list
> Linux-HA[at]lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


lars.ellenberg at linbit

Jul 3, 2009, 1:59 AM

Post #10 of 14 (631 views)
Permalink
Re: socket of lrmd [In reply to]

On Fri, Jul 03, 2009 at 09:28:20AM +0200, Dejan Muhamedagic wrote:
> Hi again Junko-san,
>
> On Fri, Jul 03, 2009 at 04:15:40PM +0900, Junko IKEDA wrote:
> > Hi,
> >
> > On Fri, 03 Jul 2009 10:59:21 +0900, Junko IKEDA <ikedaj[at]intellilink.co.jp> wrote:
> >
> > > Hi Dejan,
> > >
> > > Your patch could stop the error message from LVM RA.
> > > Many thanks!
> > > But I run Heartbeat 2.1.4 so I worry about whether 2.1.4 still have a problem about stonithd that you pointed.
> > >
> > > By the way, does closefiles() close both of the connections for syslogd and ha_logd ?
> >
> > The patch works well.
> > but, what will happen if we don't use it?
>
> There will be an unused file open.
>
> > It seems that lrmd keeps the file descriptor (3) for
> > ha_logd/syslogd while running.
>
> Right.
>
> > Will the number of the file descriptor reach its max limit without this patch?
>
> No, there's always only one connection. That's inherited from the
> parent process.

All those FDs should probably be opened with the "O_CLOEXEC" flag,
or alternatively by
flags = fcntl(fd, F_GETFD); /*error handling*/
fcntl(fd, F_SETFD, flags | FD_CLOEXEC); /*error handling*/


--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


ikedaj at intellilink

Jul 3, 2009, 3:02 AM

Post #11 of 14 (630 views)
Permalink
Re: socket of lrmd [In reply to]

Hi,

Thank you for your comment.

By the way, this is the tips for LVM RA...

It is possible to prevent the warning from LVM command,
like "File descriptor 3 (socket:[375969]) leaked on vgdisplay invocation. Parent PID 18253: /bin/sh"
if we set "LVM_SUPPRESS_FD_WARNINGS" in LVM RA.

--- /usr/lib/ocf/resource.d/heartbeat/LVM.org 2009-07-03 18:04:06.000000000 +0900
+++ /usr/lib/ocf/resource.d/heartbeat/LVM 2009-07-03 18:02:03.000000000 +0900
@@ -27,6 +27,7 @@
# Initialization:

. ${OCF_ROOT}/resource.d/heartbeat/.ocf-shellfuncs
+export LVM_SUPPRESS_FD_WARNINGS=1

#######################################################################

We can get through the LVM's warning message with this way for a while if our customer don't want to apply the patch at once.

Thanks,
Junko

On Fri, 03 Jul 2009 17:50:16 +0900, Dejan Muhamedagic <dejanmm[at]fastmail.fm> wrote:

> Hi Junko-san,
>
> On Fri, Jul 03, 2009 at 04:37:44PM +0900, Junko IKEDA wrote:
>> Hi again, :)
>>
>> Thank you for your quick reply.
>> Our customer might hesitate to apply the new patch for their running system at once.
>
> Of course.
>
>> (I don't know their upgrade plan unfortunately)
>> So I want to know whether Heartbeat can run safely without your patch for a while.
>
> Definitely.
>
>> I know that we should recommend the latest code, of course. :)
>
> Yes.


_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


dejanmm at fastmail

Jul 3, 2009, 3:10 AM

Post #12 of 14 (630 views)
Permalink
Re: socket of lrmd [In reply to]

Hi,

On Fri, Jul 03, 2009 at 10:59:04AM +0200, Lars Ellenberg wrote:
> On Fri, Jul 03, 2009 at 09:28:20AM +0200, Dejan Muhamedagic wrote:
> > Hi again Junko-san,
> >
> > On Fri, Jul 03, 2009 at 04:15:40PM +0900, Junko IKEDA wrote:
> > > Hi,
> > >
> > > On Fri, 03 Jul 2009 10:59:21 +0900, Junko IKEDA <ikedaj[at]intellilink.co.jp> wrote:
> > >
> > > > Hi Dejan,
> > > >
> > > > Your patch could stop the error message from LVM RA.
> > > > Many thanks!
> > > > But I run Heartbeat 2.1.4 so I worry about whether 2.1.4 still have a problem about stonithd that you pointed.
> > > >
> > > > By the way, does closefiles() close both of the connections for syslogd and ha_logd ?
> > >
> > > The patch works well.
> > > but, what will happen if we don't use it?
> >
> > There will be an unused file open.
> >
> > > It seems that lrmd keeps the file descriptor (3) for
> > > ha_logd/syslogd while running.
> >
> > Right.
> >
> > > Will the number of the file descriptor reach its max limit without this patch?
> >
> > No, there's always only one connection. That's inherited from the
> > parent process.
>
> All those FDs should probably be opened with the "O_CLOEXEC" flag,
> or alternatively by
> flags = fcntl(fd, F_GETFD); /*error handling*/
> fcntl(fd, F_SETFD, flags | FD_CLOEXEC); /*error handling*/

Never heard of CLOEXEC. According to the manpage it's also linux
specific.

Thanks,

Dejan

> --
> : Lars Ellenberg
> : LINBIT | Your Way to High Availability
> : DRBD/HA support and consulting http://www.linbit.com
>
> DRBD? and LINBIT? are registered trademarks of LINBIT, Austria.
> _______________________________________________
> Linux-HA mailing list
> Linux-HA[at]lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


dejanmm at fastmail

Jul 3, 2009, 3:11 AM

Post #13 of 14 (630 views)
Permalink
Re: socket of lrmd [In reply to]

Hi Junko-san,

On Fri, Jul 03, 2009 at 07:02:52PM +0900, Junko IKEDA wrote:
> Hi,
>
> Thank you for your comment.
>
> By the way, this is the tips for LVM RA...
>
> It is possible to prevent the warning from LVM command,
> like "File descriptor 3 (socket:[375969]) leaked on vgdisplay invocation. Parent PID 18253: /bin/sh"
> if we set "LVM_SUPPRESS_FD_WARNINGS" in LVM RA.
>
> --- /usr/lib/ocf/resource.d/heartbeat/LVM.org 2009-07-03 18:04:06.000000000 +0900
> +++ /usr/lib/ocf/resource.d/heartbeat/LVM 2009-07-03 18:02:03.000000000 +0900
> @@ -27,6 +27,7 @@
> # Initialization:
>
> . ${OCF_ROOT}/resource.d/heartbeat/.ocf-shellfuncs
> +export LVM_SUPPRESS_FD_WARNINGS=1
>
> #######################################################################
>
> We can get through the LVM's warning message with this way for
> a while if our customer don't want to apply the patch at once.

OK. Thanks for the tip. But I think that once it is thoroughly
tested we will just close all unnecessary files.

Thanks,

Dejan

>
> Thanks,
> Junko
>
> On Fri, 03 Jul 2009 17:50:16 +0900, Dejan Muhamedagic <dejanmm[at]fastmail.fm> wrote:
>
> > Hi Junko-san,
> >
> > On Fri, Jul 03, 2009 at 04:37:44PM +0900, Junko IKEDA wrote:
> >> Hi again, :)
> >>
> >> Thank you for your quick reply.
> >> Our customer might hesitate to apply the new patch for their running system at once.
> >
> > Of course.
> >
> >> (I don't know their upgrade plan unfortunately)
> >> So I want to know whether Heartbeat can run safely without your patch for a while.
> >
> > Definitely.
> >
> >> I know that we should recommend the latest code, of course. :)
> >
> > Yes.
>
>
> _______________________________________________
> Linux-HA mailing list
> Linux-HA[at]lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems


lars.ellenberg at linbit

Jul 3, 2009, 4:06 AM

Post #14 of 14 (629 views)
Permalink
Re: socket of lrmd [In reply to]

On Fri, Jul 03, 2009 at 12:10:17PM +0200, Dejan Muhamedagic wrote:
> > All those FDs should probably be opened with the "O_CLOEXEC" flag,
> > or alternatively by
> > flags = fcntl(fd, F_GETFD); /*error handling*/
> > fcntl(fd, F_SETFD, flags | FD_CLOEXEC); /*error handling*/
>
> Never heard of CLOEXEC. According to the manpage it's also linux
> specific.

fcntl FD_CLOEXEC is POSIX.

O_CLOEXEC is linux specific, and was introduced in linux 2.6.23 (or so
the manpage says), to avoid the race in multi-threaded applications:
t1: t2:
open
exec -----> inherited fd :(
fcntl(FD_CLOEXEC)

--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
_______________________________________________
Linux-HA mailing list
Linux-HA[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

Linux-HA users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.