SP logging to syslog can not set syslogHost

[Mark Vinkx]

Hi

I use the shibboleth SP 2.5.3 with apache 2.4 on Windows Server 2008R2 sp 1
I try to send the logs to a syslog server. I use the following logger configuration
If a use procmon I see that UPD packets are sent to 192.255.255.255:syslog. The ip adres should be that of testhost 192.168.99.18

What do I do wrong ?

log4j.rootCategory=DEBUG, syslog

log4j.category.XMLTooling.XMLObject=INFO
log4j.category.XMLTooling.KeyInfoResolver=INFO
log4j.category.Shibboleth.PropertySet=INFO
log4j.category.XMLTooling.libcurl=INFO

log4j.appender.syslog=org.apache.log4j.SyslogAppender
log4j.appender.syslog.syslogName=shibboleth
log4j.appender.syslog.portNumber=514
log4j.appender.syslog.syslogHost=testhost
#log4j.appender.syslog.layout=org.apache.log4j.BasicLayout
log4j.appender.syslog.layout=org.apache.log4j.PatternLayout
log4j.appender.syslog.layout.ConversionPattern=%d{%Y-%m-%d %H:%M:%S} %p %c %x: %m%n

[Cantor, Scott]

Post by Mark Vinkx
I use the shibboleth SP 2.5.3 with apache 2.4 on Windows Server 2008R2 sp 1
I try to send the logs to a syslog server. I use the following logger configuration
If a use procmon I see that UPD packets are sent to
192.255.255.255:syslog. The ip adres should be that of testhost
192.168.99.18
What do I do wrong ?

I've never used it.

My suggestion to anybody trying to use syslog would be to just run
something like minion to tail the log and remote the entries that way. I
would be leery in the extreme of tying the SP to a remote logging source.

But if somebody else has made it work, they're welcome to comment.

-- Scott

[Nate Klingenstein]

My preferred approach for pulling logs together across nodes is rsyslog best-effort over tcp as a secondary logging mechanism to a local on-disk something. That gives you a higher probability of capturing interesting events no matter when or where they happen or what is done to the node subsequently, without introducing a centralized single point of failure.

On Jul 30, 2014, at 10:33 AM, "Cantor, Scott" <cantor.2-ZbGKxL/***@public.gmane.org<mailto:cantor.2-ZbGKxL/***@public.gmane.org>> wrote:

My suggestion to anybody trying to use syslog would be to just run
something like minion to tail the log and remote the entries that way. I
would be leery in the extreme of tying the SP to a remote logging source.

But if somebody else has made it work, they're welcome to comment.

[Jacob Lundberg]

Hi all,

Post by Cantor, Scott
My suggestion to anybody trying to use syslog would be to just run
something like minion to tail the log and remote the entries that way.

The log tailing systems we have tried have some drawbacks when you try
to use them at scale ... eventually you find some instances that have
stopped forwarding logs or lost their place and forwarded multiple
copies of the same logs ... rsyslog is our current syslog of choice and
it has both of these problems.

Post by Cantor, Scott
But if somebody else has made it work, they're welcome to comment.

We use the syslog logger. It has some drawbacks, mostly because it
creates only one configuration instance that is shared. Probably the
issue for the OP is they have another instance of syslogging from the
same process that is created later and overwrites the destination.

If you can live within the constraints of the syslog logger it works
well. (I know this may not work for a Windows user, but) we send all
the SP logs to localhost and then rsyslog writes them both to local disk
based on string matching and over the network to our centralized log
processing service.

Here are some diffs relative to the Debian defaults:


--- /etc/shibboleth/native.logger.orig 2013-06-18 15:28:42.391040157 -0700
+++ /etc/shibboleth/native.logger 2014-03-06 19:05:45.941357897 -0800
@@ -51,4 +51,6 @@
log4j.appender.native_log=org.apache.log4j.LocalSyslogAppender
log4j.appender.native_log.syslogName=shibboleth-sp
log4j.appender.native_log.facility=3
-log4j.appender.native_log.layout=org.apache.log4j.BasicLayout
+#log4j.appender.native_log.layout=org.apache.log4j.BasicLayout
+log4j.appender.native_log.layout=org.apache.log4j.PatternLayout
+log4j.appender.native_log.layout.ConversionPattern=%p %c %x: %m%n


--- /etc/shibboleth/shibd.logger.orig 2013-06-18 15:28:51.991122948 -0700
+++ /etc/shibboleth/shibd.logger 2014-03-06 19:05:37.701284666 -0800
@@ -1,5 +1,5 @@
# set overall behavior
-log4j.rootCategory=INFO, shibd_log, warn_log
+log4j.rootCategory=INFO, shibd_log

# fairly verbose for DEBUG, so generally leave at INFO
log4j.category.XMLTooling.XMLObject=INFO
@@ -35,27 +35,21 @@

# define the appenders

-log4j.appender.shibd_log=org.apache.log4j.RollingFileAppender
-log4j.appender.shibd_log.fileName=/var/log/shibboleth/shibd.log
-log4j.appender.shibd_log.maxFileSize=1000000
-log4j.appender.shibd_log.maxBackupIndex=10
+log4j.appender.shibd_log=org.apache.log4j.LocalSyslogAppender
+log4j.appender.shibd_log.syslogName=shibd
+log4j.appender.shibd_log.facility=3
+#log4j.appender.shibd_log.layout=org.apache.log4j.BasicLayout
log4j.appender.shibd_log.layout=org.apache.log4j.PatternLayout
-log4j.appender.shibd_log.layout.ConversionPattern=%d{%Y-%m-%d %H:%M:%S} %p %c %x: %m%n
+log4j.appender.shibd_log.layout.ConversionPattern=%p %c %x: %m%n

-log4j.appender.warn_log=org.apache.log4j.RollingFileAppender
-log4j.appender.warn_log.fileName=/var/log/shibboleth/shibd_warn.log
-log4j.appender.warn_log.maxFileSize=1000000
-log4j.appender.warn_log.maxBackupIndex=10
-log4j.appender.warn_log.layout=org.apache.log4j.PatternLayout
-log4j.appender.warn_log.layout.ConversionPattern=%d{%Y-%m-%d %H:%M:%S} %p %c %x: %m%n
-log4j.appender.warn_log.threshold=WARN
-
-log4j.appender.tran_log=org.apache.log4j.RollingFileAppender
-log4j.appender.tran_log.fileName=/var/log/shibboleth/transaction.log
-log4j.appender.tran_log.maxFileSize=1000000
-log4j.appender.tran_log.maxBackupIndex=20
+log4j.appender.tran_log=org.apache.log4j.LocalSyslogAppender
+# This is broken in Shibboleth's forked log4cpp.
+#log4j.appender.tran_log.syslogName=shibd-transaction
+log4j.appender.tran_log.syslogName=shibd
+log4j.appender.tran_log.facility=3
+#log4j.appender.tran_log.layout=org.apache.log4j.BasicLayout
log4j.appender.tran_log.layout=org.apache.log4j.PatternLayout
-log4j.appender.tran_log.layout.ConversionPattern=%d{%Y-%m-%d %H:%M:%S} %p %c %x: %m%n
+log4j.appender.tran_log.layout.ConversionPattern=%p %c %x: %m%n

log4j.appender.sig_log=org.apache.log4j.FileAppender
log4j.appender.sig_log.fileName=/var/log/shibboleth/signature.log


And some rsyslog configuration:


***@server:~# cat /etc/rsyslog.d/50_action_templates.conf
$EscapeControlCharactersOnReceive off
$template showPrio,"%PRI-text%: %timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n"


***@server:~# cat /etc/rsyslog.d/60_tagselect.conf
# native log from Shibboleth SP Apache module
:programname, isequal, "shibboleth-sp" /var/log/shibboleth/shib-sp.log
& ~

# transaction log from Shibboleth SP shibd
:msg, contains, "Shibboleth-TRANSACTION" /var/log/shibboleth/transaction.log
& ~

# shibd log from Shibboleth SP shibd
:programname, isequal, "shibd" /var/log/shibboleth/shibd.log
& ~


-Jacob
--
Jacob Lundberg
Director, IT Services
jacob-***@public.gmane.org
503.290.0100 (voice)
503.973.5252 (fax)
503.901.8343 (cell)