Nuremberg, The Truth is in the Logs

Transkript

1 Nuremberg, The Truth is in the Logs

2 Jan Doberstein #OSS #bash #vi Capgemini Outsourcing Services Infrastructur Engineer wer-kennt-wen.de GmbH System Operator Linux 1blu AG System Engineer (Shared Hosting) Teamlead System Operations HostEurope GmbH Teamlead Shared Hosting Operations Postmaster 2

3 Eine Logdatei enthält das automatisch geführte Protokoll aller oder bestimmter Aktionen von Prozessen auf einem Computersystem Wichtige Anwendungen finden sich vor allem bei der Prozesskontrolle und Automatisierung. Prinzipiell werden alle Aktionen mitgeschrieben, die für eine spätere Untersuchung erforderlich sind oder sein könnten. 3

4 [18/Mar/2013:12:43: ] "GET /health/currentthroughput HTTP/1.1" " "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/ (KHTML, like Gecko) Chrome/ Safari/537.22" [18/Mar/2013:13:06: ] "GET /visuals/fetch/streamgraph?hours=2&stream_id=514 6bccd772ae72c HTTP/1.1" " user/messages" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/ (KHTML, like Gecko) Chrome/ Safari/537.22" [18/Mar/2013:13:06: ] "GET /messages HTTP/1.1" " user/messages" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/ (KHTML, like Gecko) Chrome/ Safari/537.22" Mar 19 09:30:01 sv /usr/sbin/cron[7657]: (root) CMD (/root/bin/http_connection >> /tmp/http_connection.log) Mar 19 09:30:02 sv sshd[7761]: Accepted publickey for root from port ssh2 Mar 19 09:31:01 sv /usr/sbin/cron[8098]: (/root/bin/http_connection >> /tmp/http_connection.log) Mar 19 09:30:02 sv sshd[7761]: Accepted publickey for root from port ssh2 Mar 19 09:30:03 sv sshd[7795]: Accepted publickey for root from port ssh2 Mar 19 09:31:01 sv /usr/sbin/cron[8098]: (root) CMD (/root/bin/http_connection >> /tmp/http_connection.log) 4

5 5

6 6

7 7

8 8

9 9

10 syslog ist ein De-facto-Standard zur Übermittlung von Log- Meldungen [...] Der Begriff syslog wird oft sowohl für das eigentliche syslog- Netzwerkprotokoll als auch für die Anwendung oder Bibliothek benutzt, die syslog-meldungen sendet oder empfängt. 10

11 11

12 12

13 13

14 14

15 Graylog2 enables you to unleash the power that lays inside your logs. Use it to run analytics, alerting, monitoring and powerful searches over your whole log base. [ ] Create streams for every consumer and have them always only one click away. 15

16 16

17 17

18 18

19 - Syslog - GELF The Graylog Extended Log Format 19

20 20

21 21

22 logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). 22

23 23 Input Filter Output amqp drupal_dblog eventlog exec file ganglia gelf gemfire generator heroku irc log4j lumberjack pipe redis relp sqs stdin stomp syslog tcp twitter udp xmpp zenoss zeromq alter anonymize checksum csv date dns environment gelfify geoip grep grok grokdiscovery json kv metrics multiline mutate noop split syslog_pri urldecode xml zeromq amqp boundary circonus cloudwatch datadog elasticsearch elasticsearch_http elasticsearch_river exec file ganglia gelf gemfire graphite graphtastic http internal irc juggernaut librato loggly lumberjack metriccatcher mongodb nagios nagios_nsca opentsdb pagerduty pipe redis riak riemann sns sqs statsd stdout stomp syslog tcp websocket xmpp zabbix zeromq

24 24

25 25

26 26

27 statsd graphite 27

28 statsd Metriken graphite Monitoring Zentrale Filterung Zentrale Einsicht 28

29 statsd graphite 29

30 30

31 31

32 Keine Raketenwissenschaft Jim Parsons in The Big Bang Theory 32

33 Analyse Monitoring Reporting 33

34 Credits: Jan Doberstein Infrastructur Engineer -Icons "Nerdy-Lines" by Christoph Brill - Graylog2 Gorilla from the oatmeal ( - Logstash Logo from by Jordan Font : Source Sans Pro by Paul D. Hunt

35 INGRESS for Beginners by Kris Köhntopp 18:15 Ofenbar neben Rezeption Start: 19:00, Hotel Lobby