When installing a Sumo Logic collector, you have 2 methods of configuration:
- You can continue to configure the collector via the web interface
- Easy to do and many options are just a click or drop-down away
- You can continue to configure the collector via an installed JSON file
- Full control of the configuration, version control, and the ability to use an automation system like Ansible to handle standardized/consistent deployment
But you cannot do both.
There are not a lot of good examples that I have found for configuring the collector via the JSON method and a few options available. For this blog post I am going to use one of my collectors’ configuration files (sumologic-collector.json) and break down things a bit to help out the community.
But before we dive into the JSON file let’s take a quick look at the sumo.conf file (no, these are not my real access credentials for registering a collector):
name=syslog-receiveremail=insert-email-hereaccessid=wbevLDb8wEFPV1accesskey=d1b6766f3fcb005QAijDD4LPQzcbGGaFVHqYBQEfUMoleIm4bff9274afc4aac31syncSources=/etc/sumologic-collector.jsonVery straight forward options above.
- name
- the hostname of the system, and will default to the local hostname if available
- email address associated with a Sumo Logic account, and used if you decide to use the authentication version of registration instead of the accessid/accesskey way
- accessid
- accesskey
- self-explanatory and you generate these credentials via the web interface for collectors
- syncSources
- where the JSON file is read from
Now let’s look at the basic configuration file we can start the process of breaking down the included basic JSON configuration file. The Sumo Logic documentation has the full list of options you can set.
| Variable Name | Example | Comment |
|---|---|---|
name | apache_logs | This is the _source directive and names the source |
automaticDateParsing | true | Tells the collector that it should try to do automatic date parsing. This works well for most normal items though the Fortigate systems log with a key/value layout that requires a little more work |
multilineProcessingEnabled | false | Defaults to false and is very useful for Java/Ruby stack traces |
forceTimeZone | false | If you need to force the timezone to match something that the collector is unable to decide itself |
timeZone | America/Chicago UTC | Set the timezone of the source |
defaultDateFormat | 'date='yyyy-MM-dd 'time='HH:mm:ss | Specifically define how to parse the date - this example is for a Fortigate |
encoding | UTF-8 | How should the incoming data be interpreted |
protocol | TCP UDP | If you are using a port than which protocol would you like to use |
port | 514 | Port number to listen on |
category | unix_logs | Sets the _sourceCategory flag for searching |
sourceType | Syslog LocalFile | Type of item to inject - syslog (over a port) or localfile (a local file to read |
One hint: values of true and false should not have double-quotes added but everything else requires double-quotes.
And if you did not click above to download the JSON file it is below to browse.
{ "api.version": "v1", "sources": [ { "name": "syslog_udp_514", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "America/Chicago", "encoding": "UTF-8", "protocol": "UDP", "port": 514, "category": "unix_logs", "sourceType": "Syslog" }, { "name": "syslog_tcp_514", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "America/Chicago", "encoding": "UTF-8", "protocol": "TCP", "port": 514, "category": "unix_logs", "sourceType": "Syslog" }, { "name": "syslog_udp_1514", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "America/Chicago", "encoding": "UTF-8", "protocol": "UDP", "port": 1514, "category": "juniper_logs", "sourceType": "Syslog" }, { "name": "syslog_tcp_1514", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "America/Chicago", "encoding": "UTF-8", "protocol": "TCP", "port": 1514, "category": "juniper_logs", "sourceType": "Syslog" }, { "name": "syslog_udp_1515", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "America/Chicago", "encoding": "UTF-8", "protocol": "UDP", "port": 1515, "category": "bigip_logs", "sourceType": "Syslog" }, { "name": "syslog_tcp_1515", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "America/Chicago", "encoding": "UTF-8", "protocol": "TCP", "port": 1515, "category": "bigip_logs", "sourceType": "Syslog" }, { "name": "syslog_udp_3514", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "America/Chicago", "encoding": "UTF-8", "protocol": "UDP", "port": 3514, "category": "unix_logs", "sourceType": "Syslog" }, { "name": "syslog_tcp_3514", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "America/Chicago", "encoding": "UTF-8", "protocol": "TCP", "port": 3514, "category": "unix_logs", "sourceType": "Syslog" }, { "name": "fgt_udp_4514", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "America/Chicago", "defaultDateFormat": "'date='yyyy-MM-dd 'time='HH:mm:ss", "encoding": "UTF-8", "protocol": "UDP", "port": 4514, "category": "fw_security", "sourceType": "Syslog" }, { "name": "fgt_tcp_4514", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "America/Chicago", "defaultDateFormat": "'date='yyyy-MM-dd 'time='HH:mm:ss", "encoding": "UTF-8", "protocol": "TCP", "port": 4514, "category": "fw_security", "sourceType": "Syslog" }, { "name": "ubuntu_audit_log", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "os_security", "pathExpression": "/var/log/audit/*.log" }, { "name": "ubuntu_auth_log", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "os_security", "pathExpression": "/var/log/auth.log" }, { "name": "ubuntu_cron", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "unix_logs", "pathExpression": "/var/log/cron" }, { "name": "ubuntu_dmesg", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "unix_logs", "pathExpression": "/var/log/dmesg" }, { "name": "ubuntu_daemon_log", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "unix_logs", "pathExpression": "/var/log/daemon.log" }, { "name": "ubuntu_kern_log", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "unix_logs", "pathExpression": "/var/log/kern.log" }, { "name": "ubuntu_mail_log", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "unix_logs", "pathExpression": "/var/log/mail.log" }, { "name": "ubuntu_mail_err", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "unix_logs", "pathExpression": "/var/log/mail.err" }, { "name": "ubuntu_mail_info", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "unix_logs", "pathExpression": "/var/log/mail.info" }, { "name": "ubuntu_messages", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "unix_logs", "pathExpression": "/var/log/messages" }, { "name": "ubuntu_syslog", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "unix_logs", "pathExpression": "/var/log/syslog" }, { "name": "ubuntu_secure", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "America/Chicago", "category": "os_security", "pathExpression": "/var/log/secure" }, { "name": "nginx_logs", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "UTC", "category": "nginx_logs", "pathExpression": "/var/log/nginx/*.log" }, { "name": "mirrors_udp_1516", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": false, "forceTimeZone": false, "timeZone": "UTC", "encoding": "UTF-8", "protocol": "UDP", "hostName": "mirrors-server", "port": 1516, "category": "nginx_logs", "sourceType": "Syslog" }, { "name": "apache_logs", "sourceType": "LocalFile", "automaticDateParsing": true, "multilineProcessingEnabled": false, "useAutolineMatching": true, "forceTimeZone": false, "timeZone": "UTC", "category": "apache_logs", "pathExpression": "/var/log/apache2/*.log" } ]}