Having an intelligent, scalable and easily searchable consolidation point for log files is a very useful capability. My tool of choice for this is Graylog running on Ubuntu 16.04. Graylog is an open source project that is able to ingest logs from a wide variety of sources and provides very useful visualisation, processing and alerting capabilities. Graylog uses MongoDB for log storage and Elasticsearch to provide full text search capabilities. More information is available here.
This post will go through the details of how to install a single server instance of Graylog 2.3 onto Ubuntu 16.04.
We will start with a minimal Ubuntu 16.04 instance, instructions on this are available here.
Next step is to install the required prerequisites
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install apt-transport-https openjdk-8-jre-headless uuid-runtime pwgen
Then we install MongoDB
sudo apt-get install mongodb-server
And now Elasticsearch
Add and configure the repository
sudo wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list
and now update and install
sudo apt-get update
sudo apt-get install elasticsearch
We now need to configure the cluster name. Fire up your text editor of choice and open /etc/elasticsearch/elasticsearch.yml
Look for the cluster.name value in the Cluster section
Uncomment the cluster.name, set it to graylog and save
Now we can start Elasticsearch and set it to run at startup
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch.service
sudo systemctl restart elasticsearch.service
We are now ready to install Graylog
Add and configure the repository
sudo wget https://packages.graylog2.org/repo/packages/graylog-2.3-repository_latest.deb
sudo dpkg -i graylog-2.3-repository_latest.deb
and now update and install
sudo apt-get update
sudo apt-get install graylog-server
Graylog requires a bit of configuration before we can start it, fire up your text editor of choice and open /etc/graylog/server/server.conf
First change is to set a value for the password_secret parameter. This is used to pepper the stored user passwords. (I had not heard of the term pepper in relation to cryptography before, interesting reading here).
Choose a value with at least 64 characters, you can easily generate one using the following command:
pwgen -N 1 -s 64
Now we need to configure the root_password_sha2 value.
Create a secure password and then plug it into the following command to create the hash value to enter. The secure password you created will be used to log into the web interface.
echo -n yoursecurepassword | shasum -a 256
And now we configure the values for rest_listen_uri and web_listen_uri
These need to be set to the IP or FQDN of the Graylog server. If you use an FQDN it MUST be resolvable by any client machine that is connecting.
Save the changes.
The final step is to configure Graylog to run at system startup
sudo systemctl daemon-reload
sudo systemctl enable graylog-server.service
sudo systemctl start graylog-server.service
After a minute or two you should be able to fire up your web browser, point it to the web_listen_uri (http://svr-syslog01.test.lab:9000 in our example) and be presented with the login screen.
Login with the username admin and the secure password you used previously to create the password hash.
In the next post of this series, we will enable SSL/TLS for the Web Interface and API endpoint.