Librenix
Headlines | Linux | Apps | Coding | BSD | Admin | News
Information for Linux System Administration 

Install Apache2, PHP5, MySQL on CentOS 5.7

Up
vote
Down

LAMP is short for Linux, Apache, MySQL, PHP. This tutorial shows how you can install an Apache2 webserver on a CentOS 5.7 server with PHP5 support (mod_php) and MySQL support. read more...
permapage | score:9664 | -falko, December 11, 2011

Virtual Users/Domains with Postfix/Courier/MySQL/SquirrelMail (Debian 7)

Up
vote
Down

This document describes how to install a Postfix mail server that is based on virtual users and domains, i.e. users and domains that are in a MySQL database. I'll also demonstrate the installation and configuration of Courier, so that Courier can authenticate against the same MySQL database Postfix uses. The resulting Postfix server is capable of SMTP-AUTH and TLS and quota. Passwords are stored in encrypted form in the database. In addition to that, this tutorial covers the installation of Amavisd, SpamAssassin, ClamAV, and SquirrelMail. read more...
mail this link | permapage | score:9631 | -falko, July 8, 2013

Tutorial: Install Postfix, Courier, MySQL, SquirrelMail on CentOS 6.2

Up
vote
Down

This document describes how to install a Postfix mail server that is based on virtual users and domains, i.e. users and domains that are in a MySQL database. I'll also demonstrate the installation and configuration of Courier, so that Courier can authenticate against the same MySQL database Postfix uses. The resulting Postfix server is capable of SMTP-AUTH and TLS and quota. Passwords are stored in encrypted form in the database. In addition to that, this tutorial covers the installation of Amavisd, SpamAssassin, ClamAV and SquirrelMail. read more...
mail this link | permapage | score:9606 | -falko, February 2, 2012

Tutorial: Install Lighttpd, PHP5, MySQL on CentOS 6.0

Up
vote
Down

Lighttpd is a secure, fast, standards-compliant web server designed for speed-critical environments. This tutorial shows how you can install Lighttpd on a CentOS 6.0 server with PHP5 support (through FastCGI) and MySQL support. read more...
permapage | score:9596 | -falko, December 15, 2011

Install MySQL 5.6, memcached on Ubuntu 12.10

Up
vote
Down

According to What's New in MySQL 5.6, MySQL 5.6 will bring some performance improvements over MySQL 5.5. If you want to test out MySQL 5.6, you have to install one of the development releases from the MySQL downloads page because there is no stable release yet. This tutorial explains how to install the MySQL 5.6.8rc1 (Linux Generic) development release on an Ubuntu 12.10 server. read more...
permapage | score:9506 | -falko, November 21, 2012

Virtual users with Postfix, MySQL, SquirrelMail (Mandriva 2010.0)

Up
vote
Down

This document describes how to install a Postfix mail server that is based on virtual users and domains, i.e. users and domains that are in a MySQL database. I'll also demonstrate the installation and configuration of Courier (Courier-POP3, Courier-IMAP), so that Courier can authenticate against the same MySQL database Postfix uses. The resulting Postfix server is capable of SMTP-AUTH and TLS and quota. Passwords are stored in encrypted form in the database. In addition to that, this tutorial covers the installation of Amavisd, SpamAssassin and ClamAV so that emails will be scanned for spam and viruses. I will also show how to install SquirrelMail as a webmail interface so that users can read and send emails and change their passwords. read more...
mail this link | permapage | score:9426 | -falko, March 18, 2010

Tutorial: Install Lighttpd, PHP5, MySQL on CentOS 6.4

Up
vote
Down

Lighttpd is a secure, fast, standards-compliant web server designed for speed-critical environments. This tutorial shows how you can install Lighttpd on a Centos 6.4 server with PHP5 support (through PHP-FPM) and MySQL support. PHP-FPM (FastCGI Process Manager) is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites. I use PHP-FPM in this tutorial instead of Lighttpd's spawn-fcgi. read more...
permapage | score:9359 | -falko, April 17, 2013

Tutorial: Install Nginx, PHP5, PHP-FPM, MySQL on Fedora 19

Up
vote
Down

Nginx (pronounced "engine x") is a free, open-source, high-performance HTTP server. Nginx is known for its stability, rich feature set, simple configuration, and low resource consumption. This tutorial shows how you can install Nginx on a Fedora 19 server with PHP5 support (through PHP-FPM) and MySQL support. read more...
permapage | score:9301 | -falko, August 4, 2013

Tutorial: Install Nginx, PHP5, PHP-FPM, and MySQL on CentOS 6.4

Up
vote
Down

Nginx (pronounced "engine x") is a free, open-source, high-performance HTTP server. Nginx is known for its stability, rich feature set, simple configuration, and low resource consumption. This tutorial shows how you can install Nginx on a CentOS 6.4 server with PHP5 support (through PHP-FPM) and MySQL support. read more...
permapage | score:9232 | -falko, April 1, 2013

Tutorial: Install Apache2, PHP5, MySQL on Fedora 12

Up
vote
Down

LAMP is short for Linux, Apache, MySQL, PHP. This tutorial shows how you can install an Apache2 webserver on a Fedora 12 server with PHP5 support (mod_php) and MySQL support. read more...
permapage | score:9232 | -falko, January 21, 2010

LAMP Tutorial: Apache2, PHP5, MySQL on Ubuntu 12.04 LTS

Up
vote
Down

LAMP is short for Linux, Apache, MySQL, PHP. This tutorial shows how you can install an Apache2 webserver on an Ubuntu 12.04 LTS server with PHP5 support (mod_php) and MySQL support. read more...
permapage | score:9223 | -falko, May 14, 2012

Tutorial: Install Lighttpd, PHP5, MySQL on Debian 6

Up
vote
Down

Lighttpd is a secure, fast, standards-compliant web server designed for speed-critical environments. This tutorial shows how you can install Lighttpd on a Debian Squeeze server with PHP5 support (through FastCGI) and MySQL support. read more...
permapage | score:9058 | -falko, February 28, 2011

Writing syslog messages to MySQL

Up
vote
Down

Writing syslog messages to MySQL

Written by Rainer Gerhards (2005-08-02)

Abstract

In this paper, I describe how to write syslog messages to a MySQL database. Having syslog messages in a database is often handy, especially when you intend to set up a front-end for viewing them. This paper describes an approach with rsyslogd, an alternative enhanced syslog daemon natively supporting MySQL. I describe the components needed to be installed and how to configure them.

Background

In many cases, syslog data is simply written to text files. This approach has some advantages, most notably it is very fast and efficient. However, data stored in text files is not readily accessible for real-time viewing and analysis. To do that, the messages need to be in a database. There are various ways to store syslog messages in a database. For example, some have the syslogd write text files which are later feed via a separate script into the database. Others have written scripts taking the data (via a pipe) from a non-database-aware syslogd and store them as they appear. Some others use database-aware syslogds and make them write the data directly to the database. In this paper, I use that "direct write" approach. I think it is superior, because the syslogd itself knows the status of the database connection and thus can handle it intelligently (well ... hopefully ;)). I use rsyslogd to acomplish this, simply because I have initiated the rsyslog project with database-awareness as one goal.

One word of caution: while message storage in the database provides an excellent foundation for interactive analysis, it comes at a cost. Database i/o is considerably slower than text file i/o. As such, directly writing to the database makes sense only if your message volume is low enough to allow a) the syslogd, b) the network, and c) the database server to catch up with it. Some time ago, I have written a paper on optimizing syslog server performance. While this paper talks about Window-based solutions, the ideas in it are generic enough to apply here, too. So it might be worth reading if you anticipate medium high to high traffic. If you anticipate really high traffic (or very large traffic spikes), you should seriously consider forgetting about direct database writes - in my opinion, such a situation needs either a very specialised system or a different approach (the text-file-to-database approach might work better for you in this case).

Overall System Setup

In this paper, I concentrate on the server side. If you are thinking about interactive syslog message review, you probably want to centralize syslog. In such a scenario, you have multiple machines (the so-called clients) send their data to a central machine (called server in this context). While I expect such a setup to be typical when you are interested in storing messages in the database, I do not describe how to set it up. This is beyond the scope of this paper. If you search a little, you will probably find many good descriptions on how to centralize syslog. If you do that, it might be a good idea to do it securely, so you might also be interested in my paper on ssl-encrypting syslog message transfer.

No matter how the messages arrive at the server, their processing is always the same. So you can use this paper in combination with any description for centralized syslog reporting.

As I already said, I use rsyslogd on the server. It has intrinsic support for talking to MySQL databases. For obvious reasons, we also need an instance of MySQL running. To keep us focussed, the setup of MySQL itself is also beyond the scope of this paper. I assume that you have successfully installed MySQL and also have a front-end at hand to work with it (for example, phpMyAdmin). Please make sure that this is installed, actually working and you have a basic understanding of how to handle it.

Setting up the system

You need to download and install rsyslogd first. Obtain it from the rsyslog site. Make sure that you disable stock syslogd, otherwise you will experience some difficulties.

It is important to understand how rsyslogd talks to the database. In rsyslogd, there is the concept of "templates". Basically, a template is a string that includes some replacement characters, which are called "properties" in rsyslog. Properties are accessed via the "Property Replacer". Simply said, you access properties by including their name between percent signs inside the template. For example, if the syslog message is "Test", the template "%msg%" would be expanded to "Test". Rsyslogd supports sending template text as a SQL statement to MySQL. As such, the template must be a valid SQL statement. There is no limit in what the statement might be, but there are some obvious and not so obvious choices. For example, a template "drop table xxx" is possible, but does not make an awful lot of sense. In practice, you will always use an "insert" statment inside the template.

An example: if you would just like to store the msg part of the full syslog message, you have probably created a table "syslog" with a single column "message". In such a case, a good template would be "insert into syslog(message) values ('%msg%')". With the example above, that would be expanded to "insert into syslog(message) values('Test')". This expanded string is then sent to the database. It's that easy, no special magic. The only thing you must ensure is that your template expands to a proper SQL statement and that this statement matches your database design.

Does that mean you need to create database schema yourself and also must fully understand rsyslogd's properties? No, that's not needed. Because we anticipated that folks are probably more interested in getting things going instead of designing them from scratch. So we have provided a default schema as well as build-in support for it. This schema also offers an additional benefit: rsyslog is part of Adiscon's MonitorWare product line (which includes open source and closed source members). All of these tools share the same default schema and know how to operate on it. For this reason, the default schema is also called the "MonitorWare Schema". If you use it, you can simply add phpLogCon, a GPLed syslog web interface, to your system and have instant interactive access to your database. So there are some benefits in using the provided schema.

The schema definition is contained in the file "createDB.sql". It comes with the rsyslog package. Review it to check that the database name is acceptable for you. Be sure to leave the table and field names unmodified, because otherwise you need to customize rsyslogd's default sql template, which we do not do in this paper. Then, run the script with your favourite MySQL tool. Double-check that the table was successfully created.

Next, we need to tell rsyslogd to write data to the database. As we use the default schema, we do NOT need to define a template for this. We can use the hardcoded one (rsyslogd handles the proper template linking). So all we need to do is add a simple selector line to /etc/rsyslog.conf:

*.* >database-server,database-name,database-userid,database-password

In many cases, MySQL will run on the local machine. In this case, you can simply use "127.0.0.1" for database-server. This can be especially advisable, if you do not need to expose MySQL to any process outside of the local machine. In this case, you can simply bind it to 127.0.0.1, which provides a quite secure setup. Of course, also supports remote MySQL instances. In that case, use the remote server name (e.g. mysql.example.com) or IP-address. The database-name by default is "syslog". If you have modified the default, use your name here. Database-userid and -password are the credentials used to connect to the database. As they are stored in clear text in rsyslog.conf, that user should have only the least possible privileges. It is sufficient to grant it INSERT privileges to the systemevents table, only. As a side note, it is strongly advisable to make the rsyslog.conf file readable by root only - if you make it world-readable, everybody could obtain the password (and eventually other vital information from it). In our example, let's assume you have created a MySQL user named "syslogwriter" with a password of "topsecret" (just to say it bluntly: such a password is NOT a good idea...). If your MySQL database is on the local machine, your rsyslog.conf line might look like in this sample:

*.* >127.0.0.1,syslog,syslogwriter,topsecret

Save rsyslog.conf, restart rsyslogd - and you should see syslog messages being stored in the "systemevents" table!

The example line stores every message to the database. Especially if you have a high traffic volume, you will probably limit the amount of messages being logged. This is easy to acomplish: the "write database" action is just a regular selector line. As such, you can apply normal selector-line filtering. If, for example, you are only interested in messages from the mail subsystem, you can use the following selector line:

mail.* >127.0.0.1,syslog,syslogwriter,topsecret

Review the rsyslog.conf documentation for details on selector lines and their filtering.

You have now completed everything necessary to store syslog messages to the MySQL database. If you would like to try out a front-end, you might want to look at phpLogCon, which displays syslog data in a browser. As of this writing, phpLogCon is not yet a powerful tool, but it's open source, so it might be a starting point for your own solution.

On Reliability...

Rsyslogd writes syslog messages directly to the database. This implies that the database must be available at the time of message arrival. If the database is offline, no space is left or something else goes wrong - rsyslogd can not write the database record. If rsyslogd is unable to store a message, it performs one retry. This is helpful if the database server was restarted. In this case, the previous connection was broken but a reconnect immediately succeeds. However, if the database is down for an extended period of time, an immediate retry does not help. While rsyslogd could retry until it finally succeeds, that would have negative impact. Syslog messages keep coming in. If rsyslogd would be busy retrying the database, it would not be able to process these messages. Ultimately, this would lead to loss of newly arrived messages.

In most cases, rsyslogd is configured not only to write to the database but to perform other actions as well. In the always-retry scenario, that would mean no other actions would be carried out. As such, the design of rsyslogd is limited to a single retry. If that does not succeed, the current message is will not be written to the database and the MySQL database writer be suspended for a short period of time. Obviously, this leads to the loss of the current message as well as all messages received during the suspension period. But they are only lost in regard to the database, all other actions are correctly carried out. While not perfect, we consider this to be a better approach then the potential loss of all messages in all actions.

In short: try to avoid database downtime if you do not want to experience message loss.

Please note that this restriction is not rsyslogd specific. All approachs to real-time database storage share this problem area.

Conclusion

With minumal effort, you can use rsyslogd to write syslog messages to a MySQL database. Once the messages are arrived there, you can interactivley review and analyse them. In practice, the messages are also stored in text files for longer-term archival and the databases are cleared out after some time (to avoid becoming too slow). If you expect an extremely high syslog message volume, storing it in real-time to the database may outperform your database server. In such cases, either filter out some messages or think about alternate approaches involving non-real-time database writing (beyond the scope of this paper).

The method outline in this paper provides an easy to setup and maintain solution for most use cases, especially with low and medium syslog message volume (or fast database servers).

Feedback Requested

I would appreciate feedback on this paper. If you have additional ideas, comments or find bugs, please let me know.

References and Additional Material

Revision History

Copyright

Copyright (c) 2005 Rainer Gerhards and Adiscon.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be viewed at http://www.gnu.org/copyleft/fdl.html.

mail this link | permapage | score:8952 | -rgerhards, August 4, 2005 (Updated: March 21, 2007)

Tutorial: Install Apache2, PHP5, MySQL on OpenSUSE 12.3

Up
vote
Down

LAMP is short for Linux, Apache, MySQL, PHP. This tutorial shows how you can install an Apache2 webserver on an OpenSUSE 12.3 server with PHP5 support (mod_php) and MySQL support. read more...
permapage | score:8881 | -falko, April 21, 2013

Installing Nginx, PHP5, PHP-FPM, MySQL on Debian 7

Up
vote
Down

Nginx (pronounced "engine x") is a free, open-source, high-performance HTTP server. Nginx is known for its stability, rich feature set, simple configuration, and low resource consumption. This tutorial shows how you can install Nginx on a Debian Wheezy server with PHP5 support (through PHP-FPM) and MySQL support (LEMP = Linux + nginx (pronounced "engine x") + MySQL + PHP). read more...
permapage | score:8840 | -falko, June 3, 2013

Set up Postfix, MySQL, DoveCot, DSpam, ClamAV, Postgrey on Debian Linux

Up
vote
Down

This guide describes how to set up a full email solution in Debian Linux (all code is from Debian Etch). I was asked to design a secure, scalable, portable solution for a small company. While the guide references many servers, the company only had 4 physical machines, Xen was used to virtualize the entire solution. That particular aspect of the system is not discussed in this guide, although I will try to get it into the next revision. read more...
permapage | score:8836 | -falko, October 31, 2007

Virtual Hosting with PureFTPd, MySQL on CentOS 6.4

Up
vote
Down

This document describes how to install a PureFTPd server that uses virtual users from a MySQL database instead of real system users. This is much more performant and allows to have thousands of ftp users on a single machine. In addition to that I will show the use of quota and upload/download bandwidth limits with this setup. Passwords will be stored encrypted as MD5 strings in the database. read more...
permapage | score:8823 | -falko, April 3, 2013

Tutorial: Install Lighttpd, PHP5 (PHP-FPM), MySQL on Fedora 17

Up
vote
Down

Lighttpd is a secure, fast, standards-compliant web server designed for speed-critical environments. This tutorial shows how you can install Lighttpd on a Fedora 17 server with PHP5 support (through PHP-FPM) and MySQL support. PHP-FPM (FastCGI Process Manager) is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites. I use PHP-FPM in this tutorial instead of Lighttpd's spawn-fcgi. read more...
permapage | score:8819 | -falko, August 30, 2012

Tutorial: Install Lighttpd, PHP5, PHP-FPM, MySQL on Fedora 19

Up
vote
Down

Lighttpd is a secure, fast, standards-compliant web server designed for speed-critical environments. This tutorial shows how you can install Lighttpd on a Fedora 19 server with PHP5 support (through PHP-FPM) and MySQL support. PHP-FPM (FastCGI Process Manager) is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites. I use PHP-FPM in this tutorial instead of Lighttpd's spawn-fcgi. read more...
permapage | score:8780 | -falko, August 7, 2013

Virtual Users/Domains with Postfix, Courier, MySQL, SquirrelMail (CentOS 6.3)

Up
vote
Down

This document describes how to install a Postfix mail server that is based on virtual users and domains, i.e. users and domains that are in a MySQL database. I'll also demonstrate the installation and configuration of Courier, so that Courier can authenticate against the same MySQL database Postfix uses. The resulting Postfix server is capable of SMTP-AUTH and TLS and quota. Passwords are stored in encrypted form in the database. In addition to that, this tutorial covers the installation of Amavisd, SpamAssassin and ClamAV so that emails will be scanned for spam and viruses. I will also show how to install SquirrelMail as a webmail interface so that users can read and send emails and change their passwords. read more...
mail this link | permapage | score:8772 | -falko, March 27, 2013
More articles...
Abstract Art Prints by Ray Yeargin

Recent headlines

Tahoe Least-Authority File System for secure, distributed data storage

Dual-boot Windows 7 and Ubuntu 12.04 on a PC with UEFI board, SSD and HDD

Ubuntu Edge: Is there life after an unsuccessful crowd-funding campaign?

How to install Ajenti on Ubuntu 13.04 server

Hand of Thief trojan and your favorite Linux distribution

vsftpd: Local, Virtual users with bash script maintainance

LinSSID Graphical wireless scanning for Linux

Tutorial: Fedora 19 Samba server with tdbsam

Tutorial: Debian 7 Samba Server with tdbsam

Tutorial: Automatically add a disclaimer to emails with alterMIME (Postfix on Debian 6)

StartOS 6 GNOME 3 and KDE preview

LG 8-inch G Pad 8.3 Android tablet

Tutorial: Install SugarCRM CE on Fedora 10

Encrypt mail with SSL certificates

Plasma Media Center 1.1 and digiKam 3.3

OpenShift Online: a non-developer guide

Linux Iptables Examples

How to replace Ufw with Firewalld in Linux Mint 15

4 third-party repositories for Fedora 19

Debian 6, Squid, Kerberos/LDAP authentication, Active Directory integration, Cyfin Reporter

Install Apache2, PHP5, MySQL on CentOS 5.7

Does disk encryption really protect your data?

webOS: The latest Linux distribution

Run Unity desktop on Linux Mint 12

PDNSD HowTo: A DNS Caching Personal Server

Virtual Users/Domains with Postfix/Courier/MySQL/SquirrelMail (Debian 7)

GhostBSD 2.5 review

Set up Ubuntu PV DomU via xen-image-create at Xen 3.3 Ubuntu Dom0 with Novell kernel 2.6.27

Tutorial: Webcam streaming your desktop plus audio with ffmpeg, crtmpserver, Flowplayer

Tutorial: Replace Windows with Pinguy OS 11.10

Tutorial: Run Joomla 1.7 on Nginx on Debian 6, Ubuntu 11.10

Linpus Lite 1.9 review

Tutorial: Install Postfix, Courier, MySQL, SquirrelMail on CentOS 6.2

Tutorial: USB-Over-IP server, Client on Ubuntu 10.04

Reviewing Kali Linux - the distro for security geeks

Using Multiple PHP Versions (PHP-FPM FastCGI) With ISPConfig 3 (Ubuntu 12.04)

Giada Audio tool for DJs, live performers and electronic musicians

The Coming HTML 5 Revolution in Linux

Tutorial: Install Lighttpd, PHP5, MySQL on CentOS 6.0

How to install postgresql 9.2 on Ubuntu 13.04 Server

 

Firefox sidebar

Site map

Site info

News feed

Features

Login
(to post)

Search

 
Articles are owned by their authors.   © 2000-2012 Ray Yeargin