Percona Cluster

Percona Cluster with lost quorum

Minimum number of servers for Percona/Galera cluster is 3 if quorum should be handled properly. It is ok in case all is working well, but what if you need to do some maintenance on one of the nodes? This is what happened to us: We had 3 nodes. One of the nodes had problems, so tried to restart it. Restart failed and cluster (for unknown reason) lost quorum - last 2 nodes in the cluster were maybe disconnected for short moment. So we ended up with 2 running servers, each of them was...

Mysql with thousands of databases and tables - open files limit problem

We have deployed Percona Cluster with 3 nodes and started importing hundreds of databases to it. Each database has 110 tables, so within few hours we had thousands of tables in the server. Once we reached the limit of open files, it was not possible to restart the node and xtrabackup synchronization failed with strange error (Broken pipe). It was not clear on the first look what caused the problem on the restarted node. Things cleared after we have checked the log innobackup.backup.log on t...

Speedup Mysql Memory Allocation Performance

We have been trying to optimize our Percona Cluster for the best performance and we have found out (with help of http://www.percona.com/blog/2012/07/05/impact-of-memory-allocators-on-mysql-performance/), that the default memory allocation used by operating system (Centos 6.5) is not doing the best job. Therefore we have tried jemalloc and it improved the performance of our cluster nearly twice as it was with glibc. Installation was easy: _1. in the command line as root execute: yum in...

Compare config files of all nodes in Percona cluster

During administration and performance tuning of your percona cluster you often set some parameters on one node and different parameters on another node. This is the command, which will help you to compare all configs and help you to keep them synchronized: pt-config-diff h=<node1> h=<node2> h=<node3> -u <user> -p <password> Output of the command willl looks like: 15 config differences Variable ...ualityunit.com ...ualityunit.com ...ualityunit.co =======...

Restarting Mysql node in the Load Balanced Percona Cluster

In our environment we have percona cluster with 3 nodes and load balanced with HAProxy. To correctly restart one node we need to mark the node as not accessible for application servers and just after there is no traffic, we can restart the Percona server. HAProxy is checking node with xinetd services, so the easiest way how to mark not as not accessible is to stop xinetd service and than restart the mysql process. Here is the command including sleep commands: service xinetd stop; s...