Installing MySQL Cluster on EC2

4 Jan

Here is how I installed MySQL cluster on Amazon EC2. Note: This is a basic setup that I used to test MySQL cluster. A real-world production setup would probably involve a more complex variation

Create 3 Amazon EC2 instances.

  • This post assumes 3 Amazon EC2 t1.micro instances
    • 1 t1.micro for SQL Node and Mgmt Node
    • 1 t1.micro for Data Node
    • 1 t1.micro for Data Node

On the SQL/Mgmt Node executes these steps:

Common Steps

  • Follow all steps in Common Steps on all Nodes section below.

Create the Deployment Directory and Setup Config Files

  • mkdir -p /opt/mysqlcluster/deploy
  • cd /opt/mysqlcluster/deploy
  • mkdir conf
  • mkdir mysqld_data
  • mkdir ndb_data
  • cd conf
  • gedit my.cnf and enter the following
    [mysqld]
    ndbcluster
    datadir=/opt/mysqlcluster/deploy/mysqld_data
    basedir=/opt/mysqlcluster/home/mysqlc
    port=3306

     

  • gedit config.ini and enter the following
    • NOTE: REPLACE the hostname entries below with names of the SQL/MGMT Node and Data Nodes.
      [ndb_mgmd]
      hostname=domU-12-31-39-04-D6-A3.compute-1.internal
      datadir=/opt/mysqlcluster/deploy/ndb_data
      nodeid=1
      
      [ndbd default]
      noofreplicas=2
      datadir=/opt/mysqlcluster/deploy/ndb_data
      
      [ndbd]
      hostname=ip-10-72-50-247.ec2.internal
      nodeid=3
      
      [ndbd]
      hostname=ip-10-194-139-246.ec2.internal
      nodeid=4
      
      [mysqld]
      nodeid=50

       

Initialize the Database

  • Just like any other MySQL Server, the mysqld process requires a ‘mysql’ database to be created and populated with essential system data”
  • cd /opt/mysqlcluster/home/mysqlc
  • scripts/mysql_install_db –no-defaults –datadir=/opt/mysqlcluster/deploy/mysqld_data

START MANAGEMENT NODE

  • ndb_mgmd -f /opt/mysqlcluster/deploy/conf/config.ini –initial –configdir=/opt/mysqlcluster/deploy/conf
    • MGMT SERVER RUNS LISTENING ON PORT 1186

On Data Node 1 host:

Common Steps

  • Follow all steps in Common Steps on all Nodes below.

Create NDB DATA directory

  • mkdir -p /opt/mysqlcluster/deploy/ndb_data

START DATA NODE

  • Start up Data Node passing in Private DNS name of Mgmt Node
    • ndbd -c domU-12-31-39-04-D6-A3.compute-1.internal:1186

On Data Node 2 host:

  • Repeat Steps listed for Data Node 1 above.

Back On SQL/Mgmt Node:

CHECK STATUS OF MGMT/DATA NODES

  • ndb_mgm -e show

START SQL NODE

  • mysqld –defaults-file=/opt/mysqlcluster/deploy/conf/my.cnf –user=root &
  • WAIT FOR THE FOLLOWING TO BE DISPLAYED BEFORE PROCEEDING:
    111104 12:01:50 [Note] Plugin 'FEDERATED' is disabled.
    111104 12:02:25 [Warning] NDB: server id set to zero - changes logged to bin log with server id zero will be logged with another server id by slave mysqlds
    111104 12:02:25 [Note] Starting Cluster Binlog Thread
    111104 12:02:25 InnoDB: The InnoDB memory heap is disabled
    111104 12:02:25 InnoDB: Mutexes and rw_locks use GCC atomic builtins
    111104 12:02:25 InnoDB: Compressed tables use zlib 1.2.3
    111104 12:02:25 InnoDB: Using Linux native AIO
    111104 12:02:25 InnoDB: Initializing buffer pool, size = 128.0M
    111104 12:02:25 InnoDB: Completed initialization of buffer pool
    111104 12:02:25 InnoDB: highest supported file format is Barracuda.
    InnoDB: 127 rollback segment(s) active.
    111104 12:02:26  InnoDB: Waiting for the background threads to start
    111104 12:02:27 InnoDB: 1.1.8 started; log sequence number 44233
    111104 12:02:27 [Note] Event Scheduler: Loaded 0 events
    111104 12:02:27 [Note] mysqld: ready for connections.
    Version: '5.5.15-ndb-7.2.1-gpl'  socket: '/tmp/mysql.sock'  port: 5000  MySQL Cluster Community Server (GPL)
    111104 12:02:57 [Warning] NDB : Tables not available after 30 seconds.  Consider increasing --ndb-wait-setup value
    111104 12:03:11 [Note] NDB: NodeID is 50, management server 'localhost:1186'
    111104 12:03:12 [Note] NDB Binlog: DISCOVER TABLE Event: REPL$mysql/ndb_schema
    111104 12:03:12 [Note] NDB Binlog: logging ./mysql/ndb_schema (UPDATED,USE_WRITE)
    111104 12:03:12 [Note] NDB Binlog: DISCOVER TABLE Event: REPL$mysql/ndb_apply_status
    111104 12:03:12 [Note] NDB Binlog: logging ./mysql/ndb_apply_status (UPDATED,USE_WRITE)
    2011-11-04 12:03:12 [NdbApi] INFO     -- Flushing incomplete GCI:s < 628/6
    2011-11-04 12:03:12 [NdbApi] INFO     -- Flushing incomplete GCI:s < 628/6
    111104 12:03:12 [Note] NDB Binlog: starting log at epoch 628/6
    111104 12:03:12 [Note] NDB Binlog: ndb tables writable

     

CHECK STATUS OF MGMT/DATA/SQL NODES

  • ndb_mgm -e show

SECURE MYSQL INSTALLATION

  • Note: Temporary “root” and “app” users password in testpwd. This will change once we are in production
  • mysql_secure_installation
    [root@domU-12-31-39-04-D6-A3 ndb_data]# mysql_secure_installation
    
    NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
          SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!
    
    In order to log into MySQL to secure it, we'll need the current
    password for the root user.  If you've just installed MySQL, and
    you haven't set the root password yet, the password will be blank,
    so you should just press enter here.
    
    Enter current password for root (enter for none): 
    OK, successfully used password, moving on...
    
    Setting the root password ensures that nobody can log into the MySQL
    root user without the proper authorisation.
    
    You already have a root password set, so you can safely answer 'n'.
    
    Change the root password? [Y/n] n
     ... skipping.
    
    By default, a MySQL installation has an anonymous user, allowing anyone
    to log into MySQL without having to have a user account created for
    them.  This is intended only for testing, and to make the installation
    go a bit smoother.  You should remove them before moving into a
    production environment.
    
    Remove anonymous users? [Y/n] 
     ... Success!
    
    Normally, root should only be allowed to connect from 'localhost'.  This
    ensures that someone cannot guess at the root password from the network.
    
    Disallow root login remotely? [Y/n] y
     ... Success!
    
    By default, MySQL comes with a database named 'test' that anyone can
    access.  This is also intended only for testing, and should be removed
    before moving into a production environment.
    
    Remove test database and access to it? [Y/n] y
     - Dropping test database...
    ERROR 1008 (HY000) at line 1: Can't drop database 'test'; database doesn't exist
     ... Failed!  Not critical, keep moving...
     - Removing privileges on test database...
     ... Success!
    
    Reloading the privilege tables will ensure that all changes made so far
    will take effect immediately.
    
    Reload privilege tables now? [Y/n] y
     ... Success!
    
    Cleaning up...
    
    All done!  If you've completed all of the above steps, your MySQL
    installation should now be secure.
    
    Thanks for using MySQL!

     

Setup Remote Users

[root@domU-12-31-39-04-D6-A3 ndb_data]# mysql -h 127.0.0.1 -u root -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 21
Server version: 5.5.15-ndb-7.2.1-gpl MySQL Cluster Community Server (GPL)

Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> CREATE USER 'myapp'@'%' IDENTIFIED BY 'testpwd';
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON * . * TO 'myapp'@'%' IDENTIFIED BY 'testpwd' WITH GRANT OPTION MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0 ;
Query OK, 0 rows affected (0.01 sec)

TESTS on SQL/Mgmt Node:

  • mysql -h 127.0.0.1 -u myapp -p
    mysql> create database clusterdb;use clusterdb;
    mysql> create table simples (id int not null primary key) engine=ndb;
    mysql> insert into simples values (1),(2),(3),(4);
    mysql> select * from simples;

     

TESTS from a Remote MySQL client:

  • /opt/mysqlcluster/home/mysqlc/bin/mysql -h ec2-50-17-110-253.compute-1.amazonaws.com -u w -p
    mysql> use clusterdb;
    mysql> select * from simples;

     

Shutdown Services on SQL/Mgmt Node

  • SHUTDOWN MYSQLD
    • mysqladmin -u myapp -h 127.0.0.1 -p shutdown
  • SHUTDOWN MANAGEMENT AND DATA NODES:
    • ndb_mgm -e shutdown

Common Steps on all Nodes

Uninstall any existing Mysql packages

  • service mysqld stop
  • yum remove mysql-server mysql mysql-devel

Download and Extract MySQL Cluster Binaries

Setup Executable Path Globally

  • echo ‘export MYSQLC_HOME=/opt/mysqlcluster/home/mysqlc’ > /etc/profile.d/mysqlc.sh
  • echo ‘export PATH=$MYSQLC_HOME/bin:$PATH’ >> /etc/profile.d/mysqlc.sh
  • source /etc/profile.d/mysqlc.sh
    • This will set the environment variables for the current shell

References

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: