fbpx

HA Proxy For MySQL Master – Slave

There are scenarios where we have to provide the high ability to MySQL database instances and we use the Master and Slave replication method of MySQL database.

In the same case to segregate the Read and Write database traffic. We widly use HA- Proxy. It is a feature rich open source Load blancing tool with many unique features like reverse proxy but in out case we are going to use it only for Hight aviliblity purpose.

				
					root@haproxy01:~# haproxy -v
HA-Proxy version 2.0.13-2ubuntu0.3 2021/08/27 - https://haproxy.org/

 
				
			

How to Install it?

You simply use yum or apt commands to install it

				
					sudo apt install -y haproxy
				
			

 

Check  the version 

 

 

 

 

Install Mysql Client for HA Proxy Node to communicate with mysql master and slave databases.

 

				
					apt-get install -y mysql-client
cd /etc/haproxy/
cp haproxy.cfg haproxy.cfg_org
vim haproxy.cfg

				
			
				
					root@haproxy01:~# cat /etc/haproxy/haproxy.cfg
global
	log 127.0.0.1 local0 notice
        log /dev/log    local0
  	user haproxy
	group haproxy

	# Default SSL material locations

defaults
	log global
	mode tcp
        option tcplog
	retries 2
	timeout client 30m
	timeout connect 4s
    	timeout server 30m
	timeout check 5s

listen stats
        mode http
        bind *:9201
        stats enable
        stats uri /stats
        stats realm Strictly\ Private
        stats auth admin:admin

listen mysql-cluster
       bind *:3306
       mode tcp
       option mysql-check user haproxy_user
       balance roundrobin
       server master 192.168.56.205:3306 check
       server slave1 192.168.56.206:3306 check

listen mysql-cluster1
    bind 192.168.1.208:3306
    mode tcp
    option mysql-check user haproxy_user
    balance roundrobin
    server mysql-1 192.168.1.205:3306 check
    server mysql-2 192.168.1.206:3306 check

				
			

Create HA proxy user on mysql01/205 on primary node

				
					GRANT ALL PRIVILEGES ON *.* TO 'haproxy_root'@'%' IDENTIFIED BY 'Oracle@123' WITH GRANT OPTION;

flush privileges;
				
			

Test the configuration and it should start without error & Target should come up on GUI

				
					haproxy -f /etc/haproxy/haproxy.cfg -db

systemctl restart haproxy.service
				
			

Check HA proxy GUI and see all the MySQL target is up and running fine using HA Proxy Admin link:

HA Proxy Link Structure:

http://<localhost or IP/HostName/stats

http://192.168.1.208:9201/stats

Default Credentials : 

UserName : admin

Password: admin

Read More...

Essential AWS Services for Database Administrators to Learn

Why AWS?

Cloud is becoming a vital part of Database Administration because it provides various database services & Infrastructure to run Database Ecosystems instantly. AWS (Amazon Web Services) is one of the pioneers in the Cloud according to the Gartner magic quadrant. Knowing more cloud infrastructure technologies is going to give more mileage to your Administrator career. In this article, you will find some of the AWS services which Database Administrators should know as they are basic to run Database opration.

Essential AWS Services List For Database Administrator (DBA)

Network
VPCs
Subnets
Elastic IPs
Internet Gateways
Network ACLs
Route Tables
Security Groups
Private Subnets
Public Subnets
AWS Direct Connect

Virtual Machine 
EC2
AWS Work Space

Storage
EBS
EFS
S3

Database as Services (RDS)
MySQL / MariaDB
PostgreSQL
Oracle
Micrsoft SQL Server
AWS Aurora PostgreSQL/MySQL

Database Managed Services
AWS Dynamo DB
AWS Elasticsearch 
Amazon DocumentDB

Messaging & Event Base Processing 
Apache Kafka (Amazon MSK)

 

Warehousing/ OLAP /Analytics Stagging DB
AWS Redshift

 

Monitoring 
Cloud watch
Amazon Grafana
Amazon Prometheus

 

Email Service
Amazon Simple Notification Service

Security 
IAM
Secrets Manager

Database Task Automation
AWS Batch
AWS Lambda
Cloud Formation

Command-line interface (CLI) to Manage AWS Services
AWSCLI

Migration 
Database Migration Service

Budget 
AWS Cost Explorer
AWS Budgets

Some other Services & Combination worth of Exploring

Bastion Host For DBA
MongoDB running on EC2
ELK (Elastic Search , LogStach, Kibana) running on EC2
Tunnels for Non stranded ports for Database Connections for more security
pg_pool or pg_Bouncer for PostgreSQL Databases

Stay Tuned For Latest Database, Cloud & Technology Trends

Read More >>

How to fix Rocky Linux full screen issue on Oracle Virtual Box

In this article, we will fix the rocky Linux full-screen issue on the oracle virtual box. Normally after installing rocky Linux on virtual box by default full scree don’t work and some times its annoying because it little hard to work with small screen. Let me give you few simple steps with commands which will fix this issue and you will many option to set screen resolution on rocky Linux.

 

Step 1: login as root and run following commands.

Commands:

yum install dkms gcc make kernel-devel bzip2 binutils patch libgomp glibc-headers glibc-devel kernel-headers -y

Important Note: In case if you don’t find the yum packages, configure epel yum repository for centos 8

 

Step 2: Install the virtual box guest addition. You can see the navigation from flowing screenshot

Step 3: restart your rocky Linux virtual machine

Command:

[root@rocky01 ~]# reboot

You are Done !!!

 

 

MySQL query output to csv

There are situations when we have to run the select queries on MySQL databases and redirect the output in .cav format. This is required when we have to share the data with the developer or send it to some ETL job for external tables. In addition, if you are running MySQL on AWS RDS you get limited options to get the MySQL query output to CSV. As an alternative, you can use the AWS S3 bucket but again it’s a complex and cost-involved way of getting MySQL query output to CSV. I am going to show you an easy alternative that will work on a normal Linux command prompt. Using this you can fetch a long SQL output in CSV from MySQL RDS or from on promises MySQL Database instance.

 

Step 1: log in to any jump host or bastion host

Step 2: if your SQL output is going to take a long time use screen

screen -S mysql2csv_date

Step 3 Run MySQL export batch command and sed Linux command

mysql -A -h testdb01.us-east-1.rds.amazonaws.com -u root -p --batch -e "select * from testdb01.table_x where date_created >='2021-07-01 00:00:00'  AND LENGTH(emp_id) < 30 order by date_created;" | sed 's/\t/","/g;s/^/"/;s/$/"/;s/\n//g' > mysq2csv_output.csv

Step 4: once the command is completed you will see the output in .csv under the file mysq2csv_output.csv

Hope you like the article

References: https://dev.mysql.com/doc/refman/8.0/en/select-into.html

 

Read More

Take MySQL backup From Jenkins Job

How to fix ORA-28368: cannot auto-create wallet
AWS Services and their Azure alternatives 
How to Manage AWS S3 Bucket with AWS CLI

How to connect PostgreSQL Database from PgAdmin
How to create AWS rds PostgreSQL Database
Convert pem to ppk
How to Install PuTTy on Window
How to Install and create AWS EC2 Instance using Terraform
AWS MySQL RDS Database Creation using AWSCLI
How to Create MySQL Database with AWS RDS
How to connect to AWS MySQL / MariaDB RDS or EC2 database from MySQL WorkBench

 

Take MySQL backup From Jenkins Job

Take MySQL Database Backup From Jenkins

In this post, we will explain and practically show how you can configure MySQL database backup using Jenkins jobs. in easy words, you will automate the MySQL database backup process from Jenkins GUI and MySQL dump command. Normally we use MySQL dump from CLI or using the windows option. In this case, just automate the MySQL dump triggering from a shell script which will be called by Jenkins job with parameter. MySQL dump needs few parameters to run just pass them by build with parameter option in jenkins.

 

1. Log to Jenkins default URL and port: http://192.168.56.21:8080/

If you want to install and setup Jenkins see this article

2. Go to New items

3.  Give some Name and select FreeStyle project

 

4.  In the build section click on Add build step radio button and select Execute shell.

5. Select boolean parameter for MySQL database host and database superuser password which is root password in MySQL database case.

 

6.  In this build part give the shell script name with full path and parameters like $HOST_IP & $Password

 

Shell Script

[root@master01 ~]# cat /opt/jenkins_scripts/mysql_full_db_backup.sh
#!/bin/bash
HOST_IP=$1
PASSWORD=$2
echo "starting mysql database full backup"
mysqldump -h $HOST_IP --all-databases --single-transaction --quick --lock-tables=false > /opt/jenkins_scripts/mysql_backup/full_backup_$(date +%F_%N).sql -u root -p$PASSWORD
echo "Backup has been done"

 

[su_box title=”IMP Note” box_color=”#fe2227″ title_color=”#101112″]Note: Make sure Jenkins have execute (chmod +x /opt/jenkins_scripts/mysql_full_db_backup.sh)  privilege on script[/su_box]

 

7. Let’s execute the Jenkins job

7.1  click on the the MySQL database backup job

 

7.2  Click on build with Parameter

 

7.3  Give your database server name or IP. In my case Mysql database running on the same server that’s thy, I will give localhost and database root password

 

7.4 Once your job is executed go to that job open it in console output mode and check the status of the job. And you are DONE!!!

Read More

How to fix ORA-28368: cannot auto-create wallet
AWS Services and their Azure alternatives 
How to Manage AWS S3 Bucket with AWS CLI

How to connect PostgreSQL Database from PgAdmin
How to create AWS rds PostgreSQL Database
Convert pem to ppk
How to Install PuTTy on Window
How to Install and create AWS EC2 Instance using Terraform
AWS MySQL RDS Database Creation using AWSCLI
How to Create MySQL Database with AWS RDS
How to connect to AWS MySQL / MariaDB RDS or EC2 database from MySQL WorkBench

Apache Tomcat: java.net.BindException: Permission denied (Bind failed) :443

How to fix error: Apache tomcat : java.net.BindException: Permission denied (Bind failed) <null>:443

 

Full Error:

SEVERE: Failed to initialize end point associated with ProtocolHandler ["http-bio-4443"]
java.net.BindException: Permission denied (Bind failed) <null>:4443
	at org.apache.tomcat.util.net.JIoEndpoint.bind(JIoEndpoint.java:413)
	at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:715)
	at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:452)
	at org.apache.coyote.http11.AbstractHttp11JsseProtocol.init(AbstractHttp11JsseProtocol.java:119)
	at org.apache.catalina.connector.Connector.initInternal(Connector.java:978)
	at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
	at org.apache.catalina.core.StandardService.initInternal(StandardService.java:560)
	at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
	at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:840)
	at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
	at org.apache.catalina.startup.Catalina.load(Catalina.java:642)
	at org.apache.catalina.startup.Catalina.load(Catalina.java:667)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:253)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:427)
Caused by: java.net.BindException: Permission denied (Bind failed)
	at java.net.PlainSocketImpl.socketBind(Native Method)
	at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
	at java.net.ServerSocket.bind(ServerSocket.java:390)
	at java.net.ServerSocket.<init>(ServerSocket.java:252)
	at java.net.ServerSocket.<init>(ServerSocket.java:196)
	at javax.net.ssl.SSLServerSocket.<init>(SSLServerSocket.java:136)
	at sun.security.ssl.SSLServerSocketImpl.<init>(SSLServerSocketImpl.java:71)
	at sun.security.ssl.SSLServerSocketFactoryImpl.createServerSocket(SSLServerSocketFactoryImpl.java:80)
	at org.apache.tomcat.util.net.jsse.JSSESocketFactory.createSocket(JSSESocketFactory.java:256)
	at org.apache.tomcat.util.net.JIoEndpoint.bind(JIoEndpoint.java:400)
	... 17 more

 

Why, when, and where we get this error?

 

We get this issue while starting Apache tomcat on secure port 443. The error “java.net.BindException: Permission denied (Bind failed) :443 ” is visible in /usr/share/tomcat/logs/catalina.2016-12-02.log

 

How to fix it

This error occurs when your Linux firewall levels are set to ‘enforcing‘.To fix this error make it ‘permissive

  1. Check current firewall status using the command ‘getenforce
  2. Edit the file ‘vi /etc/selinux/config’ and make it ‘permissive’ from ‘enforcing’
    # This file controls the state of SELinux on the system.
    # SELINUX= can take one of these three values:
    #       enforcing - SELinux security policy is enforced.
    #       permissive - SELinux prints warnings instead of enforcing.
    #       disabled - No SELinux policy is loaded.
    SELINUX=permissive
    # SELINUXTYPE= can take one of these two values:
    #       targeted - Targeted processes are protected,
    #       mls - Multi Level Security protection.
    SELINUXTYPE=targeted
  3. Reboot the system using ‘reboot‘ as the root
  4. After reboot check, if tomcat is up with 443 port
    netstat -plan| grep 443
  5. And check the tomcat URL with ssl/https  (https://localhost:443)

Leave a comment if you find the article helpful.

 

Read More

How to fix ORA-28368: cannot auto-create wallet
AWS Services and their Azure alternatives 
How to Manage AWS S3 Bucket with AWS CLI

How to connect PostgreSQL Database from PgAdmin
How to create AWS rds PostgreSQL Database
Convert pem to ppk
How to Install PuTTy on Window
How to Install and create AWS EC2 Instance using Terraform
AWS MySQL RDS Database Creation using AWSCLI
How to Create MySQL Database with AWS RDS
How to connect to AWS MySQL / MariaDB RDS or EC2 database from MySQL WorkBench