[MySQL生产环境复制故障修复] Last_IO_Errno: 1045 Last_IO_Error: error connecting to master 'replication@...

1 修复集群，start slave 报错如下：

mysql> stop slave;
Query OK, 0 rows affected (0.10 sec)

-- 重新设置复制参数
mysql> change master to master_host='10.24.54.18',master_port=3306,master_user='replication',master_password='xxxxxx';
Query OK, 0 rows affected, 0 warnings (0.40 sec)

mysql> start slave;
Query OK, 0 rows affected (0.01 sec)

mysql>  show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Connecting to master
                  Master_Host: 10.24.54.18
                  Master_User: replication
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000002
          Read_Master_Log_Pos: 1355028
               Relay_Log_File: mysql-relay-bin.000001
                Relay_Log_Pos: 4
        Relay_Master_Log_File: mysql-bin.000002
             Slave_IO_Running: Connecting
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1355028
              Relay_Log_Space: 107
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 1045
                Last_IO_Error: error connecting to master 'replication@10.24.54.18:3306' - retry-time: 60  retries: 86400
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 2
1 row in set (0.00 sec)

-- 报错如下： Last_IO_Errno: 1045

看问题，一开始怀疑是密码问题，但是密码都是复制上来的，没有变动，问题不大，还是看看check下别的情况吧。

[xxx@xxx-c004db1 shared]$ ping 10.24.54.18
PING 10.24.54.18 (10.24.54.18) 56(84) bytes of data.
64 bytes from 10.24.54.18: icmp_seq=1 ttl=64 time=0.321 ms
64 bytes from 10.24.54.18: icmp_seq=2 ttl=64 time=0.370 ms
^C
--- 10.24.54.18 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1417ms
rtt min/avg/max/mdev = 0.321/0.345/0.370/0.030 ms
[xxx@xxx-c004db1 shared]$ 

3 ，access 数据库端口3306，也是OK的。

[xxx@xxx-c004db1 shared]$ ssh -v -p3306 10.24.54.18 
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 10.24.54.18 [10.24.54.18] port 3306.
debug1: Connection established.
debug1: identity file /home/xxx/.ssh/xxx type -1
debug1: identity file /home/xxx/.ssh/id_rsa type -1
debug1: identity file /home/xxx/.ssh/id_dsa type -1
debug1: ssh_exchange_identification: O
^C

4，关闭防火墙service iptables stop，然后重启slave，还是报Last_IO_Errno: 1045错误。

[xxx@xxx-c004db1 shared]$ mysql -h 10.24.54.18 -ureplication -pxxx
ERROR 1045 (28000): Access denied for user 'replication'@'xxx-xxx.abn-sjl.ea.com' (using password: YES)
[xxx@xxx-c004db1 shared]$ 

6，修复：去掉空格，重新执行grant语句，再重新启动slave，OK。

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.24.54.18
                  Master_User: replication
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000660
          Read_Master_Log_Pos: 34886442
               Relay_Log_File: mysql-relay-bin.001320
                Relay_Log_Pos: 253
        Relay_Master_Log_File: mysql-bin.000660
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 34886442
              Relay_Log_Space: 452
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 2
1 row in set (0.00 sec)

总结：DBA最重要的一点：细心，与大家共勉