|
Hey guys, I was trying to make another management server in a different network. My initial cloudstack setup is running under 172.16.17.0/24 network and other cloudstack network will be 172.16.18.0/24. I have changed the mysql bind-address to 0.0.0.0 in mgmt server 1 and in the db.properties in mgmt server 2 i have changed all the hosts to this management server 172.16.17.x the the mgmt server 1. now this error is coming while connecting to the 1st mgmt server's mysql database Also I edited the management cidr for production to 172.16.0.0/16 (contains both 172.16.17.0/24 and 172.16.18.0/24) and for testbed to 192.168.0.0/16. I have made a whole setup seperately in a test bed with two management server, there both the management servers are getting connected but both the management servers are in same network 192.168.1.0/24. Please help me with this matter. |
Replies: 5 comments 23 replies
did not help ? |
|
Previously on the 1st management server I have setup the database usign this command
But for the new manaement server I have used something like this
Can it create any problem? As the management server key which is in /etc/cloudstack/management/key is different on both the management server. |
yes, the database key must be the same in the two commands moreover, deploy the db only once on the first management server (--deploy-as=root) |
|
@weizhouapache so i don't need to setup the database again in the 2nd management server? Tell me the steps which i need to follow in order to setup the new management server so both the management servers can be accessible |
correct.
|
|
@weizhouapache |
|
Hi Arpan, Thanks for sharing the detailed configuration and logs. We went through the issue carefully, and based on the behavior you described, we don't believe the fact that the two Management Servers are on different subnets is, by itself, the problem. CloudStack can run multiple Management Servers against the same CloudStack database, even when the servers are on different routed networks. The important point here is to identify what changes when MGMT-02 moves from the same subnet to The We recommend checking the following in order: 1. Verify both Management Servers are using the same CloudStack database MGMT-02 should be pointing to the existing CloudStack database used by MGMT-01. Please make sure the database was not initialized separately on MGMT-02. Also, please review the 2. Check the Management Server encryption key You mentioned that Please compare the keys on both servers: sha256sum /etc/cloudstack/management/keyFor security, please don't share the actual key publicly. If the hashes are different, we recommend correcting the configuration so both Management Servers use the appropriate shared key for the same CloudStack deployment. 3. Verify connectivity from MGMT-02 to the database From the nc -vz 172.16.17.x 3306If that succeeds, also verify actual MySQL authentication: mysql -h 172.16.17.x -u cloud -pChanging MySQL's On MGMT-01, you can verify that MySQL is listening with: ss -lntp | grep 33064. Verify routing and firewall rules between the two networks The change to: is fine from a CIDR perspective because it covers both However, the Management CIDR setting itself does not create routing between the networks. Please make sure there is an actual route and that the required traffic is permitted in both directions between: 5. Use the same-subnet test as a controlled comparison Since you already confirmed that two Management Servers work when both are on If possible, perform one more controlled test with both Management Servers temporarily placed on the same If that works, and moving MGMT-02 back to 6. Check the first error in the Management Server log The Please check the Management Server log immediately before this exception and look for the first occurrence of: That first error will usually tell us much more about the actual root cause. What we would check firstBased on the information currently available, our priority would be:
We would avoid rebuilding the environment or making further configuration changes until these checks are completed. The fact that the same setup works when both Management Servers are on the same subnet gives us a good baseline, and we can use that to isolate the exact point of failure rather than changing multiple variables at once. If you can share the sanitized Regards, **DevOps Engineer & Technical Support Engineer ** |
|
@arpanbht |
|
@weizhouapache both are running the same versions |
|
@arpanbht , can you share the contents of the version table? |
|
@DaanHoogland which content do you want? I gave the dpkg list above. Tell me what you want then I will try to give you |
|
@arpanbht |
|
@arpanbht |
|
@weizhouapache But the thing is I am trying to install the management server again by following the steps in our 1st mgmt server same as it which are listed below the management-server 1 is running fine with this setup after I installed the new server with this it is giving the same spring bean error. In the dpkg list the mgmt1 shows the cloudstack version as 4.22.1.0 but in the mgmt2 its getting upgraded to 4.22.1.1 on its own. |
|
@weizhouapache @DaanHoogland |
I think this caused the issue. could you stop cloudstack-management service on mgmt1, upgrade mgmt1 to 4.22.1.1, restart cloudstack-management service ? |
|
Ok I will try that |
|
@weizhouapache @DaanHoogland @RyanP-Testing Previously on mgmt1 during setting up database I used this command to setup the db on that management server machine
So in mgmt1 the db was running fine. Now during the mgmt2 setup I was putting the ip of this machine in the command like this
But now after a fresh installation I removed the ip option then the db got connected and management server got up and running. Thank you for your help guys 🙏. |
|
great @arpanbht , in the end the version difference was actually intended to stop you from starting the second management server, but obviously the messaging was not pointing you in the right direction. If you have any suggestions how to improve that, please do enter an issue or PR. |
|
@DaanHoogland will these two management servers load balance inbetween themselves automatically ?cause we've added both the management servers ip in the host section of the global settings |
@weizhouapache @DaanHoogland @RyanP-Testing
Hey guys I finally managed to connect both the management servers 🎉.
Previously on mgmt1 during setting up database I used this command to setup the db on that management server machine
sudo cloudstack-setup-databases cloud:cloud@localhost --deploy-as=rootSo in mgmt1 the db was running fine.
Now during the mgmt2 setup I was putting the ip of this machine in the command like this
sudo cloudstack-setup-databases cloud:cloud@localhost --deploy-as=root -i 192.168.1.105like this.But now after a fresh installation I removed the ip option then the db got connected and management server got up and running.
Thank you for your help guys 🙏.