1. Introduction to Apollo
Apollo (Apollo) is a distributed configuration center developed by Ctrip framework department. It can centrally manage the configuration of different application environments and clusters. After the configuration is modified, it can be pushed to the application end in real time. It also has standardized permissions, process governance and other characteristics, and is suitable for micro service configuration management scenarios.
2.Apollo address
Apollo official address: https://github.com/ctripcorp/apollo
Official release package address: https://github.com/ctripcorp/apollo/releases
Infrastructure
simplified model
3. prepare the Apollo configservice package
Apollo configservice package download address: https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-configservice-1.5.1-github.zip
Execute on the O & M host
[root@mfyxw50 ~]# cd /opt/src [root@mfyxw50 src]# wget https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-configservice-1.5.1-github.zip [root@mfyxw50 src]# mkdir -p /data/dockerfile/apollo-configservice [root@mfyxw50 src]# unzip apollo-configservice-1.5.1-github.zip -d /data/dockerfile/apollo-configservice/ [root@mfyxw50 src]# rm -fr /data/dockerfile/apollo-configservice/apollo-configservice-1.5.1-sources.jar #Apollo-configservice-1.5.1-sources Jar source code package is not available
4. install MariaDB database
In mfyxw10 Mfyxw Operation on COM host
Note: mysql version should be above 5.6 and mariadb version should be above 10.1
(1) Add MariaDB source
[root@mfyxw10 ~]# cat > /etc/yum.repos.d/MariaDB.repo << EOF [mariadb] name = MariaDB baseurl = https://mirrors.ustc.edu.cn/mariadb/yum/10.1/centos7-amd64/ gpgkey=https://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB gpgcheck=1 EOF
(2) Import MariaDB certificate
[root@mfyxw10 ~]# rpm --import https://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB
(3) Build cache
[root@mfyxw10 ~]# yum makecache
(4) View available MariaDB database versions
[root@mfyxw10 ~]# yum list MariaDB-server --show-duplicates Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.163.com * extras: mirrors.ustc.edu.cn * updates: mirrors.aliyun.com Available Packages MariaDB-server.x86_64 10.1.43-1.el7.centos mariadb MariaDB-server.x86_64 10.1.44-1.el7.centos mariadb MariaDB-server.x86_64 10.1.45-1.el7.centos mariadb mariadb-server.x86_64 1:5.5.65-1.el7 base
(5) Install MariaDB server version 10.1.45
[root@mfyxw10 ~]# yum -y install MariaDB-server
(6) Set MariaDB profile
/Etc/my Cnf D/server CNF file contents are as follows
[root@mfyxw10 ~]# cat > /etc/my.cnf.d/server.cnf << EOF # # These groups are read by MariaDB server. # Use it for options that only the server (but not clients) should see # # See the examples of server my.cnf files in /usr/share/mysql/ # # this is read by the standalone daemon and embedded servers [server] # this is only for the mysqld standalone daemon [mysqld] character_set_server = utf8mb4 collation_server = utf8mb4_general_ci init_connect = "SET NAMES 'utf8mb4'" # # * Galera-related settings # [galera] # Mandatory settings #wsrep_on=ON #wsrep_provider= #wsrep_cluster_address= #binlog_format=row #default_storage_engine=InnoDB #innodb_autoinc_lock_mode=2 # # Allow server to accept connections on all interfaces. # #bind-address=0.0.0.0 # # Optional setting #wsrep_slave_threads=1 #innodb_flush_log_at_trx_commit=0 # this is only for embedded server [embedded] # This group is only read by MariaDB servers, not by MySQL. # If you use the same .cnf file for MySQL and MariaDB, # you can put MariaDB-only options here [mariadb] # This group is only read by MariaDB-10.1 servers. # If you use the same .cnf file for MariaDB of different versions, # use this group for options that older servers don't understand [mariadb-10.1] EOF
/Etc/my Cnf D / MySQL clients CNF file contents are as follows
[root@mfyxw10 ~]# cat > /etc/my.cnf.d/mysql-clients.cnf << EOF # # These groups are read by MariaDB command-line tools # Use it for options that affect only one utility # [mysql] default-character-set = utf8mb4 [mysql_upgrade] [mysqladmin] [mysqlbinlog] [mysqlcheck] [mysqldump] [mysqlimport] [mysqlshow] [mysqlslap] EOF
(7) Start the MariaDB database and add it to the startup bootstrap
[root@mfyxw10 ~]# systemctl enable --now mariadb [root@mfyxw10 ~]# systmctl status mysql [root@mfyxw10 ~]# netstat -tanlp | grep mysql #View the startup port of MariaDB
(8) Set MariaDB database password
[root@mfyxw10 ~]# mysqladmin -uroot password #Set password: H@o123456 [root@mfyxw10 ~]# mysql -uroot -p #Enter the database password to log in MariaDB [(none)]> drop database test; MariaDB [(none)]> use mysql; MariaDB [mysql]> delete from user where user=''; Or use the following command to MariaDB Database initialization settings [root@mfyxw10 ~]# mysql_secure_installation
(9) Log in to the MariaDB database to check whether the encoding is UTF-8
[root@mfyxw10 ~]# mysql -uroot -p MariaDB [(none)]> \s
(10) Download and import the apollo database initialization script
Initialization data script of apollo: https://raw.githubusercontent.com/ctripcorp/apollo/v1.5.1/scripts/db/migration/configdb/V1.0.0__initialization.sql
[root@mfyxw10 ~]# wget https://raw.githubusercontent.com/ctripcorp/apollo/v1.5.1/scripts/db/migration/configdb/V1.0.0__initialization.sql -O apolloconfig.sql [root@mfyxw10 ~]# mysql -uroot -p < apolloconfig.sql
(11) View the database and tables of apollo
[root@mfyxw10 ~]# mysql -uroot -p MariaDB [(none)]> show databases; +--------------------+ | Database | +--------------------+ | ApolloConfigDB | | information_schema | | mysql | | performance_schema | +--------------------+ 4 rows in set (0.00 sec) MariaDB [(none)]> use ApolloConfigDB; MariaDB [ApolloConfigDB]> show tables; +--------------------------+ | Tables_in_ApolloConfigDB | +--------------------------+ | App | | AppNamespace | | Audit | | Cluster | | Commit | | GrayReleaseRule | | Instance | | InstanceConfig | | Item | | Namespace | | NamespaceLock | | Release | | ReleaseHistory | | ReleaseMessage | | ServerConfig | +--------------------------+ 15 rows in set (0.00 sec)
(12) Grant permissions to database users
[root@mfyxw10 ~]# mysql -uroot -p MariaDB [(none)]> grant SELECT,DELETE,UPDATE,INSERT on ApolloConfigDB.* to "apolloconfig"@"192.168.80.%" identified by "123456"; Query OK, 0 rows affected (0.00 sec) MariaDB [mysql]> select user,host from mysql.user; +--------------+-------------------+ | user | host | +--------------+-------------------+ | root | 127.0.0.1 | | apolloconfig | 192.168.80.% | | root | ::1 | | root | localhost | | root | mfyxw10.mfyxw.com | +--------------+-------------------+ 5 rows in set (0.00 sec)
(13) Modify the value of the ApolloConfigDB database
Check the default values first:
[root@mfyxw10 ~]# mysql -uroot -p
MariaDB [(none)]> use ApolloConfigDB;
MariaDB [ApolloConfigDB]> select * from ServerConfig\G;
*************************** 1. row ***************************
Id: 1
Key: eureka.service.url
Cluster: default
Value: http://localhost:8080/eureka/
Comment: Eureka service Url，Multiple service Separated by English commas
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-03 03:20:18
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-03 03:20:18
*************************** 2. row ***************************
Id: 2
Key: namespace.lock.switch
Cluster: default
Value: false
Comment: Only one person can modify the switch at a time
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-03 03:20:18
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-03 03:20:18
*************************** 3. row ***************************
Id: 3
Key: item.key.length.limit
Cluster: default
Value: 128
Comment: item key Maximum length limit
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-03 03:20:18
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-03 03:20:18
*************************** 4. row ***************************
Id: 4
Key: item.value.length.limit
Cluster: default
Value: 20000
Comment: item value Maximum length limit
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-03 03:20:18
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-03 03:20:18
*************************** 5. row ***************************
Id: 5
Key: config-service.cache.enabled
Cluster: default
Value: false
Comment: ConfigService Whether to enable the cache, which can improve performance but increase memory consumption!
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-03 03:20:18
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-03 03:20:18
5 rows in set (0.00 sec)
ERROR: No query specified
MariaDB [ApolloConfigDB]>
Modify the Value value of the ServerConig table
[root@mfyxw10 ~]# mysql -uroot -p
MariaDB [(none)]> update ApolloConfigDB.ServerConfig set ServerConfig.Value="http://config.od.com/eureka" where ServerConfig.Key="eureka.service.url";
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
MariaDB [ApolloConfigDB]> select * from ServerConfig\G;
*************************** 1. row ***************************
Id: 1
Key: eureka.service.url
Cluster: default
Value: http://config.od.com/eureka
Comment: Eureka service Url，Multiple service Separated by English commas
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-03 03:20:18
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-03 04:20:52
*************************** 2. row ***************************
Id: 2
Key: namespace.lock.switch
Cluster: default
Value: false
Comment: Only one person can modify the switch at a time
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-03 03:20:18
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-03 03:20:18
*************************** 3. row ***************************
Id: 3
Key: item.key.length.limit
Cluster: default
Value: 128
Comment: item key Maximum length limit
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-03 03:20:18
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-03 03:20:18
*************************** 4. row ***************************
Id: 4
Key: item.value.length.limit
Cluster: default
Value: 20000
Comment: item value Maximum length limit
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-03 03:20:18
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-03 03:20:18
*************************** 5. row ***************************
Id: 5
Key: config-service.cache.enabled
Cluster: default
Value: false
Comment: ConfigService Whether to enable the cache, which can improve performance but increase memory consumption!
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-03 03:20:18
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-03 03:20:18
5 rows in set (0.00 sec)
ERROR: No query specified
MariaDB [ApolloConfigDB]>
5. domain name resolution
In mfyxw10 Mfyxw Operation on COM host
(1) At od Add mysql Od COM and config Od COM domain name
[root@mfyxw10 ~]# cat > /var/named/od.com.zone << EOF
\$ORIGIN od.com.
\$TTL 600 ; 10 minutes
@ IN SOA dns.od.com. dnsadmin.od.com. (
;Please add 1 to the serial number, which means it is newer than the previous version
2020031311 ; serial
10800 ; refresh (3 hours)
900 ; retry (15 minutes)
604800 ; expire (1 week)
86400 ; minimum (1 day)
)
NS dns.od.com.
\$TTL 60 ; 1 minute
dns A 192.168.80.10
harbor A 192.168.80.50 ;Add harbor record
k8s-yaml A 192.168.80.50
traefik A 192.168.80.100
dashboard A 192.168.80.100
zk1 A 192.168.80.10
zk2 A 192.168.80.20
zk3 A 192.168.80.30
jenkins A 192.168.80.100
dubbo-monitor A 192.168.80.100
demo A 192.168.80.100
mysql A 192.168.80.10
config A 192.168.80.100
EOF
(2) Restart DNS server service
[root@mfyxw10 ~]# systemctl restart named
(3) Test domain name resolution
[root@mfyxw10 ~]# dig -t A mysql.od.com @192.168.80.10 +short 192.168.80.11 [root@mfyxw10 ~]# dig -t A config.od.com @192.168.80.10 +short 192.168.80.100
6. create a docker image of apolloconfig
Execute on the operation and maintenance host (mfyxw50.mfyxw.com)
(1) Update application github Properties
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-configservice/config/application-github.properties << EOF # DataSource spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigDB?characterEncoding=utf8 spring.datasource.username = apolloconfig spring.datasource.password = 123456 #apollo.eureka.server.enabled=true #apollo.eureka.client.enabled=true EOF
(2) Update startup SH file
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-configservice/scripts/startup.sh << EOF
#!/bin/bash
SERVICE_NAME=apollo-configservice
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-config-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_CONFIG_SERVICE_NAME=\$(hostname -i)
SERVER_URL="http://\${APOLLO_CONFIG_SERVICE_NAME}:\${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms128m -Xmx128m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="\$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="\$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="\$JAVA_OPTS -Dserver.port=\$SERVER_PORT -Dlogging.file=\$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=\$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "\$JAVA_HOME" ]] && [[ -x "\$JAVA_HOME/bin/java" ]]; then
javaexe="\$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=\$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "\$javaexe" ]]; then
version=\$("\$javaexe" -version 2>&1 | awk -F '"' '/version/ {print \$2}')
version=\$(echo "\$version" | awk -F. '{printf("%03d%03d",\$1,\$2);}')
# now version is of format 009003 (9.3.x)
if [ \$version -ge 011000 ]; then
JAVA_OPTS="\$JAVA_OPTS -Xlog:gc*:\$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ \$version -ge 010000 ]; then
JAVA_OPTS="\$JAVA_OPTS -Xlog:gc*:\$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ \$version -ge 009000 ]; then
JAVA_OPTS="\$JAVA_OPTS -Xlog:gc*:\$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="\$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="\$JAVA_OPTS -Xloggc:\$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="\$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "\$(date) ==== Starting ==== \n"
cd \`dirname \$0\`/..
chmod 755 \$SERVICE_NAME".jar"
./\$SERVICE_NAME".jar" start
rc=\$?;
if [[ \$rc != 0 ]];
then
echo "\$(date) Failed to start \$SERVICE_NAME.jar, return code: \$rc"
exit \$rc;
fi
tail -f /dev/null
EOF
(3) Writing Dockefile files
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-configservice/Dockerfile << EOF
FROM harbor.od.com/base/jre8:8u112
ENV VERSION 1.5.1
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-configservice-\${VERSION}.jar /apollo-configservice/apollo-configservice.jar
ADD config/ /apollo-configservice/config
ADD scripts/ /apollo-configservice/scripts
CMD ["/apollo-configservice/scripts/startup.sh"]
EOF
(4) Make docker image
[root@mfyxw50 ~]# cd /data/dockerfile/apollo-configservice [root@mfyxw50 apollo-configservice]# docker build . -t harbor.od.com/infra/apollo-configservice:v1.5.1
(5) Upload the created docker image to the private warehouse
[root@mfyxw50 ~]# docker login harbor.od.com [root@mfyxw50 ~]# docker push harbor.od.com/infra/apollo-configservice:v1.5.1
(6) Check whether the infra in the private warehouse already has an Apollo configservice:v1.5.1
7. provide apolloconfig configuration resource list
Execute on the operation and maintenance host (mfyxw50.mfyxw.com)
(1) Create a directory to store the apolloconfig configuration resource manifest
[root@mfyxw50 ~]# mkdir -p /data/k8s-yaml/apollo-configservice
(2) Add configuration resource list
Deployment Yaml file contents are as follows:
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-configservice/deployment.yaml << EOF
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-configservice
namespace: infra
labels:
name: apollo-configservice
spec:
replicas: 1
selector:
matchLabels:
name: apollo-configservice
template:
metadata:
labels:
app: apollo-configservice
name: apollo-configservice
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-configservice-cm
containers:
- name: apollo-configservice
image: harbor.od.com/infra/apollo-configservice:v1.5.1
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-configservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
EOF
Service Yaml file contents are as follows:
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-configservice/service.yaml << EOF
kind: Service
apiVersion: v1
metadata:
name: apollo-configservice
namespace: infra
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: apollo-configservice
clusterIP: None
type: ClusterIP
sessionAffinity: None
EOF
Ingress Yaml file contents are as follows:
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-configservice/Ingress.yaml << EOF
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: apollo-configservice
namespace: infra
spec:
rules:
- host: config.od.com
http:
paths:
- path: /
backend:
serviceName: apollo-configservice
servicePort: 8080
EOF
Configmap Yaml file contents are as follows:
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-configservice/configmap.yaml << EOF
apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-configservice-cm
namespace: infra
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config.od.com/eureka
app.properties: |
appId=100003171
EOF
8. application apolloconfig configuration resource list
Execute on any of the master nodes (mfyxw30.mfyxw.com or mfyxw40.mfyxw.com)
(1) Apply apolloconfig to configure resource list
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/configmap.yaml configmap/apollo-configservice-cm created [root@mfyxw30 ~]# [root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/deployment.yaml deployment.extensions/apollo-configservice created [root@mfyxw30 ~]# [root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/service.yaml service/apollo-configservice created [root@mfyxw30 ~]# [root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/Ingress.yaml ingress.extensions/apollo-configservice created [root@mfyxw30 ~]#
(2) Check whether the pod of apolloconfig is running
[root@mfyxw30 ~]# kubectl get pod -n infra NAME READY STATUS RESTARTS AGE apollo-configservice-5f6555448-wssq5 1/1 Running 0 51s dubbo-monitor-6676dd74cc-9hghb 1/1 Running 7 14d dubbo-monitor-6676dd74cc-rd86g 1/1 Running 6 14d jenkins-b99776c69-p6skp 1/1 Running 14 36d [root@mfyxw30 ~]#
9. access config Od COM
