Helm installs elasticsearch 6 with authentication X 7 X cluster

preface

Previously, we added es permission verification. The common method is to add HTTP password. However, it is not very friendly for use. X-pack is easy to use, but it is charged. However, after ES6, x-pack has gradually released some functions, such as the Monitor cluster monitoring function. ES6.8 and later versions of es have opened some security functions free of charge, including the security authentication function. Later versions have opened some basic authentication functions. This article introduces the use of x-pack to realize the authentication of ES cluster versions. The stand-alone version of ES does not involve certificates, and can be installed directly.

Prepare mirror

This article tag s the official image. Of course, you can also directly use the official image.

FROM  docker.elastic.co/elasticsearch/elasticsearch:6.8.6
ADD elastic-certificates.p12  /usr/share/elasticsearch/config/
RUN chown elasticsearch /usr/share/elasticsearch/config/elastic-certificates.p12

Push the awards to your own image warehouse:

$ docker tag docker.elastic.co/elasticsearch/elasticsearch:6.8.6 aresxin/elasticsearch:6.8.6 
$ docker pull aresxin/elasticsearch:6.8.6

Generate certificate

Generate the cluster certificate elastic certificates P12.
es provides a tool for generating certificates, elasticsearch certutil, which needs to be generated in the docker instance and then copied for use by the cluster.

$ docker run -dit --name=es aresxin/elasticsearch:6.8.6   /bin/bash
$ docker exec -it es /bin/bash
# Generate ca: elastic-stack-ca.p12, in container operation
$ ./bin/elasticsearch-certutil ca
# Generate cert: elastic certificates P12
$ ./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

ctrl+d exit the container and copy the certificate:

$ docker cp es:/usr/share/elasticsearch/elastic-certificates.p12 .
# Close this container
$ docker kill es
$ docker rm es

Add certificate to Kubernetes

The above generated elastic certificates The information in p12 is separated and written to the file.

$  openssl pkcs12 -nodes -passin pass:'' -in elastic-certificates.p12 -out elastic-certificate.pem

Add certificate to Kubernetes:

#   Add certificate
$ kubectl create secret -n $namespace generic elastic-certificates --from-file=elastic-certificates.p12
$ kubectl create secret -n $namespace  generic elastic-certificate-pem --from-file=elastic-certificate.pem

#   Set cluster user name and password
$ kubectl create secret -n $namespace  generic elastic-credentials \
  --from-literal=username=elastic --from-literal=password=$password

Configure helm template

ElasticSearch Master installed profile:

# Cluster name
clusterName: "es-01"
# Node name
nodeGroup: "master"

masterService: ""
# Set role
roles:
  master: "true"
  ingest: "false"
  data: "false"

replicas: 2

minimumMasterNodes: 2

esMajorVersion: "6"

# # ElasticSearch 6.8+ installs the x-pack plug-in by default, and some functions are free of charge
esConfig: 
 elasticsearch.yml: |
    network.host: 0.0.0.0
    cluster.name: "es-01"
    xpack.security.enabled: true
    xpack.security.transport.ssl.enabled: true
    xpack.security.transport.ssl.verification_mode: certificate
    xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
    xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
# Environment variable configuration, import the user name and password secret file set above
extraEnvs:
  - name: ELASTIC_USERNAME
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: username
  - name: ELASTIC_PASSWORD
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: password
        
imagePullSecrets:
  - name: xx
  - 
image: "aresxin/elasticsearch"
imageTag: "6.8.6"
imagePullPolicy: "Always"

esJavaOpts: "-Xmx1g -Xms1g"

resources:
  requests:
    cpu: "100m"
    memory: "2Gi"
  limits:
    cpu: "1000m"
    memory: "2Gi"

volumeClaimTemplate:
  accessModes: [ "ReadWriteOnce" ]
  resources:
    requests:
      storage: 50Gi

persistence:
  enabled: true
# Set the protocol, which can be configured as http and https
protocol: http
httpPort: 9200
transportPort: 9300

ElasticSearch Data installed profile:

clusterName: "es-01"
nodeGroup: "data"

masterService: "es-01-master"

roles:
  master: "false"
  ingest: "true"
  data: "true"

replicas: 2

minimumMasterNodes: 2

esMajorVersion: "6"

imagePullSecrets:
  - name: xx
  
esConfig: 
 elasticsearch.yml: |
    network.host: 0.0.0.0
    cluster.name: "es-01"
    xpack.security.enabled: true
    xpack.security.transport.ssl.enabled: true
    xpack.security.transport.ssl.verification_mode: certificate
    xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
    xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

extraEnvs:
  - name: ELASTIC_USERNAME
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: username
  - name: ELASTIC_PASSWORD
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: password
        
image: "aresxin/elasticsearch"
imageTag: "6.8.6"
imagePullPolicy: "Always"

esJavaOpts: "-Xmx1g -Xms1g"

resources:
  requests:
    cpu: "100m"
    memory: "2Gi"
  limits:
    cpu: "1000m"
    memory: "2Gi"

volumeClaimTemplate:
  accessModes: [ "ReadWriteOnce" ]
  resources:
    requests:
      storage: 1000Gi
 
persistence:
  enabled: true

protocol: http
httpPort: 9200
transportPort: 9300

Installing ES with Helm

The Helm Chart template of ElasticSearch is obtained from the official Github of ES. Its Github address is https://github.com/elastic/helm-charts.

# Helm add Elastic warehouse
$ helm repo add elastic https://helm.elastic.co
# Install ElasticSearch Master node
$ helm install elasticsearch-master -f es-master.yaml --namespace $namespace --version 6.8.6 elastic/elasticsearch

# Installing ElasticSearch Data node
$ helm install elasticsearch-data -f es-data.yaml --namespace $namespace --version 6.8.6 elastic/elasticsearch

# View resources
$ kubectl get pod -n $namespace
es-01-data-0                                  1/1     Running   0          1m
es-01-data-1                                  1/1     Running   0          1m
es-01-master-0                                1/1     Running   0          1m
es-01-master-1                                1/1     Running   0          1m

Test access:

# curl  xx:9200
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
# curl -u elastic:$password  xx:9200
{
  "name" : "es-01-data-1",
  "cluster_name" : "es-01",
  "cluster_uuid" : "8eNkuEcpSWa1tLGoSyN_Xg",
  "version" : {
    "number" : "6.8.6",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "3d9f765",
    "build_date" : "2019-12-13T17:11:52.013738Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.2",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

So far, using Helm to deploy the ES cluster with authentication is complete!

Posted by gethinw on Wed, 01 Jun 2022 12:29:21 +0530

Programs Team