Skip to content

Instantly share code, notes, and snippets.

@kurokobo

kurokobo/edgex-geneva-app-service-mqtt-tls.md

Last active August 28, 2021 05:58
Show Gist options
  • Save kurokobo/448fdbbea705a8f2e88e3f585788d1bc to your computer and use it in GitHub Desktop.
Save kurokobo/448fdbbea705a8f2e88e3f585788d1bc to your computer and use it in GitHub Desktop.
Download ZIP
Raw
edgex-geneva-app-service-mqtt-tls.md

My working sample to export to MQTTS topic using app-service-configurable insecure mode for EdgeX Foundry Geneva release

  • EdgeX Foundry Geneva release.
  • Using docker-app-service-configurable:1.2.0 in insecure mode (without Secret Store).
    • A bit of a special method to get around the bug.
  • Export to MQTTS topic using client certificate authentication.

Prepare Cert Files

In this case, use test.mosquitto.org for testing purposes.

  • CA Cert File
    • Download mosquitto.org.crt from https://test.mosquitto.org/. 
      $ mkdir certs
$ curl https://test.mosquitto.org/ssl/mosquitto.org.crt -o certs/ca.crt
$ ls -l certs
  • Client Key File
    • Generate my own key. 
      $ openssl genrsa -out certs/client.key
$ ls -l certs
    • Generate the CSR. Note that Country Name, State Name, Organization Name and Common Name have to be changed from default. 
      $ openssl req -out certs/client.csr -key certs/client.key -new
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:JP
State or Province Name (full name) [Some-State]:Tokyo
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:EdgeX Foundry
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:app-service-mqtt        
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
$ ls -l certs
  • Client Cert File
    • Get my CSR string. 
      -----BEGIN CERTIFICATE REQUEST-----
MIIClTCCAX0CAQAwUDELMAkGA1UEBhMCSlAxDjAMBgNVBAgMBVRva3lvMRYwFAYD
VQQKDA1FZGdlWCBGb3VuZHJ5MRkwFwYDVQQDDBBhcHAtc2VydmljZS1tcXR0MIIB
...
hZGwRAKfpbaW6yKdEqPq8D2roV6Y2j/4pWcC073dvTa5NWPtqtWadNbYVFrbWpZC
m8j02K4xwUpfuX5/7cpRQOoGVFTSgxx9Nfvr8pcShb+BI3xRV2w7PQc=
-----END CERTIFICATE REQUEST-----
    • Paste my CSR string to the form at https://test.mosquitto.org/ssl/.
      • The file clinet.crt is downloaded. Place this into certs dir.

Now I have these four files.

$ ls -l certs/
total 16
-rw-rw-r-- 1 kuro kuro 1452 Oct 15 06:06 ca.crt
-rw-r--r-- 1 kuro kuro 1261 Oct 15 06:12 client.crt
-rw-rw-r-- 1 kuro kuro  972 Oct 15 06:08 client.csr
-rw------- 1 kuro kuro 1675 Oct 15 06:07 client.key

The CSR will not be used anymore. I can safely remove the CSR.

$ rm certs/client.csr

Directory Structure

Finally, the files are placed as follows:

- demo
  |- docker-compose.yml
  |- mqtt-export
  |  |- configuration.toml
  |- certs
     |- ca.crt
     |- client.crt
     |- client.key

Prepare Required Files

configuration.toml

To use MQTTS, I have to pass my cert files to app-service. EdgeX Foundry has an excellent ability to override the configuration using environment variables, but currently, there is an issue in the latest edgexfoundry/docker-app-service-configurable:1.2.0 that I can't pass the value which includes the character =.

Because the cert files usually contain =, I can not use the environment variable to pass my certs. That's why I have to modify TOML file manually.

In this case, use configuration.toml for mqtt-export from the official repository.

Download it,

$ mkdir mqtt-export
$ curl https://raw.githubusercontent.com/edgexfoundry/app-service-configurable/master/res/mqtt-export/configuration.toml -o mqtt-export/configuration.toml

and then change these lines:

        [Writable.InsecureSecrets.mqtt.Secrets]
            username = ""
            password = ""
            cacert = ""
            clientcert = ""
            clientkey = ""

as follows:

        [Writable.InsecureSecrets.mqtt.Secrets]
            username = ""
            password = ""
            cacert = """-----BEGIN CERTIFICATE-----
MIIDdjCCAl6gAwIBAgIBADANBgkqhkiG9w0BAQsFADCBkDELMAkGA1UEBhMCR0Ix
FzAVBgNVBAgMDlVuaXRlZCBLaW5nZG9tMQ4wDAYDVQQHDAVEZXJieTESMBAGA1UE
...
mxXnSWchJCoy6J8IP8hjggMOVbeLG5H5LMnn4xKzqlsM5fKzCtA0n7Wqcf+TrPXB
hjesoN8T8GxSGvXMIISiZ72HqBYxBp1mGt8=
-----END CERTIFICATE-----"""
            clientcert = """-----BEGIN CERTIFICATE-----
MIIDdjCCAl6gAwIBAgIBADANBgkqhkiG9w0BAQsFADCBkDELMAkGA1UEBhMCR0Ix
FzAVBgNVBAgMDlVuaXRlZCBLaW5nZG9tMQ4wDAYDVQQHDAVEZXJieTESMBAGA1UE
...
mxXnSWchJCoy6J8IP8hjggMOVbeLG5H5LMnn4xKzqlsM5fKzCtA0n7Wqcf+TrPXB
hjesoN8T8GxSGvXMIISiZ72HqBYxBp1mGt8=
-----END CERTIFICATE-----"""
            clientkey = """-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAuYPrFTipQzNQdv3tXxlfUVd2xtfXs9KXBmLg8/QCR4+f4HZa
031mBzWVAs3619VxrN8+RN5YZ022umXIBAA4hrjRJBS7KoJ5Aa2ekonSpdTth97s
...
IecGjp8X8MdIC23jJcCX76SeevOvBcCukPnaehK1I1vTgRnQ6ANG0RTrM2G/S1/C
yIqAjI5tOVmn1HkDeK/I9wgKxpY9oLlQ2tzjdbBQbOZ1H18lds4=
-----END RSA PRIVATE KEY-----"""

Note that specify cacert with the content of ca.crt, clientcert with client.crt, and clientkey with client.key. All of them have to be quoted by a triple quote.

Because of the strict validation of whitespace and line breaks of the PEM format, I have to allow for the indentation to look funny.

Leave the rest of TOML file as its default.

docker-compose.yml

In this case, use docker-compose-geneva-mongo-no-secty.yml from the official repository.

Download it as docker-compose.yml,

$ curl https://raw.githubusercontent.com/edgexfoundry/developer-scripts/master/releases/geneva/compose-files/docker-compose-geneva-mongo-no-secty.yml -o docker-compose.yml

and then add app-service-mqtt service:

  app-service-mqtt:
    image: edgexfoundry/docker-app-service-configurable:1.2.0
    ports:
      - "127.0.0.1:48097:48097"
    container_name: edgex-app-service-configurable-mqtt
    hostname: edgex-app-service-configurable-mqtt
    networks:
      - edgex-network
    environment:
      <<: *common-variables
      edgex_profile: mqtt-export
      Service_Host: edgex-app-service-configurable-mqtt
      Service_Port: 48097
      MessageBus_SubscribeHost_Host: edgex-core-data
      Binding_PublishTopic: events
      Writable_Pipeline_ExecutionOrder: "TransformToJSON, MQTTSecretSend, MarkAsPushed"
      Writable_Pipeline_Functions_MQTTSecretSend_Parameters_brokeraddress: tcps://test.mosquitto.org:8884
      Writable_Pipeline_Functions_MQTTSecretSend_Parameters_topic: edgex/data
      Writable_Pipeline_Functions_MQTTSecretSend_Parameters_clientid: edgex
      Writable_Pipeline_Functions_MQTTSecretSend_Parameters_authmode: clientcert
      Writable_Pipeline_Functions_MQTTSecretSend_Parameters_skipverify: "true"
    volumes:
      - ./mqtt-export:/res/mqtt-export
    depends_on:
      - consul
      - data

There are some points:

  • brokeraddress, topic, clientid
    • These have to be changed depends on my environment.
  • volumes
    • It's required to pass my configuration.toml to the container.
    • Mount ./mqtt-export on my docker host as /res/mqtt-export in the container.
    • /res/mqtt-export is used by app-service because I specify mqtt-export as its profile in edgex_profile.
  • skipverify
    • It's required when I got x509: certificate signed by unknown authority error.

Test

Subscribe my topic using cert files:

$ mosquitto_sub -d -h test.mosquitto.org -t "edgex/data" -p 8884 --cafile ./certs/ca.crt --cert ./certs/client.crt --key ./certs/client.key
Client mosq/0VxxnD7xMr8ADA4SBl sending CONNECT
Client mosq/0VxxnD7xMr8ADA4SBl received CONNACK (0)
Client mosq/0VxxnD7xMr8ADA4SBl sending SUBSCRIBE (Mid: 1, Topic: edgex/data, QoS: 0, Options: 0x00)
Client mosq/0VxxnD7xMr8ADA4SBl received SUBACK
Subscribed (mid: 1): 0
...

and then start EdgeX Foundry.

$ docker-compose up -d
Creating network "demo_edgex-network" with driver "bridge"
Creating volume "demo_db-data" with default driver
Creating volume "demo_log-data" with default driver
Creating volume "demo_consul-config" with default driver
Creating volume "demo_consul-data" with default driver
Creating edgex-mongo       ... done
Creating edgex-core-consul ... done
Creating edgex-support-scheduler     ... done
Creating edgex-support-notifications ... done
Creating edgex-core-metadata         ... done
Creating edgex-core-data             ... done
Creating edgex-core-command          ... done
Creating edgex-app-service-configurable-mqtt  ... done
Creating edgex-device-rest                    ... done
Creating edgex-app-service-configurable-rules ... done
Creating edgex-sys-mgmt-agent                 ... done
Creating edgex-device-virtual                 ... done
Creating edgex-kuiper                         ... done

A short time later I got values exported to the topic to which I'm subscribing.

$ mosquitto_sub -d -h test.mosquitto.org -t "edgex/data" -p 8884 --cafile ./certs/ca.crt --cert ./certs/client.crt --key ./certs/client.key
Client mosq/0VxxnD7xMr8ADA4SBl sending CONNECT
Client mosq/0VxxnD7xMr8ADA4SBl received CONNACK (0)
Client mosq/0VxxnD7xMr8ADA4SBl sending SUBSCRIBE (Mid: 1, Topic: edgex/data, QoS: 0, Options: 0x00)
Client mosq/0VxxnD7xMr8ADA4SBl received SUBACK
Subscribed (mid: 1): 0
Client mosq/0VxxnD7xMr8ADA4SBl sending PINGREQ
Client mosq/0VxxnD7xMr8ADA4SBl received PINGRESP
Client mosq/0VxxnD7xMr8ADA4SBl received PUBLISH (d0, q0, r0, m0, 'edgex/data', ... (277 bytes))
{"id":"ebd8364b-1e24-423d-a160-89e56ce2c38e","device":"Random-Boolean-Device","origin":1602755098464914455,"readings":[{"id":"e2b73e05-693e-48ce-af8b-81706b1e6d7d","origin":1602755098458341738,"device":"Random-Boolean-Device","name":"Bool","value":"false","valueType":"Bool"}]}
Client mosq/0VxxnD7xMr8ADA4SBl received PUBLISH (d0, q0, r0, m0, 'edgex/data', ... (277 bytes))
{"id":"1f606d54-0a5c-4e7d-b723-fb1721544cde","device":"Random-Boolean-Device","origin":1602755098484135940,"readings":[{"id":"c62219a4-4dc2-4677-b670-941817f85b95","origin":1602755098478954904,"device":"Random-Boolean-Device","name":"Bool","value":"false","valueType":"Bool"}]}
Client mosq/0VxxnD7xMr8ADA4SBl received PUBLISH (d0, q0, r0, m0, 'edgex/data', ... (274 bytes))
{"id":"b70a7b14-ac40-47fb-8213-31abeebc5563","device":"Random-Integer-Device","origin":1602755103493184338,"readings":[{"id":"db7c9327-4389-415a-b80d-2f4e4dc91b9c","origin":1602755103484147140,"device":"Random-Integer-Device","name":"Int8","value":"32","valueType":"Int8"}]}
Client mosq/0VxxnD7xMr8ADA4SBl received PUBLISH (d0, q0, r0, m0, 'edgex/data', ... (278 bytes))
{"id":"f351cd0a-db03-48bd-aac1-7eafc985f47c","device":"Random-Integer-Device","origin":1602755103510821938,"readings":[{"id":"09221884-dc50-4eaf-857c-2ecda5960cda","origin":1602755103503012981,"device":"Random-Integer-Device","name":"Int16","value":"7584","valueType":"Int16"}]}
...

Troubleshoot

  • See the logs. 
    $ docker-compose logs -f app-service-mqtt
  • If I got Error parsing CA Certificate error:
    • Check the certs in configuration.toml is correct. Especially quoting, whitespace, and line breaks.
  • If I got x509: certificate signed by unknown authority error:
    • Add Writable_Pipeline_Functions_MQTTSecretSend_Parameters_skipverify: "true" to envorinment: in my docker-compose.yml.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment