Skip to content

Instantly share code, notes, and snippets.

@kurokobo

kurokobo/edgex-app-service-mqtt-azure-iot.md

Last active April 17, 2023 02:46
Show Gist options
  • Save kurokobo/fe8ff371c0bd0b3f54cef03778b0d652 to your computer and use it in GitHub Desktop.
Save kurokobo/fe8ff371c0bd0b3f54cef03778b0d652 to your computer and use it in GitHub Desktop.
Download ZIP
Example for export to Azure IoT Hub using app-service-configurable (insecure mode)
Raw
edgex-app-service-mqtt-azure-iot.md

[EdgeX Foundry] Export to Azure IoT Hub using app-service-configurable (insecure mode)

  • EdgeX Foundry Geneva / Hanoi / Ireland release
    • Using edgexfoundry/docker-app-service-configurable:1.3.1 for Geneva / Hanoi
    • Using edgexfoundry/app-service-configurable:2.0.1 for Ireland
  • Insecure mode (without Secret Store)

Table of Contents

Prepare Azure IoT Hub

  1. Create Azure IoT Hub on the Azure Portal
    1. Specify Subscription, Resource group, IoT hub name, and Region
    2. If you want to use Free Tier, move on to the Management tab and change your tier on Pricing and scale tier
    3. Press Review + create > Create
    4. Wait few minutes the deployment to be completed
  2. Register your EdgeX Foundry as "Device"
    1. Create a new device on Azure Portal
      1. In Azure Portal, go to Devices in Device management section
      2. Click + Add Device
      3. Specify Device ID as you like (in my example edgex), select Symmetrix key as Authentication type, keep Auto-generate keys checked and Enable in Connect this device to an IoT hub, then Save
    2. Get Primary key for the device
      1. Open the device you created
      2. Reveal Primary Key and keep it to use later

Prepare SAS Token

# Prepare Python
python3 -m venv .venv
. .venv/bin/activate

# Create minimal helper script
# https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-dev-guide-sas?tabs=python
cat <<EOF > create_sas_token.py
import os
from base64 import b64encode, b64decode
from hashlib import sha256
from time import time
from urllib import parse
from hmac import HMAC

def generate_sas_token(uri, key, policy_name, expiry=3600):
    ttl = time() + expiry
    sign_key = "%s\n%d" % ((parse.quote_plus(uri)), int(ttl))
    signature = b64encode(HMAC(b64decode(key), sign_key.encode('utf-8'), sha256).digest())

    rawtoken = {
        'sr' :  uri,
        'sig': signature,
        'se' : str(int(ttl))
    }

    if policy_name is not None:
        rawtoken['skn'] = policy_name

    return 'SharedAccessSignature ' + parse.urlencode(rawtoken)

if __name__ == "__main__":
    hub = os.environ["AZURE_IOT_HUB_NAME"]
    device = os.environ["AZURE_IOT_HUB_DEVICE"]
    key = os.environ["AZURE_IOT_HUB_KEY"]
    policy = os.getenv("AZURE_IOT_HUB_POLICY")
    age = int(os.environ["AZURE_IOT_HUB_AGE"])
    uri = "{}.azure-devices.net/devices/{}".format(hub, device)
    print(generate_sas_token(uri, key, policy, age))
EOF

# Generate SAS Token
export AZURE_IOT_HUB_NAME=<YOUR_IOT_HUB_NAME>
export AZURE_IOT_HUB_DEVICE=<YOUR_DEVICE_NAME>
export AZURE_IOT_HUB_KEY=<YOUR_DEVICE_PRIMARY_KEY>
export AZURE_IOT_HUB_AGE=3600
python3 ./create_sas_token.py

Keep your SAS Token to use later.

Alternatively, you can also use Azure Cloud Shell to generate SAS Token.

az iot hub generate-sas-token -n YOUR_IOT_HUB_NAME -d YOUR_DEVICE_NAME

Prepare EdgeX Foundry

Geneva / Hanoi (1.x)

This is the snippet for app-service-configurable. Note that the name of environment variable is case sensitive.

See the comments for details.

  app-service-mqtt:
    # It is recommended that you use the images from the Hanoi release for Geneva too
    # as the issue have been fixed.
    # https://github.com/edgexfoundry/go-mod-bootstrap/issues/111
    image: edgexfoundry/docker-app-service-configurable:1.3.1
    ports:
      - "127.0.0.1:48097:48097"
    container_name: edgex-app-service-configurable-mqtt
    hostname: edgex-app-service-configurable-mqtt
    networks:
      - edgex-network
    environment:
      <<: *common-variables
      edgex_profile: mqtt-export
      Service_Host: edgex-app-service-configurable-mqtt
      Service_Port: 48097
      MessageBus_SubscribeHost_Host: edgex-core-data
      Binding_PublishTopic: events

      # Requied to use MQTTSecretSend
      Writable_Pipeline_ExecutionOrder: "TransformToJSON, MQTTSecretSend, MarkAsPushed"

      # Broker Address has to be formatted as "tcps://YOUR_IOT_HUB_NAME.azure-devices.net:8883"
      # https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-mqtt-support
      Writable_Pipeline_Functions_MQTTSecretSend_Parameters_brokeraddress: tcps://iot-hub-edgex-foundry.azure-devices.net:8883

      # Topic has to be formatted as "devices/YOUR_DEVICE_NAME/messages/events/"
      # https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-mqtt-support
      Writable_Pipeline_Functions_MQTTSecretSend_Parameters_topic: devices/edgex/messages/events/

      # Client ID has to be formatted as "YOUR_DEVICE_NAME"
      # https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-mqtt-support
      Writable_Pipeline_Functions_MQTTSecretSend_Parameters_clientid: edgex

      # Auth with username and password (SAS Token)
      Writable_Pipeline_Functions_MQTTSecretSend_Parameters_authmode: usernamepassword

      # Username has to be formatted as "YOUR_IOT_HUB_NAME.azure-devices.net/YOUR_DEVICE_NAME/?api-version=2018-06-30"
      # https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-mqtt-support
      Writable_InsecureSecrets_mqtt_Secrets_username: iot-hub-edgex-foundry.azure-devices.net/edgex/?api-version=2018-06-30

      # Paste your SAS Token. SAS Token can be generated using create_sas_token.py
      Writable_InsecureSecrets_mqtt_Secrets_password: SharedAccessSignature sr=iot-hub-edgex-foundry.azure-devices.net%2Fdevices%2Fedgex&sig=zqQlB0oS41qIcpHNst%2FAyMzO%2FNWzn30hZfs%2F9chum7A%3D&se=1635064560

    depends_on:
      - consul
      - data

Then start your EdgeX Foundry, and send some data from your device. For testing purpose, device-virtual can be used to send dummy data.

# Start EdgeX Foundry
$ docker-compose up -d
...

# Check logs after sending some data from your device
$ docker logs edgex-app-service-configurable-mqtt
...
level=INFO ts=2021-10-16T08:04:20.540954927Z app=AppService-mqtt-export source=mqttsecret.go:125 msg="Connecting to mqtt server for export"
level=INFO ts=2021-10-16T08:04:20.777252528Z app=AppService-mqtt-export source=mqttsecret.go:134 msg="Connected to mqtt server for export"

Ireland (2.x)

This is the snippet for app-service-configurable.

See the comments for details.

  app-service-mqtt:
    container_name: edgex-app-mqtt-export
    depends_on:
      - consul
      - data
    environment:
      CLIENTS_CORE_COMMAND_HOST: edgex-core-command
      CLIENTS_CORE_DATA_HOST: edgex-core-data
      CLIENTS_CORE_METADATA_HOST: edgex-core-metadata
      CLIENTS_SUPPORT_NOTIFICATIONS_HOST: edgex-support-notifications
      CLIENTS_SUPPORT_SCHEDULER_HOST: edgex-support-scheduler
      DATABASES_PRIMARY_HOST: edgex-redis
      EDGEX_PROFILE: mqtt-export
      EDGEX_SECURITY_SECRET_STORE: "false"
      MESSAGEQUEUE_HOST: edgex-redis
      REGISTRY_HOST: edgex-core-consul
      SERVICE_HOST: edgex-app-mqtt-export
      TRIGGER_EDGEXMESSAGEBUS_PUBLISHHOST_HOST: edgex-redis
      TRIGGER_EDGEXMESSAGEBUS_SUBSCRIBEHOST_HOST: edgex-redis

      # Broker Address has to be formatted as "tcps://YOUR_IOT_HUB_NAME.azure-devices.net:8883"
      # https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-mqtt-support
      WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_BROKERADDRESS: tcps://iot-hub-edgex-foundry.azure-devices.net:8883

      # Topic has to be formatted as "devices/YOUR_DEVICE_NAME/messages/events/"
      # https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-mqtt-support
      WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_TOPIC: devices/edgex/messages/events/

      # Client ID has to be formatted as "YOUR_DEVICE_NAME"
      # https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-mqtt-support
      WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_CLIENTID: edgex

      # Auth with username and password (SAS Token)
      WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_AUTHMODE: usernamepassword
      
      # Skip verify to accept unknown authority
      WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_SKIPVERIFY: "true"
      
      # Username has to be formatted as "YOUR_IOT_HUB_NAME.azure-devices.net/YOUR_DEVICE_NAME/?api-version=2018-06-30"
      # https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-mqtt-support
      WRITABLE_INSECURESECRETS_MQTT_SECRETS_USERNAME: iot-hub-edgex-foundry.azure-devices.net/edgex/?api-version=2018-06-30

      # Paste your SAS Token. SAS Token can be generated using create_sas_token.py
      WRITABLE_INSECURESECRETS_MQTT_SECRETS_PASSWORD: SharedAccessSignature sr=iot-hub-edgex-foundry.azure-devices.net%2Fdevices%2Fedgex&sig=zqQlB0oS41qIcpHNst%2FAyMzO%2FNWzn30hZfs%2F9chum7A%3D&se=1635064560

    hostname: edgex-app-mqtt-export
    image: edgexfoundry/app-service-configurable:2.0.1
    networks:
      edgex-network: {}
    ports:
      - 127.0.0.1:59702:59702/tcp
    read_only: true
    security_opt:
      - no-new-privileges:true
    user: 2002:2001

Then start your EdgeX Foundry, and send some data from your device. For testing purpose, device-virtual can be used to send dummy data.

# Start EdgeX Foundry
$ docker-compose up -d
...

# Check logs after sending some data from your device
$ docker logs edgex-app-mqtt-export
...
level=INFO ts=2021-10-16T08:10:38.436078396Z app=app-mqtt-export source=mqttsecret.go:152 msg="Connecting to mqtt server for export"
level=INFO ts=2021-10-16T08:10:38.751541321Z app=app-mqtt-export source=mqttsecret.go:161 msg="Connected to mqtt server for export"

Verify your data is available on IoT Hub

You can monitor the telemetry using Cloud Shell on Azure Portal.

az iot hub monitor-events -n YOUR_IOT_HUB_NAME -d YOUR_DEVICE_NAME
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment