Azure Synapse Receiving and Passing Values from Notepad

notebook.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "source": [
        "val input =\"\""
      ],
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.livy.statement-meta+json": {
              "spark_pool": "SparkPool",
              "session_id": 46,
              "statement_id": 1,
              "state": "finished",
              "livy_statement_state": "available",
              "queued_time": "2022-05-06T10:08:47.0619711Z",
              "session_start_time": "2022-05-06T10:08:47.1072552Z",
              "execution_start_time": "2022-05-06T10:11:21.7515627Z",
              "execution_finish_time": "2022-05-06T10:11:23.7751133Z"
            },
            "text/plain": "StatementMeta(SparkPool, 46, 1, Finished, Available)"
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "input: String = \"\"\n"
          ]
        }
      ],
      "execution_count": 3,
      "metadata": {
        "jupyter": {
          "source_hidden": false,
          "outputs_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        },
        "tags": [
          "parameters"
        ]
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**To pass external paramters to Notepad**\n",
        "- Create a variable Eg:input\n",
        "- Convert it into parameters ie., hover over >> ck on \"...\" > Toggle parameters\n",
        "- While creating pipeline >New Notebooks >  settings > Add this notebook > baseparamters > \"input\" - string - abcd\n",
        "- Settings > baseparamters > set the variable to required Value"
      ],
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "println(input)"
      ],
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.livy.statement-meta+json": {
              "spark_pool": "SparkPool",
              "session_id": 46,
              "statement_id": 2,
              "state": "finished",
              "livy_statement_state": "available",
              "queued_time": "2022-05-06T10:08:47.1394043Z",
              "session_start_time": null,
              "execution_start_time": "2022-05-06T10:11:23.8849397Z",
              "execution_finish_time": "2022-05-06T10:11:24.944075Z"
            },
            "text/plain": "StatementMeta(SparkPool, 46, 2, Finished, Available)"
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n"
          ]
        }
      ],
      "execution_count": 4,
      "metadata": {
        "jupyter": {
          "source_hidden": false,
          "outputs_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Ref https://aboutdataai.com.au/2021/07/05/azure-synapse-notebooks-pass-parameter-and-return-output/\n",
        "https://docs.microsoft.com/en-us/azure/synapse-analytics/synapse-notebook-activity?tabs=classical \n",
        "\n",
        " **mssparkutils is equivalent to Dbutils of Databricks**\n",
        "```\n",
        "from notebookutils import mssparkutils\n",
        "```\n",
        " **Return**  \n",
        " \n",
        "```\n",
        "mssparkutils.notebook.exit(\"Data\")\n",
        "```"
      ],
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "code",
      "source": [
        "mssparkutils.notebook.exit(\"sending some data\")"
      ],
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.livy.statement-meta+json": {
              "spark_pool": "SparkPool",
              "session_id": 46,
              "statement_id": 3,
              "state": "finished",
              "livy_statement_state": "available",
              "queued_time": "2022-05-06T10:08:47.2001631Z",
              "session_start_time": null,
              "execution_start_time": "2022-05-06T10:11:25.0659694Z",
              "execution_finish_time": "2022-05-06T10:11:26.9044139Z"
            },
            "text/plain": "StatementMeta(SparkPool, 46, 3, Finished, Available)"
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "execution_count": 5,
          "data": {},
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ExitValue: sending some data"
          ]
        }
      ],
      "execution_count": 5,
      "metadata": {
        "jupyter": {
          "source_hidden": false,
          "outputs_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The return value can be accessed in pipeline :\n",
        "```\n",
        "@activity('NotebookName').output.status.Output.result.exitValue\n",
        "```"
      ],
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      }
    }
  ],
  "metadata": {
    "language_info": {
      "name": "scala"
    },
    "kernelspec": {
      "name": "synapse_pyspark",
      "language": "Python",
      "display_name": "Synapse PySpark"
    },
    "kernel_info": {
      "name": "synapse_pyspark"
    },
    "save_output": true,
    "synapse_widget": {
      "version": "0.1",
      "state": {}
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}