API Gateway guide to SOAP

SOAP is a big and painful topic when it comes to API gateway support. And the reason is that the SOAP protocol itself has a very flexible declarative XML format, and its specifications are unfortunately really vague and leave a lot of things open for interpretation.

In this document, we will try to cover all possible ways how you can integrate SOAP with Tyk API Gateway, from simple pass through to calling WSDL services.

Introduction to SOAP format

A SOAP message is a XML document which is used to transmit your data, and can look as simple as:

POST /HolidayService_v2/HolidayService2.asmx HTTP/1.1

Host: www.holidaywebservice.com
Content-Type: text/xml; charset=utf-8
Content-Length: length
SOAPAction: "http://www.holidaywebservice.com/HolidayService_v2/GetCountriesAvailable"

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <GetCountriesAvailable xmlns="http://www.holidaywebservice.com/HolidayService_v2/" />
  </soap:Body>
</soap:Envelope>

Message above is defined in SOAP 1.1 format, where inside SOAPAction header you specify URL of an actual action, and soap:Body contains GetCountriesAvailable XML tag, which specify arguments for the given services.

SOAP specification is quite broad and evolved with time, so we end up with a number of distinct ways on how SOAP service can be called. For example, SOAP 1.1 version requires passing SOAPAction header, while SOAP 1.2 makes it optional. There are specifications for calling SOAP services using simple GET and POST requests, using either query or form attributes.

Additionally, there is a WSDL standard: an XML document which describes how to connect and make requests to your web service, and which allows you to discover available SOAP actions and make queries using the same URL, just by specifying an operation name. Basically, SOAP messages are the data you transmit and WSDL tells you what you can do and how to make the calls.

Pass-through proxying

In this section, we want to forward SOAP messages as is without any changes, by still keeping features like analytics and authentication. It is the easiest way of configuration and does not require any specific API Gateway configuration.

If not talking about WSDL (will covert it below in separate section), SOAP routing happens either via URL Path itself, SOAPAction HTTP header or based on body content. Default routing in Tyk made by request Path, but you can use Advanced URL rewriter to do routing based on SOAPAction HTTP header, or by matching the content of the Body.

Authentification

SOAP itself does not define any standard on how to do authentication and authorization. It means that you can pick any mode like HTTP basic auth, standard token based auth (header, or inside query string), or even JWT.

In some cases credentials can be passed inside SOAP body, like in the example below:

  <Authentication>
      <Password>string</Password>
      <UserName>string</UserName>
  </Authentication>

If this is your case, you need to extract and convert this data to a format which API Gateway can understand. For example HTTP basic auth, or extract secret and embed it to a header.

Tyk API Gateway has built-in support for extracting credentials from the body, bundled to basic authentification mode, and can be enabled by setting basic_auth.body_user_regexp and basic_auth.body_password_regexp fields. Regexp for extracting username, based on provided example above, will be: <UserName>(.*)</UserName>.

If we talking about more complex cases, example above could be implemented using custom Pre JSVM plugin(https://tyk.io/docs/customise-tyk/plugins/javascript-middleware/) which convert data from body to HTTP basic auth can look as simple as:

var authMW = new TykJS.TykMiddleware.NewMiddleware({});
authMW.NewProcessRequest(function(request, session, spec) {
  var user = request.Body.match(/<UserName>([^<]+)<\/UserName>/)[1];
  var password = request.Body.match(/<Password>([^<]+)<\/Password>/)[1];
  request.SetHeaders[“Authentification”] = “Basic “ + b64enc(user + “:” + password);
  return authMW.ReturnData(request, session.meta_data);
});

REST to SOAP

A common case can be an API which use multiple sources, like another REST or SOAP services, and hides this logic from the user, providing a unified API interface. Another use case can be providing an alternative way to access your SOAP services, for example, if your team migrating to REST JSON services.

In this case, API Gateway should:

Extract data from the request: either path, payload or header
Build SOAP message based on request data
Call SOAP service and receive XML response
Transform XML response to desired format (like JSON), and respond to the user

Example using Tyk: We want define an endpoint which will return list of holidays based on country code. We going to build a wrapper around Holidays SOAP1.2 webservice, which returns a list of holidays for a given country code. When user query GET /holidays/<country-code>, it should internally invoke SOAP call, and return data converted from XML to JSON format.

Steps to implement:

Define GET /holidays/{country} endpoint in API designer
Add URL rewrite plugin, and rewrite URL to path of SOAP service, e.g. to http://www.holidaywebservice.com/HolidayService_v2/HolidayService2.asmx
Add HTTP method transform plugin, and change method from GET to POST (since this it is default way of talking with SOAP services)

Add Body Transform plugin, and set Request body template to:

<?xml version="1.0" encoding="utf-8"?>
<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope"> 
  <soap12:Body>
        	<GetHolidaysAvailable xmlns="http://www.holidaywebservice.com/HolidayService_v2/">
            	  <countryCode>{{index ._tyk_context.path_parts 2}}</countryCode>
        	</GetHolidaysAvailable>
  </soap12:Body>
</soap12:Envelope>

Note that we used {{._tyk_context.path_parts.2}} template variable to extract country code from URL.

Inside the same body transform plugin, now set Response body template, to convert response data from XML to JSON. For simplicity we can set it to {{ . | jsonMarshal }}.
Now you can call /holidays/UnitedStates and get JSON response back.

For more advanced workflows you can use VirtualEndpoint(https://tyk.io/docs/compose-apis/virtual-endpoints/) to write custom logic in Javascript, or for example, use Post plugin, written in Python or gRPC, which will do SOAP query with a library of your choice, and will respond to the user using Return overrides(https://tyk.io/docs/customise-tyk/plugins/rich-plugins/rich-plugins-work/#python-example) directive.

SOAP to REST

In some cases, you may want to extend your SOAP service with an action which internally should call another service not compatible with SOAP, like REST or similar.

Similar to REST -> SOAP, API Gateway should:

Extract data from the request: either path, XML payload or header
Build new request, for example JSON one, based on request data
Call service and receive REST/JSON response
Transform REST/JSON response to SOAP response, and respond to the user

Example using Tyk: Steps to implement this will be very similar to our approach with REST to SOAP:

If your SOAP service use HTTP path, define an endpoint with required name. If use header or body based routing, define an Advanced URL rewrite, and as a target loop to another “virtual” endpoint, which will be used for assigning plugins. If your SOAP service provides multiple operations, you can use Looping feature to define complex workflows for individual operations. See Handling complex cases(#Handling-complex-cases) section.
Add body transform plugin, and config request transform from XML to format compatible with the external service. If external service use GET instead of POST, you can use Advanced URL rewriter instead, to extract data from XML body using regexp, and add it to query attributes or URL path as needed.
Define response transformation template, which will translate JSON response from external service, to the SOAP message.

As previously, you can use VirtualEndpoint or Rich plugins to write advanced custom logic.

WSDL support

While API gateway does not directly support importing WSDL files, you can use free Apimatic transformer(https://www.apimatic.io/transformer) service to convert WSDL definition to Swagger file, which can be imported to Tyk Dashboard when creating an API.

Additionally, below you can see an example of a simple Post Python plugin which uses Zeep library to do a SOAP calls to the WSDL service, in programmatic way.

from tyk.decorators import *
from gateway import TykGateway as tyk
from zeep import Client

client = Client('http://www.webservicex.net/ConvertSpeed.asmx?WSDL')

@Hook
def PostMiddleware(request, session, spec):
    if request.URL == “/convert_speed”:     
    result = client.service.ConvertSpeed(100, 'kilometersPerhour', 'milesPerhour')    request.object.return_overrides.headers['content-type'] = 'application/json'
       request.object.return_overrides.response_code = 200
    request.object.return_overrides.response_error = “{\”speed\”: \””+result+”\”}\n”

    return request, session

Here you can find documentation on how to install such plugin.

Services with multiple operations

You can also use Python middleware like this any other language with gRPC plugins, if you prefer using SOAP library of your choice.

Summary

While SOAP acronym stands to “Simple”, you can see that its handling can be quite tricky. But the good part is that every SOAP service by its nature is an HTTP based service, and Tyk API Gateway has all the building block to deal with both simple and advanced SOAP usage.

buger/read.md