etl-data-exchange.txt

TALEND-CONTEXT sent to ETL by Qurso webservice

fiscalPeriod: format "YYYYMM01" (always the 1st day of month)

Example of payload:

{
   "callbackUrl":"http://www.example.com/some-url-to-post-results-to",
   "clientId":12344,
   "fiscalPeriod":"20170901",
   "files":[
      {
         "fileMetadata":{
            "bussinessId":"12345"
         },
         "fileS3Bucket":"quorso-development",
         "fileS3Key":"company_data_upload/files/company_4/2017/7/112/transactions-1213126-20170724T130452Z.csv",
         "fileType":"fileTypeBranchAccounts"
      },
      {
         "fileMetadata":{
            "manblo":"ertyui"
         },
         "fileS3Bucket":"quorso-development",
         "fileS3Key":"company_data_upload/files/someweird.csv",
         "fileType":"anotherType"
      }
   ]
}

ETL response in case of successfull processing

Example:

POST request to webservice to "callbackUrl" received earlier.
In case of not critical errors: save files to S3 bucket (probably, using pattern: SeoutBucket/clientId/fiscalPeriod/{...}) and post array of files with bucket name and S3 file key)
    
{
   "status":"OK",
   "clientId":1234,
   "fiscalPeriod":"20171101",
   "files":[
      {
         "fileS3Bucket":"quorso-development",
         "fileS3Key":"company_data_upload/files/company_4/2017/7/112/transactions-1213126-20170724T130452Z.csv"
      },
      {
         "fileS3Bucket":"quorso-development",
         "fileS3Key":"company_data_upload/files/someweird.csv"
      }
   ],
   "errors":[
      "Something is missing",
      "Other not critical error"
   ]
}
    
In case of failure: 

Don't put files to S3 bucket, send S3 keys of input files.
{
   "status":"ERROR",
   "clientId":1234,
   "fiscalPeriod":"20171101",
   "files":[
      {
         "fileS3Bucket":"quorso-development",
         "fileS3Key":"company_data_upload/files/company_4/2017/7/112/transactions-1213126-20170724T130452Z.csv"
      },
      {
         "fileS3Bucket":"quorso-development",
         "fileS3Key":"company_data_upload/files/someweird.csv"
      }
   ],
   "errors":[
      "Critical error",
      "Other critical error"
   ]
}