Overview
Some tasks are best run as batch processes, such as checking whether the 5 Million phone numbers in your database are still valid. A useful automation feature in the Syniverse Developer Community is that it can notify you when your batch job is complete and the results are ready. The results can then be automatically downloaded to your server.
This is simple to set up using the Syniverse Developer Community. This example shows how to set up a simple web service to receive the notification and automatically download the output file.
Pre-requisites
It is assumed that you have already registered with the Syniverse Developer Community and run the Quick Start Batch Number Lookup example. If not then do this following the instructions here
It is also assumed that you know some Python, since this example is written in Python. Ideally you have also heard of the Flask framework, though that is not essential.
Set up SDC
Set up delivery of Batch Automation Notifications
Next we want to configure the Event Manager to send notifications for completed batch jobs to your preferred end point.
First set up a delivery configuration. I recommend using https://requestb.in or similar service first to test the service. To do this create a requestbin and use the URL provided in the address field of the delivery configuration.
Next create a subscription which sends batch automation events, including job complete notifications, to this delivery configuration.
Test this by running a batch job, and once it is complete refresh the requestbin you created. This will then show you the content of the notification.
Set up your End point
Our example end point is written in Python and uses the Flask framework. It is a deliberately simple end point comprising
- A REST service created using Flask which receives POST calls from the SDC Event Manager and parses the notification to check whether it is a job complete notification.
- A simple method which parses the notification to find the file URI and uses this to download the output file. It then unzips this file and saves it to the current folder.
To host our example we are using a cloud hosting service called PythonAnywhere. This provides a hosted Python environment and infrastructure using Amazon cloud. There is even a free tier that can be used to run this demo.
Sign up at PythonAnywhere
Follow the instructions at https://www.pythonanywhere.com/ and select the free tier.
Set up the Flask Hello World Example
Using the guided Web App creation tool to create a hello world Flask App. You can do this by going to the Web Tab and hit Add a new Web App, and choose Flask and Python 2 (the version of Python the example is written in).
Alternatively there are manual instructions here.
If you want to learn more about Flask there are a lot of resources on the internet. The main resource I used to build this example is here.
Install the Batch Download Flask App
The example code is stored on GitHub here.
To install and set this up
- Open a bash console on PythonAnywhere and create a new directory
- Use git clone to download the code to the directory
- Use your preferred editor to
- Add the access token for the SDC application
- Save the changes
- Then copy batch_download_output_app.py to app.py in the flask directory, overwriting the existing Hello World Flask application.
- Reload the web application via the PythonAnywhere Web Tab.
Your end point / REST service is now live at
http://your-username.pythonaywhere.com/batchjob/v1/notification
Test the REST App
There are a few ways we can test this.
You can use Postman to test the Application from your desktop. A generic POST request should provide a HTTP 201 response saying ‘hello not an ABA job complete notification
You can also use POSTMAN to simulate the notification itself using the example notification provided on the documentation page here.
Update the Delivery Configuration
Once your REST app is set up then have one final change needed, to point the delivery configuration to the REST App.
You can do this by editing the delivery configuration and changing the address to the URL for your REST service.
Finally you can test the service by running a batch job, for example using the QuickStart Example. Once complete you should a new file created in the Flask working directory. This file will be called outputYYMMDDHHMMSS.txt where YYMMDDHHMMSS is the time stamp for when the file was downloaded.
0 Comments