Sending Messages from FileMaker to WhatsApp Users with Twilio

 

Last month I attended the Twilio Engage + Superclass events in Melbourne, Australia. This was my second Twilio Engage event after attending last year’s Sydney event, but my first Superclass session. Superclass is hands-on training for developers to learn how to use all things Twilio, with Twilio engineers on hand to help as you work through code challenges in TwilioQuest, their self-paced training video game. The use of TwilioQuest makes the day lots of fun with your awards flashing up on the big screen every time you compete a new challenge.

Even though I’ve been developing with the Twilio APIs for many years ( see my article from last year about how we’ve integrated FileMaker with Twilio for two-way messaging, voice calling, number lookups and polling for incoming messages) it’s always great to spend a day away from work with other developers and the Twilio staff to learn new skills and immerse yourself in the content without distractions.

My aim for this year’s Superclass was to spend time learning some of the other Twilio APIs that I wasn’t very familiar with, with a particular focus on the Twilio API for WhatsApp. I was aware that Twilio had recently launched the API for WhatsApp which allows you to send SMS messages directly to WhatsApp users – there are over 1.5 billion people using WhatsApp – and I was keen to see if I could get this working from fmSMS, our FileMaker solution for sending and receiving SMS messages from the FileMaker platform.

The Twilio API for WhatsApp is currently in beta and you can use their Sandbox to setup access from a Twilio number and start sending messages using their sample templates. The API works in a similar fashion to their standard Message API and I was able to get up and running quickly and send my first message from fmSMS to my WhatsApp account. Here’s a screenshot showing the message I sent from fmSMS:

and here’s how it appears in the WhatsApp app on my iPhone:

I then enabled a Webhooks for incoming messages and was then able to receive incoming messages from WhatsApp back in to fmSMS. You can learn more about pricing for the Twilio API for WhatsApp here and API documentation here.

Once the Twilio API for WhatsApp is out of beta and you can easily request your own numbers we’ll definitely be adding this as another supported Gateway for fmSMS. We’ve put together a short video showing sending and receiving messages between FileMaker and WhatsApp in action:

2018 – Another Big Year for APIs

2018 was another big year for the API (application programming interface) economy – the ProgrammableWeb directory, which tracks APIs, passed the 19,000-API mark in January of 2018 and currently sits at 20,653 on the last day of 2018. There has been an explosion in growth in the last 5 years, with more than 2,000 APIs being added every year to their directory.

Earlier this year I attended the annual Xero Developer Roadshow in Melbourne and this slide caught my attention (apologies for the resolution/quality):

Xero released their first API in 2007 – the first cloud accounting API – and now receive more than 450 million API calls each month! There was another slide showing the number of ASW Servers they deploy for handling these API calls, which from memory was in the hundreds but could have been in the thousands.

Gartner have been talking about the concept of an API economy for a number of years and that businesses should put APIs at the centre of their digital business platform. An API economy is an enabler for turning a business or organisation into a platform which is something we’ve seen with APIs like the Xero API, which now has 40,000 active developer partners and over 700 certified application partners.

2018 also saw FileMaker, Inc. ship the FileMaker Data API as part of FileMaker Server/Cloud 17 platform. This is the first REST API (Representational State Transfer) from FileMaker which makes connecting other applications and web services much easier. You can now choose your programming language of choice when connecting to the FileMaker Data API – you can even use a FileMaker client such as FileMaker Pro or FileMaker Go which we’ve done on a number of projects (see our earlier article if you’re wondering why you would do this).

We specialise in FileMaker integration here at Databuzz, which more often than not means working with APIs. We’re typically connecting FileMaker to another platform such as Xero for accounting integration or Shopify for ecommerce integration, allowing our customers to easily exchange information between different platforms and streamlining business workflows.

Looking back at the projects we’ve worked on in 2018 here’s a partial list of the APIs that we connected the FileMaker platform to over the last 12 months:

We already have a few projects lined up for 2019 which will see us working on some new APIs for the first time. Now that the FileMaker platform can easily integrate with APIs, both as a client and as a server, working on integration projects has never been easier.

If you would like to discuss integrating FileMaker with an API for your business please get in touch for a free initial consultation to discuss your requirements.

fmEcommerce Link (Shopify Edition) Now Runs Natively on FileMaker 16/17 Platform and supports Order Creation Webhooks

 

We’re pleased to announce that v1.3 of fmEcommerce Link (Shopify Edition) is now available and includes a number of new features, including native support for FileMaker Pro v16 and later. This is a free update to all existing customers (simply download using the same link that was on your original order email).

With the release of the FileMaker 16 platform we got a number of new functions and script steps, including support for working with JSON data and being able to specify supported cURL options with the Insert From URL script step. These new features meant that we were no longer dependent on plug-in functions and we could replace these with native FileMaker features.

We’ve updated all the requests for downloading and uploading to the Shopify API to use native FileMaker functions and script steps so you no longer need to use a FileMaker plug-in if you’re using FileMaker Pro v16 or higher. This means you can also use fmEcommerce Link on FileMaker Go natively (requires FileMaker Go 16 or higher).

Like our previous updates for native FileMaker functionality for our other products we’ve simply updated the scripts to check which version of FileMaker Pro, FileMaker Go or FileMaker Server is being used and if you’re using v15 or earlier we simply use the previous plug-in functionality, and if you’re using v16 or later we now use native functionality. Everything works exactly the same regardless of which version of FileMaker Pro you’re using, but we can now include support for FileMaker Go as well as having one less dependency to worry about.

This release also includes support for Shopify Webhooks – see our earlier post for a short video demonstrating how the Order Creation Webhook works. Instructions for setting up the Order Creation Webhook can be found here.

You can get all the details on our version history page – we’re continuing to work on adding native FileMaker 16 support to our other products, as well as some further updates to fmEcommerce Link (Shopify Edition) including support for processing Fulfilments in FileMaker and uploading to Shopify.

fmAccounting Link (Xero Edition) Update – Notes & History and Attachments

In Xero, you can view the history of changes made to invoices, bills, transactions, inventory items, contacts and more. This audit trail also shows any notes recorded by users in the Xero app but until recently this information was not available via the Accounting API.

Earlier this year Xero added a new endpoint to the Accounting API to allow developers to be able to download the notes/history and also add new notes. We’ve just released an update to fmAccounting Link (Xero Edition) that includes examples for downloading and creating Notes for Contacts and Invoices.

The history and notes for an Invoice in the Xero web app looks like this:

and contains important information, such as when the Invoice was sent to the customer and if they have viewed the invoice online. We’ve added a new Notes tab to display the same information in the fmAccounting Link file:

You can download all the Notes from Xero for an Invoice and a Contact and create a new Note and upload that to Xero for a Invoice and a Contact. We started adding support for the new History and Notes endpoint for Contacts and Invoices as these are two of the most popular modules, but will gradually add them to the other supported modules such as Bills, Credit Notes, Items etc.

We also took this opportunity to extend the support for working with Attachments to also include the ability to download Attachments for an Invoice and create/download Attachments for a Contact.

Here’s a video that demonstrates working with History and Notes and Attachments in fmAccounting Link (Xero Edition):

FileMaker and the Twilio API

Here at Databuzz we’re longtime fans of Twilio, the company with the strange name that offers a Cloud communications platform, providing the building blocks to add messaging, voice, and video in your web and mobile applications.

For many years we’ve been working with the Twilio API, primarily to send and receive SMS messages using our fmSMS FileMaker solution. Twilio provides great coverage and very competitive pricing and we’ve been recommending Twilio to our customers for many years – when they ask which SMS Gateway we use at Databuzz we don’t hesitate to recommend Twilio.

Earlier this year we had the chance to attend the first Twilio Engage Sydney and learn more about the company’s strategy and participate in some great product demos. Twilio are opening 2 new offices in Melbourne and Sydney and it was great to be able to meet some of their Australian based staff in person.

We’ve written about how we use the Twilio API in the past, including a demonstration for Voice Calling with FileMaker and Twilio as well as sending Emojis from fmSMS. We’ve also made some custom solutions for customers to allow them to send and receive MMS (picture messages) messages from the FileMaker platform.

Over the past few months we’ve had the opportunity to work on some additional Twilio APIs that enhance the functionality found in our fmSMS solution, including:

  • adding phone number lookup functionality via the Twilio Lookup API
  • adding the ability to poll Twilio for any incoming messages that may not have been delivered due to a local server/Internet outage when using the regular Webhooks

For some customers the ability to check a number before sending an SMS or MMS message is important as it allows you to verify that the number you are about to send a message to is indeed a mobile number. For example you might have a list of Customers and associated phone numbers, but you aren’t sure if the number is a landline number or a mobile number. Using the Twilio Lookup API you can make a request to verify a number and receive information about the carrier and optionally (where supported) the caller name, allowing you to confirm the veracity of the number.

fmSMS includes the necessary PHP files to receive incoming messages to your Twilio number and have these pushed automatically to your FileMaker Server, but there may be times when your server or Internet connection is offline but you would still like to download any missed messages into your FileMaker solution. We’ve helped customers add the ability to “poll” the Twilio Messages API for any incoming messages to their Twilio number that they might have missed during a planned or unscheduled outage.

We’ve put together a simple demonstration video showing these in action – you can watch the video inline below or directly on YouTube:

fmAccounting Link (Xero Edition) and Unit Price Rounding in Xero

By default the Xero API accepts unit prices (UnitAmount) to two decimals places – this means that if you were entering a line item on an Invoice and entered ‘1.795’ for the Unit Price it would be rounded to $1.80 in Xero. For most customers 2 decimal places is sufficient for the line item unit price, however there are some customers that do require greater precision. Xero offer a way to opt-in to 4 decimal places by adding a parameter to your API request.

For example if you wanted to create an Invoice and include 4 decimal places on the unit price for the line items you would change the URL from:

https://api.xero.com/api.xro/2.0/Invoices

to:

https://api.xero.com/api.xro/2.0/Invoices?unitdp=4

Unfortunately it wasn’t as simple as changing the URL in the appropriate fmAccounting Link scripts – even after changing the URL and including unit prices on the line items with 4 decimal places they were still being rounded to two decimals places when uploaded to Xero. We’ve come up with a solution to this to allow you to specify unit prices with 4 decimal places – you’ll need to simply install an updated version of the BaseElements plug-in (v3.3.8) and make sure you change your request URL to include the additional querystring parameter.

If you need to specify unit prices with 4 decimal places please get in touch and we’ll send you the new version of the BaseElements plug-in to use. You can get more information on the Xero developer website here:

https://developer.xero.com/documentation/api-guides/rounding-in-xero

FileMaker, cURL and HTTP Response Codes

When FileMaker Inc. released the FileMaker 16 Platform last year they included a feature I had been requesting for over 15 years – native cURL support. cURL is a command line tool and library for transferring data with URLs – you’re probably familiar with the HTTP, HTTPS, FTP and SMTP protocols. cURL is important as it allows us to interact with external web services/APIs directly from FileMaker Pro, FileMaker Go, FileMaker WebDirect and FileMaker Server. For example we can use FileMaker to send an SMS, upload an Invoice to Xero or MYOB, or download Orders from eCommerce platforms like WooCommerce or Shopify.

Starting with the introduction of the Insert From URL script step in FileMaker Pro v12 we’ve been able to interact with URLs with some limitations:

  • FileMaker Pro v12 introduced the Insert From URL script step which supported http, https, ftp, ftps, and file protocols. A field was required to download the result/response into and it didn’t support performing an HTTP/S POST request – only GET requests were supported (most APIs require a POST request to create/update records via the API). There was also no ability to customise the HTTP request, e.g. to set HTTP Headers which are often required for many APIs
  • FileMaker Pro v13 extended the Insert From URL script step to both HTTP GET and HTTP POST requests using the httppost and httpspost custom schemes that you specified when constructing the URL (e.g. “httppost://www.filemaker.com/path?fname=Bob&lname=Smith”). You still couldn’t specify HTTP Headers and still required a field for the result

Given these limitations I rarely used the Insert from URL script step until FileMaker Pro 16 was released. Since the FileMaker Pro v6 days I have been using a number of different plug-ins to perform HTTP requests – initially I had a custom plug-in developed, then switched to the Troi URL plug-in and then to the BaseElements plug-in. Databuzz sponsored the development of some specific functions that we needed and the BaseElements plug-in provided us with cURL support and a number of other related functions, such as JSON encoding and parsing and XPath for parsing XML data. The BaseElements plug-in has provided us with the functionality we needed that was missing in the FileMaker platform, but as a plug-in was required it did have a number of disadvantages:

  • FileMaker Go does not support plug-ins (you can now use the iOS SDK). You could use FileMaker Server script schedules and Perform Script on Server as a workaround but this required the FileMaker solution to be hosted by FileMaker Server so wouldn’t work for solutions running locally on the iPad or iPhone
  • running scripts under FileMaker Server required the plug-in to be installed on the server, which can be done easily as long as the appropriate permissions have been granted in the FileMaker Server Admin Console. Some hosting providers also disabled the option to install plug-ins on shared servers.

With the release of the FileMaker 16 platform came the ability to specify one or more supported cURL options as a calculation when using the Insert From URL script step, as well as the ability to specify a variable as the target (no more fields required!). This was a game changer as far as working with APIs was concerned – now the entire FileMaker Platform could natively make HTTP requests, set HTTP Headers, and encode JSON data and parse JSON. All of our integration solutions were using the BaseElements plug-in, so we started to work on adding native support for FileMaker Pro v16 users by switching from the BaseElements plug-in functions to using the native Insert From URL script step, cURL options and JSON functions.

One function for which there was no native equivalent was the BE_HTTP_Response_Code function which returns the HTTP response code value from the last request. HTTP Response Codes are issued by the server in response to a request and can be used to determine whether the HTTP request was successfully completed or if there were issues associated with the request. For example 200 is the standard response for a successful request and 401 means there was an authentication issue and the request failed. It’s important when working with APIs to check the HTTP Response Code to confirm that the request was processed successfully – the documentation for each API will usually specify what response code indicates a successful request (typically 200 or 201).

The HTTP Response Code is contained with the HTTP Response Headers which are returned by the server processing the request. You need to specify the following cURL option to get the response headers returned into a FileMaker variable:

" --dump-header $responseHeaders"

The $responseHeaders variable will then contain something like this after the Insert From URL script step has been performed:

HTTP/1.1 200 OK
Date: Wed, 13 Jun 2018 04:45:17 GMT
Server: Apache/2.4.26 (Red Hat)
X-Powered-By: PHP/5.6.35
Transfer-Encoding: chunked
Content-Type: application/json; charset=UTF-8

In the above example you can see the first line of the response headers contains the response code: “HTTP/1.1 200 OK”. Other common response codes that you might encounter include:

HTTP/1.1 100 Continue
HTTP/1.1 403 Forbidden
HTTP/1.1 201 Created
HTTP/1.1 302 Found
HTTP/1.1 301 Moved Permanently

You can use standard FileMaker functions to get the actual numerical response code from the first line of the response headers, e.g.:

Let ( [
t1 = GetValue ( $responseHeaders ; 1 ) ;
t2 = Substitute ( t1 ; " " ; "¶" ) ;
n1 = GetValue ( t2 ; 2 )
] ;

n1

) // Let

This all works fine as long as the Response Headers only contain a single response code, but sometimes the server will return multiple response codes. For example the response might include a redirect or a continue and look like this:

HTTP/1.1 100 Continue

HTTP/1.1 403 Forbidden
Server: nginx/1.14.0
Date: Wed, 13 Jun 2018 07:41:16 GMT
Connection: keep-alive

The response code we are after is the “403” value and not the “100” value, so our above calculation would return the wrong response code in this situation. We started working on a custom function that would look for the last “HTTP/1.1” string and then get the code following this, but that failed pretty quickly when working with a server that included the string “HTTP/1.1” elsewhere in the response headers.

We’ve settled on using this custom function which so far is returning the correct result, at least with the APIs and servers that we have tested it on so far. We would much rather have a native function for this which does the equivalent of what the BaseElements BE_HTTP_Response_Code function does without having to try to locate and find the last response code from the response headers. From researching this it appears that there is already a cURL option for this:

-w "%{http_code}"

but unfortunately the -w option is not one of the FileMaker supported cURL options, so for now you’ll need to parse out the response code until FileMaker add support for the -w option. We’ve added this to the FileMaker Product Ideas discussion area – if you would also like to see this feature in a future version of FileMaker platform you can vote up the idea here.

fmEcommerce Link (WooCommerce Edition) Now Runs Natively on FileMaker 16 Platform

When we first released fmEcommerce Link (WooCommerce Edition) back in 2016 the functionality required to integrate with the WooCommerce REST API did not exist in the current version of FileMaker Pro, therefore we had to use a FileMaker Pro plug-in to handle the API requests, set HTTP Headers, encode JSON and parse the JSON response. This allowed fmEcommerce Link (WooCommerce Edition) to run under FileMaker Pro v12 and later, as well as under FileMaker Server with the plug-in installed, however it prevented it from running on FileMaker Go which does not currently support plug-ins (except via the iOS App SDK).

With the release of the FileMaker 16 platform we got a number of new functions and script steps, including support for working with JSON data and being able to specify supported cURL options with the Insert From URL script step. These new features meant that we were no longer dependent on plug-in functions and we could replace these with native FileMaker features. We’re pleased to announce that v1.5 of fmEcommerce Link (WooCommerce Edition) is now available and includes a number of new features, including native support for FileMaker Pro v16 and later. This is a free update to all existing customers (simply download using the same link that was on your original order email).

We’ve updated all the requests for downloading and uploading to the WooCommerce API to use native FileMaker functions and script steps so you no longer need to use a FileMaker plug-in if you’re using FileMaker Pro v16 or higher. This means you can also use fmEcommerce Link on FileMaker Go natively (requires FileMaker Go 16) – we’ve also included an example for using Perform Script on Server to have FileMaker Server handle uploading a Product to WooCommerce.

In working on this update (which was a much bigger update than we anticipated and took much longer than we planned!) we tried to avoid making too many unnecessary schema changes. We’ve simply updated the scripts to check which version of FileMaker Pro, FileMaker Go or FileMaker Server is being used and if you’re using v15 or earlier we simply use the previous plug-in functionality, and if you’re using v16 or later we now use native functionality. Everything works exactly the same regardless of which version of FileMaker Pro you’re using, but we can now include support for FileMaker Go as well as having one less dependency to worry about.

You can get all the details on our version history page – we’re busy working on adding native FileMaker 16 support to our other products, as well as some further updates to fmEcommerce Link (WooCommerce Edition) including support for uploading Orders from FileMaker to WooCommerce.

fmAccounting Link (Xero Edition) and the Xero API TLS 1.0 Deprecation

Xero have recently announced that they will be deprecating support for TLS (Transport Layer Security) 1.0 from 30 June 2018. TLS is used to secure communication between partner apps like fmAccounting Link (Xero Edition) and the Xero API. The Xero API will now require partners apps to use TLS 1.1 as a minimum and recommend upgrading to TLS 1.2.

Xero have recently made available a test URL for partner app developers to test their apps to ensure they will continue to work after 30 June 2018. Databuzz is pleased to report that after performing a series of tests using fmAccounting Link (Xero Edition) with the test URL we encountered no issues and were able to authenticate the private application, request data from Xero and upload data to Xero successfully.

fmAccounting Link (Xero Edition) is also already using TLS 1.2 which will most likely become the mandatory minimum version at some point in the future – many of the other APIs that we work with at Databuzz have already made TLS 1.2 the minimum version.

If you have any questions about fmAccounting Link (Xero Edition) and the Xero API TLS 1.0 Deprecation please contact us.

fmAccounting Link (MYOB AccountRight Edition) and AccountRight 2018.1.1

We’ve been testing fmAccounting Link (MYOB AccountRight Edition) with AccountRight 2018.1.1 that was released this week and have not found any issues in our testing so far. We’ve been able to download data from AccountRight and upload data to AccountRight successfully so we don’t anticipate any issues with AccountRight 2018.1.1, but please let us know if you encounter any issues when working with AccountRight 2018.1.1.

MYOB have also recently added a new beta API endpoint for Quotes that will allow you to read, create, update and delete sales quotes via the API. This will work in a similar way to Invoices and Orders, and will be available for all quote layouts including Item, Miscellaneous, Professional, Service and Time Billing. Once the Quotes API endpoint is out of beta and available to all users we will add examples for working with Quotes to the fmAccounting Link (MYOB AccountRight Edition) solution as a free update for all existing customers.