FileMaker, cURL and HTTP Response Codes

When FileMaker Inc. released the FileMaker 16 Platform last year they included a feature I had been requesting for over 15 years – native cURL support. cURL is a command line tool and library for transferring data with URLs – you’re probably familiar with the HTTP, HTTPS, FTP and SMTP protocols. cURL is important as it allows us to interact with external web services/APIs directly from FileMaker Pro, FileMaker Go, FileMaker WebDirect and FileMaker Server. For example we can use FileMaker to send an SMS, upload an Invoice to Xero or MYOB, or download Orders from eCommerce platforms like WooCommerce or Shopify.

Starting with the introduction of the Insert From URL script step in FileMaker Pro v12 we’ve been able to interact with URLs with some limitations:

• FileMaker Pro v12 introduced the Insert From URL script step which supported http, https, ftp, ftps, and file protocols. A field was required to download the result/response into and it didn’t support performing an HTTP/S POST request – only GET requests were supported (most APIs require a POST request to create/update records via the API). There was also no ability to customise the HTTP request, e.g. to set HTTP Headers which are often required for many APIs
• FileMaker Pro v13 extended the Insert From URL script step to both HTTP GET and HTTP POST requests using the httppost and httpspost custom schemes that you specified when constructing the URL (e.g. “httppost://www.filemaker.com/path?fname=Bob&lname=Smith”). You still couldn’t specify HTTP Headers and still required a field for the result

Given these limitations I rarely used the Insert from URL script step until FileMaker Pro 16 was released. Since the FileMaker Pro v6 days I have been using a number of different plug-ins to perform HTTP requests – initially I had a custom plug-in developed, then switched to the Troi URL plug-in and then to the BaseElements plug-in. Databuzz sponsored the development of some specific functions that we needed and the BaseElements plug-in provided us with cURL support and a number of other related functions, such as JSON encoding and parsing and XPath for parsing XML data. The BaseElements plug-in has provided us with the functionality we needed that was missing in the FileMaker platform, but as a plug-in was required it did have a number of disadvantages:

• FileMaker Go does not support plug-ins (you can now use the iOS SDK). You could use FileMaker Server script schedules and Perform Script on Server as a workaround but this required the FileMaker solution to be hosted by FileMaker Server so wouldn’t work for solutions running locally on the iPad or iPhone
• running scripts under FileMaker Server required the plug-in to be installed on the server, which can be done easily as long as the appropriate permissions have been granted in the FileMaker Server Admin Console. Some hosting providers also disabled the option to install plug-ins on shared servers.

With the release of the FileMaker 16 platform came the ability to specify one or more supported cURL options as a calculation when using the Insert From URL script step, as well as the ability to specify a variable as the target (no more fields required!). This was a game changer as far as working with APIs was concerned – now the entire FileMaker Platform could natively make HTTP requests, set HTTP Headers, and encode JSON data and parse JSON. All of our integration solutions were using the BaseElements plug-in, so we started to work on adding native support for FileMaker Pro v16 users by switching from the BaseElements plug-in functions to using the native Insert From URL script step, cURL options and JSON functions.

One function for which there was no native equivalent was the BE_HTTP_Response_Code function which returns the HTTP response code value from the last request. HTTP Response Codes are issued by the server in response to a request and can be used to determine whether the HTTP request was successfully completed or if there were issues associated with the request. For example 200 is the standard response for a successful request and 401 means there was an authentication issue and the request failed. It’s important when working with APIs to check the HTTP Response Code to confirm that the request was processed successfully – the documentation for each API will usually specify what response code indicates a successful request (typically 200 or 201).

The HTTP Response Code is contained with the HTTP Response Headers which are returned by the server processing the request. You need to specify the following cURL option to get the response headers returned into a FileMaker variable:

" --dump-header $responseHeaders"

The $responseHeaders variable will then contain something like this after the Insert From URL script step has been performed:

HTTP/1.1 200 OK
Date: Wed, 13 Jun 2018 04:45:17 GMT
Server: Apache/2.4.26 (Red Hat)
X-Powered-By: PHP/5.6.35
Transfer-Encoding: chunked
Content-Type: application/json; charset=UTF-8

In the above example you can see the first line of the response headers contains the response code: “HTTP/1.1 200 OK”. Other common response codes that you might encounter include:

HTTP/1.1 100 Continue
HTTP/1.1 403 Forbidden
HTTP/1.1 201 Created
HTTP/1.1 302 Found
HTTP/1.1 301 Moved Permanently

You can use standard FileMaker functions to get the actual numerical response code from the first line of the response headers, e.g.:

Let ( [
t1 = GetValue ( $responseHeaders ; 1 ) ;
t2 = Substitute ( t1 ; " " ; "¶" ) ;
n1 = GetValue ( t2 ; 2 )
] ;

n1

) // Let

This all works fine as long as the Response Headers only contain a single response code, but sometimes the server will return multiple response codes. For example the response might include a redirect or a continue and look like this:

HTTP/1.1 100 Continue

HTTP/1.1 403 Forbidden
Server: nginx/1.14.0
Date: Wed, 13 Jun 2018 07:41:16 GMT
Connection: keep-alive

The response code we are after is the “403” value and not the “100” value, so our above calculation would return the wrong response code in this situation. We started working on a custom function that would look for the last “HTTP/1.1” string and then get the code following this, but that failed pretty quickly when working with a server that included the string “HTTP/1.1” elsewhere in the response headers.

We’ve settled on using this custom function which so far is returning the correct result, at least with the APIs and servers that we have tested it on so far. We would much rather have a native function for this which does the equivalent of what the BaseElements BE_HTTP_Response_Code function does without having to try to locate and find the last response code from the response headers. From researching this it appears that there is already a cURL option for this:

-w "%{http_code}"

but unfortunately the -w option is not one of the FileMaker supported cURL options, so for now you’ll need to parse out the response code until FileMaker add support for the -w option. We’ve added this to the FileMaker Product Ideas discussion area – if you would also like to see this feature in a future version of FileMaker platform you can vote up the idea here.