Connecting FileMaker Pro to FileMaker Server with the FileMaker Data API

/2 Comments/in , , , , , , , /by

FileMaker Server 16 (and FileMaker Cloud 1.16.0) included a new trial feature called the FileMaker Data API (application programming interface), which allows web services or applications to access data in hosted FileMaker solutions without needing any FileMaker client software to be installed. This is similar to the current XML and PHP APIs which have been around for over 10 years, however the Data API is a REST (Representational State Transfer) API which means you don’t need to use any particular programming language to work with the Data API.

In their Product Roadmaps over the past couple of years FileMaker Inc. have indicated that they will be investing in the Data API in future releases of FileMaker platform. Whilst they have not included the XML and PHP APIs in their list of deprecated features their focus will be on the Data API which opens up the FileMaker platform to an almost unlimited number of external apps and services.

We’ve been working with REST APIs for many years – sending SMS messages, integrating with Accounting platforms and eCommerce platforms – so we were pleased to see FileMaker Server (and now FileMaker Cloud) include a REST API as this now opens up many possibilities for integration by external web services and applications, as they don’t need to know “FileMaker” in order to make requests with the Data API. If you’re familiar with making HTTP requests (GET, POST, PUT, DELETE) and working with JSON data formats then you will have no issues working with the FileMaker Data API.

As the FileMaker Data API is a trial feature – the trial expires on September 27, 2018 and the Data API will cease to operate – we normally wouldn’t recommend our clients use this feature in any live systems as it is likely to change and you will have to upgrade to the latest version of the FileMaker platform before the trial expires in order to keep using it (after making any necessary changes to ensure compatibility with the final release of the Data API). There is also the issue of licensing – the Data API is currently free to use in the trial period, but FileMaker plan to have a licensing model in place before the end of the trial. Without knowing what the final licensing model will be makes it hard for clients to commit to using the Data API at this stage.

During a recent project we had encountered a challenging issue that turns out was perfect for the Data API. This project has a FileMaker Server v16 hosted solution with a Custom Web Publishing/PHP interface. All users access the solution using a web browser – for security reasons installing FileMaker Pro was not an option. One requirement of the project was to allow printing of labels with barcodes and picking lists – we solved this by generating PDF files via the new support for generating PDF files on FileMaker Server as well as Excel files (see our blog post from last year on how we did this). We still ended up using a local FileMaker file to import the Excel file containing the data for the shipping labels and to create the barcodes.

This process was working smoothly but was more suitable for doing large bulk shipments and involved a number of steps that you normally wouldn’t have to perform in a purely FileMaker Pro interface:

  • users would first have to find the records via the web interface that they wished to generate labels for
  • they would then click a button to call the server side script to create the PDF or Excel file
  • they would then have to download this Excel or PDF file to their local machine
  • for PDF files they would simply open them and print as needed, and for the Excel files they would open the local FileMaker file and click a button to import the Excel file which then generated the labels and barcodes

As you can see there’s a number of steps involved, whereas with a pure FileMaker Pro interface there would typically be only 2 steps:

  • find the records to print
  • click a button to print the shipping labels or picking lists

The process was also the same regardless of whether you wished to print a single shipping label or 100 labels. We had been discussing ways to streamline and improve this process, knowing that we couldn’t connect to the solution using a live FileMaker Pro connection. We were leaning towards building a integration using either the XML or PHP APIs, or using a product like RESTfm when we realised that the Data API might be another option worth considering, knowing that it was a trial feature and would stop working in September but would be a better long term investment. Now that FileMaker Pro v16 supports cURL and JSON natively it started to make even more sense, even though using FileMaker Pro to integrate with FileMaker Server via the Data API doesn’t make too much sense at first.

We built a basic prototype in a few hours to authenticate and make simple requests to get single and bulk shipping labels and picking lists and were impressed by the speed of the responses, so decided to press ahead and incorporate this functionality into the existing local FileMaker solution that users were already using to import the Excel files and print labels and barcodes. We were further able to link the web interface with this local FileMaker file through the use of FMP URLs by adding buttons on the web interface that called scripts in the local FileMaker file and passed in parameters, so the updated process to print labels and packing lists is now down to 2 steps:

  • find the records to print
  • click a button to print the shipping labels or picking lists

If you’re thinking about using FileMaker Pro as a client for the FileMaker Data API here’s some tips and tricks we learnt along the way:

  • the FileMaker Data API Reference can be found on your FileMaker Server at this URL – https://localhost/fmi/rest/apidoc/ – and contains examples for the requests and responses you can expect when authenticating, working with records and performing finds. The FileMaker 16 Data API Guide is the other main reference to working with the Data API. I recommend having both of these resources open in separate tabs when working with the Data API for the first time
  • we recommend using tools like Postman when working with the Data API to learn how to structure your requests. You can convert your working requests to cURL using the Generate Code Snippets feature which you can then copy and paste into your FileMaker Insert From URL script step and format for use with FileMaker Pro. See our previous article about about using Postman environments with the REST API.
  • the Data API is not currently a complete replacement for the XML or PHP APIs. For example you cannot insert container data or run FileMaker scripts with the Data API, or retrieve metadata about your FileMaker solution such as layout schema. The Data API does not currently return data such as the number of found records like the getFoundSetCount() method does in the PHP API, so you need to use workarounds for these.
  • date formats appear to work in a similar way to the PHP API in that you need to specify them in MM/DD/YYYY format regardless of your date formats on your server or FileMaker solution. Dates are also returned in the same MM/DD/YYYY format (e.g. 11/30/2017 for November 30th, 2017).
  • when performing a find query the fields you are searching for don’t need to be on the layout you specified in your URL, but only fields on the specified layout are returned in the response.
  • when performing a find query that results in no records found the use of the –show-error cURL option determines whether the Insert From URL script step returns an error (1631) or not. This is documented in the Supported cURL options page help page under the Handling errors heading.
  • one issue that caused us grief is an apparent bug with the use of the JSONSetElement function with spaces when used in a particular way the the cURL options. We have now standardised on specifying the JSON for each Data API request using the –data @$data syntax instead of the alternative methods
  • to help with debugging we recommend using the –trace and –dump-header options with every request

The final cURL options generally look like this for a typical request (assuming you have already authenticated and stored the access token somewhere):

"--request POST" & 
" --header " & Quote( "Content-type" & ": " & "application/json" ) & 
" --header " & Quote( "FM-data-token" & ": " & $$restAPIAccessToken ) & 
" --data @$json" & 
" --trace $$cURLTrace" & 
" --dump-header $responseHeaders"

Using FileMaker Pro as a client for the FileMaker Data API has us thinking about new ways we can leverage the Data API from an offline FileMaker Pro or FileMaker Go client, such as syncing records when using FileMaker Go on an iPhone or iPad and removing the overhead of having to have a live connection to your FileMaker Server. The Data API is perfect for uploading or downloading small changes from an offline file and once the shipping version of the Data API is available in the next version of the FileMaker platform (and the licensing model revealed) we expect to see more widespread use of the Data API, particularly for offline files running under FileMaker Go.

 

Databuzz Turns 10 ?

/2 Comments/in , , , /by

January 2018 marks the 10 year anniversary for Databuzz – we first opened our doors for business back in January, 2008. As it turned out 2008 wasn’t a particularly great year to start a new business. According to the Australian Bureau of Statistics, more than 60 percent of small businesses cease operating within the first three years of starting, and with a global financial crisis that same year I probably couldn’t have picked a worse time to start a new business.

Here I am 10 years later and happy to have come out the other side of the GFC, with many customers from our first year still with us today (and some still with us after 20 years).

When I started planning to return to the world of FileMaker Consulting as an independent developer after spending many happy years with Complete Data (which is still going strong today) I knew that I wanted to have a different name to my previous company name – Splash IT Consulting Pty Ltd – as that was no longer an accurate reflection of the business and work that we would perform.

I had first registered the name Splash IT Consulting back on January 5, 1999 when I first went solo and needed a business name to operate under as a sole trader. Having worked at Apple for many years, both in Australia and the UK, I was more of a generalist back then and offered a number of IT services, including:

  • FileMaker Pro development (I had just joined the FileMaker Solutions Alliance as it was known back then)
  • Mac OS X Server installation/configuation (sadly Apple has just announced that it has deprecated most of the macOS Server services)
  • Sherlock Plugins for Mac OS 8.5 – who remembers those!
  • Palm Pilot solutions
  • CD Burning

Here I am almost 20 years later and FileMaker Pro development is now the only service Databuzz offers and most of the other services no longer exist. Back then FileMaker Pro v5 was the latest version – FileMaker, Inc. had only recently come into existence to replace Claris Corporation. I no longer have any of the original website files for Splash IT Consulting but thanks to the magic of the Internet Archive Wayback Machine I managed to find an archive from September, 2000 (missing the main graphic):

Back in 1999 the options for searching for business names and registering them online were very limited and you had to complete the registration in person at the local Consumer Affairs registry office. I wanted to have the word ‘data’ in my name somewhere and had compiled a list of my top 3 names which I believed were available and most importantly also had the matching domain name available. I had also learnt the lesson of never having a hyphen in a domain name if you can avoid it (too many typos from customers when reading your your email address on the phone etc), as well as unfortunate emphasis some customers placed on the “sh-it” part of the name when pronouncing it.

I headed off to the local registry off knowing that if my first option was rejected then I at least had 2 other names to fall back on that I was happy with. For reasons that I can no longer recall all my names were rejected and I was left standing at the counter with a big queue of customers behind me frantically trying other names – and that’s how Databuzz was born. It was the first name I could come up with on the spot that didn’t get rejected!

I decided to keep things simple and use Databuzz as a trading name and keep Splash IT Consulting Pty Ltd as the official company name. The first Databuzz website was also a very simple 5 page static site:

which had an associated blog that ran on a platform called WordPress, which itself was only a few years old at the time. I’ve since replaced the static website with WordPress which now runs the website, blog and online shops for the Databuzz and fmSMS shops. Back in 2008 FileMaker Pro v9 was the current shipping version and FileMaker Go was still three years away from being released – FileMaker Mobile, the previous FileMaker mobile solution for the Palm OS and Pocket PC, had recently been discontinued. The original iPhone had been released in June, 2007 but was not available in Australia – it wasn’t until the iPhone 3G was released in July, 2008 that you could buy an iPhone in Australia.

It’s amazing how much change there has been in the IT industry in the past 10 years, and no doubt there will be even more change in the next 10 years as the pace of change only gets faster. Looking back there have been a number of highlights and milestones that stand out, including:

We’re continuing to update these solutions with new features based on suggestions from our customers and our own product roadmap, as well as work on some exciting new products that we hope to release some time later in 2018. I’d like to thank our customers for their support over the past 10 years (and a special mention to PEDro who we have worked with since 1999) and I look forward to working with you all again in the future and cannot wait to see what the next 10 years brings.

Databuzz 2017 End of Year Xmas Sale – 20% Off All Licenses

/0 Comments/in , , , , , , , , /by

The Databuzz End of Year Xmas Sale is now on – all Product Licenses are now 20% off, including:

The sale runs until the end of the day on the 31st December, 2017 (Australian Eastern Daylight Time). Free trial versions are available for all of our products – please contact us to request a trial version.

All our prices are in Australian dollars – if you’re purchasing from outside of Australia you’ll also get to take advantage of the low Australian dollar (1 Australian Dollar equals approximately 0.75 US Dollars).

Discounts are applied automatically – no coupons required! Click here to start shopping.

We also have a similar sale on fmSMS Licenses (20% off) – click here to visit the fmSMS website for further details.

FileMaker 16 Certified Developer

/0 Comments/in , , , , /by

Databuzz is pleased to announce that Andrew Duncan recently passed the Developer Essentials for FileMaker 16 Certification Exam and is now FileMaker Certified in v8, 9, 10, 11, 12, 13, 14, 15 and 16. FileMaker 16 Certification is the official credential offered by FileMaker, Inc.

FileMaker Certification is your validation that you are hiring an experienced FileMaker professional who has technical knowledge of the complete FileMaker product line and has passed the “Developer Essentials for FileMaker” certification exam. Being a certified developer demonstrates to clients, peers and management that you’ve achieved an essential level of knowledge, experience and skills in developing FileMaker solutions.

Webhooks are coming to Xero

/1 Comment/in , , , , , , /by

Earlier this year Xero announced that Webhooks would be coming to the Xero API, which was exciting news for Xero developers who have been asking for webhooks support for a long time (there is a request from 2010 in the Xero API Customer Feedback site!). Webhooks allow you to subscribe to certain events that happen in Xero: the Contacts endpoint was the first to receive webhooks support and Xero have said they will quickly move to Invoices and other high-use endpoints (hopefully including Payments which our customers would love).

The events that are currently support for the Contacts endpoint are:

  1. Create: A new contact has been created
  2. Update: An existing contact has been updated (including when contacts are archived)

When one of these events are triggered the Xero API sends a HTTPS POST to the webhook’s configured URL – you can then perform the appropriate actions based on the contents of the webhook payload, such as creating a new contact or updating an existing contact in FileMaker.

Xero started a closed beta for the Contacts webhooks which Databuzz was lucky enough to be invited to join and we’ve been working on ways to implement webhooks with our fmAccounting Link (Xero Edition) solution, initially using the FileMaker PHP API. Databuzz has been working with webhooks for many years, for example adding webhooks support to our fmEcommerce Link (WooCommerce Edition) solution to allow new orders to be pushed automatically, and to fmSMS to allow incoming messages to be pushed to FileMaker without requiring users to download these manually. Adding webhooks support to our fmEcommerce Link solution was fairly straightforward – we created a PHP file that uses the FileMaker PHP API and processes the webhook payload to update the existing contact or create a new Contact as required.

We still need to polish these off but we wanted to demonstrate how you could use Contacts webhooks (and Invoices and other endpoints that are supported in the future) with our fmAccounting Link (Xero Edition) solution. If Contacts are created or updated directly in Xero and you need to make the same updates in your FileMaker solution web hooks will save you having to either re-enter these manually or downloads these from Xero via the Xero API.

Check out our video below (or on YouTube) that shows how you can use webhooks to have updates from Xero pushed automatically to fmAccounting Link (Xero Edition) (or your FileMaker solution that you have integrated this into). If you have any questions about webhooks and Xero please post them in the comments below or contact us directly.

Update November 30, 2017: webhooks for Invoices are now available.

fmEcommerce Link (WooCommerce Edition) Update

/0 Comments/in , , , , /by

We’ve just released a minor update to fmEcommerce Link (WooCommerce Edition) – v1.34 – that includes one new feature and one bug fix:

  • we’ve updated the Get Orders download script to check for any previously downloaded Orders with the same WooCommerce ID and skip them if present. This makes it easer to download new orders multiple times a day without having to worry about ending up with duplicate Orders
  • we’ve fixed a bug that was failing to delete the Products Meta Data records when updating a Product from WooCommerce

Once again this is a free update to all our existing customers – you can get the full details of the changes in the release notes for fmEcommerce Link (WooCommerce Edition). Existing customers can download this new version using their existing download links from their original order – please contact us if you have any issues downloading this update.

FileMaker Cloud Joins the FileMaker 16 Platform

/0 Comments/in , , , /by

FileMaker, Inc. today announced availability of the latest version of FileMaker Cloud, its cloud platform for managing and running custom apps that runs on the Amazon Web Services Cloud. With this release (1.16.0.55), FileMaker Cloud joins the FileMaker 16 Platform and it is available in the US, Canada, Europe, Japan and Australia.

This release of FileMaker Cloud requires FileMaker 16.0 clients at a minimum – if you’re using FileMaker v15 clients you will need to upgrade to v16 in order to access hosted solutions on FileMaker Cloud 1.16.0.55. You can read the full release notes here.

The new release includes a number of integration, security and development features:

  • FileMaker Data API trial: Use FileMaker data in other popular apps and services with the REST-based FileMaker Data API in FileMaker Cloud during the trial period.
  • Tableau Web Data Connector for FileMaker: Better visualize FileMaker data with the Tableau Web Data Connector for FileMaker. The connector uses the FileMaker Data API trial to provide integration with Tableau Desktop.
  • FileMaker Admin API trial: Help manage and administer custom apps with the REST-based FileMaker Admin API trial in FileMaker Cloud. Create messages and script schedules, open and close apps, and more.
  • OAuth 2.0 support for accounts: Simplify credential management with OAuth 2.0 using third-party authentication providers. Use existing Amazon, Google or Microsoft Azure account credentials to log in to FileMaker custom apps.
  • PDF support: Generate PDFs of layouts and data from FileMaker WebDirect. Save and print PDFs of invoices, labels, badges and more, just like with FileMaker Pro.

For more information on the differences between FileMaker Cloud and FileMaker Server check out our previous article on FileMaker Cloud as well as FileMaker Inc’s comparison between the two products. This release of FileMaker Cloud brings the ability to schedule scripts via the new FileMaker Admin API trial which was one of the missing gaps when compared to FileMaker Server. FileMaker Cloud does not support Custom Web Publishing with PHP and XML and FileMaker Inc. have stated that this will not be supported by FileMaker Cloud in the future – the FileMaker Data API will be the focus as far as API integrations are concerned so developers should focus their efforts here.

fmAccounting Link (Xero Edition) and Public Application Integrations

/0 Comments/in , , , , , , /by

When we released fmAccounting Link (Xero Edition) back in 2014 we developed it to be used with a Xero API Private Application – this made the most sense at the time and still does for most customers who are only concerned with connecting their custom FileMaker business app with their own Xero organisation. Private Applications are just one of the three different types of API Applications available for the Xero API:

  • Private – connected to a single Xero organisation, doesn’t require user authentication
  • Public -connect to multiple Xero organisations, requires user authentication, 30 minute access limit
  • Partner– connect to multiple Xero organisations, requires user authentication, long term access

One of the main advantages of a Private Application is that your FileMaker users do not need to have a Xero account, so you can allow them to perform specific functions (e.g. uploading Contacts, Invoices, Payments etc from FileMaker to Xero) without having to get them access to Xero itself – the Private Application handles the authentication with Xero for them. This means you don’t have to worry about giving access to confidential business information contained in Xero to all FileMaker users.

There may be times however when you wish to only allow Xero users to upload data from FileMaker to Xero – this would require the use of a Public or Partner application. Databuzz recently took part in Xero’s XD HAX Developer Challenge 2017 and we decided to use this opportunity to explore how a Public Application integration could work. We were focussed on integrating with FileMaker Cloud which runs on AWS and wanted to have a native solution that ran on all the FileMaker clients that FileMaker Cloud supports, including FileMaker Pro for Mac/Windows, FileMaker Go for the iPhone and iPad, and FileMaker WebDirect.

As FileMaker doesn’t have the native functions to perform the OAuth 1.0a authentication that a Private Application type uses we have used a FileMaker plug-in to handle the authentication with the Xero API. FileMaker plug-ins are only supported by the Mac/Windows FileMaker Pro client applications which means they cannot be used by FileMaker Go for iOS (though you can use the iOS SDK to create an iOS application based on FileMaker Go that can include the plug-in).

Using a Public Application type integration means we could remove the dependency on the plug-in as Public applications use the standard 3 legged OAuth process where a user can authorise your application – in Xero’s case this is for 30 minutes, at which point the access token expires.

For our XD HAX Developer Challenge entry we changed the authentication method to use a Public Application integration and took advantage of a number of native FileMaker features, such as using a Card Window to handle the Xero authentication window, and the new cURL and JSON functions to handle the encoding/decoding of data and uploading/download with the Xero API. Here’s a brief video demonstrating how a Public Application integration could work with our fmAccounting Link (Xero Edition) solution:

We would love to hear if this is something that you would be interested in seeing in a future version of fmAccounting Link. We love that we can provide a native solution that runs on all FileMaker clients without a plug-in but are not sure whether customers would prefer the long term access that a private application provides and the benefit of not having to use a Xero login to authenticate.

Xero User Magazine Custom Integration Article

/0 Comments/in , , , , , , , /by

Issue 12 of the Xero User Magazine was released today in time for Xerocon London where Xero announced they have now surpassed a quarter of a million subscribers in the UK, contributing to the more one million subscribers in over 180 countries.

We’re pleased to report that this latest issue of XU Magazine includes an article by Andrew Duncan of Databuzz on ‘The Benefits of a Custom Xero Integration’. You can find the article on page 110 of Issue 12 – you can subscribe for free to the print and digital editions here. The article discusses the benefits of a custom Xero integration, particularly when there is no add-on in the Xero App Marketplace that meets you needs. We also mention examples of integrations we’ve done for customers over the past four years that have helped them eliminate double data entry, for both FileMaker solutions and other applications.

Earlier this year we were approached by an Australian Government agency that was using three different software applications to handle online sales, point of sale merchandise and ticket sales and needed to get the consolidated data into their internal financial accounting software. Xero was used to reconcile all financial data which was uploaded by Vend each night and we developed a custom web application which ran every 24 hours to download the Invoices from Xero and generate a custom XML file which was sent to the internal financial application. This has been running smoothly for many months now and the customer is very happy about the time saved from not having to do any manual re-entry of financial data.

If you would like to discuss a custom integration with Xero please get in touch.

 

fmSMS Updated for Telstra Messaging API v2

/0 Comments/in , , , , , /by

Back in 2015 Telstra, Australia’s biggest mobile phone operator, launched a beta of their SMS Messaging API and fmSMS was one of the first commercial solutions to support the Telstra API. Telstra have recently released v2 of their Messaging API which includes a number of new features including:

  • dedicated number: you can provision an Australian mobile number for your account via the API
  • send SMS and MMS messages
  • receive messages to your dedicated number
  • delivery status: query the delivery status of your messages
  • callbacks: automatic notifications of message delivery status to a callback URL
  • alphanumeric sender ID support
  • concatenation: send messages up to 1900 characters long and Telstra will automatically segment and reassemble them
  • scheduled delivery: delay the delivery of submitted messages

Now that the Messaging API is out of beta there are a number of plans to choose from. You can start with a free trial which includes 1000 free messages and a dedicated mobile number. They also have a PAYG plan with no fixed term contract and messages are 6c per message, as well as an Enterprise plan.

You can download a trial version of fmSMS from the fmSMS website to test the Telstra SMS API using their Free Trial. When you purchase fmSMS you get the PHP files to use for automatic delivery status updates and automatic replies into fmSMS. Please contact us if you have any questions about fmSMS and the Telstra Messaging API.