FileMaker Pro 9, ESS and Those Strange Box Characters

Anyone who has had to deal with exchanging data between a FileMaker Pro database and data stored in other database systems (MySQL, SQL Server, Oracle etc) will be familar with handling characters that FileMaker doesn’t appear to handle all that well. This is especially true when retrieving data via the web, e.g. an HTTP POST response back into a FileMaker field. Many of you for example are familar with the strange square box character that appears. If you’re working with ESS tables you more than likely going to have to come up with a way of dealing with these characters, which are generally ASCII characters that you normally don’t see in FileMaker, such as Line Feed, Vertical Tab and Carriage Return.

Here’s an example that we came across. If you fill out our Contact form on our website you might enter something like this:

web form

Note the returns in the Comments field. This form data is being stored in a MySQL database which we access using the great ESS feature of FileMaker Pro 9 (no more laborious copy/paste of data from an email or trying to parse the email form into the different FileMaker Pro fields!). When you view the ESS table and this field it now (at least on Windows with FileMaker Pro Advanced 9.0v3) looks like this:

ess view

Notice the square box characters that appear – it’s obviously not ideal to display these to your FileMaker users. There are a number of ways to deal with this but the way we’ve approached it in this case is to create a supplemental field in our ESS shadow table (remember you’re limited to calculation and summary fields when adding supplemental fields to an ESS table) and take advantage of a great new function that’s part of the Troi File v4.0 plugin: TrFile_AsciiValueToText function. As you’ll see shortly this makes easy work of dealing with special or invisible ASCII characters. We simply created a new calculation field with a text result using this formula:

Substitute ( comments; TrFile_AsciiValueToText( “-Unused” ; 10 ) ; “” )

Now if you look at the original MySQL field on the left and the new FileMaker supplemental calculation field on the right you’ll see the difference:

ess supplemental

ess supplementa

In the past we have tried to use the native Substitute function in FileMaker to handle these characters with mixed results. We tried copying the square box character to the clipboard and pasting into the calculation dialog:

paste into calculation formula

So far so good. Now click OK to save the formula:

click ok

 You can’t see the box anymore but it looks like something’s still there – let’s go back in to the calculation and check it:

back into formula

Hmmm that’s interesting the square box character has gone. Once you get to this point and do some replacements using the Substitute function you don’t get very far. We also noticed some other strange visual behavour. If you have a field with this same character in it and view it at 100% zoom level in Browse mode it looks like this:

browse mode 100%

Change the zoom level to less than 100% or more than 100% and you’ll see something like this:

Browse Mode 200%

Our suggestion is to find a tool that can help you identify which ASCII character you are dealing with. If you’re using the Troi File plugin make sure you upgrade to v4.0 or later so you can use the TrFile_AsciiValueToText function and simply call that within a Substitute function referencing the ASCII character you wish to replace. It’s much easier and cleaner than trying to copy/paste characters which don’t survive a copy/paste, at least on Windows.

There’s more details in this article Entering Clean Text (or: avoiding unwanted characters) and if you’re a FileMaker TechNet member search the TechTalk archives for other approaches and comments. You can get more background info on ASCII characters at this WIKI entry.

Latest FileMaker Australia Promotion

FileMaker Australia have a new promotion available – purchase 10 seats of FileMaker Pro 9 at Volume License Prices (VLA) and get a free copy of FileMaker Server 9 (RRP AUD $1645.00). Education customers get an even bigger discount. This also includes 12 months of maintenance which means you will automatically receive any upgrades for FileMaker Pro 9 and FileMaker Server 9 that you would normally have to pay for. You can get further details here – this promotion runs until 26 September, 2008 which is longer than the usual quarterly promotion FileMaker Australia usually run. Similar promotions are being run in other regions so check with your local FileMaker office.

Sample Data for FileMaker Developers

If you’re ever looking for some sample data to populate a typical FileMaker Pro contacts database here are some sources for free sample data:

  • Fake Name Generator – creates single and bulk names, addresses, phone numbers etc. Has different format options for different countries
  • – has free downloadable .csv files for 500, 5000, 50000 and 350000 records with a US centric focus

When I get a chance I’ll put together a Australian specific one and post it online here but for now you can download either of these and modify to suit your requirements.

FileMaker Developer Conference 2008

FileMaker Inc recently announced the details for this year’s FileMaker Developer Conference – DevCon 2008. This year DevCon moves back to Phoenix, Arizona after a few years in Orlando, Florida. For Australian attendees this is good news – connecting to Phoenix is much quicker than another long flight to Orlando from the US West Coast after 14 hours in a plane just getting to the US (even worse if you had to fly economy). It’s a shame the new V Australia flights aren’t taking off until December as finally we see some real competition on the route which Qantas and United have been sharing for years. I like the new timetable for the V Australia flights as well – Qantas seem to enjoy arriving in Los Angeles at 6am, having left Australia at 6am and if you’re like me you haven’t slept much on the way.

The conference is on  from July 13 to 17 at the JW Marriott Desert Ridge Resort & Spa in Phoenix, Arizona and if you register by May 16 you will save US $200.00 (registration is US $1,195.00 and increases to US $1,395 after May 16). There are 3 tracks this year: Fundamentals, Advanced and Integration. You can get all the details here. FMPug also have a Scheduler formatted for handheld/mobile devices at

Databuzz will once again be attending DevCon 2008 – this is our seventh straight year attending the FileMaker DevCon from memory – and we look forward to learning some new tricks and techniques which we can bring back and implement in our customer’s solutions.

 Andrew Duncan

Beta of FTPit Pro 2.0 released

FTPit Pro 2.0.0b1 has recently been released by the fine folks at CNS Plug-ins. I’ve been eagerly waiting for the “Pro” version of FTPit for quite a while now. I am a big fan of the SMTPit Pro and Pop3it Pro plugins for sending and receiving email from a FileMaker database respectively. You can download the beta of the FTPit Pro, which if you haven’t used it before allows you to upload/download files from/to FTP servers from your FileMaker databsae, from:

FileMaker Pro 9 Bible Book Released

A new FileMaker related book has recently been released – FileMaker Pro 9 Bible, co-authored by Australia’s very own Ray Cologon and Dennis R. Cohen. Weighing in at a massize 768 pages this will be an invaluable addition for most FileMaker developers, both beginner and experienced. I’ve ordered my copy from Amazon already and are looking forward to it’s imminent arival. Members of the FileMaker TechNet and FileMaker Business Alliance will be familiar with the quality of Ray’s contributions to the TechTalk/BizTalk mailing lists. A PDF of the table of contents is available at:

(which itself is 21 pages!) which shows the depth of content that this new book offers. I suspect this book will become one of the “must haves” for FileMaker developers worldwide (at least English speaking ones at this stage). Several example extracts are available on the publisher’s website at:,descCd-description.html

FileMaker Server and Auto Updating Plugins

As a long time user of as many plugins as I can get my hands on one of my favourite features of FileMaker Server (starting with v5.5 if my memory serves me correctly) is the ability to push plugins to FileMaker Pro and FileMaker Pro Advanced clients automatically from the server using some simple FileMaker Pro scripts. Plugin maintenance was a painful process before this, having to manually install and update new plugin versions on each client workstation.The Auto Update feature is great for:

  1. ensuring all users of your database solution have the plugin installed
  2. distributing plugin updates by making some small changes to the server and having these downloaded automatically
  3. updating plugins without having to restart FileMaker Pro/Advanced!

The documentation for setting up the Auto Update feature is covered in depth in the FileMaker Server 9 Guide to Updating Plug-ins available from the FileMaker website and the FileMaker Server CD. Essentially you need to setup some scripts that you call (typically as part of the on open or startup script) which will download the plugin if it is not installed or update the plugin if an older version of the plugin is installed (if you were to manually update an older version of the plugin you would have to quit FileMaker Pro/Advanced and re-open it if it was already open).

The Auto Update feature has worked in much the same way from Server 5.5 through Server 7, 8 and 9. However there is one small change in FileMaker Pro 9 that has caused some extra plugin maintenance work. FileMaker Pro 9 now has the ability to store plugins in 2 locations. In addition to storing plugins in the Extensions directory within the FileMaker Pro folder, for example on Windows for FileMaker Pro Advanced v9 this would be:

C:\Program Files\FileMaker\FileMaker Pro 9 Advanced\Extensions

it can also store plugins in the user’s application data directory, which is typically a hidden folder that is not subject to the same access privileges issues as the application directory might be which can cause plugins not be downloaded (the user might not have read/write access to the application directory, for example). The application data directory has different locations for all supported operating systems for FileMaker Pro 9:

  • Windows XP: C:\Document Settings\User Name\Local Settings\ApplicationData\FileMaker\Extensions
  • Windows Vista: C:\Users\User Name\AppData\Local\FileMaker\Extensions
  • Mac OS X: Macintosh HD/Users/User Name/Library/Application Support

Now that you can install plugins in more than one location you will need to learn the new rules about managing plugins in these different locations. For example what happens if you have the same plugin installed in both locations? Which one do you update?

As I encountered these problems within days of FileMaker Pro 9 being released I’ve developed a good understanding of how to manage plugins in both locations and have developed these rules:

  • Any plugin distributed by FileMaker Server (not just Server 9) to a FileMaker Pro 9 client will be downloaded to the user’s application folder. I regularly use FileMaker Pro 9 with FileMaker Server 8 and it is a function of FileMaker Pro 9 not Server 9 that causes the plugins to be downloaded to the user’s application data folder.
  • If you have the same plugin installed in both locations the one installed in the user’s application data folder takes precedence, even if it is an older version. This can be hard to troubleshoot as the user’s application data folder is typically hidden in my experience so you need to know the location and how to turn on hidden folders so you can find it in the first place.
  • If you are attempting to update a plugin that is installed in both locations it will be downloaded to the user’s application data folder and any older versions will be moved to the Saved folder like normal. However if an older version of the plugin is currently installed in the FileMaker extensions folder you will have to quit/relaunch FileMaker Pro for the newer downloaded version of the plugin to be loaded by FileMaker Pro 9. This is one of the downsides to this new feature, mostly for developers that work with plugins and are used to manually putting them into the FileMaker Pro extensions folder. For most workstations all plugins will be downloaded and updated without incident in the user’s application data folder.

The information about the change to downloading to the user’s application data folder is tucked away in the FileMaker Server 9 readme – in my opinion it would be more appropriate to include it in the FileMaker Pro 9 readme. There’s some useful information about Auto Update and FileMaker Server on the Troi website and the 24u website as well.

P.S: I needed to locate the user’s application data folder quickly on different workstations so I wrote this script (which uses the Troi File plugin naturally!) to get the location to the user’s application data folder (currently only for Windows XP and Vista, haven’t done a Mac version yet sorry). Haven’t worked out the best way to format scripts in a WordPress blog post so sorry for the formatting – if you know how to do this please let me know!


#This script gets the path to the local application data extensions folder that FileMaker Pro 9 and FileMaker Server 9 use for downloading

plugins automatically


Enter Browse Mode


#First check to see if single user or multi user as paths will be different


If [ Get( MultiUserState) <> 2 ]


#They are single user so just open the local application Extensions folder for the currently running version of FileMaker Pro


Set Variable [ $VarTroiPath; Value:TrFile_Launch( “” ; TrFile_GetPathTo( “-CurrentAppFolder” ) & “Extensions\\” ; ) ]


If [ $VarTroiPath 0 ]


Show Custom Dialog [ Title: “Error – Opening Folder”; Message: “There was an error opening the plugins folder!”; Buttons: OK ]


Exit Script [ ]


End If

Exit Script [ ]


End If


#First need to determine if current user is on Windows XP or Vista as the paths are different


If [ Abs ( GetAsNumber ( Get ( SystemVersion ) = 5.1) ) ]


#It’s Windows XP SP 2


Set Variable [ $VarPath1; Value:Right ( ( Get(DesktopPath) ) ; (Length ( Get(DesktopPath) ) – 27)) ]


Set Variable [ $VarPath2; Value:Left ($VarPath1 ; (Length ($VarPath1 ) -9 )) ]


#First check to see if the folder actually exists as it might not yet exist if no plugins have been downloaded to this location yet

Set Variable [ $VarFolderExists; Value:TrFile_Exists( “-Unused” ; “C:\Documents and Settings\\” & $VarPath2 & “\Local

Settings\Application Data\FileMaker\Extensions\\” ) ]


#Will return 0 if it doesn’t exist otherwise will return 1


If [ $VarFolderExists = 0 ]


#Doesn’t exist so just open local FileMaker Pro extensions folder instead


Set Variable [ $VarTroiPath; Value:TrFile_Launch( “” ; TrFile_GetPathTo( “-CurrentAppFolder” ) & “Extensions\\” ; ) ]




#Does exist so open that one

Set Variable [ $VarTroiPath; Value:TrFile_Launch( “” ; “C:\Documents and Settings\\” & $VarPath2 & “\Local Settings\Application

Data\FileMaker\Extensions\\” ; ) ]


End If

If [ $VarTroiPath 0 ]


Show Custom Dialog [ Title: “Error – Opening Folder”; Message: “There was an error opening the plugins folder!”; Buttons: OK ]


Exit Script [ ]


End If

Else If [ Abs ( GetAsNumber ( Get ( SystemVersion ) = 6) ) ]


#It’s Windows Vista


Set Variable [ $VarPath1; Value:Right ( ( Get(DesktopPath) ) ; (Length ( Get(DesktopPath) ) – 10)) ]


Set Variable [ $VarPath2; Value:Left ($VarPath1 ; (Length ($VarPath1 ) -9 )) ]


#First check to see if the folder actually exists as it might not yet exist if no plugins have been downloaded to this location yet

Set Variable [ $VarFolderExists; Value:TrFile_Exists( “-Unused” ; “C:\Users\\” & $VarPath2 &

“\AppData\Local\FileMaker\Extensions\\” ) ]


#Will return 0 if it doesn’t exist otherwise will return 1


If [ $VarFolderExists = 0 ]


#Doesn’t exist so just open local FileMaker Pro extensions folder instead


Set Variable [ $VarTroiPath; Value:TrFile_Launch( “” ; TrFile_GetPathTo( “-CurrentAppFolder” ) & “Extensions\\” ; ) ]




#Does exist so open that one


Set Variable [ $VarTroiPath; Value:TrFile_Launch( “” ; “C:\Users\\” & $VarPath2 & “\AppData\Local\FileMaker\Extensions\\” ; ) ]


End If

If [ $VarTroiPath 0 ]


Show Custom Dialog [ Title: “Error – Opening Folder”; Message: “There was an error opening the plugins folder!”; Buttons: OK ]


Exit Script [ ]


End If

End If

Commit Records/Requests


FileMaker Server 9 – server-side ScriptMaker scripts

One of the great new features of FileMaker Server 9 is the ability to schedule regular ScriptMaker scripts (i.e those that you create with FileMaker Pro/Pro Advanced). FileMaker Server has had the ability to perform system-level scripts since at least v5.5 as far as I can recall (e.g. Windows batch scripts for Windows and AppleScripts for Mac OS) but v9 allows you to schedule ScriptMaker scripts as well. This allows administrators to schedule batch processes to take place on the server or after business hours, for example for batch processing records for end of day processing.

The only limitation is that, like with Instant Web Publishing, the scripts must be “web compatible”:

The screenshot above shows the standard ScriptMaker script dialog box. In the bottom left hand corner there is a checkbox to indicate web compatability. With this ticked all web compatible scripts are left in black and non web compatible scripts are greyed out. So remember when creating scripts that you intend to schedule on Server 9 you must include only the web compatible script steps as indicated. I haven’t done any tests to determine what happens with non web compatible script steps to see if they are simply ignored or the script stops executing at that point.

Whilst working on the finishing touches to our new CRM product Data Manager I came across the idea of utilising the ability to schedule ScriptMaker steps in FileMaker Server 9 which will be the standard base version for multi user clients. We have routines that perform batch processes and use the Troi Dialog plugin to display a progress bar whilst the script is running to give feedback to the user. Unfortunately this means the user cannot use FileMaker whilst the script is running, for example when sending a bulk email to 500 customers. If it isn’t urgent that the email is sent immediately we will have the ability to set the batch operation to be scheduled so that the sending is actually performed by FileMaker Server, which will not interupt the user and can be performed out of business hours (less disruption to Internet connection) and makes better use of the Server’s resources. You can even use the new email notification feature to get feedback via email about the scripts operation, however you will need to build in some interface to allow clients to review batch operations to check for sending errors (e.g. Internet was down). Most plugins use the Set Field script step which is web compatble so it’s easy to use plugins that don’t rely on keyboard/mouse input and dialog boxes.

You can get more information on Server 9 script scheduling at:

FileMaker Web Viewer and Data URL’s

Whilst working on our new CRM (Customer Relationship Management) solution – Data Manager – this week we made an interesting discovery with the way the Web Viewer works. We wanted to have a way to preview an html email that was being composed in Data Manager so the user could see how it will look to the recipient, just as it does in Outlook etc when composing a new email. We also didn’t want to have to keep exporting files and then opening the exported file as that creates unnecessary clutter for the developer and the customer (though with FM Pro 9 you can use the new Get(TemporaryPath) function to dump files into the user’s temp folder. We investigated using a Data URL with a Web Viewer as this would provide a “live” view of the content. This works well for plain text type emails with no inline images as you can simply reference the fields using something like:

“data:text/html,” &

Substitute ( DM_Email::email_text ; “¶”; “<br>”)

This replaces any FileMaker carriage returns with the HTML equivalent.

However if you’re using email stationery like you can in Microsoft Outlook you might be referencing one or more inline images (gif, jpeg, etc) which are stored in a folder on the hard drive, server etc. This makes things a bit more complicated. Fortunately we have a table in Data Manager that stores all the inline images for each user which stores the path, file name and a copy of the image in a container field as well. Whilst scratching our heads trying to work out how we can do a substitute calculation to change any image references which might be:

<img src=”banner.gif” mce_src=”banner.gif”

<img border=”0″ src=”clickhere_btn.jpg” mce_src=”clickhere_btn.jpg”

we made an accidental discovery. If you export the contents of the container fields to the user’s temp folder using the Get(TemporaryPath) function the Web Viewer will somehow “discover” the images and show them when displaying the preview of the html email. This made things a lot easier for us as we don’t have to deal with complex substitutions in the html source (we’ve only tested this on Windows XP and have since heard this doesn’t work for Mac users but we haven’t had a chance to test on a Mac yet). We’re also big fans on the SMTPit Pro plugin which has a nifty function for converting rich text into HTML on the fly which makes this so much easier.

You can read more about the Data URL scheme at:

There’s also some good info on FileMaker and Data URL’s in the Web Viewer with some example files at:

New Website Launched

Our new website was launched this week. We’re still working on the content side of things and some additional functionality that will involve PHP and MySQL (we will be able to access these MySQL databases using the External SQL datasources feature of FileMaker Pro 9). Stayed tuned!