Being in a precipitous mood, I decided to plunge in and get my clichés wet. The first step was to attempt to use the new provider with Jackie Ng‘s FDO Toolbox, my go-to GUI tool for examining FDO data sets. I followed these steps:

1. Downloaded the 32 bit FDO 3.5 version of the new SpatiaLite provider from the SL-King website.
2. Copied all of the files into my C:\Program Files\FDO Toolbox\FDO directory.
3. Copied the contents of the providers_kingspatialite_entry.xml file into the providers.xml file before the closing FeatureProviderRegistry tag, modifying the FDO version string to 3.5.
4. Discovered that something had a dependency on the MS Visual C++ 7.1 runtime, so I found msvcr71.dll and msvcp71.dll on another PC and copied them into the same directory

With this in place, I was able to quickly connect to the SpatiaLite demo database:

And browse and visualize the data:

Confidence buoyed, I decided to take it a step further and attempt to connect to the same data in MapGuide. The initial steps were the same as for FDO Toolbox (adding the provider to the MapGuide FDO folder).

Once configured, I was able to quickly add a connection to the test database, but I had to manually hack in a coordinate system reference and map extents. This is due to an incompatibility between SpatiaLite and MapGuide’s coordinate system handling, and I’m hoping that the WKT projection support in the next version of SpatiaLite allows Haris to fix this.

With the data connection and its spatial context override in place, creating layer and map definitions was dead easy, and allowed me to quickly get to data visualization.

Next Up? Making the SpatiaLite FDO provider work with GeoREST. Out of time for tonight though…

UPDATE (2010-09-01): I was able to get the SpatiaLite provider working with the current preview version of GeoREST with very little difficulty, and the next official build of GeoREST will probably include this provider. If you’d like to play around with it earlier, you can download a totally unsupported package of GeoREST with Spatialite from my temp files area for a limited time only :)

-J

Notes:

1. FDO is a geospatial object/relational mapping framework, allowing consistent programmatic access to dozens of GIS data formats. It is used by MapGuide Open Source, FDO Toolbox, GeoREST, Autodesk’s geospatial products like Topobase, MapGuide Enterprise and AutoCAD Map 3D, and is also included in applications like Safe Software‘s FME and 1Spatial‘s MapRelate.

2. SpatiaLite takes SQLite‘s concept of database-in-a-file, and turns it into geodatabase-in-a-file, offering a full suite of geospatial query abilities in a lightweight extension to SQLite. FDO already has a robust SQLite provider that uses its own open geometry format, metadata, and spatial functions. In contrast, SL-King‘s new provider is intended for users who prefer to use the SpatiaLite geometry format, and takes advantage of SpatiaLite’s built-in GIS capabilities.

-J

Over the past month, developers at Safe Software and the author of the FDO SQLite provider have put some time into ensuring that the SQLite provider will work properly with FME 2009. Reading worked fine out of the box, but writing required a bit of effort. FME needed datastore creation and schema writing added to their generic FDO writer, and the FDO SQLite provider needed to account for the way that FME writes to multiple schemas.

Here’s how you can take advantage of this provider in FME (and in other FDO 3.3 consumers, such as MapGuide Open Source 2.0):

• Download the unofficial binaries for the SQLite provider from my site
• Open this zipfile and copy the SQLiteProvider.dll file into your FDO directory (default c:\Program Files\FME\plugins\fdo\)
• Make a backup of the providers.xml file in that directory, and then edit the original, adding the contents of the sqlite_provider_entry.xml file in an appropriate location.

Once this installed, writing to SQLite from within FME is dead easy…

1. Add new FDO Destination Dataset:

2. Go to Settings and specify OSGeo.SQLite.3.3 as the provider name:

3: Specify the filename you want to write to:

4. Optionally, set a spatial reference system, and click on OK:

That’s it; now you can start adding tables to your SQLite file as you would any other destination dataset in FME!

As far as I know, Safe will not be distributing the SQLite provider directly with FME 2009 (it’s still in beta) primarily because the provider is not officially being released for FDO 3.3, and partially because the provider is still under heavy development. Fear not, though. I am building this provider against the 3.3 branch as often as necessary, and will post binaries as I do.

The relative ease with which this format was supported by FME can be attributed to Safe’s foresight in exposing FDO directly, rather than just using it behind-the-scenes in their SDF3 writer. They also allow FME to act as an FDO provider, which enables users of products that use FDO for their data layer (such as AutoCAD Map 3D) to access the full range of formats that FME supports.

-J

In this submission, a fix for the patch we add to the SQL engine. One character is the difference between working code and epic fail…

This submission switches the provider to compile against SQLite 3.6.2. Apart from making universe collapse imminent, this comes with a nice performance boost of up to 2x for feature reads.

;)

-J

With release 0.50 of FDO Toolbox, Jackie has done it again by allowing users to quickly take data from both spatial (FDO) data sources, and non-spatial (OLEDB) data sources, join them together, and write out performance-optimized files. This is a huge boon for folks that need to do this and can’t afford best-of-breed proprietary tools like Safe Software’s FME.

Jackie has really impressed me with the rapid development of this tool as well as his focus on providing value in three distinct areas. First, FDO Toolbox has a great GUI for FDO data transfer and administration. Second, the command line capabilities allow you to set up scheduled translations to keep your SDF files in sync with your corporate data stores. Finally, FDO Toolbox has a minimal profile and can easily be used by install scripts that need to load data, register FDO providers, and other tasks during an automated application installation.

What’s next on the horizon? Jackie’s recent post on an FDO plug-in for SharpMap provides a hint… spatial data inspection coming soon to FDO Toolbox! Now if there was only a way of plugging FDO Toolbox into MapGuide Maestro to transform data and either create packages or load data directly into MapGuide. ;)

-J

The closest I’ve got to Informix is seeing it in an ESRI employee’s custom command prompt (“Informix Rules>” or something like that) at the 2001 UC, but I’m sure that this is great news for corporations that have data stored via the Spatial DataBlade. FDO increases transparency for Informix data, allowing simple web-based publishing with MapGuide Open Source, and updates and map production using desktop applications such as Autodesk Map 3D.

-J

With this tool, you can read KML files from MapGuide, and read/write KML using FDO2FDO or Autodesk Map 3D. This provider is still early in development, so get your feedback in now while you can still have an impact on how it works when it’s released.

As an aside, this is the first (soon to be) open source project I know of which uses Google’s libkml.

-J

I first heard about FDO Toolbox when I got a Google Alert about Jackie Ng’s initial blog post back on July 9th. Since that time, Jackie has posted seven new point releases. That’s seven releases in eleven nineteen days, people!.

Obviously, this insane development pace can’t continue forever, but up until now Jackie has been working furiously with a good number of fixes and new features in each release. Many of the enhancements and modifications have come from external suggestions, so if you try it out and find that it’s missing something or think it could be doing something better, don’t hesitate to make a suggestion.

-J

Some interesting entries have been nominated. In particular, FDO and Fusion (both open source applications) are competing alongside other prominent applications in the Innovator Award category.

Full disclosure: My work on earth.nanaimo.ca (built with MapGuide Open Source technology) is nominated for the Public Enterprise category. Please only vote for it if you think it’s the most deserving solution in this category.

-J

Traian Stanev recently proposed the creation of an SQLite provider for FDO. He was quickly arm-wrestled into supporting something close to OGC’s Simple Features for SQL specification, and working with Frank Warmerdam hammered out a GIS spec for SQLite that would work for both OGR and FDO. The beauty is that it’s a single file and can be read by any of the existing SQLite tools.

Traian completed initial development of the SQLite provider a couple weeks ago and Frank expanded OGR’s SQLite support to understand this common specification (this work is in the GDAL/OGR trunk for inclusion in the 1.6 release). These implementations have different strengths. The FDO provider was written to be blazing fast, features an in-memory spatial index, and writes to the FDO binary format. The OGR driver was written for maximum portability and allows writing WKT and WKB. Both implementations will read all three geometry formats and understand the dimension and projection information stored in the OGC-derived metadata tables.

You can download a totally unofficial build of the FDO provider from my website if you want to try it out with MapGuide 2.0 or maybe even Autodesk Map 3D 2009. I have successfully tested it in MapGuide with WKT, WKB, and FGF data. Adding this provider to MapGuide is easy:

• Drop the three files in the zipfile into your Server/bin/fdo directory
• Edit your main providers.xml file to include the SQLite provider using the included XML snippet
• Restart MapGuide

You will need some data. Testing can be done with SQLite files from the OGR sample data directory, but you will eventually want to use your own. It’s fairly simple to convert SDF and SHP. Open up a command window in your Server/bin/fdo directory and type something like:

SQLiteConverter.exe c:\src.sdf c:\dest.db

When creating a new data connection to this file, the provider only takes one configuration parameter: the full path to the file. If you run into any bugs, please post them on the FDO Trac instance.

OGR users that are tracking the trunk build can also try this out. With some amazement, I recently found that the enhancements to this driver had already been documented… obviously OGR places a premium on timely docs. ogr2ogr allows you to do a similar import operation, probably something like (untested):

ogr2ogr -f "SQLite" -dsco FORMAT=WKB dest.db src.shp


You can use additional ogr2ogr arguments to ensure that destination spatial reference and geometry type are written to the metadata tables.

Interestingly enough, a common SQLite GIS specification has been kicked around for quite some time. Last year it was discussed on the OSGeo Discuss mailing list, and more recently further discussion was held on the PostGIS mailing list and a wiki page was set up to collaborate on this idea. Obviously, there is considerable interest within the community. My personal hope is that this specification helps the idea of SQLite as a GIS data store take off.

One area where it could be improved is some kind of integration with Alessandro Furieri’s SpatiaLite extension for SQLite that allows common RDBMS GIS functionality in a native SQLite interface. Unfortunately, neither Frank nor Traian had the cycles to integrate this extension’s data format into the specification or the code at this point. Maybe we’ll get lucky and Alessandro will decide to somehow support this spec, but if not I hope there will be some convergence in the long run.

I know that there was more that I wanted to say, but it’s getting late and I don’t even have time to cut the extra gunk out of this post. Happy SQLiting!

-J