James’ recent post about the GIS Interchange File reminded me that I’ve been meaning to discuss some recent activity on the SQLite front in both FDO and OGR.

Traian Stanev recently proposed the creation of an SQLite provider for FDO. He was quickly arm-wrestled into supporting something close to OGC’s Simple Features for SQL specification, and working with Frank Warmerdam hammered out a GIS spec for SQLite that would work for both OGR and FDO. The beauty is that it’s a single file and can be read by any of the existing SQLite tools.

Traian completed initial development of the SQLite provider a couple weeks ago and Frank expanded OGR’s SQLite support to understand this common specification (this work is in the GDAL/OGR trunk for inclusion in the 1.6 release). These implementations have different strengths. The FDO provider was written to be blazing fast, features an in-memory spatial index, and writes to the FDO binary format. The OGR driver was written for maximum portability and allows writing WKT and WKB. Both implementations will read all three geometry formats and understand the dimension and projection information stored in the OGC-derived metadata tables.

You can download a totally unofficial build of the FDO provider from my website if you want to try it out with MapGuide 2.0 or maybe even Autodesk Map 3D 2009. I have successfully tested it in MapGuide with WKT, WKB, and FGF data. Adding this provider to MapGuide is easy:

  • Drop the three files in the zipfile into your Server/bin/fdo directory
  • Edit your main providers.xml file to include the SQLite provider using the included XML snippet
  • Restart MapGuide

You will need some data. Testing can be done with SQLite files from the OGR sample data directory, but you will eventually want to use your own. It’s fairly simple to convert SDF and SHP. Open up a command window in your Server/bin/fdo directory and type something like:

SQLiteConverter.exe c:\src.sdf c:\dest.db

When creating a new data connection to this file, the provider only takes one configuration parameter: the full path to the file. If you run into any bugs, please post them on the FDO Trac instance.

OGR users that are tracking the trunk build can also try this out. With some amazement, I recently found that the enhancements to this driver had already been documented… obviously OGR places a premium on timely docs. ogr2ogr allows you to do a similar import operation, probably something like (untested):

ogr2ogr -f "SQLite" -dsco FORMAT=WKB dest.db src.shp

You can use additional ogr2ogr arguments to ensure that destination spatial reference and geometry type are written to the metadata tables.

Interestingly enough, a common SQLite GIS specification has been kicked around for quite some time. Last year it was discussed on the OSGeo Discuss mailing list, and more recently further discussion was held on the PostGIS mailing list and a wiki page was set up to collaborate on this idea. Obviously, there is considerable interest within the community. My personal hope is that this specification helps the idea of SQLite as a GIS data store take off.

One area where it could be improved is some kind of integration with Alessandro Furieri’s SpatiaLite extension for SQLite that allows common RDBMS GIS functionality in a native SQLite interface. Unfortunately, neither Frank nor Traian had the cycles to integrate this extension’s data format into the specification or the code at this point. Maybe we’ll get lucky and Alessandro will decide to somehow support this spec, but if not I hope there will be some convergence in the long run.

I know that there was more that I wanted to say, but it’s getting late and I don’t even have time to cut the extra gunk out of this post. Happy SQLiting!

-J

Related posts

8 Responses to “SQLite for FDO with Sugar-Free OGR Synchronicity”

  1. 1Simon O'Keefe on May 8, 2008 at 4:56 am:

    For a moment, I though all my prayers had been answered with SpatiaLite. I have a need to routinely produce a list of properties and their corresponding planning zones, and it needs to run at many different sites. I’ve ruled out PostGIS because I don’t want to set it up at 20 different sites.

    The SpatiaLite documentation appears to indicate that it only supports geographic operators that are based on an object’s minimum bounding rectangle (MBR), and not the actual object. To correctly determine a property’s planning zone, I would need it to directly compare each property’s object against each planning zone’s object, all of which may be quite irregularly shaped.

    Does anyone know if these latest SQLite/FDO/OGR developments would expand the available geographic operators to enable these direct objects comparisons, or if there are any other suggestions for doing this.

    Thanks,

    Simon

  2. 2Jason Birch on May 9, 2008 at 3:14 pm:

    Hi Simon,

    Sorry for the delayed reply. The FDO implementation of SQLite support supports all kinds of spatial intersection queries, though I don’t think that this was initially intended. See this message:

    http://www.nabble.com/RE%3A-FDO-RFC-16—FDO-Provider-for-SQLite-p16578916.html

  3. 3Simon O'Keefe on May 12, 2008 at 6:55 am:

    Thanks Jason.

    Does any of this make it possible to do the kind of geometric operations I’m after in a batch file? I’m just trying to get some quick results without learning a programming language. Am I being a bit optimistic?

    Cheers,

    Simon

  4. 4Jason Birch on May 12, 2008 at 8:10 pm:

    No, I’m afraid that you’re stuck with learning a programming language if you want to do any of these operations via FDO. You could do it in a console app (C#) or Python (haven’t personally tested this), or you could do it from within MapGuide using the PHP or Java APIs but that’s a large install.

    I’m not sure what’s possible with other open source applications; an email to the OSGeo Discuss mailing list might get you some responses.

    http://lists.osgeo.org/mailman/listinfo/discuss

  5. 5Simon O'Keefe on May 14, 2008 at 6:24 am:

    Thanks for the info Jason. I’ll look at other open source alternatives. I might even go for a centrally administered PostGIS setup.

    Cheers,

    Simon

  6. 6zac spitzer on May 15, 2008 at 1:10 am:

    Jason, any idea what i have to do to get ogr2ogr running with sqlite & oracle oci support?

    fwtools doesn’t have oci and dropping the plugin dll & the oracle instance client in C:\Program Files\FWTools2.1.0\bin\gdalplugins does get recognized :(

    I downloaded the gdalwin32-1.5 and it will recognize the oci stuff in the gdalplugins folder but it doesn’t have sqlite support….sigh!

  7. 7Jason Birch on May 17, 2008 at 11:13 pm:

    Sorry Zac, no clue. I’m sure the folks on #gdal could help pretty quick though, or maybe on the FWTools mailing list?

Trackbacks / Pingbacks:

  1. The GIS Interchange File

    Pingback on May 6, 2008 at 9:38 am