I Heart KML Schema!

Update: this information is now out of date because of improvements to the KML format in version 2.2. You can read about this in a later post and in Google’s great tutorial on extended data.

Recently, Chris over at FortiusOne has written a couple articles, focusing largely on the Schema tag in KML. One thing in particular from this article got me thinking:

Google Earth will display your custom fields in the markers that get displayed in a Schema-based KML file, but it’s still up to you to make a pretty-looking description if you want, and if you want to include some of the data in your custom fields, you’ll have to duplicate the information which isn’t ideal.

This certainly seems like a bit of a roadblock for the good old Schema tag. It really breaks the whole separation of content from style idea, which surprised me because this separation had been accommodated in other KML tags. So I started hacking… and found that this isn’t really a limitation. One word: BalloonStyle.

Using this tag, you can easily use Schema to transport relatively pure attributes, while pointing to style information located elsewhere in the document or, if you’re in the mood, in a different document entirely.

Here’s some data in a relatively simple schema:

 <Newt>
   <name>Sammy G</name>
   <styleUrl>http://www.jasonbirch.com/files/kml_schema_style.kml#newt_style</styleUrl>
   <Point>
     <coordinates>1.75,1.75,0</coordinates>
   </Point>
   <newt_id>36</newt_id>
   <newt_breed>Common Orange Slitherer</newt_breed>
   <newt_slime_factor>7.2</newt_slime_factor>
   <newt_tail_length>6</newt_tail_length>
 </Newt>

Which can be combined with the following BalloonStyle:

 <Style id="newt_style">
  <BalloonStyle>
   <text><![CDATA[
    <h1>$[name]</h1>
    <table>
     <tr>
      <td>Breed:</td>
      <td>$[newt_breed]</td>
     </tr>
     <tr>
      <td>Tail Length:</td>
      <td>$[newt_tail_length]</td>
     </tr>
     <tr>
      <td>Slime Factor:</td>
      <td>$[newt_slime_factor]</td>
     </tr>
     <tr>
      <td colspan="2"><img src=
       "http://www.jasonbirch.com/files/jason_small.jpg?id=$[newt_id]"
       /></td>
     </tr>
    </table>
   ]]></text>
  </BalloonStyle>
 </Style>

To give you the fully separated content/style shown below:

Sammy G Meets World

Now, I think that’s downright cool. I can have my cake and eat it too. If you want to see this example live in Google Earth ™, just click on the image above. You can also view the source of the content document, or of the style document. I’ll leave it as an exercise for the reader to determine whether this technique can be used with more complex tags than SimpleField, such as SimpleArrayField and ObjField :)

I am aware of one other application (Safe Software’s FME) that is using the Schema tag to pass around attributes, and have really liked the idea ever since I first saw it in Rebecca Moore’s anti-logging KML file. It’s easy to use, doesn’t require external documents, and combined with other KML goodness allows you to pack styling around with your fully attributed data.

There is a nasty rumour floating around that Schema has been deprecated. It isn’t listed as such in the KML docs yet, and I certainly hope that it isn’t too late to ensure continued support for this handy tag from Google.

-J

FDO-a-GoGo

Time for something slightly different…

Feature Data Objects (FDO) is an open source C++ based spatial data abstraction library, allowing potential access to a large amount of spatial functionality including advanced items like rasters, complex geometry and long transactions. This plugs into a lot of different formats using a common API, and managed (.Net) and unmanaged interfaces are provided.

FDO was originally authored by Autodesk for inclusion in the Map 3D product, and then granted to the Open Source Geospatial Foundation (OSGeo) as a sub-project of MapGuide Open Source. For more history of FDO, please check out the osgeo.org FDO History page. While three providers (Oracle, MSSQL, and Raster) contained proprietary Autodesk technology and were not open sourced, many providers were. The initial providers included in the fdo.osgeo.org repositories included providers for SDF, ESRI SHP, MySQL, ESRI ArcSDE, ODBC, OGC WFS, and OGC WMS.

One of the main reasons that I’m writing about this now is that this fledgling open source project really seems to be picking up some steam. Since the initial release, Autodesk (Traian) has added a basic FDO provider for OGR, and Frank Warmerdam seems to be getting close to completion of the GDAL-based open source raster provider. Refractions Research (Mateusz Loskot) has been working under contract to my employer, the City of Nanaimo, to create a high-function PostGIS provider. And, just today, Haris Kurtagic announced on the MapGuide Users mailing list that the SL-KING Oracle provider would be moving into the OSGeo FDO project as an open source provider. This is great news for users that want to use MapGuide with Oracle XE, since this platform is not supported by the Autodesk Oracle provider. Now all we need is for someone to write an FDO Provider for the open source MS SQL Spatial package, and all of the common databases will be covered.

On a somewhat different front, FDO is already being supported in a limited extent by Safe Software it its FME product, as this is what is allowing them to write to the new SDF format. A little bird told me that this support would be extended by providing an FDO provider for FME (think of it as an open API for reading over 100 formats) and, in the longer term, a generic FDO reader/writer for FME, allowing for you to write your data out in an optimal format for the FDO providers.

My fondest wish is that other GIS application vendors take advantage of this data abstraction library and integrate it into their products. Imagine having full access to all of these formats for free. Access to SDF alone is a great benefit as it allows you to have a highly capable local geodata store, which supports rich geometry, multiple tables, and spatial indexing all in a single file. Did I mention that it’s free? :)

There is a slight downside. While most of the providers, including the OGR and King.Oracle providers, are available for the current release of MapGuide, the PostGIS and GDAL providers will only be avaliable once MapGuide is built to work with FDO version 3.2. If you’re ambitious, you can always download and compile the latest versions of FDO and MapGuide yourself. (it’s not that bad…), otherwise you’ll have to wait a bit.

-J

Closed, and open, and better, oh my!

I have a love/hate relationship with Autodesk.

One on hand, I love their MapGuide product, especially the new MapGuide Open Source / MapGuide Enterprise 2007 applications.

On the other hand, I hate the DWG file format. With each release, I have to worry about whether I’ll be able make the engineering department’s work available to the rest of the organisation, and that is exactly what Adena’s recent pointer to Evan Yares’ comments about the new DWG format had me doing.

After sitting on that for a few days, and starting to read some of the news that is leaking out about Autocad 2007 (the blogs listed by Shaan Hurley mostly seem focused on the new 3D environment) I was starting to get pretty disappointed. After all, there are some pretty compelling features in there and I’m going to get some major pressure to move our organisation to 2007.

Something seemed odd to me though. There was no real information on Autodesk Map 3D 2007 in those blogs. I had received a short blurb from my reseller about the new releases, but had not really read through it. So, I went back and did. Then I went on a hunt for more information on Map 3D 2007 and found some on the Taylor Technologies site. Wow!

It seems Autodesk has really focussed on GIS rather than CAD with this release. A long time limitation of Map was its reliance on the DWG format. Link Templates and Object Data were both pretty unreliable and inflexible ways of storing GIS data. Oracle Spatial was a pretty good option if you could afford it, but the recently added ArcSDE support seemed difficult to implement. Autodesk Map 2007 appears to blow most of these constraints away. Here’s why:

New read/write capabilities without conversion:

  • MySQL Relational Database
  • Microsoft SQL Server Relational Database (not sure how they’re storing spatial info in MSSQL)
  • ESRI SHP Files
  • Spatial Data File (SDF)

New direct access capabilities:

  • JPEG2000
  • Web Map Services (WMS)
  • Web Feature Service (WFS)

Extra goodies that got thrown in include enhanced support for DWF, what look to be great new cartographic capabilities, and the ability to publish to MapGuide Enterprise 2007.

With that list, who could ask for anything more? Well… me :) I believe that all of these new formats are provided by Autodesk’s new open source FDO spatial data abstraction layer. Now, unless I’m seeing things, Autodesk has indicated that they will be releasing a PostGIS FDO provider with MapGuide Open Source. It’s possible that this provider did not make it into the 2007 release. Maybe Service Pack 1? :) Another feature that I would really like to see, and it may be in there already because I do not have access to the beta, is the ability to open MapGuide Enterprise 2007 DWF eMaps as background layers. This would allow for consistant corporate styling without the limitations of WMS, at least as a stopgap until MapGuide offers SLD support.

To bring us back to the start of this post, the one item on this list that I think may get overlooked is support for the Spatial Data File (SDF) format. SDF is a single-user database format, similar to ESRI’s personal geodatabase. It is built on top of SQLite, is fully open, and is already supported by MapGuide Open Source and (yay!) Safe Software‘s current FME betas. Now we can choose to use enterprise spatial data stores (Oracle, ArcSDE, PostGIS, MSSQL) for corporate data sets that warrant it, and use the SDF format for smaller projects that do not justify that level of effort. This removes the requirement for Autodesk GIS users to utilize DWG as anything but an an import/export format and working environment.

In municipal government, CAD tools are often the preferred way of maintaining spatial information, but in many cases this has left us with large interoperability barriers. Autodesk has made a huge leap in allowing us to choose the right geospatial tool for the job.