Category Archives: Code

MapGuide REST Extension. Feedback wanted!

Posted on by

Traditionally, access to municipal geospatial data has been application-centric, either through a web interface with a steep learning curve, or through data downloads or services for use in desktop GIS.  While these are both much better than not providing access at all, neither strategy is particularly effective in getting the data to the people most interested in it: local residents.  This limitation has become painfully clear in an age when you can find almost any other information by typing a few words into a search engine.  Also, in the absence of ready access to our data, citizens are spending countless hours re-creating it, when they could instead be adding value and stimulating the economy. 

By linking to open source components such as MapGuide, FDO, POCO, CTemplate, and libkml, the MapGuide REST framework is very simple and powerful.  With absolutely no programming, a site administrator can build out customised HTML and KML representations of their data, and enable access to default GeoJSON and FDO-based XML representations.  Pair these representations with static HTML pages that expose "service metadata", and you have data that is easily crawlable by search engine spiders, and easily understood by anyone wanting to access the data for other applications.  Enough talk; let’s go over some examples from the City of Nanaimo’s live web site.

Data URI:  http://maps.nanaimo.ca/data/ 

This is where all of the City’s RESTful GeoData will be exposed.  Currently, there is a single resource type exposed for Property information, but more is on the way and will be interlinked when it makes sense.  The index page for this directory is a static HTML page pointing to various features of the Property resource type.

Property Resources:  http://maps.nanaimo.ca/data/property/ 

This currently displays a page that allows users to search by one of three attributes by HTML forms, and contains links to OpenSearch (with draft Geo) documents that define some of the other search and representation capabilities.

OpenSearch link:

<link 
  rel="search" 
  type="application/opensearchdescription+xml" 
  title="Properties by Street" 
  href="/data/property/search_by_street.xml"
>

As well, there is a link to a Street Index which acts as a form of sitemap, providing web crawlers with an equal-length trip to every property in the City.

Example Property HTML Representation:  http://maps.nanaimo.ca/data/property/123170.html

As much as this looks like a regular web page, it is cram-packed with features that make it distinguishable as a geospatial resource.  Most visibly, it displays an image of the property’s shape in context, and provides a link to the KML representation of that property.  Digging a little bit deeper, the page header contains links to alternate representations:

<link 
  rel="alternate" 
  type="application/vnd.google-earth.kml+xml" 
  title="KML representation of 2143 AARON WAY" 
  href="http://maps.nanaimo.ca/data/property/123170.kml" >
<link 
  rel="alternate" 
  type="application/json" 
  title="GeoJSON representation of 2143 AARON WAY" 
  href="http://maps.nanaimo.ca/data/property/123170.json" >
<link 
  rel="alternate" 
  type="text/xml" 
  title="XML representation of 2143 AARON WAY" 
  href="http://maps.nanaimo.ca/data/property/123170.xml" >

and some simple geotags:

<meta name="ICBM" content="49.18882, -123.9785">
<meta name="geo.region" content="CA-BC">
<meta name="geo.placename" content="Nanaimo">
<meta name="geo.position" content="49.18882;-123.9785">

Finally, within the page content itself, you will find both adr and geo microformats.

<div class="adr">
  <span class="extended-address"></span>
  <span class="street-address">2143 AARON WAY</span>, 
  <span class="locality">NANAIMO</span>,
  <span class="region">BC</span>
  <span class="country-name" style="display:none;">CANADA</span>
</div>

<div class="geo">
  <span class="latitude">49.18882</span>, 
  <span class="longitude">-123.9785</span>
</div>

Example Property KML Representation:  http://maps.nanaimo.ca/data/property/123170.kml

The KML representation uses Atom Author and Link elements to define the publisher of the data, and to store the KML’s URI and that of its alternate HTML representation.  It also uses a combination of ExtendedData and BalloonStyle entities to separate data from presentation, though I have also included a link to the HTML representation in the Description entity for earlier clients.  Finally, it uses external Style and Schema definitions to reduce the size of the file and to maintain consistency across the site.  I’d encourage you to view the source of the KML file to see how these elements work together.

Example JSON Representation:  http://maps.nanaimo.ca/data/property/123170.json

Example XML Representation:  http://maps.nanaimo.ca/data/property/123170.xml

Example PNG Representation:  http://maps.nanaimo.ca/data/property/123170.png

The native GeoJSON and FDO XML representations are provided by the extension.  No customization is possible, which ensures a consistent implementation from site to site.  The PNG representation is also native (not template), but allows you to specify a MapGuide map definition and a layer for selection.

"Endpoints" (because I’m not sure what to call them):

http://maps.nanaimo.ca/data/property/index.kml

This is a static file that allows KML clients to interact with the property information dynamically through View Based Refresh.  Basically, this file tells Google Earth to request the .kmz endpoint below using a BBOX element that will be used to return only features in view, or a warning if the viewport is too large.

http://maps.nanaimo.ca/data/property/.html

http://maps.nanaimo.ca/data/property/.kmz

http://maps.nanaimo.ca/data/property/.json

http://maps.nanaimo.ca/data/property/.xml

These allow users to perform queries on the data.  There are quite a few variations possible, but they basically support standard FDO filters, bbox, count, and start GET parameters.  There are also some funky parameter patterns that allow for easy searching from HTML forms, in the format: filter_[0-9]_(and|or)_(equal|like|likeleft|likeright|lt|gt)_attributename

http://maps.nanaimo.ca/data/property/.schema

This is a special "endpoint" that allows a smart client to determine the data schema for the XML representations of the feature.

Applications

Now, providing read-only representations of these features is pretty cool on its own, and that is all that the City of Nanaimo is prepared to do at this point.  However, the extension can be configured to allow full RESTful data access:  GET, POST, PUT, DELETE.  Combined with a yet-to-be-released FDO REST provider, users of applications that implement FDO (such as AutoCAD Map 3D, Safe Software‘s FME, 1Spatial‘s MapRelate and Radius Studio, and MapGuide) will be able to read, write, update, and delete data from this extension the same way as they would from local data stores.  I was experimenting with this against an internal server using Map 3D, and it was pretty cool to be able to edit data directly from a web site.

Applications that implement GeoJSON, such as Safe Software’s FME can also take advantage of this format to read and (untested) write to the exposed data.

json001

The GeoJSON representation has a considerable upside for light-weight mobile data editing, for instance in an OpenLayers application running inside Google Chrome (or another Gears-enabled browser) and for minimal-packet data access from mobile phone applications.

All of these are really cool applications from a geospatial professional standpoint, but they don’t really address the primary goal of the original requirement:  enabling easier access to spatial information by residents.  That’s OK though, just by making these representations available and well-linked, the search engines will pick them up and do this part of the work for me. 

This site has been active for less than two weeks, and Google is already ranking some of the property reports first in the results for generic searches within British Columbia and specific searches elsewhere.  For instance, when I search from home for "881 Hunter St" (one of the properties Google has indexed) it comes up as number 1.  When Haris searches from Slovenia, that isn’t the case, but if he searches for "881 Hunter St, Nanaimo, BC" then it does.  Most municipalities have high authority for their location name, and this is a great way of taking advantage of that authority to provide a higher level of service to our residents.

Weaknesses

The current implementation has a few issues.  Unfortunately these will require a substantial refactoring to resolve, so we’re looking at another month or so before we have a public-ready build of the code. Some of these problems are as follows:

  • HTTP status codes are often not correct.  For instance, a search that returns a single feature should redirect to that representation rather than displaying the single feature under the search representation.
  • Fine control over methods is required, such as enabling/disabling specific methods, requiring SSL, requiring certain group access, etc.
  • Existing configuration system grew in odd ways as the code was modified, and needs to be de-Frankenstein-ed
  • More control over URI patterns needs to be extended to users, for instance to allow the creation of -edit representations that return pre-populated HTML forms.
  • Need revision numbering or data checksum to reduce chance of overlapping writes.
  • Need parameter/http header overrides for method (to deal with firewalls) and format (to allow selection from HTML forms)
  • Other miscellaneous issues.

Updates

Based on feedback, Haris has enabled JSONP for the .json representations. Just add the argument _callback to the URI, with the name of your desired function as the value. e.g.: http://maps.nanaimo.ca/data/property/.json?filter_1_and_equal_street=ROSSTOWN+ROAD&_callback=rosstown

Call for Feedback

If you have a chance to look at the current implementation and see any areas where it could be improved, we’re really interested in making changes required to meet REST best practices.  We’ve been trying to pay attention to current guidance, but without a real implementation it has been hard to truly understand what is needed.

Thanks!

-J

Listing MapGuide Data File Locations

Posted on by

Over on the MapGuide Users mailing list, Zac said: "it would be nice to add a text file listing the unmanaged resources"

Happy to oblige! :)

OK, not quite, but I’ve created a simple command-line PHP script that talks to the MapGuide repository and outputs a CSV containing the ResourceID and File Path for all data sources that I’ve recognised as file-based. I’ll leave it as an exercise for the reader to filter this to only show unmanaged data sources.

Sample output:

C:\MgTest\listpaths>php cli_listfiles.php admin

“Library://Test/Data/MyImages.FeatureSource”,”C:\TestData\img_ecw”
“Library://Test/Data/MyImages2.FeatureSource”,”C:\TestData\img_ecw”
“Library://Test/Data/MyImages3.FeatureSource”,”C:\TestData\img_sid”
“Library://Test/Data/ABC123.FeatureSource”,”C:\TestData\img_ecw\”
“Library://Test/Data/MyImages4.FeatureSource”,”C:\TestData\img_sid\”
“Library://ImageTest/Data/ImageTest.FeatureSource”,”C:\ImageTest”
“Library://ImageTest/Data/Tiled.FeatureSource”,”C:\ImageTest\”
“Library://SdfTest/Data/bc_border.FeatureSource”,”%MG_DATA_FILE_PATH%”

I think this shows how easy it is to throw around the XML that MapGuide uses in its repositories.

-J

Student Coder? Love Maps? Need Cash? Click Here!

Posted on by

For a limited time only, students who submit a great proposal to OSGeo for Google’s Summer of Code have the chance to:

  • work on awesome open source geospatial software. Location is the new Web 2.0, and open source is where it’s at.
  • earn the adoration and respect of their peers (Ohh, look at Jenny… she’s an UberCoder)
  • interact with and study sought-after geogeeks in their natural habitat
  • make a bit of money ($4500 USD – certain restrictions apply)

This offer won’t last long. If you or someone you love might be interested, please call now:

http://wiki.osgeo.org/index.php/2007_SoC_Merged_Ideas

Seriously, if you’re interested, now is a good time to act. We haven’t received very many applications yet, and they need to be in by the 26th (the deadline was extended). More information on the summer of code can be found here:

http://code.google.com/soc/

And the student application guide is here:

http://groups.google.com/group/google-summer-of-code-announce/web/guide-to-the-gsoc-web-app-for-student-applicants

-J

Using MultiGeometry for Mouseover Effects in Google Earth KML

Posted on by

Well, this has been sitting in my half-finished pile for a long time, so I figured I’d better polish it up and kick it out the door (I’m practicing for my kids). There are probably other sites that have documented this, but here’s my version.

Google’s KML format has some really strong styling elements built into it, similar to HTML/CSS. In a single <Style> tag, you can specify how points, lines, and polygons should display in the Google Earth browser. For instance: 

<Style id="parcelNormal">
  <IconStyle>
    <scale>0.8</scale>
    <Icon>
      <href>http://maps.google.com/mapfiles/kml/pal4/icon56.png</href>
    </Icon>
  </IconStyle>
  <LabelStyle>
    <scale>0</scale>
  </LabelStyle>
  <LineStyle>
    <color>ff4090ff</color>
    <width>10</width>
  </LineStyle>
  <PolyStyle>
    <color>00000000</color>
  </PolyStyle>
</Style>

This allows you to define polygons, points, and lines, all referencing the same style definition. Definitely a space saver over having to define the style against each feature: 

<Placemark>
  <name>238 Franklyn St.</name>
  <styleUrl>#parcelNormal</styleUrl>
  <Point>
    <coordinates>-123.938788334,49.1646409443,0</coordinates>
  </Point>
</Placemark>
<Placemark>
  <name>238 Franklyn St.</name>
  <styleUrl>#parcelNormal</styleUrl>
  <Polygon>
    <outerBoundaryIs>
      <LinearRing>
        <coordinates>
          -123.93904811,49.16434558280001,0
          -123.939274779,49.16457225540001,0
          -123.939105984,49.1646449658,0
          -123.93924107,49.1647800536,0
          -123.938876261,49.16493630590001,0
          -123.938424057,49.1646143977,0
          -123.93904811,49.16434558280001,0
        </coordinates>
      </LinearRing>
    </outerBoundaryIs>
  </Polygon>
</Placemark>

And this is what it looks like:

Basic Styling

There is also a <StyleMap> entity that allows you to specify a style to apply when a feature is active and when a feature is inactive:

<StyleMap id="parcel">
  <Pair>
    <key>normal</key>
    <styleUrl>#parcelNormal</styleUrl>
  </Pair>
  <Pair>
    <key>highlight</key>
    <styleUrl>#parcelHighlight</styleUrl>
  </Pair>
</StyleMap>

Unfortunately, the only feature type that triggers this normal/highlight switch is the point:

Highlight Style on Point

The secret to working around this is to provide a point feature as part of every line or polygon Placemark that you want to be affected by a mouseover. Then, when the user hovers over the point, the line or polygon is affected by the stylemap that applies to that composite feature. The key to doing this is a “wrapper” called MultiGeometry. MultiGeometry allows you to have multiple geometries tied to a single feature. For example:

<Placemark>
  <name>238 Franklyn St.</name>
  <styleUrl>#parcel</styleUrl>
  <MultiGeometry>
    <Point>
      <coordinates>-123.938788334,49.1646409443,0</coordinates>
    </Point>
    <Polygon>
      <outerBoundaryIs>
        <LinearRing>
            <coordinates>
              -123.93904811,49.16434558280001,0
              -123.939274779,49.16457225540001,0
              -123.939105984,49.1646449658,0
              -123.93924107,49.1647800536,0
              -123.938876261,49.16493630590001,0
              -123.938424057,49.1646143977,0
              -123.93904811,49.16434558280001,0
            </coordinates>
        </LinearRing>
      </outerBoundaryIs>
    </Polygon>
  </MultiGeometry>
</Placemark>

As you can see, both the point and the polygon are affected:

Highlight Style on MultiGeometry

This trick is useful for those hand-crafting their KML, but it is also possible to build this kind of construct in some of the KML generating tools out there. For instance, in Safe Software‘s FME, you can generate a point at the centroid of a polygon with a CenterPointReplacer, and then merge the polygons and their centroids using an Aggregator before writing this out into KML format (pdf). You can even generate the StyleMap elements (see format docs for details). In MapGuide Open Source, I use the GetPointInRegion() method to determine the centroid of the polygon, and build my MultiGeometry from this and the polygon coordinates. Most other GIS software has similar functionality.

The final example shown here is available for download in KML, XML and TXT formats.

-J

MapGuide Open Source Package Utilities

Posted on by

MapGuide Open Source uses a binary XML data structure to store its site configuration. This contains things such as data1, layer definitions, map definitions, web layouts, and just about everything else. The existing data management tools don’t provide an easy way to reference external data2, but the advantages far outweigh this disadvantage.

The recent 1.0.0 release of MapGuide comes with a great feature that allows you to import and export portions of this XML data store, including any referenced spatial data, into something called a “package”. A package is just a zipfile with an .mgp extension, which contains XML data and any referenced files.

The only problem with this feature is that it could only be accessed via the web-based site administrator. I wanted to use this ability to automate data transfers from my intranet site to my externally hosted internet site, and really didn’t want to be staying up until midnight each night. Fortunately, MapGuide Open Source is, well, open source. The site administration interface is written in PHP, and uses the same well-documented web API as the web maps.

Armed with a hammer (what else would you use to drive screws?) I attacked the site administration import/export code, and came up with a couple PHP utilities for importing and exporting packages from the command line using the php CLI.

cli_loadpackage.php
Usage: php cli_loadpackage.php AdministratorPassword PackageName.mgp

cli_makepackage.php {watch line wrap}
Usage: php cli_makepackage.php AdministratorPassword PackageName.mgp Library://FolderPath "Package Description"

You can download mgpackageutils.zip from my site, but only with the understanding that I accept no responsibility for your use of these tools. Even if you turn into a chicken. This archive contains the PHP utilities, a readme file, a copy of the LGPL license, and a couple DOS batchfiles for testing:

http://www.jasonbirch.com/files/mgpackageutils.zip

Next job? Create a utility to unzip the .mgp file, rsync the results to the external site, and zip it back up before importing on the other side. I’ll just file that at the bottom of my 10-page TODO list…

Footnotes:

1. Although the data itself could potentially be stored as a byte stream in the XML file, spatial data is typically stored within the Library repository on the server.

2. At some point I hope to be able to provide some tools to define external data sources, based on a conversation on the MapGuide Users List)

.