Archive for category Formats
GeoRSS and Google Maps for Fire Response Notifications
Posted by Jason Birch in Formats, Google, Nanaimo on February 18, 2008
One of the developers in my section (Chris McLuckie) has been working on a replacement for our creaky old fire incident notification system, and launched the new City of Nanaimo Fire Daily Calls page last week. If you’re interested, you can read the official press release (pdf).
Basic improvements include:
- automatic pull from our FDM Computer Aided Dispatch (CAD… a popular acronym) database, removing requirement for dispatchers to manually update this list
- publication of the incidents in multiple formats (GeoRSS, HTML… KML planned for an intermediate update)
- integration with Google Maps
This is what the query interface for incidents looks like:
This is the embedded Google Map (using the GGeoXML class to hit the GeoRSS feed) showing incidents for the selected day:
This is the basic statistics interface allowing users to see incident activity for a date range (which also has a similar Google Map embedded):
And this is what the GeoRSS feed looks like in Google Maps
And now for the technical stuff…
Chris has put together a fairly strong process for extracting and displaying this information publicly, consisting of several components:
- A monthly FME process that reads the current 9-1-1 streets shape file and places it into a normalized database structure (one-to-many between lines and coordinates). This allows easy formatting of coordinates for different output formats (GeoRSS, KML, etc) in native ASP.Net.
- A SQL Server Integration Services job that pulls initial incident data from our CAD database, as well as updated information from the RMS (records management system), generalizes it to block ranges to reduce privacy concerns, and writes it our publicly accessible webserver.
- An ASP.Net web application that transforms an XML serialized data set into GeoRSS (and eventually other formats) using XSLT.
- An ASP.Net web application that provides the rudimentary user interface, including the incident query mechanism, incident statistics, and Google Maps integration.
The current applications are accessed via IFRAMEs because although our standard for web apps on our main site has changed from ASP to ASP.Net, our web site migration is still under way. Once the web site is redeveloped, these will be standard non-encapsulated web apps.
This is definitely just a starting point for us; the framework that has been used for this application was designed so that we can add other data sources that make sense for GeoRSS syndication, such as recent business licenses, building permits, etc. This aligns with our website redevelopment, where we are using RSS/Atom as an alternate access mechanism wherever possible.
As an initial project, there are certainly limitations with this implementation. For instance, we haven’t defined an API for pulling down specific categories or date ranges from the RSS feed. Also, because the GGeoMap class doesn’t expose properties of specific features, we were unable to link the incident rows with the map (pan and pop-up). There are third-party interfaces to Google Maps (GeoXML, EGeoXML) that work around this, and of course the option of just creating the lines ourselves, but we were trying to keep coding and dependencies to a minimum. There is a ticket in the Google Maps API issue tracker for this, so hopefully it will be addressed eventually…
Ideally we’d be using a spatially-enable database (such as PostGIS) as the underlying data store for this application, but we don’t have PostGIS in place on our public webserver yet.
-J
KML Schema Rides Again
Posted by Jason Birch in Formats, Google Earth, Tutorial on September 8, 2007
I just read the news about the new ExtendedData tag in KML 2.2. With one mighty stroke of the pen, Google has saved the Schema tag, and my sanity along with it!
What does it mean? Basically: KML can still act as a self-contained data exchange format, while getting rid of the nasty part of the original <Schema> tag that defined new elements on the fly.
To illustrate the changes, I’ll take you through my previous example of Sammy G Newt. Here he is in glorious colour under the new ExtendedData system:
The first part of this new system is defining the schema; you can see how to do this here:
<Schema name="newt" id="newt_schema">
<SimpleField name="id" type="int">
<displayName><![CDATA[<b>ID</b>:]]></displayName>
</SimpleField>
<SimpleField name="breed" type="string">
<displayName><![CDATA[<b>Breed</b>:]]></displayName>
</SimpleField>
<SimpleField name="slime_factor" type="double">
<displayName><![CDATA[<b>Slime Factor</b>:]]></displayName>
</SimpleField>
<SimpleField name="tail_length" type="int">
<displayName><![CDATA[<b>Tail Length</b>:]]></displayName>
</SimpleField>
<SimpleField name="relative_id" type="string" />
</Schema>
Looks pretty basic, right? Not much has changed in the Schema tag, except that has name and id attributes, and there is now an optional displayName element for each field.
OK, now that you’ve got the schema, you want to create a BalloonStyle that takes advantage of that schema. Here’s mine:
<BalloonStyle>
<bgColor>ffffaa90</bgColor>
<textColor>ffffffff</textColor>
<text><![CDATA[
<h1>$[name]</h1>
<table>
<tr>
<td>$[newt/breed/displayName]</td>
<td>$[newt/breed]</td>
</tr>
<tr>
<td>$[newt/tail_length/displayName]</td>
<td>$[newt/tail_length]</td>
</tr>
<tr>
<td>$[newt/slime_factor/displayName]</td>
<td>$[newt/slime_factor]</td>
</tr>
<tr>
<td colspan="2"><img src="http://www.jasonbirch.com/files/jason_small.jpg?id=$[newt/id]" /></td>
</tr>
</table>
<a href="#$[newt/relative_id];balloonFlyto">Please visit my Sister!</a>
]]></text>
</BalloonStyle>
Can I hear the ah-ha’s?
There are a couple neat things here. First, it’s pretty obvious that if you want the data, you use the format $[nodeName/fieldName], and if you want the display name you use $[nodeName/fieldName/displayName]. Look close at the <a> tag though… I am using an identifier stored in the extended data area to link to a different record in the same KML file, using its ID. This will be great for “previous” and “next” applications among other things. This could just as easily have been used to point to a different Placemark within a remote KML file.
Now that we have a schema and a style, we can create some some content that references these:
<Placemark id="sammyg">
<name>Sammy G</name>
<styleUrl>#newt_style_boy</styleUrl>
<ExtendedData>
<SchemaData schemaUrl="#newt_schema">
<SimpleData name="id">36</SimpleData>
<SimpleData name="breed">Common Orange Slitherer</SimpleData>
<SimpleData name="slime_factor">7.2</SimpleData>
<SimpleData name="tail_length">6</SimpleData>
<SimpleData name="relative_id">phyllisk</SimpleData>
</SchemaData>
</ExtendedData>
<Point>
<coordinates>1.75,1.75,0</coordinates>
</Point>
</Placemark>
So, what’s special about this? Two things. First, I didn’t magically create any new KML tags on the fly. Second, you can see that I am referencing both the style and the schema by URL. This means that you can either store all of your schema, markup, and code in one file, or you can break them out into as many individual files as you would like. You can have a look at my completed example here:
One File:
Three files:
Anyway, this last-minute addition to KML 2.2 has made me pretty happy. If you have any questions or comments about it, be sure to chime in on the official thread.
-J
P.S. Another thing that I noticed in perusing the documentation is the ability to use atom tags to link from a KML entity back to its web representation. This is important because it adds another vector for Google to use when assigning relevance to KML files in spatial search. I’m not an AtomPub expert, but I would imagine that it could also be used to allow a smart client to update features on the fly?
Kilroy Was AtomPub
Posted by Jason Birch in Formats, Loose Integration on September 6, 2007
I was happy to hear Charile’s news of the first successful Geo Web Rest interoperability day. I had seen Christopher’s post about a GeoRSS/AtomPub demo over at MetaCarta Labs earlier, and just had to spend some time checking it out.
Being the cultured guy that I am, I chose to engage is some highly artistic feature editing in this demo:
The OpenLayers demo is highly responsive. The editing tools are intuitive, and even the node insertion feature is well implemented. What is most impressive though is what is going on in the background. Every time you insert/update a feature, an AtomPub operation is triggered. The data that is stored via these operations is (of course) also available as a GeoRSS feed, allowing you to view it in any RSS browser, or even in Google Maps:
If you look close, you will see that Kilroy’s nose’s shape changed between the two screen shots. In the two minutes it took me to get back to take a screenshot of my beautiful artwork in the original interface, some vandal had come by and given him a huge proboscis. :)
I did suffer from some disappointment in my experimentation, though not with this service. I tried to pull this feed through Yahoo Pipes and do something interesting with it, and belatedly remembered how poor their GeoRSS support is. If anyone from Yahoo is listening: please add support for full GeoRSS geometries. Points don’t cut it any more.
Anyway, in my opinion this represents the future of GIS interoperability. Look at how AtomPub ties in with some of the service chaining (featureserver/spatialreference/yahoopipes/fme) stuff I mentioned yesterday. As richer client tools such as Google Earth and other proprietary and open source desktop GIS begin to play ball, I believe that we will see a revolution in social mapping. Anyone not paying attention had better smarten up. It’s no longer about ignoring the elephant; now it’s about getting out from under the steamroller.
-J
JSON and GeoJSON in FME
Posted by Jason Birch in Data Transformation, FME, Formats, Loose Integration, Tutorial on September 5, 2007
Many of you know about JSON, an object serialization scheme that has rapidly gained acceptance in AJAX-style applications. What you may not know is that there is an effort to standardise the representation of JSON-ified spatial features, known as GeoJSON.
FME is usually quick to support new formats (like KML and GeoRSS) but this time Safe has surpassed themselves, getting early JSON and GeoJSON support into their betas before the GeoJSON specification has reached a release version. Tonight I took some time out to play with this new support.
The basis of JSON support in FME is provided by two new read/write formats: JSON and GeoJSON. These new formats are augmented by two new transformers: JSONExploder and JSONExtractor. To get started, I’m going to show you how to extract data from a JSON source I happen to have lying around (it’s publicly available at Yahoo Pipes), which looks like this:
When you first import this data source into FME, it is imported with the top level of attributes broken out. In this case, Pipes returns a top-level object with several attributes (link, description, etc) that you can see in my test workspace below:
Now, this isn’t much use, because my features are hidden inside the “items” attribute. In order to get them out, I first need to explode my single object into multiple features. The new JSONExploder transformer comes to the rescue here:
Now, I have a unique feature for each of my feed items, but I really want some of the nested attributes. In particular, I want the description from the root of the item, and the nested y:location["lat"] and y:location["lon"] attributes. The JSONExtractor makes it easy to pull these out into new attributes:
And once adding a couple more of these, each of my features has some nice attributes attached to it, which I could then turn into points if I wanted:
OK, so that’s kinda cool from a straight ETL standpoint. I can take in JSON, mess with it, and then pump it out into whatever format I want. But the fun stuff is when you start getting into GeoJSON. Fortunately there are a couple early adopters, Christopher Schmidt and Howard Butler, who gave me some feeds to play around with. The first of these comes from Christopher’s super-flexible FeatureServer application (check it out, it’s open source):
The features displayed on this OpenLayers map can be easily downloaded from FeatureServer in GeoJSON format (or KML, or GeoRSS, or whatever). The URL for the GeoJSON representation is:
http://featureserver.org/featureserver.cgi/scribble?format=geojson
Pulling this into FME is as simple as creating a new FME data source, and specifying the URL:
As you can see, you can then treat this data like any other spatial data source:
Now, for a final example… Howard has a GeoJSON resource collection of counties in Iowa, accessible in a pattern something like this:
http://geoservices.hobu.biz/political/json/johnson
Now we could take this feature, in its source projection of UTM Zone 15N NAD83, but Howard’s put together a really nifty (non-commercial use only, unless you want to pay Howard some $$$) JSON-based web processing projection service. Not only that, but he’s also made it smart enough to interpret projections referenced locally, but also from the oh-so-cool (and built as a collaborative effort between Christopher and Howard) SpatialReference.org. So, all you need to do is feed it the URL of your source data, the url of your source CRS (http://spatialreference.org/ref/epsg/26915/) and the url of your destination CRS (http://spatialreference.org/ref/epsg/4326/). Like so:
http://geoservices.hobu.biz/project/?url=%22http://geoservices.hobu.biz/political/json/johnson%22&inref=http://spatialreference.org/ref/epsg/26915/proj4/&outref=http://spatialreference.org/ref/epsg/4326/proj4/
And, as just another link in the dynamic web chain, FME can read this transformed JSON feature:
Now, for desktop FME users, this gives us “Pipes on Steroids”: all the mashup flexibility of Yahoo Pipes, with the huge format support and rich processing model of FME. As cool as this is, I think the real power will be seen whenever Safe integrates this functionality into their Server product. It will allow them to play well on both the “enterprise” traditional GML/WFS/etc level and on the neogeography JSON/GeoRSS/KML mashup level with a single product from a single (or multiple if you want) data source. For organisations that want turn-key interoperability solutions, FME Server is going to rock your world.
-J
P.S. I’m thinking about getting a personalized plate that says GEO JSN :)
I Heart KML Schema!
Posted by Jason Birch in Formats, Google Earth on June 29, 2007
Update: this information is now out of date because of improvements to the KML format in version 2.2. You can read about this in a later post and in Google’s great tutorial on extended data.
Recently, Chris over at FortiusOne has written a couple articles, focusing largely on the Schema tag in KML. One thing in particular from this article got me thinking:
Google Earth will display your custom fields in the markers that get displayed in a Schema-based KML file, but it’s still up to you to make a pretty-looking description if you want, and if you want to include some of the data in your custom fields, you’ll have to duplicate the information which isn’t ideal.
This certainly seems like a bit of a roadblock for the good old Schema tag. It really breaks the whole separation of content from style idea, which surprised me because this separation had been accommodated in other KML tags. So I started hacking… and found that this isn’t really a limitation. One word: BalloonStyle.
Using this tag, you can easily use Schema to transport relatively pure attributes, while pointing to style information located elsewhere in the document or, if you’re in the mood, in a different document entirely.
Here’s some data in a relatively simple schema:
<Newt>
<name>Sammy G</name>
<styleUrl>http://www.jasonbirch.com/files/kml_schema_style.kml#newt_style</styleUrl>
<Point>
<coordinates>1.75,1.75,0</coordinates>
</Point>
<newt_id>36</newt_id>
<newt_breed>Common Orange Slitherer</newt_breed>
<newt_slime_factor>7.2</newt_slime_factor>
<newt_tail_length>6</newt_tail_length>
</Newt>
Which can be combined with the following BalloonStyle:
<Style id="newt_style">
<BalloonStyle>
<text><![CDATA[
<h1>$[name]</h1>
<table>
<tr>
<td>Breed:</td>
<td>$[newt_breed]</td>
</tr>
<tr>
<td>Tail Length:</td>
<td>$[newt_tail_length]</td>
</tr>
<tr>
<td>Slime Factor:</td>
<td>$[newt_slime_factor]</td>
</tr>
<tr>
<td colspan="2"><img src=
"http://www.jasonbirch.com/files/jason_small.jpg?id=$[newt_id]"
/></td>
</tr>
</table>
]]></text>
</BalloonStyle>
</Style>
To give you the fully separated content/style shown below:
Now, I think that’s downright cool. I can have my cake and eat it too. If you want to see this example live in Google Earth ™, just click on the image above. You can also view the source of the content document, or of the style document. I’ll leave it as an exercise for the reader to determine whether this technique can be used with more complex tags than SimpleField, such as SimpleArrayField and ObjField :)
I am aware of one other application (Safe Software’s FME) that is using the Schema tag to pass around attributes, and have really liked the idea ever since I first saw it in Rebecca Moore’s anti-logging KML file. It’s easy to use, doesn’t require external documents, and combined with other KML goodness allows you to pack styling around with your fully attributed data.
There is a nasty rumour floating around that Schema has been deprecated. It isn’t listed as such in the KML docs yet, and I certainly hope that it isn’t too late to ensure continued support for this handy tag from Google.
-J
-
You are currently browsing the archives for the Formats category.
User Map
Recent Comments