Status Report for 12/05/2007

Why Client-based Dynamic Caching for Map Rendering
- Comparison with Google Map’s tiling approach

With the client-based caching, besides removing the repeated processing jobs, we utilize the locality principles and develop efficient load balancing algorithm for sharing unpredicted workload among the worker nodes. implementation details are given for geo-data rendering, but principles are addressed for the general science domains.

I compare my approach with Google Map's tiling approach and present why and when we need such an algorithm.

Here is the 2-page document:
http://complexity.ucs.indiana.edu/~asayar/thesis/client-based-caching1.pdf

Status Report for 11/25/2007

I have written my thesis's *draft* introduction chapter [see the link]

http://complexity.ucs.indiana.edu/~asayar/thesis/intro5.pdf

Title: Federated Service-oriented Geographic Information Systems

Outline:
1. Introduction
......1.1. Motivation
......1.2. Why federated Service-oriented Design
............1.2.1. Architectural Design Features
............1.2.2.High-performance Design Features
......1.3. Summary of Contributions
......1.4. Research Issues
......1.5. Organization of Dissertation

Status Report for 11/14/2007

I have been writing a manuscript
Here is the abstract:

Geospatial information is critical to the effective and collaborative decision making in earth-related disaster planning, crisis management and early-warning systems. The decision making in GIS increasingly relies on analyses of spatial data in map-based formats. Maps are complex structures composed of layers created from distributed heterogeneous data and computation resources belonging to the separate virtual organizations from various expert skill levels.

We propose infrastructure for understanding and managing the production of knowledge from distributed observation, simulation and analysis through integrated data-views in the form of multi-layered map images. Infrastructure is based on common data model, standard GIS Web-Service components and a federator. Federator federates GIS services and enables unified data access/query and display over integrated data-views. Integrated data-views are defined in federator’s capability metadata as composition of layers provided by standard GIS Web-Services. Our grid approach is based on the “WS-I+”; we start from core Web Service standards and build outward on other applicable W3C and OASIS standards.

Status Report for 09/05/2007

I have been writing my thesis's performance evaluation chapter
Titled: Performance Issues, and measurements and analysis.
I am almost done with the implementation, and currently analysing and measuring the system performance.

Here is the outline (table of contents) of the chapter. Once I am done with it I will upload the document.

Status Report for 08/08/2007

My Thesis Topic:

High Performance, Federated and Service-Oriented Geographic Information Systems


Introduction:

Geographic Information Systems (GIS) [1, 2] is basically a collection of computer hardware and software for capturing, managing, analyzing, and displaying all forms of geographically referenced data.

General purpose of GIS is extracting information/knowledge from the raw geo-data. The raw-data is collected from sensors, satellites or any other ways and stored in databases or file systems. The data goes through the filtering and rendering services and, presented to the end-users in human recognizable formats such as images, graphs, charts etc. A well-known example of GIS is map viewers which process layers of geospatial data to create map images. GIS are used in a wide variety of tasks such as urban planning, resource management, emergency response planning in case of disasters, crisis management and rapid response etc.

Over the past decades, GIS has evolved from traditional centralized systems to distributed systems [3]. Centralized systems provide an environment for stand-alone applications in which data sources, rendering and processing services are all tightly coupled and application specific. Therefore, they are not capable of allowing seamless interaction with the other data or processing/rendering services. These “deficiencies of centralized systems” and “improvements in the internet technologies” encouraged the academia, governments and businesses to start using distributed system approaches. Distributed systems are composed of autonomous hosts that are connected through a computer network. They aim sharing of data and computation resources, and collaboration on large scale applications.

The main challenge in sharing of data and computation resources and integrating GIS data is the heterogeneity of different sources. It has been studied from the perspective of how to resolve the semantic differences between heterogeneous data sources, mapping different schemas, and providing standard service and query interfaces. The adoption of GIS open standards for online service and data model solve the heterogeneity problems to some extent. OGC and ISOTC-211 are the well-known universally standards we use for this respect.

A large variety of different data sets are available in various specialized repositories, and users and geo-science applications would like to access these distributed heterogeneous data sources through uniform service interfaces enabling unified querying from a single access point. In the literature, these requirements are explained as “federation” which is initially used by database community [29, 30] to federate heterogeneous databases. Federation is basically established by working out the interoperability and inter-service communication issues among the distributed heterogeneous sources.

Distributed GIS systems typically handle large volume of datasets. Therefore the transmission, processing and visualization/rendering techniques used need to be responsive to provide quick, interactive feedback. There are some characteristics of GIS services and data that make it difficult to design distributed GIS with satisfactory performance. One of them is that services provided by a GIS typically require heavy CPU usage due to the complex computation involved in the underlying computational geometry. Another one is that GIS services often transmit large resulting datasets such as structured data, images, or large files in tabular-matrix formats.

In this thesis, we will first research architectural design requirements of federated GIS framework (in accordance with Service Oriented Architecture (SOA)) composed of Web Service components of commonly accepted GIS Open standards. Secondly, we propose a novel grid-enabled [10, 11] aggregator map services (federator) for optimizing the performance and responsiveness of the federated Service-Oriented GIS.

For complete thesis proposal document
See : http://complexity.ucs.indiana.edu/~asayar/proposal/proposal10.pdf

[1] GIS Research at Community Grids Lab, Project Web Site: http://www.crisisgrid.org.
[2] Ahmet Sayar, Marlon Pierce, Geoffrey Fox OGC Compatible Geographical Information Services Technical Report (Mar 2005), Indiana Computer Science Report TR610
[3] Peng, Z.R. and M. Tsou, Internet GIS: Distributed Geographic Information Services for the Internet and Wireless Networks. 2003: Wiley.
[10] Fox, G. and M. Pierce. Web Service Grids for iSERVO. in International Workshop http://www.eps.s.u-tokyo.ac.jp/jp/COE21/events/20041014.pdf on Geodynamics: Observation, Modeling and Computer Simulation University of Tokyo Japan October 14 2004. 2004
[11] Fran Berman, Geoffrey C, Fox, Anthony J. G. Hey., Grid Computing: Making the Global Infrastructure a Reality. John Wiley, 2003.
[29] McLeod and Heimbigner (1985). “A Federated architecture for information management", ACM Transactions on Information Systems Vol 3, Issue 3: 253-278.
[30] Sheth and Larson (1990). "Federated Database Systems for Managing Distributed, Heterogeneous, and Autonomous Databases". ACM Computing Surveys Vol 22, No.3: 183-236.

Status Report for 07/03/2007

TENTATIVE OUTLINE OF MY REARCH WORK :

Title:
Semantic-coupling of Data and Computation Grid Sources in GIS

1. Introduction

2. Background and Definitions
............-GIS (Geographic Information Systems)
............-Web Services (GIS Web Services)
............-Geospatial Standards
........................o OGC, ISO/TC-211
............-General Data Types
........................o vector data and raster data
........................o and standard common data model -GML
3. Coupling data and computation grid sources in GIS
............-Architecture
............-3layer structured display
............-Components and their interactions
............-Inter-service communications
............-Autonomic capability upgrade
............-Components chaining through capability metadata
............-Sci-Plotting Service
............-Motivating use cases
........................o IEISS
........................o ServoGRID geo-science applications (PI and VC)
4. Making the system internally semantic
............-Using OGC standards for
........................ o WMS and WFS
........................ o Capability metadata (RDF and WSDL like structure)
............-Common data model (GML)
............-Common service descriptions and interfaces
5. Adapting third party heterogeneous data sources semantically
............-Making the heterogeneous data and services compatible
............-Horizontal mediation
......................o Heterogeneous communication protocols
...................... o Enabling inter-service communication among
................................. - The Gris-enabled vs Non-Grid OGC services
.................................- SOAP vs HTTP-GET/POST
............-Vertical mediation (data heterogeneity)
......................o Enable integration of third party data
......................o WFS-based mediation
................................ -XLink-WFS
6. Extending the system with quality of services
............-Interactive decision making tools
........................o portal and browser based user interfaces
........................o comprehensible data display (maps composed of layers)
........................o interactive data querying
....................................... through three-layer structured display
........................o enabling application based grouping of data sets
........................o AJAX + GIS Web Services Integration framework
............-Time-series data animations (map animations)
............-Streaming map movies architectures
7. Performance issues for large scale geo-science applications
............-Data transfer (between WFS and WMS)
........................o streaming
........................o load balancing -through service capability metadata
....................................query decomposition and results composition
............-Data handling (semi-structured common data model -GML):
........................o Parsing (in WMs)
....................................o pull parsing
........................o Rendering and handling large scale data
....................................o caching + prefetching
8. Performance Tests
9. Contributions
10. Future Work

Status Report for 05/30/2007

We summarize the common GIS problems in a general context (Option 2) as
..........-....geospatial (or geographic) data access and integration
..........-....coupling the data grid with service grid
..........-....unified queries for integrated data
..........-....integrated data display
..........-....interactive smart visualization tools
..........-....performance
and discuss and propose our approaches to these problems.

These issues are undeniably the crucial points of the numerous research and development efforts. Especially the problems related to the data and storage heterogeneites are being addressed by a number of groups and organizations some of which also offer solutions to the application level interoperability issues.

We generally focus on the issues in terms of GIS and geographic data, but findings and recommendations are relevant for any other science domains and data types.

Below I summarized these issues as my thesis prospectus:

*****************************************************
Summary of my thesis prospectus:

We investigate the issues pertaining to the traditional Geographic Information Systems (GIS)approaches and propose solutions to these problems based on modern Service Oriented Grids approaches. As in general science domains, GIS requires decision making and situation assessment based on integrated data display. We generally focus on the issues in terms of GIS and geographic data, but findings and recommendations are relevant for any other science domains and data types.

GIS is a system of computer software, hardware, and data used to manipulate, analyze, and graphically present a potentially wide array of information associated with geographic locations. GIS’s powerful ability to integrate different kinds of information about a physical location can lead to better informed decisions about public investments in infrastructure and services—including national security, law enforcement, health care, and the environment—as well as a more effective and timely response in emergency situations. However, long-standing challenges to data sharing and integration need to be addressed before the benefits of geographic information systems can be fully realized. Our focus regarding data integration is different from the Database or digital library communities. We deal with the integration at the higher level than they do and we try to utilize their approaches at the bottom level by proposing generic mediator services.

Our work is about developing a Web Services architecture that provides coupling of scientific geophysical applications with archival data through the innovative interactive smart decision making tools. This work can be detailed in a couple of sub-research areas such as: accessing and querying heterogeneous data provided by heterogeneous storages with unified query structures, developing GIS data services, considering performance issues of transferring, parsing and rendering of large geographic data, and composition of GIS services.

In the light of the explanations above we categorize our work as below:

1. Heterogeneous and distributed data integration (Data Grid) through mediator services
.........o Different data types
.........o Different storage types

2. Coupling Geo-Science Grid with Data Grid
.........o Integrating Web Map Services with Geo-Science Grid
.........o Enabling decision making through integrated data display (3-layered display structure)
.........o Creating view-level integration structure. View is abstracted as layers in GIS domain.
.........o Creating generic plotting Web Services (Sci-Plot) in order to couple Geo-Science Grid outputs with its inputs (from data grid) at the view level.

3. Interactive and smart decision making tools
.........o Coupling interface for browser based remote access
.........o Data/information display
.........o Interactive querying and mining the data
.........o Visualization and analysis of the data and Science Grid simulation outputs
.........o Movies and animations tools over time-series data

4. Performance
.........o Accessing remote large data sets provided by geographically distributed data vendors.
.........o Transferring, integrating, processing and interpreting data.
.........o Proposing: High-performance streaming data services through messaging middleware.
.........o Proposing: Advanced pre-fetching, caching and load balancing techniques.

*****************************************************
Special note to Professor Geoffrey: As you realized I was confused with options 1 and 2. My previous document was according to the idea you named as Option-2. The above statements are summary of my upcoming document in option-2. Once I am done with it I will send to you.

Pre-fetching Concept in Brief

1. Pre-fetching Concept in Brief

Proposed system (mentioned in previous documents as grid oriented map services) enables Aggregator WMS to run in combination of streaming (or non-streaming) and pre-fetching (or on-demand) data transfer modes.

In order to run it in pre-fetched mode Aggregator WMS's "DATACCESS" entry in the properties file should be set as below

#--------------------------------
# DATA ACCESSING MODE #
#--------------------------------
DATACCESS = pre-fetched
WMSTYPE = streaming

In order for the pre-fetching algorithm to work properly, data should be fetched as a whole.No constraint should be defined in the query.

Aggregator WMS queries and transfers the data by using Web Feature Service's (we call it DSFS in general) getFeature Web Service interface. WFS provides data in GML common data model (we call it DSL in general)

In our motivating domain (GIS) one of the criteria define the constrant in the query is bounding box and it is defined in the getFeature request as below.

<gml:coordinates>minx,miny maxx,maxy</gml:coordinates>

in order to get whole data we set this criteria in its widest range as it is shown below:

<gml:coordinates>-180,-90 180,90</gml:coordinates>

Or alternatively,
Remove this constraint tag from the getFeature request


Why do you do pre-fetching?:
We do pre-fetching to get rid of the poor performance of transferring XML based feature data. According to our proposed framework, pre-fetching is done over Layer set 2 data (see the previous documents for three layered structures). Layer set 2 data is represented in Common Data Model (CDM). As CDM we use GML.

The names of the data to be pre-fetched are obtained from the capabilities file. Here are the example layer-set 2 data names listed in capability file:
California_Faults
States_Boundaries
World_Seismic

These data are pre-fetched into seperate files of different names. File names are same as corresponding data names that they will be fetched in.
California_Faults.xml
State_Boundaries.xml
World_Seismic.xml

There will be two seperate locations for the pre-fetched data. One is temporary which will be active during prefetchin the data. Another is stable which will be used for serving the clients' requests. When the data transfer is done to the temporary location, all the data at that location will be moved to stable location. Reading and writing the data files at the stable locations will be synchronized to keep the data files consistent. This cycle will be repeating.

Requests from clients contain some constrants. So, querying the pre-fetched data is handled at the Aggregator server side at which pre-fetching initialized. Query is basically done by using Pull Parsing techniques and XPATH queries over pre-fetched data.


2. Coding Level
-how to define the task and periodicity of it to fetch the data?

It should not be less than the time to transfer one set of layer-set 2 data.
So from our experience for the data listed above, it should be more than 2 hours.Since these data sets are not updated often we plan to set the periodicity once avery day.

what technology do you use to pre-fetch the data?
Timer
TaskTimer Java class libraries.


//TimerTask definition:
TimerTask task = new TimerTask() {
...... public void run() {
............ Vector CDMdataList = new Vector();
............ CDMdataList = getListOfGMLDataInCapability();

............ String tempDatastore = applpath + "/prefetchedData";
............ String usedDatastore = applpath + "/prefetchedDataUsed";

............ //Fetching all the data in CDM format (GML) - with NB
............ fd.FetchDataWithStreamingNBip,NBport,NBtopic,
........................................wfs_address,tempDatastore,CDMdataList);

............ //after pre-fetching is done move the data to stable storage
............ fd.moveData(tempDatastore, usedDatastore);
...... }
}

//Running the Task defined above:
Timer timer = new Timer();
timer.schedule(task, 0, 40000);
//sets timer to run once every three days

Timer class schedules the specified task for repeated fixed-delay execution, beginning after the specified delay. Subsequent executions take place at approximately regular intervals separated by the specified period



3. Possible Challenges:

One minor challenge with storing the data in file system is that some systems allow a limitid size for user created files such as:

LINUX(RedHat)...........................SOLARIS
2GB{512B block size}................... 1 TB
8192GB{8KB block size} ............... 2 GB {=<2.5.1}

Even if operating system does not have limited file size constraint, then file size will be contrant by the hard dizk size.

Another challenge is synchronization of the two storages (temporary for fetching the data and stable for answering the client requests while fetching is happening). This challenge is easier to solve.

Status Report for 05/02/2007

1. I have implemented "pre-fetching" according to the proposed architecture that I have mentioned in my previous posts. You can test the performance by going to demo page.
http://toro.ucs.indiana.edu:8083/aaa/maptools/newmap.jsp

Notes for simple instructions to test:

__testing fault and state data
__At the top left drop down list select "California Faults"
............then select layers named "California:faults" and "State:Boundaries"
............then use map tools (zooming, move etc.) at the bottom of the map image

__testing earthquake seismic data
__At the top left drop down list select "Pattern Informatics"
............then select layers named "World:Seismic" and "State:Boundaries"
............then use map tools (zooming, move etc.) at the bottom of the map image

For the stress test please zoom out to whole world and in secion "time interval for seismic data" set minimum magnitude as 0, and set "from time" to 01/01/1900 and "to time" to 12/31/2006.

Please do not use buttons "Create Map Movie" and "Plot Pi Output" (FTB)


2. I have answered one of user questions about "Google Maps" and our "Grid-oriented mapping services" and, posted it to my blog. Please check my previous post.

Our Grid-oriented Map Servers vs. Google Map Servers

Question: How are you different from Google Maps?
Can you use only Google maps (without any mediators) in Geo-Science Applications?

Google Map Services enables navigating into an Earth framework for map/satellite display in images and applying dynamic browser based geo-processing services.

Instead of competing, Google Map provides complementary services and layers to our integration framework. We use its earth map/satellite images as Layer-Set 1 (see my previous posts) to create more comprehensible map images in our integration framework.

We mostly deal with the performance issues at Layer set 2.Google Map and NASA OnEarth WMS map servers provide layer set 1 in tolerable performance level.

Our main focus is integrating Mapping Services with Geo-science grids . Outcomes of these integration is three layer structured map images. Geo-Science applications use scientific data (geo-data, spatial data) provided by different vendors in differetn format. Because of this, we mainly deal with the interoperability of services and data. In order to solve the data interoperability problem we use OGC defined standard GML specifications. GML is XML encoding of the scientific vector data. Using XML causes performance problems which is not related to the problems Google people deal with such as ransferring parsing rendering and displaying XML based large data (GB in size).

Actually Google enables overlaying vector data (Layer-Set 2) over their satellite images or maps but when you tried to use Google map API with geo-science any XML encoded science data system will be stuck. it is because of that Google map API uses DOM parsers. DOM enables parsing XML documents of limited sizes.

How Google Map Server works:

Here, I summarized their approach. You can find more detailed architecture description on the web.
----------------------------------------------------------------------------------
Google cut the whole world's satellite image into pieces and call them tiles. The vital question is how to formulate the accepted requests and regions composed of tiles. Google Map provides two different maps, (1) Google maps and (2) Google satellite images.


1. Google Maps:

They use three parameters, xcoord, ycoord and zoom factor to formulate requests and regions.

A sample request to mt1 server at zoom level 9:
http://mt1.google.com/mt?n=404&v=w2.12&amp;amp;amp;amp;amp;amp;amp;amp;amp;x=130&y=93&zoom=9

Zoom factor gets values from 17 (fully zoomed out) to 0 (maximum definition).
x coordinate takes values from -90 to 90
y coordinate takes values from -180 to 180

At a factor 17, the whole earth is in one tile where x=0 and y=0. At a factor 16, the earth is divided in 2x2 parts, where 0<=x<=1 and 0<=y<=1. and at each zoom step, each tile is divided into 4 parts. So at a zoom factor Z, the number of horizontal and vertical tiles is 2^(17-z)

Google uses 4 servers to balance the load. these are mt0, mt1, mt2 and mt3. And each tile is a 256X256 png picture.

To find a tile from the cached tiles:

latitude=90-latitude;
longitude=180+longitude;

double latTileSize=180/(pow(2,(17-zoom)));
double longTileSize=360/(pow(2,(17-zoom)));

//Tile coordinates:
int tilex=(int)(longitude/longTileSize);
int tiley=(int)(latitude/latTileSize);

(2) Google Satellite Images:

A sample request to kh0 server at zoom level 6:
http://kh0.google.com/kh?n=404&v=8&amp;amp;amp;amp;amp;t=trtqtt

the length of parameter t defines the zoom level.

To see the whole globe, just use 't=t'. This gives a single tile representing the earth. For the next zoom level, this tile is divided into 4 quadrants, called, clockwise from top left : 'q' 'r' 's' and 't'. To see a quadrant, just append the letter of that quadrant to the image you are viewing. For example :'t=tq' will give the upper left quadrant of the 't' image. And so on at each zoom level...

Google uses 4 servers to balance the load. these are kh0, kh1, kh2 and kh3. And each tile is a 256X256 png picture.

To find a tile from the cached tiles:

double xmin=-180; double xmax=180;
double ymin=-90; double ymax=90;
double xmid=0; double ymid=0;

string location="t";

//google use a latitude divided by 2;
double halflat = latitude / 2;
for (int i = 0; i < zoom; i++)
{
xmoy = (xmax + xmin) / 2;
ymoy = (ymax + ymin) / 2;
if (halflat > ymoy) //upper part (q or r)
{
ymin = ymoy;
if (longitude < xmoy)
{ /*q*/
location+= "q";
xmax = xmoy;
}
else
{/*r*/
location+= "r";
xmin = xmoy;
}
}
else //lower part (t or s)
{
ymax = ymoy;
if (longitude < xmoy)
{ /*t*/
location+= "t";
xmax = xmoy;
}
else
{/*s*/
location+= "s";
xmin = xmoy;
}
}
}

Status Report for 04/18/2007

(1) I have proposed an integration framework for Web Map Services and Geo-Science Grids,
then,
(2) I have evaluated current performance of the system and figured out the bottleneck regarding large XML based data transfer and handling (parsing rendering etc.),
and,
(3) proposed an architecture for the improvements.
and finally,
(4) evaluated the expected results

Architecture is summarized in three orderly steps. These are
. Pre-fetching (Runs in a predefined periods asynchronously and independently)
Run-time:
. Caching
. Cached-data extraction and Rectangulation on un-cached data
. Merging the extracted cached data with the returned map images in response to rectangles

Please for more detailed information see this paper
http://complexity.ucs.indiana.edu/~asayar/proposal/CurrentResearchWork1.pdf

FAQ About Sci-Plotting Services

SCI-Plotting Services (%60 completed)
-- Implementing in Web Service principles
-- Data Conversion
-- Layers compatible to be overlaid on WMS maps or Google Maps


53. What is Sci-Plotting service?
It is a rendering service like WMS providing layers integratable to WMS maps but not map images. They provide layers created from geo-science simulation outputs in tabular format. For the core functionality we use Dislin scientific-data plotting libraries. Dislin is a plotting library containing functions for displaying data graphically as curves, graphs, pie-charts, 3-D color plots, surfaces and contours. All these services are wrapped as Web Services and integrated into the general visualization system. This is a service consists of the functionalities provided as Web Services through which any geo-science application output data can be interpreted, rendered and displayed as images and graphs.

54. Can you give a sample output layer Sci-Plotting service provides?
In the three layer set structured architecture, Layer-1 and Layer-2 come from WMS and, Layer-3 comes from Sci-Plotting services.

In order to get better view of the figure, please click on it.

55. Did you use Sci-Plotting services in any real geo-science application?
Yes we did. GeoFEST is a two- and three-dimensional finite element software package for the modeling of solid stress and strain in geophysical and other continuum domain applications. GeoFEST uses stress-displacement finite elements to model stress and strain due to elastic static response to an earthquake event in the region of the slipping fault, the time-dependent viscoelastic relaxation, and the net effects from a series of earthquakes. Sample output snapshot at Q59.

56. What are the inputs and outputs of Sci-Plotting services?
Currently raw data (input data) is in the column format and located on an HTTP address which can be accessed by using HTTP protocol. It is actually a plain text file. URL to this file is given as parameter to the request for plotting. For each different application there is a data conversion filter to make the data understandable to the service. For example for PI applications Sci-Plotting server is given a file URL to plot in which data is in tabular format. One column shows X coordinate other column shows Y coordinate and other columns shows the values to be plotted. In the future we plan to use VOTables to represent tabular data.
The VOTable format is an XML representation of the tabular data. The VOTable format’s aim is making the online Sci-Data both interoperable and scalable. The interoperability is encouraged through the use of XML standards. When the data is encoded in XML, data interpretation and validation will be easier. During the machine to machine communication, data encapsulation in any kind of messages such as SOAP will also be easier. Application specific metadata can also be embedded in to the VOTable. Metadata can be about the owner of the data, application properties of the data, authentication etc. There is also a price for encoding data in XML. When the data encoded in XML, each data element is tagged. The tagging causes overheads in terms of the volume. Increasing in the data volume basically causes increasing in the transfer time and the processing time.
Return type is image map layers to be overlaid on other maps or graphs showing statistic information to be displayed separately. Most of the output types are application dependent so far but since our implementation packages are open source you can download and modify it in accordance with your application and needs.

57. What is the most challenging part in creating Sci-Plotting services?
Input data comes from variety of different Geo-Science applications to be plotted. Each of these applications output different number of records, columns, variables, and different data structures. Most of the applications outputs thousands or even millions of lines of records. Storing, updating and reusing of the data are cumbersome. Some times, application requires some restrictions and properties on the data. In that case, there should be some properties and attributes about the archived data to check against these requirements. For example, some data are valid just for some specific time intervals or they might have expiration dates for their validations. Furthermore, data on the net may require authentication for access. The remote data needs to be accessed by using various arbitrarily complex protocols by using the URL syntax, “protocol://location”.
The VOTable format seeks answers and solutions to the problems and challenges explained in the previous paragraph. The main goal is making the online geo-data both interoperable and scalable. The interoperability is encouraged through the use of XML standards. When the data encoded in XML, data interpretation and validation will be easier. During the machine to machine communication, data encapsulation in any kind of messages such as SOAP will also be easier. Metadata about the data can also be embedded in to the data. Metadata can be about the owner of the data, application properties of the data, authentication etc. Everything is not perfect when we used the XML encoded data. There is also a price for it. When the data encoded in XML, each data element is tagged. The tagging causes overheads in terms of volume. Increasing in the data volume basically causes increasing in the transfer time and the processing time.


58. How do you use core Dislin libraries through Web Service Interfaces?
Dislin is originally written in Fortran and C but they have also their corresponding native libraries. Native libraries allow us call codes written in any programming language from a program written in the Java language by declaring a native Java method, loading the library that contains the native code, and then calling the native method. We embedded Dislin libraries into our plotting Web Services which is written in Java by using Java Native Interfaces and native methods. In order to install it into our whole system we needed to publish the plotting services as Web Services. Therefore, we first created service description files (WSDL) and publish them by using java axis web services container.
Service stack is composed of native method declarations. All the declared native methods have their own corresponding native implementation routines in the loaded native libraries. Routines can be accessed from any java class through the ServiceStack class. You can see the shrunk version of the ServiceStack class as below. Since all the methods are declared as static, class is assumed to be static as well. So, sample contour calls from any java class will be as below;

// before calling ServiceStack routines first import ServiceStack class
ServiceStack.contour(x-array, x-length, y-array, y-length, data-array, data-length, z-level);

public class ServiceStack
{
public ServiceStack () { }
public static native void contri(float af[], float af1[], float af2[], int i, int ai[], int ai1[], int ai2[]);
public static native void abs3pt(float f, float f1, float f2, float af[]);
public static native void contour(float af[], int i, float af1[], int j, float af2[], float f);
public static native void angle(int i);
/*
Native routine correspondences of all the scientific data plotting libraries
*/
static
{
System.loadLibrary("javaNativeCodeLibraries");
}
}


59. How do you invoke the Sci-Plotting services? Can you give sample snapshot from your outputs?
We initially created below user interface as WMS Client. It was enabling just getting maps from WMSs. Later, we enhanced it with smart map tools and movie streaming interfaces. Finally, we have added and integrated new interfaces for invoking scientific plotting Interfaces in accordance with other interacted GIS services.
GUI and map tools enable users to dynamically interact with the Dislin libraries and GIS visualization services. Each action by the user causes the changes in the parameter values and new request to the corresponding services. Depending on the application and user needs, plotting layers can be shown on another pop-up browser or just overlaid on other layers. All these kinds of service parameters are defined in the requests.

Pattern Informatics Geo-Science Application snapshot

In order to get better view of the figure, please click on it.

GeoFEST Geo-Science Application snapshot

In order to get better view of the figure, please click on it.

FAQ About Creating Streaming Map Movies

Building Architectural Framework for Streaming Map Movies:

- Publishing streams through JMF libraries.
- Enhanced SMT with movie capabilities
- Easy to display movie streams
........... from Collaboration sessions
........... from standalone JMF client program

46. What is streaming map movie?
Visualizing changes over time is achieved by integrating temporal information on a map. Usually the result is a series of static maps showing certain themes at different moments. In addition to creating static maps, WMS also has the ability to combine the static maps correspond to a specific time interval data and combine them in an animated movie. Movies created by WMS are composed of a certain number of frames. Each frame represents a static map that corresponds to a time frame defined in request.

47. Why do you need to create map movie?
Standard map servers produce static images, but many types of geographic data are time dependent. In order to understand geographic phenomena and characteristics of temporal data it is necessary to examine how these patterns change over time for these types of data.

48. How does WMS create a streaming map movie?
Movies in general are composed on static frames. Frames are actually images. Movies are obtained by playing these static frames sequentially. In our case of map movies, frames are static map images created for a specific bounding box and a specific time interval. Sequential frames are created based on successive time intervals. Range of interval is given by the client from the user interface. See below figure, time interval is set to “P1Y”.

In order to get better view of the figure, please click on it.

Movies are created just for time-dependent data. WMS notify the clients for what layers it can create movie streams through its capabilities file. If a client makes a request to get a movie for a specific layer, to succeed, this layer should have a time dimension defined under this layer element in the capabilities file. Here is an example dimension tag under the layer tag in capability. Layer is created based on a data which is provided yearly.

In order to get better view of the figure, please click on it.

The last value “P1Y” (in the Dimension tag element above) defines the periodicity of the data collection. According to this value, WMS cuts the whole time (“from/to/periodicity”) into multiple values and for each time slice, it makes a "getFeature” request to WFS to get feature data in GML. After receiving GML, WMS creates static map images for each time slice. These images will be frames of the result map movie.
Clients should make the “getMap” standard request to WMS to get the movie for a specific layer but the “format” variable should be set to “movie/ ” and “time” variable should be set to a value in an appropriate format. The last value in the Dimension tag element (P1Y) is defined based on the user’s selection from the drop-down list shown in above figure. (See also a sample request for the movie in Q49 below).
Below figure is the BIG PICTURE for the creation of the map movies. In order to get further information for the map movies please see the *draft* technical report
http://complexity.ucs.indiana.edu/~asayar/gisgrids/docs/movie_jmf.doc.

In order to get better view of the figure, please click on it.

49. Can you give a sample movie request sent to WMS?
The request is actually “getMap” request with specific parameters set to specific values. We might need to request a map movie based on the layer created from a data. WMS defines this layer in its capabilities file with the “dimension” tag as displayed in Q48. In order for WMS to create map movie we need to send a “getMap” request as below. Here we request an mpeg movie created from frames created with a yearly data. First frame starts with the year 1987 and last frame ends with the year 1992. So, movie will be created from five frames.

In order to get better view of the figure, please click on it.

50. Can you define any attributes such as frame rate for the movie streams published by WMS?
The map video stream has several parameters that can be adjusted. These parameters affect the quality of the produced map video stream. Among these configurable parameters are frame rate and video format of the stream, update rate of the map images in the video stream.
In our current implementation, we define these attributes in hard coding but in the future we will take it from the smart map tools interface. This is necessary because; some clients might not be capable of visualizing video streams with low frame rate or can visualize them with very low quality because of their connection or some other issues. They might need to adjust the attributes and get the best result.

51. How do you display map videos?
When the frames (static maps) corresponding to separate time slices are created, WMS publishes them to a Real Time Protocol (RTP) session as streams. RTP sessions are defined with IP Address and Port Number couples. Movie streams published to an RTP session can be visualized by any client connecting to the same RTP session (same IP and port number defined by the publisher).
If you want to display the movie streams with a standalone program (not in collaboration environment such as AccessGrid and Global MMCS), then download and install JMStudio software (http://java.sun.com/products/java-media/jmf/). From the user interface select “File” menu and click on “open RTP session” and give IP, port and time to live (ttl) values that you already know there is one WMS publishing map movie streams to this session.

52. Do you have any sample map movie to see online?
Yes, see the below link. Movie at the below link is obtained by screen recording of the real PI Geo-Science application’s output. Movie is composed of yearly seismic earthquake records. The minimum magnitude is 4.0 (PI spec parameter), and frames are composed of static maps created yearly based.http://complexity.ucs.indiana.edu/~asayar/gisgrids/html/work/AutoScreenRecorder_01.mpg

FAQ About Smart Map Tools (SMT)

Smart Map Tools
- WMS Client
- Interactive data querying
- Streaming map movies capability for the time-dependent feature data
- Integrating AJAX Approach
...... Layer based integration of Google maps and OGC’s WMS maps
....... Overlaying features from WFS over Google Maps

31. What is SMT?
SMT is acronym of Smart Map Tools. We call it smart because tools allow users to manipulate data, create interactive queries and analyze the spatial information. SMT also enables Geo-Science application results interrelated with maps at the “layer” level. SMT also allows creating movie streams or animations from time dependent data.

32. What functionalities does it provide?
In order to give brief idea about what SMT provides, I give development stages from the initial to the current.
1. I first implemented it as “WMS Client”. At that time, SMT was just getting map images and displaying through standard map tools enabling zooming, moving and calculation distance between two selected points on the map etc.
2. I have added interactive query capability. Users are able to query data just by clicking on a specific data on the map. They do not write any complicated SQL queries or any other type of scripts.
3. I have added project based functionality to give additional interpretation to the data.
4. Embedding additional interfaces for the layers based on time dependent data in order to create streaming map movies. User is given option of defining time intervals such as per one year or per one month etc.
5. I created AJAX based approach. It enabled layer based integration of Google maps and OGC’s WMS maps at the “layer” level and, overlaying features from WFS over Google Maps.
6. NOW – I am trying to create a flexible framework enabling integration of all these Map services with Geo-Science Grids. it includes interacting with Sci-Plotting services to plot Geo-Science Grids simulation outputs over Google maps or WMS maps, interacting with HPSearch Job Manager in accordance with Sci-Plotting services. Sci-Plotting services (by Sayar) and HPSearch Job manager (by Gadgil) are two of CGL projects.

33. Can you give a snapshot from SMT?
This is a snapshot from the very early stage of our SMT implementation. For more advanced one please see the snapshot at Q59. Explanations in orange framed boxes give the basic functionality of the SMT. It is implemented by using Java Server Pages (JSP), JavaScript and DHTML technologies.

In order to get better view of the figure, please click on it.

34. Do you have any GIS demo running?
We have a demo running at http://156.56.104.187:8083/aaa/maptools/newmap.jsp. This is my implementation of user interface for the WMS Client and our proposed Map Services and Geo-Science grids integration framework.
Here is the user manual for instructions about how to use smart map tools to get and manipulate maps and run geo-science applications through smart map tools interactively.
http://complexity.ucs.indiana.edu/~asayar/gisgrids/docs/usermanual.pdf

35. What is the streaming map video?
Another capability SMT provides is the set of tools enabling creation of the streaming map movies. Visualizing changes over time is achieved by integrating temporal information on a map. Usually the result is a series of static maps showing certain themes at different moments. In addition to creating static maps, WMS also has the ability to combine the static maps correspond to a specific time interval data and combine them in an animated movie. Movies created by WMS are composed of a certain number of frames. Each frame represents a static map that corresponds to a time frame defined in request. Please see the Section 1.2.3 for more detailed information about the streaming map movies.

Integrating AJAX Approach to SMT (Q36 to Q45):

36. What is AJAX?
AJAX (Asynchronous JavaScript and XML) is a web development model for the browser based web applications. AJAX is a style of web application development that uses a mix of modern web technologies to provide a more interactive user experience. AJAX is not a technology. It is an approach to web applications that includes a couple of technologies. These are JavaScript, HTML, Cascading Style Sheets (CSS), Document Object Model (DOM), XML and XSLT, and XMLHttpRequest as messaging protocol.
AJAX drew some attention in the public, after Google started to develop some new applications with it. Some of the major products Google has introduced over the last year by using AJAX model are Google Groups, Google Suggests, and Google Maps. Besides the Google products Amazon also have used AJAX approach in its search engine application.

37. What inspired you to use AJAX and Web Services together for the GIS Vis Systems?
AJAX provides high interactivity and performance results for the browser based web applications. AJAX and Web Services are XML Based structures, so that they are able to leverage each others strength. Both AJAX and Web Services uses widely-used and well-known technologies such as XML and HTTP. Last but not least is that they are not competing but complementary technologies.

38. What are the advantages of using AJAX in your GIS Visualization systems?
AJAX has a couple of advantages for the browser based web applications developers. It eliminates the stop-start nature of interactions, user interactions with the server happen asynchronously, data can be manipulated without having to render the entire page again and again in the web browser, and requests and responses over the XMLHttpRequest protocol are structured XML documents.
Using AJAX approach in our system enables users to be able to integrate Google Maps with the Geo-science data in the form of layers (from WMS) or plotting (feature data from WFS). Regarding WFS, it provides vector data (points or line strings) in the GML format encoded in XML. GML contains content and presentation information for the feature data. Google maps are able to be plotted by using its own libraries after extracting the presentation data (OGC call them as Geometry elements such as points and line strings) through parsing of the GML.
By layering or/and plotting geo-science data on the Google maps, we obtain more detailed and informatory maps. As an example, I can see if my house is constructed on an earthquake fault through layering and in right zoom level.

39. What are the disadvantages of using AJAX and Web Services together (if there is…)?
This framework introduces some extra work. Extra work mostly comes from the conversion of parameters to be able to make compatible requests to remote Web Services. In order to make valid requests, the proxy server should be deployed locally and client stubs for Web Service invocations should be created before running the application.

40. What are the challenges in your architecture?
The biggest challenge is the handling the messages going through different protocols before accessing the receiver. Web Services and AJAX use different protocols for the data transfer. AJAX uses HTTP based XMLHttpRequest protocol and Web Services use SOAP over HTTP protocol (they also use SMP but in our implementations we use SOAP). They have no common message protocols to transfer messages and data.

41. Can you use Google Maps to display geo-science data directly instead of using your integration approach?
First of their concern is commercial not scientific. It is not efficient for integrating and displaying scientific data over the Google Maps by using their libraries. There are a couple of reasons for that but the most important one is that they use DOM parsing internally and it is not memory and performance efficient when you need to display data larger then about 4MB. With our approach we can display even 30MB of XML based geo-science data in a reasonable time period.

42. What is the general architecture (approach for the involving AJAX)?
The major contribution of the proposed architecture is that users interact with the Web Service based GIS services through the AJAX enabled GIS client user interface.
AJAX uses HTTP GET/POST requests (through JavaScript’s XMLHttpRequest) for the message transfers (see (A) in Figure). Web Services use Simple Object Access Protocol (SOAP) as a communications protocol (see (B) in Figure). In order to be able to integrate these two different message protocols, we convert the message format from one protocol to another. Since there is no ready to use common protocol to handle messages communications between AJAX and Web Services, we implemented a simple message conversion technique (see (C) in Figure). This essentially works by having the XMLHttpRequest communicate with a servlet, which in turn acts as a client to a remote Web service. This allows us to easily convert between SOAP invocations and HTTP POSTS. It also has the benefit of avoiding JavaScript sandbox limitations: normally the XmlHttpRequest object in the browser can only interact with its originating Web server.

In order to get better view of the figure, please click on it.

43. How do you integrate Google Maps with WMS?
This is actually a usage scenario of the general architecture explained in Q42 with some extensions. In order to fulfill this task we need to use base architecture.
Please see the figure below in order to understand the architecture in general before reading below explanations. There are two different path working in parallel by the given user parameters created by the client actions. Actions are interpreted by the browser through the Google Mapping tools on Layer 1. JavaScript captures these actions by ActionListeners and Google Binding APIs and gives to Layer-2 object.
Each layer is represented as JavaScript object. In our architecture we can utilize all the WMS Web Service interfaces such as “getMap”, “getCapabilities” and “getFeatureInfo”. The client does not need to make rendering or mapping jobs to create the map image. The map is consists of two layers one is coming from Google and other is coming from WMS. The layer returned by WMS is in a ready to use format such as JPEG or PNG or TIFF. The Layer returned by Google is a JavaScript object. In order to overlay these, we convert the layer coming from WMS into a JavaScript object before overlaying.

In order to get better view of the figure, please click on it.

44. How do you integrate Google Maps with WFS?
This is actually a usage scenario of the general architecture explained in Q42 with some extensions. In order to fulfill this task we need to use base architecture.
Before reading the answer please see the figure below in order to better understand the architecture. WFS provide feature data in vector format and vector data are encoded in GML according to OGC WFS specifications. GML is an XML encoding for the transport and storage of geographic information, including both the geometry and properties of geographic features.Client makes “getFeature” request and, the GML file encoded in XML is returned in a SOAP envelope. After getting a response, the client extracts geometry elements. The most important and commonly used geometry elements are Points, LineStrings, LinearRings, and Polygons. Even though Google Mapping API supports just two of them, Points and LineStrings, the other geometry elements can also be converted to these two types with minor updates. Having extracted and obtained geometry elements, these elements are plotted over the Google Map by using “GPoints” and “GPolylines” objects and the “mapOverlay” function of the Google Map API. By setting returned GML’s non-geometry elements and using ‘GMarker” object of the Google API, this architecture also provides the “getFeatureInfo” functionalities of the OGC WMS services. All these tasks are achieved by using XMLHttpRequest API and JavaScript functionalities.

In order to get better view of the figure, please click on it.

45. Did you apply your architecture to any project? If so, can you give us sample screen shots of your outputs?
Figure-a is about Google and WMS integration. The layer is coming from WMS is overlaid on Google map. This is a real output from the IESS application done in los Alamos National Lab. Layers coming from WMS are consists of two feature data. These are Natural gas and electric power pipelines.Figure-b is about Google and WFS integration. Application is done at Community Grids Lab (CGL) for Pattern Informatics, earthquake forecast application. In this sample application output we show California Earthquake seismic data plotted on Google maps as GIcons. You can even query data interactively by clicking on it as shown on the figure.

In order to get better view of the figure, please click on it.

Status Report for 04/04/2007

SUMMARY OF MY RESEARCH

Please first see "FAQ About OGC Compatible GIS Systems - Definitions" Posted: 03/25/07

(See FAQ About OGC Compatible Map Services - Development Posted: 03/28/07)
I. Building OGC Compatible Web Map Services (WMS) with Web Service Principles
<> Integration with WFSs and other WMSs
<> Implementing as Web Services
<> Capability-Based WMS Federations
<> Performance issues with archived data access and rendering

(See FAQ About Smart Map Tools (SMT) Posted: 04/04/07)
II. Smart Map Tools (SMT) Tools
<>WMS Client
<>Interactive data querying
<>Streaming map movies capability for the time-dependent feature data
<>Integrating AJAX Approach
.........Layer based integration of Google maps and OGC’s WMS maps
........ Overlaying features from WFS over Google Maps

(See FAQ About Creating Streaming Map Movies Posted: 04/04/07)
III. Building Architectural Framework for Creating Streaming Map Movies:
<>Publishing streams through JMF libraries.
<>Enhanced SMT with movie capabilities
<>Easy to display movie streams
..........from Collaboration sessions
......... from standalone JMF client program

(See FAQ About Sci-Plotting Services Posted: 04/04/07)
IV. SCI-Plotting Services (%60 completed)
<>Implementing in Web Service principles
<>Data Conversion (from … to VOTable)
<>Provides layers compatible to be overlaid on WMS maps a and/or Google Maps

FAQ About OGC Compatible Map Services - Development

Building OGC Compatible Web Map Services (WMS) with Web Service Principles

- Integration with WFSs and other WMSs
- Implementing as Web Services
- Capability-Based WMS Federations
- Performance issues with archived data access and rendering

13. Why did you implement GIS services in Web Services principles? What is the gain of extending OGC GIS as Web Services?
Web Services give us a means of interoperability between different software applications, running on a variety of platforms. A Web Services support interoperable machine-to-machine interaction over a network.
Distribution: It will be easier to distribute geospatial data and applications across platforms, operating systems, computer languages, etc.
Integration: It will be easier for application developers to integrate geospatial functionality and data into their custom applications.
Infrastructure: The GIS industry could take advantage of the huge amount of infrastructure that is being built to enable the Web Services architecture – including development tools, application servers, messaging protocols, security infrastructure, workflow definitions, etc.Easy to extend: Web Services have lots of specifications. By using any configuration of these specifications we will make our implementation more secure, more robust and fault tolerant as Web Service specs improve themselves.

14. OGC introduces capability concept to describe services. What are the differences between WSDL and capability?
WSDL is Web Service Description Language. Web Service is software that makes services available on a network using technologies such as HTTP, XML and SOAP. Web Service enables loosely coupling of services and interoperability. WSDL describes Web Services’ service functionality in operations and messages. WSDL enables access to functions through describing data encodings, location and communication protocols WSDL enables client to invoke the service’s operations through generating of stubs and skeletonsSO, Web Services are service based, in contrast capability is service+data related. Below figure shows the WSDL structure roughly with fundamental tag elements.

In order to get better view of the figure, please click on it.

See the capability structure represented with its main tag elements in FAQ for Users section Q12.

15. OGC introduces capability concept to describe services. What is the difference between metadata and capability?
Metadata is data describing data. Data might be paper, article, image, sound, spatial, temporal etc. To give an example, suppose data is article, so metadata might be consists of “author”, “title”, “subject”, “source type”, “format” and possibly more. Metadata enables describing and sharing information between different data sources and applications.
SO, metadata is data based, in contrast capability is service+data related.
Here is a sample metadata describing a resource created by merging Dublin Core in RDF.

In order to get better view of the figure, please click on it.

16. How is the data stored, will it be in GML, what database is mostly used?
The Web Feature Service starts processing user requests by extracting the database query from the request (encoded as a GML Filter – see below figure) and uses this to construct a corresponding MySQL query. The results of the database querying process are used to create a GML FeatureCollection which usually contains multiple features.Sample query “getFeature” encoded in XML (WMS sends to WFS to get feature data):

In order to get better view of the figure, please click on it.

17. How does WMS get the data from WFS and in what format?
WMS uses basic WFS services. These are getCapabilities, describeFeatureType, and getFeature. Invocation is done through client stubs created at the WMS side according to WFS’s WSDL file before run-time. Before invocation WMS also sends a getCapabilities requests to WFS to learn which feature types WFS provides and what operations are supported on each feature type. When WMS receives any getMap or getFeatureInfo request, it creates a getFeature (see the figure in previous question -16-) request and sends it to WFS. As a response WFS sends back the set of result features meeting the search criteria to WMS as a GML encoded in XML. In case of using streaming WMS, the result feature set is returned in streaming through NaradaBrokering messaging middleware in the form of byte arrays. In case of using pure Web Service version the result set is returned as XML document in the built-in format

. We prefer to use streaming data approach by using NaradaBrokering.


18. Can you give a sample “getCapabilities” request for Web Map Server?
WMS receives getCapabilities request in below format and returns its capabilities listed in Q12.

19. Can you give a sample “getMap” request for Web Map Servers?
WMS received getMap request and returns a map in a form defined in the tag “format” which is under the tag “image”.

20. Can you give a sample “getFeatureInfo” request for Web Map Servers?
WMS receives getFeatureInfo request and returns content of the feature data falling in close neighborhood (small bbox) of the clicked point on the map. Return type is mostly html or text.

21. Why do you prefer using streaming data approach through NaradaBrokering ?
In case of sending the result GML set in the form of string causes some problems when the GML is larger than some amount of size. Since the WFS returns the resulting XML document as an , this has to be constructed in memory and the size will depend on several parameters such as the system configuration and memory allocated to the Java Virtual Machine etc. Consequently there will be a limit on the size of the returned XML documents. For these reasons we have investigated alternative ways for data transport and researched use of topic based publish-subscribe messaging systems for streaming the data. Our research on NaradaBrokering shows that it can be used to stream large amount of data between nodes without significant overhead. Additional capabilities such as reliable messaging and support for different transport protocols already inherent in NaradaBrokering show that it is a powerful yet easy to integrate messaging infrastructure. For these reasons we have developed a novel Web Map Service and Web Feature Service that integrate Open Geospatial Consortium specifications with Web Service-SOAP calls and NaradaBrokering messaging system.

22. How does data streaming from WFS to WMS work to display a scientific data?
The clients make the requests with standard SOAP messages but for retrieving the results a NaradaBrokering subscriber class is used. Through first request to web service getFeature, WMS gets the topic (publish-subscribe for a specific data), IP and port on which WFS is streaming requested data. Second request is done by NaradaBrokering Subscriber. Even whole data is not received by WMS; WMS can draw the map image with the returned science data. This depends on the WMS’s implementation.

23. In order to get further information about the feature data displayed on the map, I selected “i” map tool and clicked on the feature data on the map and saw the content data displayed in a pop-up window. How does it work?
This is done through invoking “GetFeatureInfo” interface of WMS. GML is consists of content and presentation parts. WMS’s “GetFeatureInfo” interface enables displaying of content part of the data falling in a small area. This small area is around the point that user defines by clicking on the map and formulated by WMS Client as bbox and embedded in getFetatureInfo query. Area calculation in bbox changes depending on the zoom level. If you can’t get any result you should change your resolution level by zoom-in or out. When WMS get GetFeatureInfo, it creates getFeature request and sends it to WFS. Upon receiving returned GML, it extracts the content part (all but geometry elements) and converts it to HTML by using deployed XSLT and XSLT machine. It then returns HTML file to the client. The returned HTML document’s format is defined in XSLT.

24. How do you convert feature data into HTML?
When WMS get GetFeatureInfo, it creates getFeature request and sends it to WFS. Upon receiving returned GML, WMS extracts the content part (all but geometry elements) and converts it into HTML by using deployed XSLT and XSLT machine. It then returns HTML file to the client. The returned HTML document’s format is defined in XSLT.

25. How do you transfer and display large amount of scientific data (such as 60MB) in XML format?
In order to increase the performance we use innovative techniques. Regarding the transfer of large data sets see my explanation in Q21 and Q22. Regarding the parsing and rendering the returned GML, we use Pull Parsing technique (see Q26).

26. Why do you use pull parsing? What are the advantages over DOM and SAX parsing techniques?
In case of using pull parsing, the parser only parses what is asked for by the application rather than passing all events up to the client application. The pull approach of this parsing model results in a very small memory footprint (no document state maintenance required – compared to DOM), and very fast processing (fewer unnecessary event callbacks - compared to SAX).
Pull parsing does not provide any support for validation. This is the main reason that it is much faster than its competitors. If you are sure that data is completely valid, or if validation errors are not catastrophic to your system, or you can trust validation on the server, then using XML Pull Parsing gives the highest performance results.
Since we know that returned data is GML and know all possible tag names of the possible geometry elements, it is very advantageous using pull parsing.
XML Pull Parsing Web Site: http://www.xmlpull.org.
Please see the below article where XPP2 document model is compared with other leading Java based XML parsing implementations. http://www-128.ibm.com/developerworks/xml/library/x-injava/index.html

27. How do you convert GML to SVG?
We convert GML to SVG by using XSLT.

28. How does WMS send the map images to the requester (WMS Client)?
There two ways one is a common pure OGC approach through HTTP and other is through SOAP messages using client stubs for the getMap interface.
In case of using HTTP, result is sent to client through java’s servlet classes. For example result is written to client port by using HttpServletResponse’s java.io.PrintWriter object. Client does not need to write any client stub or anything. He just put the URL on the web browser in a correct format defined in OGC specifications, and gets the map on the browser.In case of using SOAP, WMS returns maps in image MIME types such as image/jpeg, before returning WMS converts maps into javax.activation.DataHandler object and attaches it to SOAP message. Client needs to create stubs before runtime in order to invoke Web Service interface of “getMap” by using online WSDL file of the WMS.

29. How do you create map images from the GML data after getting it from WFS? And, how do you overlay vector data from WFS over another layers coming from other WMSs?
There are twp possible options to create a map from the WMS point of view. One is creating SVG first from the GML through XSLT and converting SVG to any image type by using java libraries. Another is extracting geometry elements by using Pull parsing technique and drawing and displaying the map by java Image object. From our experience and performance tests we figured out that second approach has higher performance results then the first approach.
Web Map and Feature Services cooperate to create map images. XML-represented geographical features obtained from Web Feature Services are converted by Web Map servers into images using matrix-pixel (raster data), or points, lines and polygons (vector data). The basic operations related to mappings include discovering the physical address of the data providers, transferring the data from data servers and rendering it to create a map.The figure below helps to explain a few more questions here (Ex. Q17, Q28 and Q30). Information Service (IS) in the figure is not completed exactly. Currently, we read the addresses of WFS and other Map services providing requested data from the WMS capability file. In the future, we plan to use any Information Service completing the architecture illustrated below.

In order to get better view of the figure, please click on it.

30. How does WMS provide services through two different protocol in your implementation, one is over HTTP and other as Web Services?
The difference of these two is at the interface level. I have written a request handler for both protocols. Request handlers get the request (in case of Web Services, they are structured XML format – getMap, getFeature or getCapabilities) and parse and extract all the parameters. After having request name and related parameters, remaining jobs are all handled through the same classes and handlers.

FAQ About OGC Compatible GIS Systems - Definitions

Key Terms and Definitions :

1. What is GIS?

Geographical Information Systems (GIS) introduce methods and environments to visualize, manipulate, and analyze geospatial data. The nature of the geographical applications requires seamless integration and sharing of spatial data from a variety of providers. Interoperability of the services across organizations and providers is the main goal for GIS.

2. What is OGC?

The Open Geospatial Consortium (OGC) defines a number of standards (both for data models and for online services) that have been widely adopted in the Geographical Information System (GIS) community. OGC is a non-profit, international standards organization that is leading the development of standards for geographic data related operations and services. OGC has variety of contributors from different areas such as private industry and academia to create open and extensible software application programming interfaces for GIS.

The most important OGC specifications are WMS, WFS, WCS and GML.

3. What are the main OGC services in order to create the simplest GIS?

In simplest case, you should have a database (MySQL, Oracle or any other), Web Feature Server providing data from this database and Web Map Server to create map images by rendering data coming from WFS or/and any other WMS. See the below figure for the simplest representation of an OGC compatible GIS systems.

4. Is there any relation between ISO (International Standard Organization) and OGC?
OGC has a close relationship with ISO/TC 211 (Geographic Information/Geomatics). The OGC abstract specification is being progressively replaced by volumes from the ISO 19100 series under development by this committee. Further, the OGC standards Web Map Service and Simple Features are ISO standards. GML will soon be approved as an ISO standard.

5. What is WMS?
Web Map Service produces maps from geographic data. A map is not the data itself. Maps create information from raw geographic data. Maps are generally rendered in pictorial formats such as jpeg, GIF, png. WMS also produce maps from vector-based graphical elements in Scalable Vector Graphics (SVG).
WMS provides three operations protocols (GetCapabilities, GetMap, and GetFeatureInfo) in support of the creation and display of registered and superimposed map like views of information that come simultaneously from multiple sources that are both remote and heterogeneous.
Basically there are two types of WMS defined in the specifications. These are basic WMS and SLD-enabled WMS. For the basic WMS, there are three operations defined. These are getCapabilities, getMap, GetFeatureInfo.
WMS publishes its ability and data holdings in its capabilities document. This document is encoded in XML. WMS classifies its geographic data holdings in the “Layers” and gives information about the styles available for these Layers. Each layer can have sub layers and the sub layers can have different styling defined for them.
A Web Map Service is usually not invoked directly. More often, it is invoked by a client application that provides the user with interactive controls. This client application may or may not be web-based

6. What is WFS?
Web Feature Service (WFS) is an intermediary server sitting in front of the archived data kept in the Databases or files systems and, provides data manipulation operations.
Clients interact with a Web Feature Service by submitting database queries encoded in Open Geospatial Consortium Filter Encoding Implementation and in compliance with the Open Geospatial Consortium’s Common Query Language.
As a minimal requirement a basic Web Feature Service should be able to provide requested geographical information as Geographic Markup Language feature collections. However, more advanced versions also support “create, update, delete and lock operations”. Three operations must be supported by a basic Web Feature Service: GetCapabilities, DescribeFeatureType and GetFeature.

7. What is GML?
GML is an XML encoding for the transport and storage of geographic information, including both the spatial (attributes) and non-spatial (geometric) properties of geographic features. GML defines a data encoding in XML that allows geographic data and its attributes to be moved between disparate systems with ease XML encoding of geospatial information. It is human-readable and separates contents from presentation. Presentation is basically represented under the “geometry” tag elements.
If some one wants to define a GML schema for a feature, he should import the Geometry schema (geometry.xsd) the Feature Schema (feature.xsd) and the XLinks schema (xlinks.xsd) as base schemas.
WMS requests feature data from the WFS in the form of GML. WFS store and serve the geospatial data encoded in GML.

8. What is feature?
A feature is an abstraction of a real world phenomenon; it is a geographic feature if it is associated with location relatives to the earth such as faults, rivers, roads, lakes. All the feature instances should be created according to the schema files and encoded in GML.

9. What are the vector and raster data?
Vector data deals with discrete phenomena, each of which is conceived of as a feature. The spatial characteristics of a discrete real world phenomenon are represented by a set of one or more geometric primitives (points, curves, surfaces, or solids). Other characteristics of the phenomenon are recorded as feature attributes. Usually, a single feature is associated with a single set of attribute values.
Raster data deals with real world phenomena that vary continuously over space. It contains a set of values, each associated with one of the elements in a regular array of points or cells. It is usually associated with a method for interpolating values at spatial positions between the points or within the cells.

10. What is coverage data?
OGC uses the term “coverage” to refer to any data representation that assigns values directly to spatial position. Coverage is a feature that associates positions within a bounded space (its spatiotemporal domain) to feature attribute values (its range). Examples include a raster image, a polygon overlay, or a digital elevation matrix. The spatio-temporal domain of coverage is a set of geometric objects described in terms of direct positions. Commonly used spatio-temporal domains include point sets, grids, collections of closed rectangles, and other collections of geometric objects.

11. What is spatial and geo-spatial data?
Spatial data are a kind of data that pertains to the space occupied by objects. Example spatial data from the real world are cities, rivers, roads, states, crop coverage, mountain ranges etc. In the implementation these are represented by points, lines, rectangles, surfaces, volumes and etc. Spatial data have some common characteristics. These type of data are geometric data and in high dimensions. These data can be either discrete (vector) or continuous (raster). GIS applications are applied on these types of data.
Geospatial data are spatial data associated with a location relative to the Earth.

12. What is the term “capability” in OGC used for?
Each GIS service defined in OGC has a capability file encoded in XML. There is no common format for the capability valid across all the OGC defined services. All the OGC services has getCapability service interface in order to return capabilities metadata. Capabilities function as “service” metadata, providing information about what the service offers. Clients to the services determine whether they can work with that server based on its capabilities.