DataCove Viewer Implementation

  • Author
    Katharina SCHLEIDT
  • up
    0 users have voted.

    Once the services for the INSPIRE Themes SU and PD were operational we designed a simple online viewer to test the usability of the services as specified by INSPIRE. This work was kicked off a DanubeHack2 in Bratislava in December of 2016, but could not be completed as the effort required was a bit more than the 2 days of the hackathon (~15 Person Days); it was finalized in the following weeks.

    The viewer allows the user to select a specific statistical measure, then further specify for which classifications under this measure the data should be provided (i.e. by gender, male or female or total). In addition, the user will be able to select multiple statistical measures and perform simple calculations between the values provided for these measures; this allows for one measure to be set in relation to a different one, i.e. number of asylum applications per population. This functionality was chosen as we believe it reflects the minimum requirements likely to be posed to INSPIRE Services by an application.

    The DataCove Statistical Viewer was implemented in JavaScript utilizing jQuery and AJAX; OpenLayers was used for the visualization of maps. The following stored queries were implemented on the PD service for this applications:

    • List of statistical measures: In order to provide the user with a selection list of possible statistical measures, we must know what statistical measures are available from this service. 
    • List of classifications and valid classification items per statistical measure, so the user can select which classification to display
    • List of featureIds for features corresponding to the selection criteria (statistical measure and year, optionally also country)

    Once the user has selected a statistical measure, together with the desired classifications and year, this request is passed on to the PD_Filter middleware, that accesses the data for the featureIds provided, filters out the data corresponding to the selected classifications, and returns this data in a simple JSON encoding.

    Once more than one statistical measure has been loaded into the system, the user is given the opportunity to specify simple arithmetic operations to be performed between statistical measures; examples are relativizing the number of asylum applications by the resident population, or comparing the correlation between Education vs. Unemployment. The individual statistical measures are assigned to variable names (sequential letters, a, b, c...), the same is done for aggregate measures stemming from previous calculations. The user can then enter a simple algebraic formula (allowed symbols: + - * / ( ) ), the calculation is performed for all spatial objects available from all statistical measures, and displayed as an additional measure.

    Various problems were encountered in the development process, the following list provides an overview of both the problems and workarounds used to enable implementation:

    • No way found to get list of distinct values for a specific field in WFS. While it would be possible to request a specific element (in this case the measure) from all available features using the WFS GetPropertyValue request, this does not provide distinct values, but rather repeats the same value for each corresponding feature instance available, requiring the client application to filter this mass of data down to singular values. The first workaround was creating a view based on a select distinct, and defining a simple feature based on this view. While this works for a query on the id, it is not possible to further filter this simple feature. We finally ended up creating a dedicated DB table for this purpose, and updating it whenever we add new data to the system.
    • Problems with the syntax for stored queries on complex features (slightly different than for simple features). For implementation, we just defined simple features based on the same tables as the complex ones. Once we were done we found the solution for stored queries on complex features - the full XPath including the base feature must be provided (simple features support relative XPaths)
    • Requests for multiple complex features crashes Geoserver - While getFeature works for multiple features (featureId=F1, F2), it will only do this a few times. Then no complex features are returned. Strange is that simple features and WMS continue to work.
      Workaround: request features individually
    • GeoServer restarts cause zombie DB connections; after a few restarts all DB connections are used, GeoServer behaviour becomes interesting ;)
      Current suggestions: use JNDI and reload App Schema config from the GeoServer GUI (or just restart everything on a regular basis)