project logo

pycsw Quickstart

pycsw is an OGC CSW server implementation written in Python.

pycsw implements clause 10 (HTTP protocol binding (Catalogue Services for the Web, CSW)) of the OpenGIS Catalogue Service Implementation Specification, version 2.0.2. Initial development started in 2010 (more formally announced in 2011).

pycsw allows for the publishing and discovery of geospatial metadata. Existing repositories of geospatial metadata can be exposed via OGC:CSW 2.0.2.

pycsw is Open Source, released under an MIT license, and runs on all major platforms (Windows, Linux, Mac OS X).

pycsw is installed by default on the OSGeo-Live DVD. This QuickStart will describe how to:

  • perform a fresh installation of pycsw
  • test pycsw installation through the tester application
  • edit pycsw’s Capabilities document and display this information in a browser
  • create ISO 19139 XML metadata records (through INSPIRE Metadata Editor or GIMED)
  • load the created XML metadata files in pycsw database
  • install QGIS CSW Client plugin
  • perform discovery of data through the pycsw Catalogue Service within QGIS

Installation

System Requirements

pycsw requires the following supporting libraries:

  • lxml (version >= 2.2.3) for XML support
  • SQLAlchemy (version >= 0.0.5) for database bindings
  • Shapely (version >= 1.2.8) for spatial query / geometry support
  • pyproj (version >= 1.8.9) for coordinate transformations

Installing from Source

Download pycsw the latest version or fetch svn trunk:

$ svn co https://pycsw.svn.sourceforge.net/svnroot/pycsw pycsw

Ensure that CGI is enabled for your install directory. For example, on Apache, if you setup pycsw in /srv/www/htdocs/pycsw (where your URL will be http://host/pycsw/csw.py), add the following to httpd.conf:

<Location /pycsw/>
 Options FollowSymLinks +ExecCGI
 Allow from all
 AddHandler cgi-script .py
</Location>

Installing on Ubuntu/Xubuntu/Kubuntu

In order to install pycsw to an Ubuntu based distribution, one can use the OSGeo-Live installation script located in pycsw/etc/dist:

$ cd pycsw/etc/dist
$ sudo ./install_pycsw.sh

The script installs the dependencies (Apache, lxml, sqlalchemy, shapely, pyproj) and then pycsw to /var/www.

Installing on openSUSE

In order to install the OBS package in openSUSE 12.1, one can run the following commands as user root:

$ zypper -ar http://download.opensuse.org/repositories/Application:/Geo/openSUSE_12.1/ GEO
$ zypper -ar http://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_12.1/ python
$ zypper refresh
$ zypper install pycsw

For earlier openSUSE versions change 12.1 with 11.4. For future openSUSE version use Factory. Rolling version Tumbleweed is also supported. pycsw is included in the official Application::Geo repository of OpenSUSE Build Service.

Tester Application

To run the pycsw tester, use the pycsw launcher from the Web Services group, or open Firefox and navigate to http://localhost/pycsw/tester/index.html:

../../_images/pycsw_tester_startup.png

By selecting the left drop-down list, the user can see various predefined POST requests, encoded as XML, that can be sent to pycsw:

../../_images/pycsw_tester_selection.png

For example, by selecting “apiso/DescribeRecord”, and pushing the “Send” button, a description of the ISO Application Profile record is presented on the right panel:

../../_images/pycsw_tester_describe_apiso_record.png

By selecting “GetCapabilities-SOAP” and pushing the “Send” button, a SOAP request is sent to the server to advertise their web service capabilities:

../../_images/pycsw_tester_soap_capabillities.png

Also, the user can search for data records, performing a spatial bounding box query, by selecting “GetRecords-filter-bbox” and editing the coordinates in the XML request:

../../_images/pycsw_tester_getrecords_bbox_filter.png

The user can go through all the available requests and perform various requests from this testing application. A more robust and graphical way to perform such queries is described in a following chapter of this tutorial.

Capabilities Document and Configuration

The capabilities of the pycsw installation can be found at http://localhost/pycsw/csw.py?service=CSW&version=2.0.2&request=GetCapabilities.

../../_images/pycsw_getcapabilities_response.png

To edit the web service metadata, included in the capabilities document, the user can modify the file /var/www/pycsw/default.cfg under the tag [metadata:main]. If the user wants to enable INSPIRE Discovery Service support, the metadata under the tag [metadata:inspire] has to be filled in and the enabled property has to be set to true.

For full functionality and configuration options, please consult the documentation on the pycsw website.

Metadata Creation

Using European Open Source Metadata Editor (EUOSME)

Metadata files for a spatial dataset or a spatial web service can be created through the open source implementation of inspire directive for metadata, European Open Source Metadata Editor (EUOSME). This application can be found at http://inspire-geoportal.ec.europa.eu/editor/. Source code is available at https://joinup.ec.europa.eu/svn/euosme/trunk

../../_images/pycsw_euosme_homepage.png

The user fills the mandatory metadata fields going through the application tabs, adding information like the name of the data owner, keywords, resource location on the web, geographic location (using a bounding box or the name of a country) etc.

../../_images/pycsw_euosme_metadata_input.png ../../_images/pycsw_euosme_metadata_bbox.png

After the user has added all the information available, must push the validation button on top of the page, so that a check to be performed for consistency with the INSPIRE directive.

../../_images/pycsw_euosme_save_metadata.png

After a sucessful validation, the XML file can be saved to the local disk and viewed through a text editor or a browser.

../../_images/pycsw_euosme_xml.png

Using GIMED

GIMED is a Free Software for creating and editing INSPIRE compliant metadata XML files. To install the application, use the following commands from a terminal:

$ wget http://downloads.sourceforge.net/project/gimed/GIMED_v1.3.0.tar.gz
$ tar zxvf GIMED_v1.3.0.tar.gz
$ mono ./GIMED_v1.3.0/bin/GIMED.exe

The user can follow the same workflow as in EUOSME to add and edit metadata, without the need for an internet connection.

../../_images/pycsw_gimed.png

Loading Metadata

Metadata Repository Setup

pycsw supports the following databases:

  • SQLite3
  • PostgreSQL
  • MySQL

In OSGeo-Live, a simple SQLite setup has been implemented. The database can be found in /var/www/pycsw/data/cite

In case of a new db, the setup process is:

$ cd /path/to/pycsw
$ export PYTHONPATH=`pwd`
$ python ./sbin/setup_db.py sqlite:////path/to/records.db

Importing Metadata

$ cd /path/to/pycsw
$ export PYTHONPATH=`pwd`
$ python ./sbin/load_records.py /path/to/records sqlite:////path/to/records.db

This will import all *.xml records from /path/to/records into records.db and configure the repository to expose queryables as per Table 53 of OGC:CSW.

../../_images/pycsw_import_metadata.png

QGIS CSW Client Installation

The HTTP request/response mechanism is not friendly enough to the end user in order to perform queries to the Catalogue Service. For this reason, there are several CSW Clients in the form of web applications like the INSPIRE Geoportal or GeoNetwork. For this tutorial, we will use the QGIS OGC Catalogue Service Client plugin.

To install the plugin:

$ cd ~
$ cd .qgis/python/plugins
$ svn co https://qgiscommunitypl.svn.sourceforge.net/svnroot/qgiscommunitypl/python/plugins/qgcsw/trunk qgcsw

Data Discovery

QGIS CSW Plugin

Start QGIS from the Desktop GIS group and go to “Manage Plugins”

../../_images/pycsw_qgis_plugin.png

Enable the CSW plugin from the list

../../_images/pycsw_qgis_plugin_enable.png

Then select the CSW button from the toolbar and launch CSW Client

../../_images/pycsw_qgis_csw_plugin_open.png

Add the pycsw server by pressing the “New” button and type in http://localhost/pycsw/csw.py

../../_images/pycsw_qgis_csw_plugin_add_server.png

The user can add some default servers using the “Add default servers” button and also get the capabilities of the server using “Server info” button

../../_images/pycsw_qgis_csw_plugin_server_info.png

Perform search using the catalogue, either by a string value or with a bounding box

../../_images/pycsw_qgis_csw_plugin_search.png

Tester Application

Discovery of data can be also performed through the Tester application by setting the appropriate requests eg. any text search with the string “imagery” leads to discovering the xml data created previously by EUOSME

../../_images/pycsw_tester_discovery.png

For more information on pycsw, please consult the documentation on the pycsw website.

Copyright & Disclaimer