1. Prerequisites

1.1 MySQL

If the mysql-server and mysql-client packages have not been installed on the deployment system, install them now via apt-get. When prompted for a password by the installer, set a new password for the root user.

Deployable artifacts with default configuration for both the network node and annotation processor can be obtained from the sourceforge downloaded site (links available below in later steps), otherwise make sure that the subversion and maven2 packages are installed via apt-get on the development machine (the machine on which you plan on building FilteredPush) if you are planning on building the deployable artifacts. With maven installed you will also have to configure the settings.xml file containing the FilteredPush maven repository (see the following for more details).

Use svn to check out and deploy the FP-Medium-SCAN configuration to its default location in /etc/filteredpush:

Create the messagestore database and user. Grant the messagestore user appropriate privileges on the messagestore database:

Next, create the fpannotationprocessor database and the fpweb user. Grant the fpweb user all privileges on the fpannotationprocessor database:

Exit mysql and import the schema dump found at /etc/filteredpush/schema/sql/messagestore.sql into the messagestore database.

Also import the fpannotationprocessor sql dump found at /etc/filteredpush/schema/sql/fpannotationprocessor.sql into the fpannotationprocessor database:

1.2 MongoDB

Install mongodb via apt-get install and start the server as a service:

See the following page for more information about setting up access control with an admin account: http://docs.mongodb.org/manual/core/access-control/

1.3 Fedora

Install fedora (see http://fedora-commons.org/)

2. Jena Fuseki SPARQL Endpoint

2.1. Installing Fuseki

Additional documentation can be found at: http://jena.apache.org/documentation/serving_data/index.html

Download and extract the latest version of the Jena Fuseki sparql endpoint from http://www.apache.org/dist/jena/binaries/ (look for the latest version of jena-fuseki-0.x.x-distribution.tar.gz).

Move the extracted Fuseki directory to the installation directory /usr/share/fuseki/jena-fuseki-0.2.5:

Use the assembler config (uses TDB as the triple store) provided in /etc/filteredpush/misc/tdb-assembler.ttl by copying it to the Fuseki directory you just extracted to:

The example fuseki startup script can be found at /etc/filteredpush/misc/fuseki. If necessary, edit the file so that FUSEKI_HOME is set to the directory you installed fuseki to. Also make it executable via chmod and use update-rc.d to install it as a startup script.

NOTE: On Debian I reveived the following warning: insserv: warning: script 'fuseki' missing insserv: warning: script 'fuseki' missing LSB tags and overrides and overrides. If you want to add the LSB tags that the warning refers to you can find more info here http://wiki.debian.org/LSBInitScripts/

Start fuseki and visit http://localhost:3030/ in your browser:

2.2. SPARQL Query Primer

If you need to execute SPARQL queries on Fuseki manaully you can do so from the web interface. First select the Control Panel link on the main page (under Server Management) and select the "/AnnotationStore" dataset (the dataset is supplied as an argument when the startup script in /etc/init.d starts fuseki and the files are stored in the directory configured in the tdb-assembler.ttl file in FUSEKI_HOME). On the page that follows, you can launch sparql queries, updates or upload rdf/xml. Some useful queries are listed below.

In the query textarea:

in the update textarea:

For the queries, selecting XML from the Output dropdown on the form will probably give the most human readable results. Additionally, you may specify an xslt file for styling the xml result for browser viewing (a value of /xml-to-html.xsl is supplied by default, this file is located in your FUSEKI_HOME/pages directory).

It may also be useful to enable logging to a file in the log4j.properties for fuseki. If you wish to do this now, uncomment the following lines in the log4j.properties file (located at /usr/share/fuseki/jena-fuseki-0.2.5/log4j.properties):

Create the logs directory in /usr/share/fuseki/jena-fuseki-0.2.5:

2.3 Securing Fuseki

The deployment and configuration steps presented in the previous sections do not setup access restrictions on the Fuseki triplestore. Both the update and query endpoints are exposed to external clients. If you want to restrict access to fuseki from outside of localhost you can configure Fuseki (which is running in a bootstrapped Jetty servlet container) to only accept connections from localhost on a specified port. Provided that SparqlPuSH or the AccessPoint is installed on the same host, only the network node will have access to the triplestore. External clients must be authenticated by SparqlPuSH/AccessPoint before the updates are made to the triplestore.

If you would like to open access to the Fuseki query endpoint (i.e. http://localhost:3030/AnnotationStore/query) but not upload or updates you can configure mod_proxy in apache to allow access to the endpoints selectively. The steps for this are detailed below:

Obtain the jetty config for Fuseki from /etc/filteredpush/misc/jetty.xml and put this in the FUSEKI_HOME directory (/usr/share/fuseki/jena-fuseki-0.2.5/). Make edits if necessary but the defaults should restrict access to localhost on port 3030:

More detailed info about the jetty configuration can be found at http://wiki.eclipse.org/Jetty/Reference/jetty.xml

Stop the Fuseki server if it is currently running and edit the startup script (/etc/init.d/fuseki). Add the --jetty-config=jetty.xml option to the invocation of fuseki-server

Restart fuseki and confirm that external access is not possible (http://<hostname>:3030).</hostname>

If you only wish for applications running on localhost to invoke the fuseki (via query, upload, update) then you can stop here. Otherwise, if you would like to expose only certain endpoints while restricting others (for example you want to allow queries from the outside but no changes/additions to data), you can follow the rest of the configuration steps below.

The following steps are based on the documentation found at http://wiki.eclipse.org/Jetty/Tutorial/Apache

First enable mod_proxy in apache via the following:

This should have created a proxy.conf file in /etc/apache2/mods-enabled. This file contains some default configuration, edit it and replace everything within <IfModule>...</IfModule> with the following:

Next, add the following line to the default site in /etc/apache2/sites-available and make sure it is enabled (via a2ensite default):

Restart apache and check that outside access to the query interface is enabled (http://<hostname>/fuseki/AnnotationStore/query should give an error from fuseki since no query is specified in the request. If you get an error from apache instead this means that it is still not accessible)

With outside access no longer possible you may not be able to get at the Fuseki control panel to test queries and updates. You could run the ruby scripts (located in your FUSEKI_HOME) provided by the Jena Fuseki project on the localhost instead (see http://jena.apache.org/documentation/serving_data/index.html#script-control for how to use the scripts. also see http://jena.apache.org/documentation/serving_data/soh.html for more info.)

3. GlassFish Application Server

3.1. Installation

Make sure that the version of Java that your system is using is the Oracle Java SE 6 JDK and not OpenJDK (check this in a terminal window with java -version). If you need to install the Oracle JDK, download Java 6 from the Oracle site and use the update-alternatives utility (see Installing Oracle Java SE 6 for more details)

Create the glassfish user and group. Add the current user along with any other users that should have glassfish admin privileges to the new group.

Log out and log in again with the current user to update the groups. Next, download the Glassfish 3.1.2.2 zip file from http://www.oracle.com/technetwork/middleware/glassfish/overview/index.html. Unzip this file and move the glassfish3 directory to /usr/share/:

Copy the JDBC MySQL driver included with glassfish (mysql-connector-java-commercial-5.1.14-bin.jar) to the glassfish/lib directory:

Change ownership of the glassfish directory to user glassfish and group glassfish:

Copy the example startup script for glassfish from /etc/filteredpush/misc/glassfish to /etc/init.d/ and make the script executable via the chmod command.

Edit the script and make sure that the JAVA_HOME variable is set to point to the base directory of the Oracle JDK 6. Once you have finished editing the script, install it using the update-rc.d command in terminal. This command will allow the script to automatically start the server on boot and stop it during shutdown:

Start Glassfish via the startup script. The first time you start Glassfish you will be prompted to set the administrative password for the domain:

If you plan on using the admin control panel remotely via a web browser, you must configure the admin user and password and secure access via the asadmin utillity from shell:

3.2 Configuration

Once glassfish has started, navigate to the admin control panel at http://localhost:4848 in a browser window. Enter your admin username and password (default username is admin).

On the left side of the control panel page select Resources > JDBC > JDBC Connection Pools and create a new connection pool with the name "mysql_messagestore". Select "java.sql.ConnectionPoolDataSource" as the Resource Type and "MySQL" as the Database Driver Vendor from the dropdowns before clicking next.

In step two, you should configure the User, Password, DatabaseName, URL and Url properties for the messagestore database:

Once you have done this click the finish button. Select the connection pool again from the list and on the edit page click the "Ping" button to test your configuration (you should get a "Ping Succeeded message").

Now from the list to the left of the control panel, select Resources > JDBC > JDBC Resources and create a new resource with the name "jdbc/mysql_messagestore". Select the mysql_messagestore connection pool you just created from the "Pool Name" dropdown box and click OK to finish.

NOTE: if you end up using a different name for your connection pools you must edit the persistence.xml file in the src/main/resources/META-INF directory of the projects that use JPA.

Now we repeat the process for the fpannotationprocessor database and fpweb user. Create a new connection pool with the name "mysql_fpannotationprocessor" with Resource Type "java.sql.ConnectionPoolDataSource" and Database Driver Vendor "MySQL" in Resources > JDBC > JDBC Connection Pools. Configure the properties on the next page using the following values:

After clicking the Finish button, select the connection pool from the list and click "Ping" to test the connection.

Create a new JDBC resource with the name "jdbc/fpannotationprocessor" in Resources > JDBC > JDBC Resources and select the mysql_fpannotationprocessor connection pool from the "Pool Name" dropdown before clicking OK to finish.

4. Deploying FP-EAR

Obtain the latest copy of the FP-EAR.ear file from the sourceforge downloads page or by building the FP-JavaSOA project according to the development documentation here.

Login to the Glassfish admin control panel by opening the following link in a browser window: http://localhost:4848/

Once you have logged in, click the "Deploy an Application" option under Deployment in Common Tasks or select Applications on the left and click the Deploy button in the view that appears.

Select the "Packaged File to Be Uploaded to the Server" option if you are uploading and deploying the ear file (easier for deployments on localhost) and click "Choose File" to select the FP-EAR.ear file.

Otherwise, if you have uploaded the ear file to the deployment target machine already (using sftp for example), select the "Local Packaged File or Directory That Is Accessible from GlassFish Server" and click "Browse Files" to open up the Glassfish file chooser and browse to the location of the ear file.

Click OK, deployment may take a little while (especially if you are uploading the file via the web browser). Once deployed, the Access Point, Messaging, Knowledge, Analysis, Annotation Generator, and Annotation Processor components will all be deployed. The wsdl for the Access Point soap webserice (the entry point to filteredpush) is hosted at the following location (hostname and port values given are specific to localhost deployments): localhost:8081/FPNetworkAccessPointService/FPNetworkAccessPoint?wsdl

5. Deploying Annotation Processor in Tomcat (Optional)

If you are installing the Annotation Processor web application alongside an instance of Specify on a different host than that of the accesspoint and network node, you may want to deploy the annotation processor to Tomcat instead of Glassfish.

Otherwise, you can skip ahead to the next set of instructions and configure the instance of the Annotation Processor that was deployed with the EAR in previous steps.

Install Tomcat on the Annotation Processor host via apt-get install

Obtain the stand-alone FP-AnnotationProcessor-Web war from the sourceforge download site or by configuring and building the module according to the development instructions.

Once you have built the war file via maven, copy it from the target directory of the FP-AnnotationProcessor-Web project to the tomcat webapps directory to deploy.

Restart Tomcat and browse to the annotation processor homepage http://localhost:8080/FP-AnnotationProcessor-Web/pages/home.faces and login with the default username and password, "t" and "tp".

6. Configuring Oauth for Map to Local

6.1. Deploy specifyweb and fpmapperweb

The following instructions cover how to configure oauth for the map to local data feature of the Annotation Processor using Specify. This requires that you have the latest version of Specify installed: http://specifysoftware.org/. These instructions also assume that you have following all of the previous deployment steps necessary for FP-Medium.

First obtain and deploy the fpmapperweb and specifyweb war files to tomcat. These files are hosted on the sourceforge site at the following url:

https://sourceforge.net/projects/filteredpush/files/oauth/

Once the web applications have been deployed you should see them unpacked to the fpmapperweb and specifyweb directories located in the Tomcat webapps directory (i.e. /var/lib/tomcat6/webapps)

6.2. Configure the Consumer (fpmapperweb)

Edit the consumer.properties file in the following location:

Enter the url to the specifyweb application deployed in Tomcat:

Use the following defaults for the properties listed below:

Enter a value for consumer key and consumer secret

6.3. Configure the Producer (specifyweb)

Now edit the hibernate config in specifyweb:

Use the default value for dialect and driver

Set the values for the specify database url, username and password

NOTE: You may need to edit ./specifyweb/WEB-INF/classes/org/filteredpush/specify/auth/jdbc.properties and set the dbuser and dbpassword properties

Now edit the provider.properties to match consumer.properties in fpmapperweb:

Add (or update) the entry of the form consumerKey=consumerSecret (using the values you entered previously in consumer.properties)

View http://localhost:8080/fpmapperweb/ in a browser and click the Say Hello to Specify link to test oath and specify. Type in your specify username and password, you should now see the following message:

Hi, I'm the Specify "Hello Servlet." I was called like this: ...

7. Client Helper Libraries and Configuration

7.1 Client Authentication

Generate public/private key pairs for each client you wish to be authorized to load annotations into the triple store:

The above will generate two files: privkey.pem (which should reside with the client and be placed somewhere outside the directory root of the server) and newcert.pem (contains public key, a copy of this should be stored on the same server as SparqlPuSH somewhere the application can access it)

Create the keystore and move it to /etc/filteredpush/auth. The name option in the command below corresponds to the alias for the keypair in the keystore.:

The annotation processor config.properties file (found in /usr/share/glassfish3/glassfish/domains/domain1/applications/FP-EAR/FP-AnnotationProcessor-Web-0.0.1-SNAPSHOT_war/WEB-INF/classes or /var/lib/tomcat7/webapps/FP-AnnotationProcessor-Web/WEB-INF/classes/ ) contains a pkAlias property that references the private key alias (the argument for the name option when creating the keystore or adding a keypair to the pkcs12 file) for the Annotation Processor's client authorization key. Change this property to match the alias for the Annotation Processor in the keystore. The Annotation Processor will use this key to sign all FPMessages it sends to the accesspoint.

FP-Core contains the keystore.properties file used by the AccessPoint when authenticating clients. Update the value for keystore.password with the password to the keystore (when creating the keystore for the first time you are prompted to enter a password).

7.2 PHP Client Libraries

The FilteredPush libraries for php clients can be checked out from the sourceforge svn:

Clients must be configured (via edits to fp/FPConfig.php) to use the network components. Edit this file and set the X509_CERTIFICATE, PRIVATE_KEY and NETWORK_FACADE properties.

X509_CERTIFICATE should be set to the file path of the certificate (pem file) containing the public key for the client and PRIVATE_KEY should be set to the file path of the pem file containing the client's private key. Set the NETWORK_FACADE property to either FPLiteFacade (for FP-Lite deployments) or FPMediumFacade (for FP-Medium deployments).

The rest of the defaults should work with the default single-node deployment of FilteredPush. A summary of all the configuration options can be found below:

• RDFHANDLER_ENDPOINT - the url for the annotation webservice, used for creating new identification annotation rdf/xml
• FPNODE_ENDPOINT - this is the AccessPoint SOAP webservice as part of FP-Medium (for an FP-Lite deployment the default can be used for now)
• SPARQLPUSH_SERVER - sparqlpush server uri
• SPARQLPUSH_CLIENT - sparqlpush client uri
• DS - the dataset that fuseki was started with in the startup script
• SPARQL_ENDPOINT - the uri to the fuseki endpoint
• RESULT_XSLT - the xsl for styling query results and the annotations shown on the Annotations tab in Symbiota (on the Occurrence Record form).
• X509_CERTIFICATE - the pem file (newcert.pem from the example above) that contains the public key for this client
• PRIVATE_KEY - the pem file used by this client (privkey.pem) for signing the rdf/xml
• NETWORK_FACADE - Current network implementation to use (see classes/fp/facades), choices are FPLiteFacade and FPMediumFacade

Once configured, we can deploy the fp directory containing the config and the libraries in FP-PHP-Library/fp to /usr/share/php. Clients (such as Symbiota and Morphbank) will use these libraries when interacting with both FP-Medium and FP-Lite