1. Prerequisites

1.1 MySQL

If the mysql-server and mysql-client packages have not been installed on the deployment system, install them now via apt-get. When prompted for a password by the installer, set a new password for the root user.

$ sudo apt-get install mysql-server mysql-client

Deployable artifacts with default configuration for both the network node and annotation processor can be obtained from the sourceforge downloaded site (links available below in later steps), otherwise make sure that the subversion and maven2 packages are installed via apt-get on the development machine (the machine on which you plan on building FilteredPush) if you are planning on building the deployable artifacts. With maven installed you will also have to configure the settings.xml file containing the FilteredPush maven repository (see the following for more details).

Use svn to check out and deploy the FP-Medium-SCAN configuration to its default location in /etc/filteredpush:

$ sudo svn checkout svn://svn.code.sf.net/p/filteredpush/svn/trunk/FP-Configuration/FP-Medium-SCAN/config/filteredpush /etc/filteredpush

Create the messagestore database and user. Grant the messagestore user appropriate privileges on the messagestore database:

CREATE DATABASE messagestore; CREATE USER 'messagestore'@'localhost' IDENTIFIED BY 'password'; GRANT ALL PRIVILEGES ON messagestore.* TO 'messagestore'@'localhost';

Next, create the fpannotationprocessor database and the fpweb user. Grant the fpweb user all privileges on the fpannotationprocessor database:

mysql> CREATE DATABASE fpannotationprocessor; mysql> CREATE USER 'fpweb'@'localhost' IDENTIFIED BY 'password'; mysql> GRANT ALL PRIVILEGES ON fpannotationprocessor.* TO 'fpweb'@'localhost';

Exit mysql and import the schema dump found at /etc/filteredpush/schema/sql/messagestore.sql into the messagestore database.

$ mysql -umessagestore -p -D messagestore < /etc/filteredpush/schema/sql/messagestore.sql Enter password:

Also import the fpannotationprocessor sql dump found at /etc/filteredpush/schema/sql/fpannotationprocessor.sql into the fpannotationprocessor database:

$ mysql -ufpweb -p -D fpannotationprocessor < /etc/filteredpush/schema/sql/fpannotationprocessor.sql

1.2 MongoDB

Install mongodb via apt-get install and start the server as a service:

$ sudo apt-get install mongodb $ sudo service mongodb start

See the following page for more information about setting up access control with an admin account: http://docs.mongodb.org/manual/core/access-control/

1.3 Fedora

Install fedora (see http://fedora-commons.org/)

2. Jena Fuseki SPARQL Endpoint

2.1. Installing Fuseki

Additional documentation can be found at: http://jena.apache.org/documentation/serving_data/index.html

Download and extract the latest version of the Jena Fuseki sparql endpoint from http://www.apache.org/dist/jena/binaries/ (look for the latest version of jena-fuseki-0.x.x-distribution.tar.gz).

Move the extracted Fuseki directory to the installation directory /usr/share/fuseki/jena-fuseki-0.2.5:

$ tar -xzvf jena-fuseki-0.2.5-distribution.tar.gz $ sudo mkdir /usr/share/fuseki $ sudo mv jena-fuseki-0.2.5 /usr/share/fuseki

Use the assembler config (uses TDB as the triple store) provided in /etc/filteredpush/misc/tdb-assembler.ttl by copying it to the Fuseki directory you just extracted to:

$ cp /etc/filteredpush/misc/tdb-assembler.ttl /usr/share/fuseki/jena-fuseki-0.2.5/

The example fuseki startup script can be found at /etc/filteredpush/misc/fuseki. If necessary, edit the file so that FUSEKI_HOME is set to the directory you installed fuseki to. Also make it executable via chmod and use update-rc.d to install it as a startup script.

NOTE: On Debian I reveived the following warning: insserv: warning: script 'fuseki' missing insserv: warning: script 'fuseki' missing LSB tags and overrides and overrides. If you want to add the LSB tags that the warning refers to you can find more info here http://wiki.debian.org/LSBInitScripts/

$ sudo cp /etc/filteredpush/misc/fuseki /etc/init.d/ $ sudo chmod +x /etc/init.d/fuseki $ sudo update-rc.d fuseki defaults

Start fuseki and visit http://localhost:3030/ in your browser:

/etc/init.d/fuseki start

2.2. SPARQL Query Primer

If you need to execute SPARQL queries on Fuseki manaully you can do so from the web interface. First select the Control Panel link on the main page (under Server Management) and select the "/AnnotationStore" dataset (the dataset is supplied as an argument when the startup script in /etc/init.d starts fuseki and the files are stored in the directory configured in the tdb-assembler.ttl file in FUSEKI_HOME). On the page that follows, you can launch sparql queries, updates or upload rdf/xml. Some useful queries are listed below.

In the query textarea:

1) SELECT * {?s ?p ?o} (select all triples currently in the default un-named graph) 2) SELECT * WHERE { GRAPH ?g {?s ?p ?o} } LIMIT 10 (select all triples in any named graph besides those in the default graph)

in the update textarea:

1) CLEAR ALL (clear all triples from the triplestore) 2) CLEAR GRAPH <name of graph here> (clear all triples in any given named graph) 3) LOAD <http: //yourhost/data/somefile.rdf> (load all the triples from the rdf file hosted at the address specified)

For the queries, selecting XML from the Output dropdown on the form will probably give the most human readable results. Additionally, you may specify an xslt file for styling the xml result for browser viewing (a value of /xml-to-html.xsl is supplied by default, this file is located in your FUSEKI_HOME/pages directory).

It may also be useful to enable logging to a file in the log4j.properties for fuseki. If you wish to do this now, uncomment the following lines in the log4j.properties file (located at /usr/share/fuseki/jena-fuseki-0.2.5/log4j.properties):

log4j.rootLogger=INFO, FusekiFileLog ... log4j.appender.FusekiFileLog=org.apache.log4j.DailyRollingFileAppender log4j.appender.FusekiFileLog.DatePattern='.'yyyy-MM-dd log4j.appender.FusekiFileLog.File=logs/fuseki-log log4j.appender.FusekiFileLog.layout=org.apache.log4j.PatternLayout log4j.appender.FusekiFileLog.layout.ConversionPattern=%d{HH:mm:ss}%-5p %-20c{1} :: %m%n

Create the logs directory in /usr/share/fuseki/jena-fuseki-0.2.5:

$ sudo mkdir /usr/share/fuseki/jena-fuseki-0.2.5/logs

2.3 Securing Fuseki

The deployment and configuration steps presented in the previous sections do not setup access restrictions on the Fuseki triplestore. Both the update and query endpoints are exposed to external clients. If you want to restrict access to fuseki from outside of localhost you can configure Fuseki (which is running in a bootstrapped Jetty servlet container) to only accept connections from localhost on a specified port. Provided that SparqlPuSH or the AccessPoint is installed on the same host, only the network node will have access to the triplestore. External clients must be authenticated by SparqlPuSH/AccessPoint before the updates are made to the triplestore.

If you would like to open access to the Fuseki query endpoint (i.e. http://localhost:3030/AnnotationStore/query) but not upload or updates you can configure mod_proxy in apache to allow access to the endpoints selectively. The steps for this are detailed below:

Obtain the jetty config for Fuseki from /etc/filteredpush/misc/jetty.xml and put this in the FUSEKI_HOME directory (/usr/share/fuseki/jena-fuseki-0.2.5/). Make edits if necessary but the defaults should restrict access to localhost on port 3030:

$ sudo cp /etc/filteredpush/misc/jetty.xml /usr/share/fuseki/jena-fuseki-0.2.5/

More detailed info about the jetty configuration can be found at http://wiki.eclipse.org/Jetty/Reference/jetty.xml

Stop the Fuseki server if it is currently running and edit the startup script (/etc/init.d/fuseki). Add the --jetty-config=jetty.xml option to the invocation of fuseki-server

./fuseki-server --jetty-config=jetty.xml --desc=tdb-assembler.ttl --update /AnnotationStore 2>&1 &

Restart fuseki and confirm that external access is not possible (http://<hostname>:3030).</hostname>

If you only wish for applications running on localhost to invoke the fuseki (via query, upload, update) then you can stop here. Otherwise, if you would like to expose only certain endpoints while restricting others (for example you want to allow queries from the outside but no changes/additions to data), you can follow the rest of the configuration steps below.

The following steps are based on the documentation found at http://wiki.eclipse.org/Jetty/Tutorial/Apache

First enable mod_proxy in apache via the following:

$ sudo a2enmod proxy_http

This should have created a proxy.conf file in /etc/apache2/mods-enabled. This file contains some default configuration, edit it and replace everything within <IfModule>...</IfModule> with the following:

# Turn off support for true Proxy behaviour as we are acting as # a transparent proxy ProxyRequests Off # Turn off VIA header as we know where the requests are proxied ProxyVia Off # Turn on Host header preservation so that the servlet container # can write links with the correct host and rewriting can be avoided. ProxyPreserveHost On # Set the permissions for the proxy <Proxy *> AddDefaultCharset off Order deny,allow Allow from all </Proxy> # Turn on Proxy status reporting at /status # This should be better protected than: Allow from all ProxyStatus On <Location /status> SetHandler server-status Order Deny,Allow Allow from all </Location>

Next, add the following line to the default site in /etc/apache2/sites-available and make sure it is enabled (via a2ensite default):

ProxyPass /fuseki/AnnotationStore/query (if your dataset is not named AnnotationStore, replace this in both urls)

Restart apache and check that outside access to the query interface is enabled (http://<hostname>/fuseki/AnnotationStore/query should give an error from fuseki since no query is specified in the request. If you get an error from apache instead this means that it is still not accessible)

With outside access no longer possible you may not be able to get at the Fuseki control panel to test queries and updates. You could run the ruby scripts (located in your FUSEKI_HOME) provided by the Jena Fuseki project on the localhost instead (see http://jena.apache.org/documentation/serving_data/index.html#script-control for how to use the scripts. also see http://jena.apache.org/documentation/serving_data/soh.html for more info.)

3. GlassFish Application Server

3.1. Installation

Make sure that the version of Java that your system is using is the Oracle Java SE 6 JDK and not OpenJDK (check this in a terminal window with java -version). If you need to install the Oracle JDK, download Java 6 from the Oracle site and use the update-alternatives utility (see Installing Oracle Java SE 6 for more details)

Create the glassfish user and group. Add the current user along with any other users that should have glassfish admin privileges to the new group.

$ sudo adduser --system --no-create-home glassfish --group $ sudo usermod -a -G <user< glassfish

Log out and log in again with the current user to update the groups. Next, download the Glassfish zip file from http://www.oracle.com/technetwork/middleware/glassfish/overview/index.html. Unzip this file and move the glassfish3 directory to /usr/share/:

$ sudo unzip ogs- $ sudo mv glassfish3 /usr/share/

Copy the JDBC MySQL driver included with glassfish (mysql-connector-java-commercial-5.1.14-bin.jar) to the glassfish/lib directory:

$ sudo cp /usr/share/glassfish3/mysql-connector-java-commercial/mysql-connector-java-commercial-5.1.14-bin.jar /usr/share/glassfish3/glassfish/lib/

Change ownership of the glassfish directory to user glassfish and group glassfish:

$ sudo chown -R glassfish:glassfish /usr/share/glassfish3

Copy the example startup script for glassfish from /etc/filteredpush/misc/glassfish to /etc/init.d/ and make the script executable via the chmod command.

$ sudo cp /etc/filteredpush/glassfish /etc/init.d/ $ sudo chmod +x /etc/init.d/glassfish

Edit the script and make sure that the JAVA_HOME variable is set to point to the base directory of the Oracle JDK 6. Once you have finished editing the script, install it using the update-rc.d command in terminal. This command will allow the script to automatically start the server on boot and stop it during shutdown:

$ sudo update-rc.d glassfish defaults

Start Glassfish via the startup script. The first time you start Glassfish you will be prompted to set the administrative password for the domain:

$sudo /etc/init.d/glassfish start Starting Glassfish: This domain requires an administrative password to be set before the domain can be started. Please specify an administrative password. Enter an admin password for user "admin"

If you plan on using the admin control panel remotely via a web browser, you must configure the admin user and password and secure access via the asadmin utillity from shell:

$ sudo /usr/share/glassfish3/glassfish/bin/asadmin enable-secure-admin --port 4848

3.2 Configuration

Once glassfish has started, navigate to the admin control panel at http://localhost:4848 in a browser window. Enter your admin username and password (default username is admin).

On the left side of the control panel page select Resources > JDBC > JDBC Connection Pools and create a new connection pool with the name "mysql_messagestore". Select "java.sql.ConnectionPoolDataSource" as the Resource Type and "MySQL" as the Database Driver Vendor from the dropdowns before clicking next.

In step two, you should configure the User, Password, DatabaseName, URL and Url properties for the messagestore database:

User: messagestore Password: <password> DatabaseName: messagestore URL: jdbc:mysql://:3306/messagestore Url: jdbc:mysql://:3306/messagestore (note that there are two properties for the url)

Once you have done this click the finish button. Select the connection pool again from the list and on the edit page click the "Ping" button to test your configuration (you should get a "Ping Succeeded message").

Now from the list to the left of the control panel, select Resources > JDBC > JDBC Resources and create a new resource with the name "jdbc/mysql_messagestore". Select the mysql_messagestore connection pool you just created from the "Pool Name" dropdown box and click OK to finish.

NOTE: if you end up using a different name for your connection pools you must edit the persistence.xml file in the src/main/resources/META-INF directory of the projects that use JPA.

Now we repeat the process for the fpannotationprocessor database and fpweb user. Create a new connection pool with the name "mysql_fpannotationprocessor" with Resource Type "java.sql.ConnectionPoolDataSource" and Database Driver Vendor "MySQL" in Resources > JDBC > JDBC Connection Pools. Configure the properties on the next page using the following values:

User: fpweb Password: <password> DatabaseName: fpannotationprocessor URL: jdbc:mysql://:3306/fpannotationprocessor Url: jdbc:mysql://:3306/fpannotationprocessor

After clicking the Finish button, select the connection pool from the list and click "Ping" to test the connection.

Create a new JDBC resource with the name "jdbc/fpannotationprocessor" in Resources > JDBC > JDBC Resources and select the mysql_fpannotationprocessor connection pool from the "Pool Name" dropdown before clicking OK to finish.

4. Deploying FP-EAR

Obtain the latest copy of the FP-EAR.ear file from the sourceforge downloads page or by building the FP-JavaSOA project according to the development documentation here.

Login to the Glassfish admin control panel by opening the following link in a browser window: http://localhost:4848/

Once you have logged in, click the "Deploy an Application" option under Deployment in Common Tasks or select Applications on the left and click the Deploy button in the view that appears.

Select the "Packaged File to Be Uploaded to the Server" option if you are uploading and deploying the ear file (easier for deployments on localhost) and click "Choose File" to select the FP-EAR.ear file.

Otherwise, if you have uploaded the ear file to the deployment target machine already (using sftp for example), select the "Local Packaged File or Directory That Is Accessible from GlassFish Server" and click "Browse Files" to open up the Glassfish file chooser and browse to the location of the ear file.

Click OK, deployment may take a little while (especially if you are uploading the file via the web browser). Once deployed, the Access Point, Messaging, Knowledge, Analysis, Annotation Generator, and Annotation Processor components will all be deployed. The wsdl for the Access Point soap webserice (the entry point to filteredpush) is hosted at the following location (hostname and port values given are specific to localhost deployments): localhost:8081/FPNetworkAccessPointService/FPNetworkAccessPoint?wsdl

5. Deploying Annotation Processor in Tomcat (Optional)

If you are installing the Annotation Processor web application alongside an instance of Specify on a different host than that of the accesspoint and network node, you may want to deploy the annotation processor to Tomcat instead of Glassfish.

Otherwise, you can skip ahead to the next set of instructions and configure the instance of the Annotation Processor that was deployed with the EAR in previous steps.

Install Tomcat on the Annotation Processor host via apt-get install

$ sudo apt-get install tomcat7

Obtain the stand-alone FP-AnnotationProcessor-Web war from the sourceforge download site or by configuring and building the module according to the development instructions.

Once you have built the war file via maven, copy it from the target directory of the FP-AnnotationProcessor-Web project to the tomcat webapps directory to deploy.

$ cp FP-JavaSOA/FP-Modules/FP-AnnotationProcessor-Web/target/FP-AnnotationProcessor-Web.war /var/lib/tomcat7/webapps/

Restart Tomcat and browse to the annotation processor homepage http://localhost:8080/FP-AnnotationProcessor-Web/pages/home.faces and login with the default username and password, "t" and "tp".

6. Configuring Oauth for Map to Local

6.1. Deploy specifyweb and fpmapperweb

The following instructions cover how to configure oauth for the map to local data feature of the Annotation Processor using Specify. This requires that you have the latest version of Specify installed: http://specifysoftware.org/. These instructions also assume that you have following all of the previous deployment steps necessary for FP-Medium.

First obtain and deploy the fpmapperweb and specifyweb war files to tomcat. These files are hosted on the sourceforge site at the following url:


cp fpmapperweb.war /var/lib/tomcat6/webapps cp specifyweb.war /var/lib/tomcat6/webapps

Once the web applications have been deployed you should see them unpacked to the fpmapperweb and specifyweb directories located in the Tomcat webapps directory (i.e. /var/lib/tomcat6/webapps)

6.2. Configure the Consumer (fpmapperweb)

Edit the consumer.properties file in the following location:

vi ./fpmapperweb/WEB-INF/classes/net/oath/example/consumer/webapp/consumer.properties

Enter the url to the specifyweb application deployed in Tomcat:

filteredpush.serviceProvider.baseURL = http://localhost:8080/specifyweb/

Use the following defaults for the properties listed below:

filteredpush.serviceProvider.requestTokenURL = request_token filteredpush.serviceProvider.userAuthorizationURL = authorize filteredpush.serviceProvider.accessTokenURL = access_token filteredpush.serviceProvider.name = filteredpush

Enter a value for consumer key and consumer secret

filteredpush.consumerKey = myKey filteredpush.consumerSecret = mySecret

6.3. Configure the Producer (specifyweb)

Now edit the hibernate config in specifyweb:

vi ./specifyweb/WEB-INF/classes/hibernate.properties

Use the default value for dialect and driver

hibernate.dialect = org.hibernate.dialect.MySQLDialect hibernate.connection.driver_class = com.mysql.jdbc.Driver

Set the values for the specify database url, username and password

hibernate.connection.url = jdbc:mysql://localhost:3306/specify hibernate.connection.username = specify hibernate.connection.password = specify

NOTE: You may need to edit ./specifyweb/WEB-INF/classes/org/filteredpush/specify/auth/jdbc.properties and set the dbuser and dbpassword properties

Now edit the provider.properties to match consumer.properties in fpmapperweb:

vi ./specifyweb/WEB-INF/classes/net/oath/example/provider/core

Add (or update) the entry of the form consumerKey=consumerSecret (using the values you entered previously in consumer.properties)

View http://localhost:8080/fpmapperweb/ in a browser and click the Say Hello to Specify link to test oath and specify. Type in your specify username and password, you should now see the following message:

Hi, I'm the Specify "Hello Servlet." I was called like this: ...

7. Client Helper Libraries and Configuration

7.1 Client Authentication

Generate public/private key pairs for each client you wish to be authorized to load annotations into the triple store:

$ openssl req -x509 -nodes -newkey rsa:2048 -out newcert.pem -outform PEM -days 1825

The above will generate two files: privkey.pem (which should reside with the client and be placed somewhere outside the directory root of the server) and newcert.pem (contains public key, a copy of this should be stored on the same server as SparqlPuSH somewhere the application can access it)

Create the keystore and move it to /etc/filteredpush/auth. The name option in the command below corresponds to the alias for the keypair in the keystore.:

$ openssl pkcs12 -export -in newcert.pem -inkey privkey.pem -out keystore.p12 -name keystore $ sudo mv keystore.p12 /etc/filteredpush/auth

The annotation processor config.properties file (found in /usr/share/glassfish3/glassfish/domains/domain1/applications/FP-EAR/FP-AnnotationProcessor-Web-0.0.1-SNAPSHOT_war/WEB-INF/classes or /var/lib/tomcat7/webapps/FP-AnnotationProcessor-Web/WEB-INF/classes/ ) contains a pkAlias property that references the private key alias (the argument for the name option when creating the keystore or adding a keypair to the pkcs12 file) for the Annotation Processor's client authorization key. Change this property to match the alias for the Annotation Processor in the keystore. The Annotation Processor will use this key to sign all FPMessages it sends to the accesspoint.

FP-Core contains the keystore.properties file used by the AccessPoint when authenticating clients. Update the value for keystore.password with the password to the keystore (when creating the keystore for the first time you are prompted to enter a password).

7.2 PHP Client Libraries

The FilteredPush libraries for php clients can be checked out from the sourceforge svn:

$ svn checkout svn://svn.code.sf.net/p/filteredpush/svn/trunk/FP-PHP-Library FP-PHP-Library

Clients must be configured (via edits to fp/FPConfig.php) to use the network components. Edit this file and set the X509_CERTIFICATE, PRIVATE_KEY and NETWORK_FACADE properties.

X509_CERTIFICATE should be set to the file path of the certificate (pem file) containing the public key for the client and PRIVATE_KEY should be set to the file path of the pem file containing the client's private key. Set the NETWORK_FACADE property to either FPLiteFacade (for FP-Lite deployments) or FPMediumFacade (for FP-Medium deployments).

The rest of the defaults should work with the default single-node deployment of FilteredPush. A summary of all the configuration options can be found below:

Once configured, we can deploy the fp directory containing the config and the libraries in FP-PHP-Library/fp to /usr/share/php. Clients (such as Symbiota and Morphbank) will use these libraries when interacting with both FP-Medium and FP-Lite

$ cp -r FP-PHP-Library/fp /usr/share/php