GSIP 36

GSIP 36 - Resource - Publishing Split and Virtual Configuration

Overview

This proposal is deferred, as it is being done more incrementally, starting in gsips 66 and 67

Motivation Proposal Backwards Compatability Feedback Voting Links

Proposed By

Justin Deoliveira

Assigned to Release

The release that this proposal will be implemented for.

State

Under Writing, Under Discussion, In Progress, Completed, Rejected, Deferred

Motivation

The essence of this proposal is to decouple resources in geoserver from what gets published. In this context the term resource refers to a feature type or coverage, a shapefile, a postgis table, a geotiff, etc… The term published refers to exposing a resource via a service such as WFS, WMS, or WCS.

Currently the two concepts are tightly bound together. As soon as you add a new feature type that feature type is published via all services. This prevents the following use cases:

• Publishing a resource under one service but not another. An example here would be wanting to publish a resource via WMS, but wanting to turn WFS off to prevent access to the underlying attribution.

• Publishing the same resource multiple times under separate layers. An example here is publishing the same physical database table under different layers, perhaps intended for different services. And doing so in such as way that data such as connection pools, and locks are shared among the two layers.

A separate but related concept is the idea of “virtual configuration”. This idea refers to the ability to allow for multiple configurations within a single geoserver instance. In loose terms this is the equivalent of the mapfile for MapServer, and the ability to specify a mapefile on the fly in a request.

A couple of the use cases being:

• Running a development and a production configuration in the same server
• Grouping published data logically
• Limiting a service to subset of resources, ie limiting its capabilities document

Terminology

For this proposal it is necessary to establish some consistent terminology.

• resource - Physical dataset, feature type or coverage.
• store - Container for a resource, can be a database in the case of PostGIS, can be a file or a directory in the case of Shapefiles or GML.
• workspace - Top level container for stores, a mechanism for grouping data.
• layer - A published resource, published by a service. Contains all publishing information including styles, namespace uri, metadata like title and description, etc…
• map - A container for published layers
• profile - A virtual GeoServer configuration.

Proposal

This proposal is laid out into the following steps:

1. Add MapInfo
2. Add GeoServerProfile
3. UI work
4. Port services to LayerInfo
5. Implement thread-local catalog view

Add MapInfo

Currently the MapInfo interface is a dummy object in the catalog. It is there but does nothing. There are two major bits of work in this phase:

1. Beefing up the MapInfo interface to contain all the properties needed
2. Adding appropriate methods to Catalog

Currently MapInfo looks like:

interface MapInfo {

    /**
     * Id of the map.
     */
    String getId();

    /**
     * The name of the map.
     */
    String getName();

    /**
     * Flag indicating if the map is enabled.
     */
    boolean isEnabled();

    /**
     * The layers that compose the map.
     */
    List<LayerInfo> getLayers();
}

The following methods needs to be added:

/**
 * List of SRS's available for this map.
 */
List<String> getSRS();

/**
 * List of styles available for this map.
 */
List<StyleInfo> getStyles();

---

Wether these methods are actually needed is still under a bit of discussion.

---

After the MapInfo interface looks like how we want it, methods must be added to the Catalog to qualify layer lookups with a map. In other words, all getLayer…() methods need to be overloaded with methods that take MapInfo as a parameter.

---

If it is deemed that a Map should be container for styles as well, then similar steps will need to be taken for all getStyle…() methods as well.

---

Furthermore, the idea of a default map (similar to default workspace) must be added as well.

interface Catalog {

  /**
   * Default map
   */
  MapInfo getDefaultMap();
  void setDefaultMap( MapInfo map );

  /**
   * Layer lookup by map methods
   */
  LayerInfo getLayerByName( String mapName, String name );
  LayerInfo getLayerByName( MapInfo map, String name );
  List<LayerInfo> getLayersByMap( String mapName );
  List<LayerInfo> getLayersByMap( MapInfo map);
  List<LayerInfo> getLayersByResource( String mapName, ResourceInfo resource );
  List<LayerInfo> getLayersByResource( MapInfo map, ResourceInfo resource );
  List<LayerInfo> getLayersByStyle( String mapName, StyleInfo style );
}

Note that all the unqualified lookup methods for layer will stay in tact. They will be qualified by the default map.

Add GeoServerProfile

Current the GeoServer interface looks like:

interface GeoServer {
  /**
   * The global geoserver configuration.
   */
  GeoServerInfo getGlobal();

  /**
   * The logging configuration.
   */
  LoggingInfo getLogging();

  /**
   * GeoServer services.
   */
  Collection<? extends ServiceInfo> getServices();

  /**
   * The factory used to create configuration object.
   */
  GeoServerFactory getFactory();

  /**
   * The catalog.
   */
  Catalog getCatalog();

  /**
   * Returns all configuration listeners.
   */
  Collection<ConfigurationListener> getListeners();
}

This step involves coming up with a new interface GeoServerProfile and copying many of the methods from GeoServer to it, namely:

• getGlobal ()
• getLogging ()
• getServices ()

In addition, a profile will also contain a list of maps. This allows a user to specify which maps should be available when a particular profile is active.

• getMaps ()

interface GeoServerProfile {
 /**
   * The global geoserver configuration.
   */
  GeoServerInfo getGlobal();

  /**
   * The logging configuration.
   */
  LoggingInfo getLogging();

  /**
   * GeoServer services.
   */
  Collection<? extends ServiceInfo> getServices();

  /**
   * The maps for this profile.
   */
  List<MapInfo> getMaps();

}

GeoServer will then contain a list of profiles, and two “special” profiles. The default profile will be the fallback or global profile. The local profile will be a thread/context specific profile (see … for details).

interface GeoServer {
  /**
   * list of profiles
   */
  List<GeoServerProfile> getProfiles();

  GeoServerProfile getDefaultProfile();
  GeoServerProfile getLocalProfile();


  GeoServerProfile getActiveProfile();
}

The method getActiveProfile () is defined as follows:

public GeoServerProfile getActiveProfile() {
  if ( getLocalProfile() != null ) {
     return getLocalProfile();
  }

  return getDefaultProfile();
}

Basically any client code (service code, ui code, etc…) should always call getActiveProfile ().

UI Work

All the UI pages and functionality used to expose the new MapInfo and GeoServerProfile constructs in the UI. More details on this to come. But at this point this bit of work can go on in parallel to the rest of the steps.

Port services to LayerInfo

At this point the data structures are in place, but none of the rest of the system is using them, most importantly the services. This step involves de-coupling services from resources directly and instead working from layers.

The first part of this step is to move all metadata that currently lives on the ResourceInfo to the LayerInfo interface. This includes:

• ReferencedEnvelope boundingBox ();
• String getPrefixedName ();
• Name getQualifiedName ();
• List getAlias ();
• NamespaceInfo getNamespace ();
• String getTitle ();
• String getAbstract ();
• String getDescription ();
• List getKeywords ();
• List getMetadataLinks ();
• ReferencedEnvelope getLatLonBoundingBox ();
• ReferencedEnvelope getBoundingBox () throws Exception;
• String getSRS ();
• CoordinateReferenceSystem getCRS () throws Exception;
• ProjectionPolicy getProjectionPolicy ();

Then it is a matter of changing the services over to use the methods on LayerInfo. This is the biggest chunk of work in the process, but it mostly mechanical. All the methods that look up a resource are more or less mirrored by methods for looking up a layer.

Also to note is that most of the WMS service code goes though LayerInfo already to get to resource. So most of the porting for that module will be simply removing the call to the getResource () middle man.

Some examples:

WCS

Before:

CoverageInfo info = catalog.getCoverageByName(name);
String title = info.getTitle();

After:

LayerInfo info = catalog.getLayerByName(name);
String title = info.getTitle();

WMS

Before:

CoordinateReferenceSystem crs = layer.getResource().getCRS();

After:

CoordinateReferenceSystem crs = layer.getCRS();

WFS

Before:

FeatureTypeInfo info = catalog.getFeatureTypeByName(name);
NamespaceInfo ns = info.getNamespace();

After:

LayerInfo info = catalog.getLayerByName(name);
NamespaceInfo ns = layer.getNamespace();

Implement thread-local Catalog View

This is the final step of the process, and where things get interesting. Up until this point the entire system should more or less function the way it does today. What this step involves is allowing the client to specify a parameter which filters down the view of the catalog that the rest of the system will see.

There are two types of views. The first is a view driven by a specific profile. This view will only allow access to layers (and the resources they publish) that are contained with the list of maps specified by the profile. The second type of profile is similar in nature but driven by a single map.

Both views will be wrappers around the real catalog, much like how the secure catalog works.

class MapCatalogView extends AbstractDecorator<Catalog> implements Catalog {

    MapInfo map;

    public MapVatalogView(MapInfo map, Catalog catalog) {
        super(catalog);
        this.map = map;
    }

    ...
    public List<LayerInfo> getLayers() {
       //filter out only layers which are contained by the map
       return map.getLayers();
    }

    public List<LayerInfo> getLayerByName(String name) {
       //qualify the name with the map
       return catalog.getLayerByName( map, name );

    }

    ...
}

The class ProfileCatalogView will be similar in nature. The two may share a common super class, but that is left as an implementation detail.

Once the views are written, it becomes a question of how to wire them up. In the case of a map, which map is used will be driven by the client making the request. For example a client may wish to specify a particular map in a capabilities request such as:

http://.../geoserver/wms?map=foo&request=getcapabilities

Which makes a WMS GetCapabilities request against a single map.

In the case of a profile, the profile can be specified in a couple of ways:

• As a request parameter, similar to the map example given above. Example:

  http://.../geoserver/wms?profile=bar&request=getcapabilities

1. As a session variable, via the user interface

It has also been suggested that the profile be specified as part of the context path, for instance:

http://.../geoserver/bar/wms?request=getcapabilities

However to support this will require some modifications to the ows dispatcher.

To support these use cases the thread local catalog view must be set on the org.geoserver.config.GeoServer singleton instance:

class GeoServerImpl {

  ThreadLocal<Catalog> localCatalog = new ThreadLocal();
  Catalog catalog;

  public void setLocalCatalog(Catalog catalog) {
      localCatalog.set( catalog );
  }

  public void clearLocalCatalog() {
      localCatalog.remove();
  }

  public Catalog getCatalog() {
      if ( localCatalog.get() != null ) {
          return localCatalog.get();
      }
      return catalog;
  }
}

Setting the local catalog for a request can be achieved a number of ways. The most direct would be via a servlet Filter. The following is a simplified example:

class CatalogViewFilter implements Filter {

  public init(FilterConfig config) {

  }

  public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) {
     GeoServer gs = GeoServerExtensions.bean( GeoServer.class );     
     Catalog catalog = gs.getCatalog(); //the raw catalog

     String mapName = request.getParameter( "map" );
     if ( mapName != null ) {
        MapInfo map = catalog.getMap( mapName );
        gs.setCatalog( new MapCatalogView( map, catalog ) );
     }
     else {
       String profileName = request.getParameter( "profile" );
       GeoServerProfile profile = gs.getProfile( profileName );
       gs.setCatalog( new ProfileCatalogView( profile, catalog ) );
     } 

     chain.doFilter( request, response );
  }

  void destroy() {
      GeoServer gs = GeoServerExtensions.bean( GeoServer.class );
      gs.clearLocalCatalog();
  }
}

The above could also be implemented as a spring HandlerInterceptor. The benefit of which would be access to dependency injection like a regular spring bean.

Setting the profile from the UI is different, and uses a session based approach. Basically a session will have the notion of a current profile. Any time the GeoServer singleton is accessed the session must be checked for a profile, and the thread local view must be set accordingly.

class GeoServerApplication {
  ...
  GeoServer getGeoServer() {
     GeoServer gs = getBeanByType( GeoServer.class );
     GeoServerSession session = (GeoServerSession) Session.get();
     GeoServerProfile profile = session.getProfile();
     if ( profile != null ) {
        gs.setLocalProfile( profile );
        gs.setLocalCatalog( new CatalogProfileView( profile, gs.getCatalog() );
     }

     return gs;
  }
}

Feedback

Backwards Compatibility

State here any backwards compatibility issues.

Voting

Andrea Aime: Alessio Fabiani Jody Garnett: Jody Garnett: Saul Farber: Rob Atkinson:

Links

JIRA Task Email Discussion Wiki Page