archive-gh-me/geoserver-cloud
1
0
Fork 0
mirror of https://github.com/geoserver/geoserver-cloud.git synced 2026-02-01 16:46:47 +00:00
Code Issues Projects Releases Wiki Activity

Initial project documentation page

Browse Source
This commit is contained in:
Gabriel Roldan 2021-02-15 05:07:53 -03:00
parent 0336a8eb47
commit 878cc7b849
No known key found for this signature in database
GPG Key ID: E7A133EF88684A50
11 changed files with 428 additions and 152 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -58,7 +58,7 @@ Browse to [http://localhost:9090](http://localhost:9090)
Please read [the contribution guidelines](CONTRIBUTING.md) before contributing pull requests to the CN GeoServer project.
<!-- Follow the [developer's guide](CONTRIBUTING.md) to know more about the project's technical details. -->
Follow the [developer's guide](docs/develop/index.md) to know more about the project's technical details.
## Status

1
docs/deploy/index.md Normal file
View File

@ -0,0 +1 @@
# Cloud Native GeoServer Deployment Guide

177
docs/develop/index.md Normal file
View File

@ -0,0 +1,177 @@
# Developer's Guide
# Contents
{:.no_toc}
* Will be replaced with the ToC, excluding the "Contents" header
{:toc}
# Technology Overview
With *GeoServer* being a traditional, [Spring Framework](https://spring.io/) based, monolithic servlet application, a logical choice has been made to base the *GeoServer* derived microservices in the [Spring Boot](https://spring.io/projects/spring-boot) framework.
Additionally, [Spring Cloud](https://spring.io/projects/spring-cloud) technologies enable crucial capabilities such as [dynamic service discovery](https://spring.io/projects/spring-cloud-netflix), [externalized configuration](https://spring.io/projects/spring-cloud-config), [distributed events](https://spring.io/projects/spring-cloud-bus), [API gateway](https://spring.io/projects/spring-cloud-gateway), and more.
Only a curated list of the [vast amount](http://geoserver.org/release/stable/) of GeoServer extensions will be supported, as they are verified and possibly adapted to work with this project's architecture.
# System Architecture
The following diagram depicts the System's general architecture:
![Cloud Native GeoServer Architecture Diagram](../img/gs_cloud_architecture_diagram.svg "Cloud Native GeoServer Architecture Diagram")
> - Hexagons represent microservices;
> - coloured rectangles, logical groupings of components;
> - lines connecting a group to another component: connector applies to all services of the outgoing end, to all components of the incoming end;
> - white rectangles, components that are platform/deployment choices. For example:
> - "Event bus" could be a cloud provider's native service (event queue), or a microservice implementing a distributed event broker;
> - "Catalog/Config backend" is the software compoent used to access the catalog and configuration. Might be a microservice itself, catalog/config provider for "data directory", database, or other kind of external service store, catalog/config backend implementations;
> - "Catalog/Config storage" is the storage mechanism that backs the catalog/config software component. Might be a shared "data directory" or database, a "per instance" data directory or database, and so on, depending on the available catalog/config backend implementations, and how they're configured and provisioned;
> - "Geospatial data sources" is whatever method is used to access the actual data served up by the microservices.
Does that mean *GeoServer*'s `.war` is deployed several times, with each instance exposing a given "business capability"?
**ABSOLUTELY NOT**.
Each microservice is its own self-contained application, including only the GeoServer dependencies it needs. Moreover, care has been taken so that when a dependency has both required and non-required components, only the required ones are loaded.
> Note the above diagram represents the overall system's architecture. This is not a deployment diagram. Deployment involves choice of platforms, configurations, and more; without affecting the general architecture.
> Some microservices/components, though planned and represented in the architecture diagram, have not yet been developed/integrated. For instance: the logging, tracing, and monitoring components, as well as the GWC and WPS microservices.
## Components Overview
* Front services:
* Gateway
* Monitoring
* Infrastructure:
* Discovery
* Config
* Event bus
* Logging
* Tracing
* Cache
* GeoServer:
* Catalog
* OWS services
* REST API service
* Web-UI service
* GWC service
# Building
## Requirements:
* Java >= 11 JDK
* [Maven](https://maven.apache.org/) >= `3.6.3`
* [Docker](https://docs.docker.com/engine/install/) version >= `19.03.3`
* [docker-compose](https://docs.docker.com/compose/) version >= `1.26.2`
*CN GeoServer* uses [Apache Maven](http://maven.apache.org/) (included) for a build system.
You need to have [docker](https://www.docker.com/) and [docker-compose](https://docs.docker.com/compose/install/) installed.
## Build
> The main branch follows GeoServer's main branch, currently `2.19-SNAPSHOT`.
> The stable branch builds against a released GeoServer version.
> Building master against the latest stable version (`2.18.2`) is no longer possible due to binary incompatible between `2.18.x` and `2.19.x`. Once 2.19.0 is released, it'll be possible to build against a stable version activating the `geoserver_stable_version` profile as follows:
> `./mvnw clean install -P geoserver_stable_version`
To build the applications run the following command from the root project directory:
./mvnw clean install
That will compile, run unit and integration tests, install maven artifacts to the local maven repository, and create all microservices docker images.
The maven build uses the `com.spotify:dockerfile-maven-plugin` maven plugin to build the microservice docker images.
The simple build command above creates the following docker images:
```bash
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
org.geoserver.cloud/gs-cloud-catalog 0.2-SNAPSHOT cd7159216be8 About an hour ago 406MB
org.geoserver.cloud/gs-cloud-config-service 0.2-SNAPSHOT 28f4f4f9ff35 25 hours ago 332MB
org.geoserver.cloud/gs-cloud-database 0.2-SNAPSHOT 0022bb2d2a1e 6 weeks ago 491MB
org.geoserver.cloud/gs-cloud-discovery-service 0.2-SNAPSHOT 827e3ebde911 25 hours ago 334MB
org.geoserver.cloud/gs-cloud-gateway 0.2-SNAPSHOT 55ecab20b51e 25 hours ago 330MB
org.geoserver.cloud/gs-cloud-restconfig-v1 0.2-SNAPSHOT d1aa8a3495a1 About an hour ago 432MB
org.geoserver.cloud/gs-cloud-wcs 0.2-SNAPSHOT 580b2336ab02 About an hour ago 416MB
org.geoserver.cloud/gs-cloud-web-ui 0.2-SNAPSHOT da1e714ff851 About an hour ago 461MB
org.geoserver.cloud/gs-cloud-wfs 0.2-SNAPSHOT 6f296b3ba198 About an hour ago 427MB
org.geoserver.cloud/gs-cloud-wms 0.2-SNAPSHOT 294ab913aaf4 About an hour ago 439MB
org.geoserver.cloud/gs-cloud-wps 0.2-SNAPSHOT 07135b861814 About an hour ago 440MB
```
To run the build without building the docker images, disable the `docker` maven profile:
```bash
$ ./mvnw clean install -P\!docker
```
# Running
## docker-compose
Now run the docker composition as follows, the first time it might need to download some additional images for the `rabbitmq` event broker and the `postgresql` config database:
```bash
$ docker-compose up -d
```
Run `docker-compose logs -f` to watch startup progress of all services.
Watch the output of `docker-compose ps` until all services are healthy:
```bash
$ docker-compose ps
Name Command State Ports
-----------------------------------------------------------------------------------------------------------------
gscloud_catalog_1 dockerize -wait http://con ... Up (healthy)
gscloud_config_1 dockerize -wait http://dis ... Up (healthy)
gscloud_database_1 docker-entrypoint.sh postgres Up (healthy) 0.0.0.0:5432->5432/tcp
gscloud_discovery_1 /bin/sh -c exec java $JAVA ... Up (healthy) 0.0.0.0:8761->8761/tcp
gscloud_gateway_1 dockerize -wait http://con ... Up (healthy) 0.0.0.0:9090->8080/tcp
gscloud_rabbitmq_1 docker-entrypoint.sh rabbi ... Up 15671/tcp, 0.0.0.0:15672->15672/tcp, ...
gscloud_rest_1 dockerize -wait http://con ... Up (healthy)
gscloud_wcs_1 dockerize -wait http://con ... Up (healthy)
gscloud_webui_1 dockerize -wait http://con ... Up (healthy)
gscloud_wfs_1 dockerize --timeout 60s -w ... Up (healthy)
gscloud_wms_1 dockerize -wait http://con ... Up (healthy)
```
Now you can access all front-services (`wms`, `wfs`, `wcs`, `rest`, and `webui`) through the `gateway` service at [http://localhost:9090](http://localhost:9090)
## Running a service in development/debug mode
Running a single service in "local" mode (that is, outside the docker composition) can be done either through the command line or through the IDE.
First, make sure at least the essential infrastructure services are running:
```bash
$ docker-compose up -d discovery rabbitmq config database catalog gateway
```
> The `gateway` service is not essential, but useful to check it's correctly proxy'ing requests to your locally running services as well as the ones in the docker composition.
To run a specific service through the command line, for example, `wfs-service`, run:
```bash
$ ./mvnw -f services/wfs spring-boot:run -Dspring-boot.run.profiles=local
```
To run a service through the IDE, execute the specific application class (for example, `org.geoserver.cloud.wfs.app.WfsApplication`), which is a regular Java class with a `main()` method, passing the JVM argument `-Dspring-boot.run.profiles=local`.
The "local" spring profile in each `config/<service>.yml` file sets a different hard-coded port for each service, which aids in debugging a locally running service:
* `catalog-service`: [9100](http://localhost:9100)
* `wfs-service`: [9101](http://localhost:9101)
* `wms-service`: [9102](http://localhost:9102)
* `wcs-service`: [9103](http://localhost:9103)
* `wps-service`: [9100](http://localhost:9104)
* `restconfig-v1`: [9105](http://localhost:9105)
* `web-ui`: [9106](http://localhost:9106)
At startup time, as configured in its `src/main/resources/bootstrap.yml` file, the service will contact the `discovery-service` at the default `http://localhost:8761/eureka` location, given there's no `eureka.server.url` configuration property set (which is otherwise provided by `docker-compose.yml`).
Since `docker-compose.yml` exposes the `discovery-service` at the local port `8761`, that's all the service being run locally needs to engage in the cluster. The discovery service will provide it with the location of any other service it needs to contact, starting with the `config-service`, where it will ultimatelly get the rest of the application configuration from.

20
docs/develop/services/catalog-service.md Normal file
View File

@ -0,0 +1,20 @@
# Cloud Native GeoServer Catalog service
Spring Webflux reactive microservice that exposes the GeoServer *catalog*, *global*, and *resources* configuration
objects through a RESTful API to other microservices, in order to abstract out the microservices that require access
to the catalog from the actual catalog backend and implementation.
**Docker image**: `cloudnativegeoserver/gs-cloud-catalog`.
**Service name**: `catalog-service`.
This is the logical service name by which web clients will get the actual instances addresses from the [discovery-service](discovery-service.yml) and perform client-side load balancing against when interacting with the service.
## Service Configuration
## Client Configuration
## Developing

46
docs/develop/services/discovery-service.md Normal file
View File

@ -0,0 +1,46 @@
# Cloud Native GeoServer Discovery service
The Discovery Service maintains a registry of the location and health status of other services.
Any other service participating in the cluster will register to the Discovery Service at start up, and de-register
at graceful shut down time.
Inter-service communication will then be automatically load balanced to all available service instances of a given type.
The most common scenario is when for High Availability or performance reasons, there are several instances of a specific service,
hence incoming requests passing through the [Gateway Service](gateway-service.yml) get served by a different instance in a round-robin
fashion.
**Docker image**: `cloudnativegeoserver/gs-cloud-discovery-service`.
**Service name**: `discovery-service`.
This is the logical service name.
Since we're using a ["discovery first bootstrap"](https://docs.spring.io/spring-cloud-config/docs/2.2.7.RELEASE/reference/html/#discovery-first-bootstrap)
approach to service orchestration, other services won't use this service name to locate the discovery service, but a fixed list of service addresses need to be provided. See the "Client Configuration" section bellow for more details.
## Reference documentation
* Spring Cloud Netflix [reference](https://docs.spring.io/spring-cloud-netflix/docs/2.2.7.RELEASE/reference/html/)
* Discovery First Bootstrap [Config Client](https://docs.spring.io/spring-cloud-config/docs/2.2.7.RELEASE/reference/html/#discovery-first-bootstrap)
## Service Configuration
## Client Configuration
## Developing
In the default docker composition, there's only one instance of the discovery service. In a High Availability deployment scenario,
more instances can be launched, al
Since the discovery service are fixed entry points, we're setting up two peer aware eureka instances for HA.
# Browse to http://localhost:8761 and http://localhost:8762 to verify they see each
# other and all services are registered at both eureka instances.
# See http://cloud.spring.io/spring-cloud-static/spring-cloud.html#_peer_awareness
https://docs.spring.io/spring-cloud-netflix/docs/2.2.7.RELEASE/reference/html/

17
docs/develop/services/gateway-service.md Normal file
View File

@ -0,0 +1,17 @@
# Cloud Native GeoServer Gateway service
**Docker image**: `cloudnativegeoserver/gs-cloud-gateway`.
**Service name**: `gateway-service`.
This is the logical service name by which web clients will get the actual instances addresses from the [discovery-service](discovery-service.yml) and perform client-side load balancing against when interacting with the service.
## Service Configuration
## Client Configuration
## Developing

14
docs/develop/services/resconfig-v1-service.md Normal file
View File

@ -0,0 +1,14 @@
# Cloud Native GeoServer REST configuration API v1 service
Spring Boot/Cloud microservice that exposes GeoSever [REST API](https://docs.geoserver.org/stable/en/user/rest/).
**Docker image**: `cloudnativegeoserver/gs-cloud-restconfig-v1`.
**Service name**: `restconfig-v1`.
Logical service name by which the [gateway-service](gateway-service.yml) will get the actual instances addresses from the [discovery-service](discovery-service.yml) and perform client-side load balancing against when interacting with the service.
## Configuration
## Developing

273
docs/index.md
View File

@ -2,10 +2,14 @@
*Cloud Native GeoServer* is [GeoServer](http://geoserver.org/) ready to use in the cloud through dockerized microservices.
This project is an opinionated effort to split *GeoServer*'s geospatial services and API offerings as individually deployable components of a [microservices based architecture](https://microservices.io/).
The *Cloud Native GeoServer* project splits the GeoServer geospatial services and API offerings into individually deployable components of a [microservices based architecture](https://microservices.io/). This project provides clear constructive changes to GeoServer when required, based on prior experience with GeoServer clustering solutions.
As such, it builds on top of existing *GeoServer* software components, adapting and/or extending them in an attempt to achieve functional decomposition by business capability; which roughly means each OWS service, the Web UI, the REST API, and probably other components such as the *Catalog and Configuration subsystem*, become self-contained, individually deployable and scalable micro-services.
*Cloud Native GeoServer* is built with GeoServer software components. Compontents are adapted and/or extended in a [functional decomposition by business capability](https://microservices.io/patterns/decomposition/decompose-by-business-capability.html). The result is each service (OWS service, the Web UI, the REST API ) is available as a self-contained, individually deployable and scalable micro-services.
> Q: Does that mean *GeoServer*'s `.war` is deployed several times, with each instance exposing a given "business capability"?
>
> Absolutely not, this is not a clusterd GeoSever approach and does not use war files.
>Each microservice is its own self-contained application, including only the GeoServer compoents needed for the service. Many GeoServer components provide a lot of functionality (such as different output formats). In these cases, care is taken to only load the functionality that is needed for a light-weight experience.
# Contents
{:.no_toc}
@ -13,16 +17,129 @@ As such, it builds on top of existing *GeoServer* software components, adapting
* Will be replaced with the ToC, excluding the "Contents" header
{:toc}
# Background and motivation
GeoServer is the most widely used and deployed Open Source geospatial server in the world. It allows publishing, transforming, and editing geospatial data from a wide number of formats, through an extense number of user-facing services. Besides these out-of-the-box capabilities, it also counts with over 40 supported [extensions](http://geoserver.org/release/stable/), plus a bigger number of experimental extensions (so-called "community modules").
Being a [monolithic application](https://en.wikipedia.org/wiki/Monolithic_application), [installation and configuration](https://docs.geoserver.org/latest/en/user/installation/war.html) of GeoServer is very easy, as all its different components are bundled together, offerring a simple, cohesive approach to software delivery.
People in charge of deploying and maintaining GeoServer instances usually face some very common challenges when it comes to server dimensioning, configuration, system health monitoring and corrective actions; difficulties that get incremented as systems need to ensure a certain capacity to handle request load, service availability, and overall performance.
Traditional deployments, though feassible of providing [high availability](https://en.wikipedia.org/wiki/High_availability), tend to require a fixed number of application instances, a single-master/multiple-slaves cluster architecture, and/or the installation and configuration of additional components to keep instances in sync.
Ensuring a given load capacity often results in over-provisioning of server resources to handle peaks on demand, which can have a negative impact on an organization's budget, especially if deploying on a public cloud provider such as Amazon AWS, Microsoft Azure, or Google Cloud Platform.
Though a gateway/load balancer can be put in front and still make the cluster instances [auto-scalable](https://en.wikipedia.org/wiki/Autoscaling), all instances expose the whole set of functionalities GeoServer provides, despite which service(s) are under high demand, which can lead to unnecessary consumption of resources, opening attack vectors, and losing availability of unrelated capabilities when some component fails.
# Vision
*Cloud Native GeoServer* is a sibling project to GeoServer, targeting organizations and system administrators in seek of a steady, prescribed way of provisioning GeoServer capabilities in a dynamic environment, where each business capability can be enabled, configured, dimensioned, and deployed independently.
*Cloud Native GeoServer* is not a re-write of GeoServer. On the contrary, it builds on top of existing *GeoServer* software components, adapting and/or extending them, feeding from, and contributing back to GeoServer's mainline development. Traditional GeoServer installations are still recommended for simple deployments that can/need to expose all its capabilities.
This project reformulates how these capabilities are bundled together, in an attempt to achieve [functional decomposition by business capability](https://microservices.io/patterns/decomposition/decompose-by-business-capability.html); which roughly means each OWS service, the Web UI, the REST API, and probably other components such as the *Catalog and Configuration subsystem*, become self-contained, individually deployable and scalable micro-services, loosely coupled through an [event based architecture](https://en.wikipedia.org/wiki/Event-driven_architecture).
These containerized applications allow deployment strategies to vary from single-server docker composition, to multi-node deployments using [docker swarm](https://docs.docker.com/engine/swarm/) or [Kubernetes](https://kubernetes.io/).
# Goals and benefits
* Posibility to assess and provide guidelines for proper dimensioning of services based on each one's resource needs and performance characteristics
* Independent evolvability of services
* Externalized, centralized configuration of services and their sub-components
* Service isolation allows the system to keep working in the event of some specific service becoming unavailable
* Ability to implement per service auto-scaling
* Ability to implement continuous delivery workflows
* Location transparency of service instances in a dynamic environment. A "gateway" service acts as a single entry point to all requests, dispatching them to the appropriate micro-services in a round-robin load balanced automatic configuration
* Centralized service logging, request tracing, and monitoring
<!--
"Cloud Native" applications
- [ ] note: familiarity with GeoServer is assumed.
- [x] mention: budget factor in cloud providers and how (auto-)scaling and resourcing can reduce the bill
- [x] mention: possibility to assess and provide guidelines for proper dimensioning of services based on each one's resource needs and capacity
- [x] mention: Location Transparency achieved by discovery service and dynamic registration/deregistration of services + client side load balancing
- [x] mention: traditional deployments, though feasible of providing HA and even auto-scaling, tend to require fixed instance/locations. Though a gateway/load balancer can be put in front and still make the cluster instances elastic, there's still the factor of instantiating a full service each time.
- [x] isolation, independent evolvability
- [ ] target platforms
- [x] per-service resourcing (cpu/ram/storage)
- [ ] continuous delivery (cf push?)?
- [ ] per-service (auto-)scaling
- [ ] multi-cloud? (cloud foundry/k8's/docker/swarm) paradigm shift from multi-servlet engine deploy (tomcat/jetty/websphere/etc)
-->
# Quick Start
The following instructions are meant as quick start up guide to run the system as a docker composition in a single machine. For more advanced deployment options and strategies, check out the [Deployment Guide](deploy/index.md).
Make sure you have `docker` and `docker-compose` installed:
* [Docker](https://docs.docker.com/engine/install/) version >= `19.03.3`
* [docker-compose](https://docs.docker.com/compose/) version >= `1.26.2`
Download this [docker-compose.yml](deploy/docker-compose/stable/docker-compose.yml) file to a directory in your computer. For example:
```bash
$ mkdir gscloud && cd gscloud
$ wget "https://raw.githubusercontent.com/camptocamp/geoserver-microservices/v0.2.0/docs/deploy/docker-compose/stable/docker-compose.yml" -O docker-compose.yml
```
Open a terminal and enter the directory where you just downloaded that file, and run `docker-compose pull` to fetch the docker images from [Dockerhub](https://hub.docker.com/u/cloudnativegeoserver/):
```bash
$ docker-compose pull
Pulling rabbitmq ... done
Pulling database ... done
Pulling discovery ... done
Pulling config ... done
Pulling gateway ... done
Pulling catalog ... done
Pulling wfs ... done
Pulling wms ... done
Pulling wcs ... done
Pulling rest ... done
Pulling webui ... done
```
Then start the services with this command:
```bash
$ docker-compose up -d
```
Wait for the services to start up, check with `docker-compose ps` until they're healthy. May some service had exited, re-run `docker-compose up -d`, initial startup may cause timeouts as services compete for resources on a single machine.
Then browse to:
- [http://localhost:8761](http://localhost:8761/) to access the Discovery service registry page.
- [http://localhost:9090](http://localhost:9090/) for the GeoServer UI. The GateWay service will take care of proxying requests to the appropriate microservice.
From the usability point of view, there should be no appreciable differences with a traditional GeoServer deployment.
What's going on inside the local-maching cluster you've just run is that each GeoServer business capability is running as a separate process in a separate docker container, and they're all in synch in a loosely coupled way using a message driven event-bus.
Experiment dynamic service scaling and registration with `docker-compose scale <service>=<instances>`, for example:
```bash
docker-compose scale wfs=3 wcs=0
Starting gscloud_wfs_1 ... done
Creating gscloud_wfs_2 ... done
Creating gscloud_wfs_3 ... done
Stopping and removing gscloud_wcs_1 ... done
```
# Technology Overview
With *GeoServer* being a traditional, [Spring Framework](https://spring.io/) based, monolithic servlet application, a logical choice has been made to base the *GeoServer* derived microservices in the [Spring Boot](https://spring.io/projects/spring-boot) framework.
With *GeoServer* being a traditional, [Spring Framework](https://spring.io/) based, monolithic Servlet/WebMVC application, a logical choice has been made to base the *GeoServer* derived microservices in the [Spring Boot](https://spring.io/projects/spring-boot) framework.
Additionally, [Spring Cloud](https://spring.io/projects/spring-cloud) technologies enable crucial capabilities such as [dynamic service discovery](https://spring.io/projects/spring-cloud-netflix), [externalized configuration](https://spring.io/projects/spring-cloud-config), [distributed events](https://spring.io/projects/spring-cloud-bus), [API gateway](https://spring.io/projects/spring-cloud-gateway), and more.
Only a curated list of the [vast amount](http://geoserver.org/release/stable/) of GeoServer extensions will be supported, as they are verified and possibly adapted to work with this project's architecture.
# System Architecture
# System Architecture Overview
The following diagram depicts the System's general architecture:
@ -32,153 +149,19 @@ The following diagram depicts the System's general architecture:
> - coloured rectangles, logical groupings of components;
> - lines connecting a group to another component: connector applies to all services of the outgoing end, to all components of the incoming end;
> - white rectangles, components that are platform/deployment choices. For example:
> - "Event bus" could be a cloud provider's native service (event queue), or a microservice implementing a distributed event broker;
> - "Event bus" could be a cloud infrastructure provider's native service (event queue), or a microservice implementing a distributed event broker;
> - "Catalog/Config backend" is the software compoent used to access the catalog and configuration. Might be a microservice itself, catalog/config provider for "data directory", database, or other kind of external service store, catalog/config backend implementations;
> - "Catalog/Config storage" is the storage mechanism that backs the catalog/config software component. Might be a shared "data directory" or database, a "per instance" data directory or database, and so on, depending on the available catalog/config backend implementations, and how they're configured and provisioned;
> - "Geospatial data sources" is whatever method is used to access the actual data served up by the microservices.
Does that mean *GeoServer*'s `.war` is deployed several times, with each instance exposing a given "business capability"?
**ABSOLUTELY NOT**.
Each microservice is its own self-contained application, including only the GeoServer dependencies it needs. Moreover, care has been taken so that when a dependency has both required and non-required components, only the required ones are loaded.
> Note the above diagram represents the overall system's architecture. This is not a deployment diagram. Deployment involves choice of platforms, configurations, and more; without affecting the general architecture.
> Some microservices/components, though planned and represented in the architecture diagram, have not yet been developed/integrated. For instance: the logging, tracing, and monitoring components, as well as the GWC and WPS microservices.
# Developer's Guide
## Components Overview
* Front services:
* Gateway
* Monitoring
* Infrastructure:
* Discovery
* Config
* Event bus
* Logging
* Tracing
* Cache
* GeoServer:
* Catalog
* OWS services
* REST API service
* Web-UI service
* GWC service
# Building
## Requirements:
* Java >= 11 JDK
* [Maven](https://maven.apache.org/) >= `3.6.3`
* [Docker](https://docs.docker.com/engine/install/) version >= `19.03.3`
* [docker-compose](https://docs.docker.com/compose/) version >= `1.26.2`
*CN GeoServer* uses [Apache Maven](http://maven.apache.org/) (included) for a build system.
You need to have [docker](https://www.docker.com/) and [docker-compose](https://docs.docker.com/compose/install/) installed.
## Build
> The main branch follows GeoServer's main branch, currently `2.19-SNAPSHOT`.
> The stable branch builds against a released GeoServer version.
> Building master against the latest stable version (`2.18.2`) is no longer possible due to binary incompatible between `2.18.x` and `2.19.x`. Once 2.19.0 is released, it'll be possible to build against a stable version activating the `geoserver_stable_version` profile as follows:
> `./mvnw clean install -P geoserver_stable_version`
To build the applications run the following command from the root project directory:
./mvnw clean install
That will compile, run unit and integration tests, install maven artifacts to the local maven repository, and create all microservices docker images.
The maven build uses the `com.spotify:dockerfile-maven-plugin` maven plugin to build the microservice docker images.
The simple build command above creates the following docker images:
```bash
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
org.geoserver.cloud/gs-cloud-catalog 0.2-SNAPSHOT cd7159216be8 About an hour ago 406MB
org.geoserver.cloud/gs-cloud-config-service 0.2-SNAPSHOT 28f4f4f9ff35 25 hours ago 332MB
org.geoserver.cloud/gs-cloud-database 0.2-SNAPSHOT 0022bb2d2a1e 6 weeks ago 491MB
org.geoserver.cloud/gs-cloud-discovery-service 0.2-SNAPSHOT 827e3ebde911 25 hours ago 334MB
org.geoserver.cloud/gs-cloud-gateway 0.2-SNAPSHOT 55ecab20b51e 25 hours ago 330MB
org.geoserver.cloud/gs-cloud-restconfig-v1 0.2-SNAPSHOT d1aa8a3495a1 About an hour ago 432MB
org.geoserver.cloud/gs-cloud-wcs 0.2-SNAPSHOT 580b2336ab02 About an hour ago 416MB
org.geoserver.cloud/gs-cloud-web-ui 0.2-SNAPSHOT da1e714ff851 About an hour ago 461MB
org.geoserver.cloud/gs-cloud-wfs 0.2-SNAPSHOT 6f296b3ba198 About an hour ago 427MB
org.geoserver.cloud/gs-cloud-wms 0.2-SNAPSHOT 294ab913aaf4 About an hour ago 439MB
org.geoserver.cloud/gs-cloud-wps 0.2-SNAPSHOT 07135b861814 About an hour ago 440MB
```
To run the build without building the docker images, disable the `docker` maven profile:
```bash
$ ./mvnw clean install -P\!docker
```
# Running
## docker-compose
Now run the docker composition as follows, the first time it might need to download some additional images for the `rabbitmq` event broker and the `postgresql` config database:
```bash
$ docker-compose up -d
```
Run `docker-compose logs -f` to watch startup progress of all services.
Watch the output of `docker-compose ps` until all services are healthy:
```bash
$ docker-compose ps
Name Command State Ports
-----------------------------------------------------------------------------------------------------------------
gscloud_catalog_1 dockerize -wait http://con ... Up (healthy)
gscloud_config_1 dockerize -wait http://dis ... Up (healthy)
gscloud_database_1 docker-entrypoint.sh postgres Up (healthy) 0.0.0.0:5432->5432/tcp
gscloud_discovery_1 /bin/sh -c exec java $JAVA ... Up (healthy) 0.0.0.0:8761->8761/tcp
gscloud_gateway_1 dockerize -wait http://con ... Up (healthy) 0.0.0.0:9090->8080/tcp
gscloud_rabbitmq_1 docker-entrypoint.sh rabbi ... Up 15671/tcp, 0.0.0.0:15672->15672/tcp, ...
gscloud_rest_1 dockerize -wait http://con ... Up (healthy)
gscloud_wcs_1 dockerize -wait http://con ... Up (healthy)
gscloud_webui_1 dockerize -wait http://con ... Up (healthy)
gscloud_wfs_1 dockerize --timeout 60s -w ... Up (healthy)
gscloud_wms_1 dockerize -wait http://con ... Up (healthy)
```
Now you can access all front-services (`wms`, `wfs`, `wcs`, `rest`, and `webui`) through the `gateway` service at [http://localhost:9090](http://localhost:9090)
## Running a service in development/debug mode
Running a single service in "local" mode (that is, outside the docker composition) can be done either through the command line or through the IDE.
First, make sure at least the essential infrastructure services are running:
```bash
$ docker-compose up -d discovery rabbitmq config database catalog gateway
```
> The `gateway` service is not essential, but useful to check it's correctly proxy'ing requests to your locally running services as well as the ones in the docker composition.
To run a specific service through the command line, for example, `wfs-service`, run:
```bash
$ ./mvnw -f services/wfs spring-boot:run -Dspring-boot.run.profiles=local
```
To run a service through the IDE, execute the specific application class (for example, `org.geoserver.cloud.wfs.app.WfsApplication`), which is a regular Java class with a `main()` method, passing the JVM argument `-Dspring-boot.run.profiles=local`.
The "local" spring profile in each `conifg/<service>.yml` file sets a different hard-coded port for each service, which aids in debugging a locally running service:
* `catalog-service`: [9100](http://localhost:9100)
* `wfs-service`: [9101](http://localhost:9101)
* `wms-service`: [9102](http://localhost:9102)
* `wcs-service`: [9103](http://localhost:9103)
* `wps-service`: [9100](http://localhost:9104)
* `restconfig-v1`: [9105](http://localhost:9105)
* `web-ui`: [9106](http://localhost:9106)
At startup time, as configured in its `src/main/resources/bootstrap.yml` file, the service will contact the `discovery-service` at the default `http://localhost:8761/eureka` location, given there's no `eureka.server.url` configuration property set (which is otherwise provided by `docker-compose.yml`).
Since `docker-compose.yml` exposes the `discovery-service` at the local port `8761`, that's all the service being run locally needs to engage in the cluster. The discovery service will provide it with the location of any other service it needs to contact, starting with the `config-service`, where it will ultimatelly get the rest of the application configuration from.
Follow the [Developer's Guide](develop/index.md) to learn more about the System's design and how to get started contributing to it.
# Deployment Guide
Chek out the [Deployment Guide](deploy/index.md) to learn about deployment options, configuration, and target platforms.

11
services/catalog/README.md
View File

@ -1,8 +1,7 @@
# GeoServer Catalog micro-service
# Cloud Native GeoServer Catalog service
Microservice that exposes the geoserver catalog through a web interface (RESTful so far) to other microservices, in order to abstract out the microservices that require access to the catalog from the actual catalog backend and implementation.
Spring Webflux reactive microservice that exposes the GeoServer *catalog*, *global*, and *resources* configuration
objects through a RESTful API to other microservices, in order to abstract out the microservices that require access
to the catalog from the actual catalog backend and implementation.
--
REVISIT:
- Looks like commons-logging is blocking the netty eventloop threads. Nonetheless achieving a 17k reqs/s throughput with 2 worker threads and the default Netty event loop threads (2 * cores)
- Reduce/make configurable the number of threads in the Netty event loop. We should be using just a few, but the default of 2*cores is high when there are many cores
Follow the service [documentation](../../docs/develop/services/catalog-service.md) and keep it up to date .

5
services/restconfig/README.md Normal file
View File

@ -0,0 +1,5 @@
# Cloud Native GeoServer REST configuration API v1 service
Spring Boot/Cloud microservice that exposes GeoSever [REST API](https://docs.geoserver.org/stable/en/user/rest/).
Follow and the service [documentation](../../docs/develop/services/resconfig-v1-service.md) and keep it up to date.

14
support-services/discovery/README.md Normal file
View File

@ -0,0 +1,14 @@
# Cloud Native GeoServer Discovery service
The Discovery Service maintains a registry of the location and health status of other services.
Any other service participating in the cluster will register to the Discovery Service at start up, and de-register
at graceful shut down time.
Inter-service communication will then be automatically load balanced to all available service instances of a given type.
The most common scenario is when for High Availability or performance reasons, there are several instances of a specific service,
hence incoming requests passing through the [Gateway Service](gateway-service.yml) get served by a different instance in a round-robin
fashion.
Follow the service [documentation](../../docs/develop/services/discovery-service.md) and keep it up to date.