diff --git a/doc/index.rst b/doc/index.rst index 530eddb..d18b99f 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -54,6 +54,14 @@ Functions functions/index +Update +-------------------------------------------------------------------------------- + +.. toctree:: + :maxdepth: 2 + + update + Tutorials -------------------------------------------------------------------------------- diff --git a/doc/tutorials/compression.rst b/doc/tutorials/compression.rst new file mode 100644 index 0000000..7b07801 --- /dev/null +++ b/doc/tutorials/compression.rst @@ -0,0 +1,98 @@ +****************************************************************************** +Schema and compression +****************************************************************************** + +This tutorial is an introduction for investigating XML schemas and playing with +compression of patches. + +------------------------------------------------------------------------------ +Compression type +------------------------------------------------------------------------------ + +The compression of a patch may be retrieved through its XML schema but it's +also stored in the patch itself. Of course, both needs to be consistent so +updating an existing schema is hardly discouraged and may lead to errors. + +In the first case, the XML schema needs to be parsed with ``xpath`` function to +retrieve the ``pc:metadata`` tag of a specific patch: + +.. code-block:: sql + + WITH tmp AS ( + SELECT pc_pcid(pa) + AS _pcid + FROM airport + LIMIT 1 + ) + SELECT unnest( + xpath( + '/pc:PointCloudSchema/pc:metadata/Metadata/text()', + schema::xml, + array[ + ['pc', 'http://pointcloud.org/schemas/PC/'], + ['xsi', 'http://www.w3.org/2001/XMLSchema-instance'] + ] + ) + ) + FROM tmp,pointcloud_formats + WHERE pcid=tmp._pcid; + --> dimensional + + +A much easier way to retrieve the compression type is to take a look to the +JSON summary of the patch: + +.. code-block:: sql + + SELECT pc_summary(pa)::json->'compr' FROM airport LIMIT 1; + --> dimensional + +------------------------------------------------------------------------------ +Create a new schema +------------------------------------------------------------------------------ + +A schema is just a XML document and may be manually inserted into the +``pointcloud_formats`` table directly from a file. We can also duplicate an +existing schema and tweak some parameters. + +For example, we can create a new schema without compression and based on the +schema ``pcid=1``: + +.. code-block:: sql + + INSERT INTO pointcloud_formats (pcid, srid, schema) + SELECT 2, srid, regexp_replace(schema, 'dimensional', 'none', 'g') + FROM pointcloud_formats + WHERE pcid=1; + +------------------------------------------------------------------------------ +Transform a patch +------------------------------------------------------------------------------ + +Thanks to the ``pc_transform`` function, we can transform the underlying data +of a patch to match a specific schema. So if we want to remove the dimensional +compression from an existing patch, we can use the schema with ``pcid=2`` +previously created. + +In this particular case, the transformed patch doesn't have compression +anymore: + +.. code-block:: sql + + SELECT pc_summary(pc_transform(pa, 2))::json->'compr' FROM airport LIMIT 1; + --> none + +So a new table of uncompressed patches may be easily created: + +.. code-block:: sql + + CREATE TABLE airport_uncompressed AS SELECT pc_transform(pa, 2) AS pa FROM airport; + + SELECT pc_summary(pa)::json->'compr' FROM airport_uncompressed LIMIT 1; + --> none + + SELECT pc_astext(pc_patchavg(pa)) FROM airport LIMIT 1; + --> {"pcid":1,"pt":[65535,0,0,0,0,0,0,0,0,30744,25999,17189,728265,4.67644e+06,299.08]} + + SELECT pc_astext(pc_patchavg(pa)) FROM airport_uncompressed LIMIT 1; + --> {"pcid":2,"pt":[65535,0,0,0,0,0,0,0,0,30744,25999,17189,728265,4.67644e+06,299.08]} diff --git a/doc/tutorials/index.rst b/doc/tutorials/index.rst index fd30446..18b56bb 100644 --- a/doc/tutorials/index.rst +++ b/doc/tutorials/index.rst @@ -11,3 +11,4 @@ pgPointcloud. :maxdepth: 1 storing + compression diff --git a/doc/tutorials/storing.rst b/doc/tutorials/storing.rst index 267a377..26dd8dd 100644 --- a/doc/tutorials/storing.rst +++ b/doc/tutorials/storing.rst @@ -11,7 +11,7 @@ Start Docker container First we download the latest tag of the pgPoincloud Docker image: -.. code-block:: +.. code-block:: bash $ docker pull pgpointcloud/pointcloud @@ -25,13 +25,13 @@ For a basic usage, we have to define two environment variables: Then we can start a new container: -.. code-block:: +.. code-block:: bash $ docker run --name pgpointcloud -e POSTGRES_DB=pointclouds -e POSTGRES_PASSWORD=mysecretpassword -d pgpointcloud/pointcloud Extensions are automatically created in the new database named ``pointclouds``: -.. code-block:: +.. code-block:: bash $ docker exec -it pgpointcloud psql -U postgres -d pointclouds -c "\dx" List of installed extensions @@ -55,9 +55,9 @@ Run PDAL pipeline For the need of the tutorial, we can download sample data from the `PDAL`_ organization: -.. code-block:: +.. code-block:: bash - $ wget https://github.com/PDAL/data/raw/master/liblas/LAS12_Sample_withRGB_Quick_Terrain_Modeler_fixed.laz /tmp + $ wget https://github.com/PDAL/data/raw/master/liblas/LAS12_Sample_withRGB_Quick_Terrain_Modeler_fixed.laz -P /tmp Thanks to the ``pdal info`` command, we can obtain some information on the dataset: @@ -68,7 +68,7 @@ To configure the json PDAL pipeline, we need to set up the ``connection`` parameter for the ``pgpointcloud`` writer. To do that, the Docker container IP adress on which the PostgreSQL database is running is necessary: -.. code-block:: +.. code-block:: bash $ docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' pgpointcloud 172.17.0.2 @@ -76,27 +76,27 @@ adress on which the PostgreSQL database is running is necessary: So the ``pipeline.json`` file looks like: -.. code-block:: +.. code-block:: json { - "pipeline":[ - { - "type":"readers.las", - "filename":"/tmp/LAS12_Sample_withRGB_Quick_Terrain_Modeler_fixed.laz" - }, - { - "type":"filters.chipper", - "capacity":"400" - }, - { - "type":"writers.pgpointcloud", - "connection":"host='172.17.0.2' dbname='pointclouds' user='postgres' password='mysecretpassword' port='5432'", - "table":"airport", - "compression":"none", - "srid":"32616" - } - ] -} + "pipeline":[ + { + "type":"readers.las", + "filename":"/tmp/LAS12_Sample_withRGB_Quick_Terrain_Modeler_fixed.laz" + }, + { + "type":"filters.chipper", + "capacity":"400" + }, + { + "type":"writers.pgpointcloud", + "connection":"host='172.17.0.2' dbname='pointclouds' user='postgres' password='mysecretpassword' port='5432'", + "table":"airport", + "compression":"dimensional", + "srid":"32616" + } + ] + } The PDAL pipeline can finally be execute with ``pdal pipeline pipeline.json`` and an ``airport`` table is created. @@ -111,7 +111,7 @@ Configure connection service file To facilitate the access to the database hosted on the Docker container, we can configure the PostgreSQL connection service file: -.. code-block:: +.. code-block:: bash [pgpointcloud] host=172.17.0.2 @@ -122,13 +122,13 @@ configure the PostgreSQL connection service file: Then we can explore the content of the new ``airport`` table: -.. code-block:: +.. code-block:: bash $ psql service=pgpointcloud psql (12.3) Type "help" for help. - pointclouds=# select count(*) from airport; + pointclouds=# SELECT COUNT(*) FROM airport; count ------- 9529 diff --git a/doc/update.rst b/doc/update.rst new file mode 100644 index 0000000..50b1260 --- /dev/null +++ b/doc/update.rst @@ -0,0 +1,42 @@ +.. _update: + +****************************************************************************** +Update +****************************************************************************** + +pgPointcloud extension +------------------------------------------------------------------------------ + +Once a new version of pgPointcloud installed, you may want to update your +databases where the extension is already in use. The first thing to compare is +the version currently used with versions actually available on your system: + +.. code-block:: sql + + SELECT pc_version(); + --> 1.1.1 + + SELECT version FROM pg_available_extension_versions WHERE name ='pointcloud'; + --> 1.1.1 + --> 1.2.1 + + +Then you can update to the latest version with ``ALTER EXTENSION pointcloud +UPDATE`` or target a specific version: + +.. code-block:: sql + + ALTER EXTENSION pointcloud UPDATE TO '1.2.1'; + + SELECT pc_version(); + --> 1.2.1 + + +.. warning:: + + The GHT compression has been removed in the 1.2.0 version. Unfortunately, + you have to remove the compression on your tables before updating the + extension from 1.1.x to a higher version. Some information are available in + the `Schema and compression`_ tutorial. + +.. _`Schema and compression`: /https://pgpointcloud.github.io/pointcloud/tutorials/compression.html#schema-and-compression