36 Commits

Author SHA1 Message Date
Alan D. Snow
90375aa99a TST: ensure datasets closed (#3444) 2025-12-05 11:35:39 -06:00
Evan McClain
a9411f8d81
Handle case where a MemoryFile is passed into open (#3145)
This was being improperly passed into the `hasattr(fp, 'read')` branch

Test with a small SRTM 3 file.
2024-08-23 15:02:25 -06:00
Sean Gillies
87a582a391
Update code to Python 3.9 usage (#3132)
* Run pyupgrade --py39-up

* Run darker on previous pyupgrade changes
2024-08-10 18:22:54 -06:00
Sean Gillies
1c54c18cec Reformat memoryfile tests, change "epsg" to "EPSG". 2023-12-19 18:44:44 -07:00
Sean Gillies
5e96a133a9
Operations on closed MemoryFile and ZipMemoryFile raise ValueError (#2989) 2023-12-19 18:40:26 -07:00
Ryan Grout
10ca62add3
Prevent interpreter segfaults with closed MemoryFile. (#2870)
* Guard file operations on MemoryFile.

Calling file operations on null pointer crashes Python interpreter.

* Update closed error message.

Remove special case of tell on closed file.

* Test I/O operations on closed MemoryFile.

---------

Co-authored-by: Sean Gillies <sean.gillies@gmail.com>
2023-12-19 18:03:29 -07:00
Ryan Grout
b83e208f1b
Close derived datasets automatically when MemoryFile is closed (#2869)
* Ensure that any derived datasets are also closed.

When MemoryFile is cleaned up we should automatically close any derived datasets.

* Add tests.
2023-12-19 13:49:16 -07:00
Sean Gillies
ccca0cb9be Merge tag '1.3.7' into main 2023-05-22 18:05:41 -06:00
Sean Gillies
089cd160f6
Turn down logging level for non-existing files in the Python file VSI plugin (#2827)
* Use GDAL_PAM_ENABLED=False in combo with FilePath

Resolves #2826

* Turn down logging level in Python File VSI plugin callback

* Rename test, add missing details to FilePath docstring

* Note fix for 2826
2023-05-15 17:56:27 -06:00
Sean Gillies
55df3c7890 Clean up MemoryFile /vsimem/ dirs recursively 2023-01-10 17:20:48 -07:00
Sean Gillies
b00a6a8a9e Clean up MemoryFile /vsimem/ dirs recursively 2023-01-10 17:16:49 -07:00
Alan D. Snow
51efb0d7a8
LNT: Use flake8 to find undefined variables & unused imports (#2693) 2023-01-10 12:48:09 -06:00
Alan D. Snow
840ea6305d
CLN: Remove unnecessary GDALVersion runtime checks (#2685)
Co-authored-by: Even Rouault <even.rouault@spatialys.com>
2022-12-27 13:20:54 -06:00
Mike Taves
6eee5577de
Replace IOError alias with OSError (#2280) 2022-04-05 19:28:27 -05:00
Sean Gillies
7aa94643bd
rasterio.open returns a dataset object for all kinds of input (#2371)
* rasterio.open returns a dataset object for all kinds of input

Resolves #2360.

* Make dataset _env an ExitStack, everywhere

* propagate keyword arguments to delegates

* Remove unused function
2022-01-06 09:34:19 -07:00
David Hoese
80a67d6275
Add Python file-like VSI Plugin (#2141)
* Initial proof of concept for Python file-like VSI handler

* Add GDAL VSI Plugin definitions to gdal.pxi [skip ci]

* Add proof of concept pyvsi with GDAL Plugin API [skip ci]

* Add global filesystem state to PyVSI plugin

* Add close function handler to pyvsi plugin

* Move cython language definition to cython module

* Switch _pyvsi extension to use cpp compile options

* Add pyvsi_ prefix to GDAL callback functions to avoid collisions

* Add basic information to _pyvsi module docstring

* Remove ability to open PyVSIFile as writable

* Remove unnecessary code from PythonVSIFile

* Match new code style used in MemoryFile

* Fix PythonVSIFile argument passing to __init__

* Update memoryfile tests to use pytest fixtures instead of hardcoded paths

* Add read method to PythonVSIFile

* Add PythonVSIFile tests

* Fix xfail reason on pyvsi test

* Add concurrent pyvsi test

* Add flexible PythonVSIFile plugin importing for GDAL<3.0

* Remove legacy 'ext' keyword argument for PythonVSIFile interface

* Rename "PyVSI" to "FilePath" and remove unnecessary methods

* Rename other instances of pyvsi to filepath

* Rename other instances of pyvsi to filepath
2021-11-13 12:25:57 -07:00
Yann-Sebastien Tremblay-Johnston
11ae663ff4
Warp add rpcs (#2031)
* [warp_add_rpcs] initial update to reproject to support rpcs

* catch kwargs in _calculate_default_transform

* fix allocation of crs to wrong dataset (was src, now dst) in _calculate_default_transform. Specifically where we went from GDALCreateGenImgProjTransformer -> GDALCreateGenImgProjTransformer2

* pass kwargs to calculate_default_transform (to be passed to transformer

* update docstrings

* update documentation

* add debug messages for kwargs used to set Transformer options

* Don't capitalize RPC_DEM transformer option value, as it might be a path

* includes existing changes from https://github.com/mapbox/rasterio/pull/2030, better support for computation of calculate_default_transform with just gcps or rpcs

* adds testse

* Remove an empty line

Mainly to trigger a new build on travis-ci.com

* add new test file (Largely derived from RGB.byte.tif)

* Fix incorrectly encoding RPC metadata as transform options

Was checking for transformer option RPC_DEM in the RPC metadata by mistake

Co-authored-by: Sebastien Tremblay-Johnston <sebastien.tremblayjohnston@mdacorporation.com>
Co-authored-by: Sean Gillies <sean@mapbox.com>
Co-authored-by: Sean Gillies <sean.gillies@gmail.com>
2020-11-13 18:26:58 -07:00
Sean Gillies
8ae606de72
Allow arbitrary seek to support multi-part writes (#1935)
* Allow arbitrary seek to support multi-part writes

Resolves #1926

The implementation of MemoryFileBase has changed quite a bit: we
are eliminating custom code and are relying more on the VSI API.

Also: memory files exist now within a dedicated directory, making it
easier to ensure that we're cleaning up auxiliary files.

* Add tests confirming the fix for #1926
2020-05-23 13:28:49 -06:00
Sean Gillies
7d316a5437
Require width/height when opening a MemoryFile in w+ mode (#1752)
Resolves #1748
2019-08-21 17:39:53 -06:00
Sean Gillies
eb6549ac62 Resolves #1659 2019-03-22 15:59:55 -06:00
Sean Gillies
928c31c21e Remove unused mode and xfail mark for vrt test 2018-12-11 15:11:15 -07:00
Sean Gillies
87bd0c80a0
Make default modes r+ and w+ for MemoryFile datasets (#1487)
* Make default modes r+ and w+

* Add a JPEG test and a fix for the bug it uncovered
2018-10-02 16:34:50 -06:00
Sean C. Gillies
ea61719f1d Hide vsi_path add a failing vsimem + VRT test 2018-06-13 15:39:11 -06:00
Sean C. Gillies
304b62c920 wrap view in bytesarray for python 2.7 compat 2018-04-16 13:14:39 -06:00
Sean C. Gillies
b5b63817f3 Disallow bytearrays when creating memory files 2018-04-16 12:46:54 -06:00
Brendan Ward
fded0b1d44 Converted a few test xfails to skip, dropped or modified a few others 2017-11-30 20:13:16 -08:00
Brendan Ward
e0f62813e4 Moved comments to reason, skipped test_memoryfile module 2017-11-30 07:01:52 -08:00
Brendan Ward
bf8d4f3509 More consistent GDAL version handling in tests 2017-11-29 08:59:31 -08:00
Sean Gillies
b3b01e07e6 Add a test of using an LZW-compressed tiff in memoryfile 2017-05-25 15:46:59 +02:00
Sean Gillies
c47604df08 Add a test of MemoryFile failure with not-bytes 2017-02-02 17:39:19 +01:00
Sean Gillies
24a07e4f63 Removal of is_zip(), variable renaming, improved docstrings 2017-02-02 17:23:03 +01:00
Sean Gillies
e27ae34698 Add read-only ZipMemoryFile and more tests 2017-02-01 10:55:45 +01:00
Sean Gillies
3909ae704c MemoryFile() can take a file object parameter
And its open() method can take a path, allowing access to a
raster within a zipfile, such as a VRT.
2017-01-31 23:59:43 +01:00
Sean Gillies
99975e8397 Lightweight context managers around file objects and memory files
Resolves #964
2017-01-30 15:05:03 +01:00
Sean Gillies
f948aeba09 Two new tests to assist feature discussion in #964 2017-01-25 12:02:51 +01:00
Sean Gillies
42c3f36db0 MemoryFile (#915)
* Implementation of MemoryFile.write

* Complete initial impl of MemoryFile and tests

* Adjustment in tests and read() for lack of bytes in Python 2.

* Relax file size assertion, skip tests for GDAL<2.0

* Write a __len__ backed by VSIGetMemFileBuffer()

* Remove leftover except clause

* Add a 'closed' attribute to MemoryFile.

And various code improvements suggested by @brendan-ward.

* Rename check() to exists() and remove entry from pxd

* Revert change to DatasetBase, use case handled in MemoryFile
2016-10-24 16:20:59 +02:00