9 Commits

Author SHA1 Message Date
Sean Gillies
87a582a391
Update code to Python 3.9 usage (#3132)
* Run pyupgrade --py39-up

* Run darker on previous pyupgrade changes
2024-08-10 18:22:54 -06:00
Sean Gillies
5db194fe99
Sidecar support for Python openers (#3032)
* Failing test with a .msk sidecar

* POC of sidecar support for Python openers

Towards resolving #3027

* Register opener for the parent directory/folder of the given URL

Handling the error cases properly gets all our tests passing again.

* Add opener adapter classes and a factory

Still a lot of redundant code here, needs refactoring.

* Remove unused import

* Don't raise exceptions when files aren't found

Just return 1 or NULL

* Add two tests of using fsspec filesystem objs and fix a bug.

* Allow testing with tiledb.VFS

* More tests of fsspec http filesystem and .msk sidecar

* Module clean up

* Add mtime to stats, docstrings to abstract class

* Expand documentation

* Reassign name_list in loop

* Return -1 on stat failure, remove useless except clauses

* Don't set an error when OpenFile doesn't resolve

* Add eof callback
2024-02-29 10:48:18 -07:00
Sean Gillies
90fcd571e2
Deprecate FilePath and CRS.is_valid (#2919)
* Deprecate FilePath

Resolves #2911

* Deprecate CRS.is_valid

Resolves #1624

---------

Co-authored-by: Sean Gillies <seangillies@Seans-MacBook-Air.local>
2023-12-19 13:20:05 -07:00
Sean Gillies
b9716ce3c9
Python file opener VSI plugin (#2898)
* Rough draft of a new VSI plugin

Resolves #2888 maybe

* Getting closer

* Working implementation!

The missing piece was keeping a reference to opened files.

* Add more tests

* Add more documentation about opener

* Improvements based on feedback from @rouault

* Update docstrings

* Update the VSI topic in docs

* Don't accept fsspec OpenFile instances as opener

Instead, we'll make a way for rasterio.open to accept them as the
"fp" positional argument

* Store contexts of vsipyopener file objects

Eliminates the fsspec OpenFile special case

* Strip file mode for the zipfile case

* Update rasterio/_vsiopener.pyx

Co-authored-by: Alan D. Snow <alansnow21@gmail.com>

---------

Co-authored-by: Sean Gillies <seangillies@Seans-MacBook-Air.local>
Co-authored-by: Alan D. Snow <alansnow21@gmail.com>
2023-08-31 11:46:00 -06:00
Sean Gillies
0560836d5f
Improve compatibility of FilePath with GDAL's VSI system (#2856)
* Improve compatibility of FilePath with GDAL's VSI system

Resolves #2850

* Pyparsing 3.1 has changes

* Store and track file objects, not file wrappers
2023-06-26 11:01:31 -06:00
Sean Gillies
089cd160f6
Turn down logging level for non-existing files in the Python file VSI plugin (#2827)
* Use GDAL_PAM_ENABLED=False in combo with FilePath

Resolves #2826

* Turn down logging level in Python File VSI plugin callback

* Rename test, add missing details to FilePath docstring

* Note fix for 2826
2023-05-15 17:56:27 -06:00
Sean Gillies
ac69d215a9 Set position of the underlying stream to 0 in _filepath
Resolves #2550
2022-10-10 11:35:10 -06:00
Sean Gillies
1041fe2c43 Follow up on #2141 2021-11-13 12:33:33 -07:00
David Hoese
80a67d6275
Add Python file-like VSI Plugin (#2141)
* Initial proof of concept for Python file-like VSI handler

* Add GDAL VSI Plugin definitions to gdal.pxi [skip ci]

* Add proof of concept pyvsi with GDAL Plugin API [skip ci]

* Add global filesystem state to PyVSI plugin

* Add close function handler to pyvsi plugin

* Move cython language definition to cython module

* Switch _pyvsi extension to use cpp compile options

* Add pyvsi_ prefix to GDAL callback functions to avoid collisions

* Add basic information to _pyvsi module docstring

* Remove ability to open PyVSIFile as writable

* Remove unnecessary code from PythonVSIFile

* Match new code style used in MemoryFile

* Fix PythonVSIFile argument passing to __init__

* Update memoryfile tests to use pytest fixtures instead of hardcoded paths

* Add read method to PythonVSIFile

* Add PythonVSIFile tests

* Fix xfail reason on pyvsi test

* Add concurrent pyvsi test

* Add flexible PythonVSIFile plugin importing for GDAL<3.0

* Remove legacy 'ext' keyword argument for PythonVSIFile interface

* Rename "PyVSI" to "FilePath" and remove unnecessary methods

* Rename other instances of pyvsi to filepath

* Rename other instances of pyvsi to filepath
2021-11-13 12:25:57 -07:00