Error reading from an Arrow ArrayStream on a GeoParquet layer with an attribute filter defined
Error attempting to reproduce in R the examples for Source Coop DEM-Terrain at https://source.coop/walkthru-earth/dem-terrain (using the V1 files that have `geometry` column):
* reading from the ArrayStream is done by `nanoarrow`
* a schema validation error occurs when reading from the Parquet layer if it has an attribute filter defined
* same result if the Parquet file is opened as a SQL layer instead (with same WHERE filter)
* no error and the data are read as expected if no attribute filter is defined (attributes look correct and geometries returned as WKB)
Also, setting an attribute filter on a GeoPackage layer does not generate the error when reading via ArrayStream (tested against the NZ Building Outlines GPKG used for benchmarks at https://firelab.github.io/gdalraster/articles/vector-read-benchmarks.html).
The code below was run in a container from `ubuntu-full-latest` GDAL 3.13dev. The same result was seen with a local build of GDAL 3.12.1.
### Error with attribute filter set:
```r
library(gdalraster)
#> GDAL 3.13.0dev-12f6982db49c570208274f5ad9aa5aa2bf86b5eb (released 2026-03-08), GEOS 3.12.1, PROJ 9.9.0
packageVersion("nanoarrow")
#> [1] ‘0.8.0’
set_config_option("AWS_NO_SIGN_REQUEST", "YES")
set_config_option("AWS_REGION", "us-west-2")
options(nanoarrow.warn_unregistered_extension = FALSE)
f <- "/vsis3/us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/v1/h3/h3_res=5/data.parquet"
(lyr <- new(GDALVector, f))
#> C++ object of class GDALVector
#> Driver : (Geo)Parquet (Parquet)
#> DSN : /vsis3/us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/h3_res=5/data.parquet
#> Layer : data
#> CRS : WGS 84 (EPSG:4326)
#> Geom : POINT
lyr$getFieldNames()
#> [1] "h3_index" "lat" "lon" "elev" "slope" "aspect" "tri"
#> [8] "tpi" "geometry"
lyr$setAttributeFilter("lat BETWEEN 35 AND 45 AND lon BETWEEN -10 AND 5")
(stream <- lyr$getArrowStream())
#> <nanoarrow_array_stream struct<h3_index: string, geometry: geoarrow.wkb{binary}, lat: float, lon: float, elev: float, slope: float, aspect: float, tri: float, tpi: float>>
#> $ get_schema:function ()
#> $ get_next :function (schema = x$get_schema(), validate = TRUE)
#> $ release :function ()
arr <- stream$get_next()
#> Error in nanoarrow_array_set_schema(array, schema, validate = validate) :
#> Expected string array buffer 0 to have size >= 27 bytes but found buffer with 0 bytes
lyr$close()
```
### No error without the attribute filter:
```r
library(gdalraster)
#> GDAL 3.13.0dev-12f6982db49c570208274f5ad9aa5aa2bf86b5eb (released 2026-03-08), GEOS 3.12.1, PROJ 9.9.0
set_config_option("AWS_NO_SIGN_REQUEST", "YES")
set_config_option("AWS_REGION", "us-west-2")
options(nanoarrow.warn_unregistered_extension = FALSE)
f <- "/vsis3/us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/v1/h3/h3_res=5/data.parquet"
(lyr <- new(GDALVector, f))
#> C++ object of class GDALVector
#> Driver : (Geo)Parquet (Parquet)
#> DSN : /vsis3/us-west-2.opendata.source.coop/walkthru-earth/dem-terrain/h3_res=5/data.parquet
#> Layer : data
#> CRS : WGS 84 (EPSG:4326)
#> Geom : POINT
(stream <- lyr$getArrowStream())
#> <nanoarrow_array_stream struct<h3_index: string, geometry: geoarrow.wkb{binary}, lat: float, lon: float, elev: float, slope: float, aspect: float, tri: float, tpi: float>>
#> $ get_schema:function ()
#> $ get_next :function (schema = x$get_schema(), validate = TRUE)
#> $ release :function ()
# without filter
arr <- stream$get_next()
typeof(arr)
#> [1] "externalptr"
class(arr)
#> [1] "nanoarrow_array"
d <- as.data.frame(arr)
nrow(d)
#> [1] 65536
head(d)
#> h3_index
#> 1 85000133fffffff
#> 2 850001a3fffffff
#> 3 850001abfffffff
#> 4 850001b3fffffff
#> 5 85001803fffffff
#> 6 85001813fffffff
#> geometry
#> 1 01, 01, 00, 00, 00, 00, 00, 00, 40, 5b, 89, 40, 40, 00, 00, 00, 00, f0, 16, 54, 40
#> 2 01, 01, 00, 00, 00, 00, 00, 00, e0, f0, ea, 3f, 40, 00, 00, 00, c0, 5c, 0c, 54, 40
#> 3 01, 01, 00, 00, 00, 00, 00, 00, 80, d2, 65, 40, 40, 00, 00, 00, 20, df, 0d, 54, 40
#> 4 01, 01, 00, 00, 00, 00, 00, 00, 60, 98, a9, 3f, 40, 00, 00, 00, a0, 43, 03, 54, 40
#> 5 01, 01, 00, 00, 00, 00, 00, 00, e0, 97, 40, 49, 40, 00, 00, 00, a0, df, 41, 54, 40
#> 6 01, 01, 00, 00, 00, 00, 00, 00, 20, 0d, f7, 48, 40, 00, 00, 00, 80, f2, 39, 54, 40
#> lat lon elev slope aspect tri tpi
#> 1 80.35840 33.07310 0.5000000 NA NA NA NA
#> 2 80.19316 31.91774 194.3317871 2.870463 324.89664 17.608427 -0.2735468
#> 3 80.21674 32.79549 348.1593933 1.951790 66.51669 6.001579 -1.4144241
#> 4 80.05100 31.66248 21.9455624 4.065749 225.69536 20.571117 -6.9174838
#> 5 81.02927 50.50463 0.7928278 NA NA NA NA
#> 6 80.90543 49.93009 18.2626114 1.200685 215.51103 8.230590 0.6501796
stream$release()
lyr$close()
```
关闭于 2026-03-28 8 条评论