Modernizing R’s web-mapping capabilties

Tim Appelhans

The problem

Various web-mapping packages:

  • leaflet - CPU, 2D only, struggles with large data

  • mapdeck - GPU, no globe projection, mapbox token required

  • deckgl - GPU, no globe projection, variety of basemaps

  • mapgl - GPU, globe projection, support for maplibregl


all of these use geojson data encoding
* multiple copies and (de-)serialization steps

The solution

deck.gl supports direct upload of binary data to the GPU GeoArrow is a high-performace binary memory layout

copy and (de-)serialization overhead is minimized

The plan

geoarrowWidget (CRAN)

  • small dedicated package for high-performance data transfer from R memory to the browser
  • also supports direct access to cloud data (without reading into R memory) - e.g. (Geo)Parquet
  • essentially provides convenience wrappers for htmltools::htmlDependency() and htmltools::attachDependencies()
  • provides JavaScript libraries
    geoarrow-js apache-arrow-js    parquet-wasm 
        "0.3.3"        "21.1.0"         "0.7.1"

geoarrowWidget - usage pattern

On the R side:

  • attachGeoarrowDependencies() or attachParquetWasmDependencies()
  • attachData()
Code 
wgt = someWidgetPckage::some_widget(..., elementId = "some-widget-id")
wgt = geoarrowWidget::attachGeoarrowDependencies(widget = wgt)
wgt = geoarrowWidget::attachData(
  widget = wgt
1  , data = data
2  , file = file
3  , url = url
  , name = "<name>"
)
1
R memory data (nanoarrow_array_stream)
2
local file ((Geo)Arrow or (Geo)Parquet)
3
remote URL ((Geo)Arrow or (Geo)Parquet)
Code 
let data_fl = document.getElementById(
  "<name>" + "-geoarrowWidget-attachment"
);

fetch(data_fl.href)
  .then(result => Arrow.tableFromIPC(result))
  .then(arrow_table => {
    // this is where the JS code will go to do something with the
    // loaded geoarrow data. Here, we simply print to console...
    
    console.log(arrow_table);
  })

See vignette 'Use geoarrowWidget with an existing widget' for a complete example.

geoarrowWidget - usage pattern ctd.

Data is “attached” to the <head> of the HTML page via <link>

Code 
<head>
<meta charset="utf-8"/>
<style>body{background-color:white;}</style>
<script src="lib/htmlwidgets-1.6.4/htmlwidgets.js"></script>
<link href="lib/maplibre-gl-5.19.0/maplibre-gl.css" rel="stylesheet" />
<script src="lib/maplibre-gl-5.19.0/maplibre-gl.js"></script>
<link href="lib/mapbox-gl-draw-1.5.0/mapbox-gl-draw.css" rel="stylesheet" />
<script src="lib/mapbox-gl-draw-1.5.0/mapbox-gl-draw.js"></script>
<script src="lib/freehand-mode-1.0.0/freehand-mode.js"></script>
<script src="lib/rectangle-mode-1.0.0/rectangle-mode.js"></script>
<script src="lib/radius-mode-1.0.0/radius-mode.js"></script>
<link href="lib/maplibre-gl-geocoder-1.5.0/maplibre-gl-geocoder.css" rel="stylesheet" />
<script src="lib/maplibre-gl-geocoder-1.5.0/maplibre-gl-geocoder.min.js"></script>
<script src="lib/mapbox-gl-globe-minimap-1.2.1/bundle.js"></script>
<script src="lib/pmtiles-4.4.0/pmtiles.js"></script>
<script src="lib/h3j-h3t-0.9.2/h3j_h3t.js"></script>
<link href="lib/maptiler-geocoding-control-2.1.7/style.css" rel="stylesheet" />
<script src="lib/maptiler-geocoding-control-2.1.7/maplibregl.umd.js"></script>
<script src="lib/turf-7.3.0/turf.min.js"></script>
<script src="lib/turf-operations-1.0.0/turf-operations.js"></script>
<script src="lib/html2canvas-1.4.1/html2canvas.min.js"></script>
<script src="lib/screenshot-1.0.0/screenshot.js"></script>
<link href="lib/legend-interactivity-1.0.0/legend-interactivity.css" rel="stylesheet" />
<script src="lib/legend-interactivity-1.0.0/legend-interactivity.js"></script>
<script src="lib/maplibregl-binding-0.4.5.9000/maplibregl.js"></script>
<link href="lib/layers-control-1.0.0/layers-control.css" rel="stylesheet" />
<script src="lib/deckglScatterplot-0.0.1/addGeoArrowDeckglScatterplotLayer.js"></script>
<script src="lib/deck.gl-9.3.2/dist.min.js"></script>
<script src="lib/parquet2arrow-0.7.1/parquet2arrow.js" type="module"></script>
<script src="lib/apache-arrow-js-21.1.0/Arrow.es2015.min.js"></script>
<script src="lib/geoarrow-js-0.3.3/geoarrow.umd.min.js"></script>
1<link id="scatter-geoarrowWidget-attachment" rel="attachment" href="lib/scatter-0.0.1/scatter.arrow"/>
<script src="lib/deck.gl-geoarrow-0.3.1/dist.umd.min.js"></script>
<link href="lib/deckglgeoarrowHelpers-0.0.1/css/deckglgeoarrow.css" rel="stylesheet" />
<script src="lib/deckglgeoarrowHelpers-0.0.1/deckglgeoarrowHelpers.js"></script>
2<link id="cities-geoarrowWidget-attachment" rel="attachment" href="https://raw.githubusercontent.com/geoarrow/geoarrow-data/v0.2.0/natural-earth/files/natural-earth_cities_native.parquet"/>
</head>
1
attached local data
2
attached remote data

deckglgeoarrow

Exposes the deck.gl-geoarrow JavaScript library to R

  • Fast: copies binary buffers directly to the GPU using deck.gl’s binary data interface.
  • Memory-efficient: no intermediate data representation and no garbage-collector overhead.
  • Full layer customization: Use the same layer properties as in the upstream deck.gl layer documentation.

more features explained here

deckglgeoarrow

Implemented so far:

  • addGeoArrowScatterplotLayer - point data
  • addGeoArrowPathLayer - line data
  • aggGeoArrowPolygonLayer - polygon data

Plus helper functions for styling and layer management

deckglgeoarrow

To be implemented (soon):

  • all layers supported by deck.gl-geoarrow
    • a5, arc, column, geohash, h3, heatmap, point-cloud, s2, solid-polygon, text, trips
  • support for flatGeobuf
  • shiny support
  • support for interactivity provided by mapgl (e.g. legends)

deckglgeoarrow

1M random points + ~11k polygons + ~155k lines

Kudos to…

  • Kyle Barron - deck.gl-gearrow
  • Dewey Dunnington - all things Arrow
  • Josiah Parry - valuable feedback on the R-Consortium grant application
  • r-spatial crew

Thank you!