Scalability

The following sections explain how to work efficiently with large geospatial datasets through chunked execution with Dask and multiprocessing, downsampling to reduce data before or during an operation, all while considering file formats and cloud storage.

For measured comparisons between execution backends and tools to inspect your own workflows, see Benchmarking.

• Usage and good practices
  • Using Dask through accessors
  • Using Multiprocessing through GeoUtils objects
  • Good practices with chunked and lazy operations
• Concept definitions
  • Deferred I/O and implicit loading
  • Chunked execution
  • Lazy execution
• Supported operations
  • Table summary
  • Preliminary reads
  • Computation breakpoints
• Implementation strategies
  • Summary
  • Logic of implementations
• Downsampling and overviews
  • Downsampling when opening
  • Downsampling when plotting
  • Downsampling within operations
  • Raster overviews
  • Point cloud levels of detail
• Formats and cloud storage
  • Choosing a file format
  • Reading from cloud storage
  • Reading and writing in chunks
• Multiprocessing configuration
  • Using MultiprocConfig
  • Advanced block functions