Using the Segment-Geospatial Python Package with ArcGIS Pro¶

The notebook shows step-by-step instructions for using the Segment Anything Model (SAM) with ArcGIS Pro. Check out the YouTube tutorial here and the Resources for Unlocking the Power of Deep Learning Applications Using ArcGIS. Credit goes to Esri.

Installation¶

1. Open Windows Registry Editor (regedit.exe) and navigate to Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem. Change the value of LongPathsEnabled to 1. See this screenshot. This is a known issue with the deep learning libraries for ArcGIS Pro 3.1. A future release might fix this issue.

2. Navigate to the Start Menu -> All apps -> ArcGIS folder, then open the Python Command Prompt.

3. Create a new conda environment and install mamba and Python 3.9.x from the Esri Anaconda channel. Mamba is a drop-in replacement for conda that is mach faster for installing Python packages and their dependencies.

   conda create conda-forge::mamba esri::python --name samgeo

4. Activate the new conda environment.

   conda activate samgeo

5. Install arcpy, deep-learning-essentials, segment-geospatial, and other dependencies (~4GB download).

   mamba install arcpy deep-learning-essentials leafmap localtileserver segment-geospatial -c esri -c conda-forge

6. Activate the new environment in ArcGIS Pro.

   proswap samgeo

7. Close the Python Command Prompt and open ArcGIS Pro.

8. Download this notebook and run it in ArcGIS Pro.

Import libraries¶

In [ ]:

import os
import leafmap
from samgeo import SamGeo

%matplotlib inline

import os import leafmap from samgeo import SamGeo %matplotlib inline

Download sample data¶

In this example, we will use the high-resolution aerial imagery from the USDA National Agricultural Imagery Program (NAIP). You can download NAIP imagery using the USDA Data Gateway or the USDA NCRS Box Drive. I have downloaded some NAIP imagery and clipped them to a smaller area, which are available here.

In [ ]:

workspace = os.path.dirname(arcpy.env.workspace)
os.chdir(workspace)
arcpy.env.overwriteOutput = True

workspace = os.path.dirname(arcpy.env.workspace) os.chdir(workspace) arcpy.env.overwriteOutput = True

In [ ]:

leafmap.download_file(
    url="https://github.com/opengeos/data/blob/main/naip/buildings.tif",
    quiet=True,
    overwrite=True,
)

leafmap.download_file( url="https://github.com/opengeos/data/blob/main/naip/buildings.tif", quiet=True, overwrite=True, )

In [ ]:

leafmap.download_file(
    url="https://github.com/opengeos/data/blob/main/naip/agriculture.tif",
    quiet=True,
    overwrite=True,
)

leafmap.download_file( url="https://github.com/opengeos/data/blob/main/naip/agriculture.tif", quiet=True, overwrite=True, )

In [ ]:

leafmap.download_file(
    url="https://github.com/opengeos/data/blob/main/naip/water.tif",
    quiet=True,
    overwrite=True,
)

leafmap.download_file( url="https://github.com/opengeos/data/blob/main/naip/water.tif", quiet=True, overwrite=True, )

Initialize SAM class¶

Specify the file path to the model checkpoint. If it is not specified, the model will to downloaded to the working directory.

In [ ]:

sam = SamGeo(
    model_type="vit_h",
    sam_kwargs=None,
)

sam = SamGeo( model_type="vit_h", sam_kwargs=None, )

Automatic mask generation¶

Specify the file path to the image we downloaded earlier.

In [ ]:

image = "agriculture.tif"

image = "agriculture.tif"

You can also use your own image. Uncomment and run the following cell to use your own image.

In [ ]:

# image = '/path/to/your/own/image.tif'

# image = '/path/to/your/own/image.tif'

Segment the image and save the results to a GeoTIFF file. Set unique=True to assign a unique ID to each object.

In [ ]:

sam.generate(image, output="ag_masks.tif", foreground=True, unique=True)

sam.generate(image, output="ag_masks.tif", foreground=True, unique=True)

If you run into GPU memory errors, uncomment the following code block and run it to empty cuda cache then rerun the code block above.

In [ ]:

# sam.clear_cuda_cache()

# sam.clear_cuda_cache()

Show the segmentation result as a grayscale image.

In [ ]:

sam.show_masks(cmap="binary_r")

sam.show_masks(cmap="binary_r")

Show the object annotations (objects with random color) on the map.

In [ ]:

sam.show_anns(axis="off", alpha=1, output="ag_annotations.tif")

sam.show_anns(axis="off", alpha=1, output="ag_annotations.tif")

Add layers to ArcGIS Pro.

In [ ]:

m = leafmap.arc_active_map()

m = leafmap.arc_active_map()

In [ ]:

m.addDataFromPath(os.path.join(workspace, "agriculture.tif"))

m.addDataFromPath(os.path.join(workspace, "agriculture.tif"))

In [ ]:

m.addDataFromPath(os.path.join(workspace, "ag_annotations.tif"))

m.addDataFromPath(os.path.join(workspace, "ag_annotations.tif"))

Convert the object annotations to vector format, such as GeoPackage, Shapefile, or GeoJSON.

In [ ]:

in_raster = os.path.join(workspace, "ag_masks.tif")
out_shp = os.path.join(workspace, "ag_masks.shp")

in_raster = os.path.join(workspace, "ag_masks.tif") out_shp = os.path.join(workspace, "ag_masks.shp")

In [ ]:

arcpy.conversion.RasterToPolygon(in_raster, out_shp)

arcpy.conversion.RasterToPolygon(in_raster, out_shp)

Segment waterbodies¶

In [ ]:

image = "water.tif"

image = "water.tif"

In [ ]:

sam.generate(image, output="water_masks.tif", foreground=True, unique=True)

sam.generate(image, output="water_masks.tif", foreground=True, unique=True)

In [ ]:

# sam.clear_cuda_cache()

# sam.clear_cuda_cache()

In [ ]:

sam.show_masks(cmap="binary_r")

sam.show_masks(cmap="binary_r")

In [ ]:

sam.show_anns(axis="off", alpha=1, output="water_annotations.tif")

sam.show_anns(axis="off", alpha=1, output="water_annotations.tif")

In [ ]:

m.addDataFromPath(os.path.join(workspace, "water.tif"))

m.addDataFromPath(os.path.join(workspace, "water.tif"))

In [ ]:

m.addDataFromPath(os.path.join(workspace, "water_annotations.tif"))

m.addDataFromPath(os.path.join(workspace, "water_annotations.tif"))

In [ ]:

in_raster = os.path.join(workspace, "water_masks.tif")
out_shp = os.path.join(workspace, "water_masks.shp")

in_raster = os.path.join(workspace, "water_masks.tif") out_shp = os.path.join(workspace, "water_masks.shp")

In [ ]:

arcpy.conversion.RasterToPolygon(in_raster, out_shp)

arcpy.conversion.RasterToPolygon(in_raster, out_shp)

Automatic mask generation options¶

There are several tunable parameters in automatic mask generation that control how densely points are sampled and what the thresholds are for removing low quality or duplicate masks. Additionally, generation can be automatically run on crops of the image to get improved performance on smaller objects, and post-processing can remove stray pixels and holes. Here is an example configuration that samples more masks:

In [ ]:

sam_kwargs = {
    "points_per_side": 32,
    "pred_iou_thresh": 0.86,
    "stability_score_thresh": 0.92,
    "crop_n_layers": 1,
    "crop_n_points_downscale_factor": 2,
    "min_mask_region_area": 100,
}

sam_kwargs = { "points_per_side": 32, "pred_iou_thresh": 0.86, "stability_score_thresh": 0.92, "crop_n_layers": 1, "crop_n_points_downscale_factor": 2, "min_mask_region_area": 100, }

In [ ]:

sam = SamGeo(
    model_type="vit_h",
    sam_kwargs=sam_kwargs,
)

sam = SamGeo( model_type="vit_h", sam_kwargs=sam_kwargs, )

In [ ]:

sam.generate("agriculture.tif", output="ag_masks2.tif", foreground=True)

sam.generate("agriculture.tif", output="ag_masks2.tif", foreground=True)

In [ ]:

sam.show_masks(cmap="binary_r")

sam.show_masks(cmap="binary_r")

In [ ]:

sam.show_anns(axis="off", alpha=0.5, output="ag_annotations2.tif")

sam.show_anns(axis="off", alpha=0.5, output="ag_annotations2.tif")