NAIP on AWS

The National Agriculture Imagery Program (NAIP) acquires aerial imagery during the agricultural growing seasons in the continental United States. This “leaf-on” imagery typically ranges from 60 centimeters to 100 centimeters in resolution and is available from the naip-analytic Amazon S3 bucket as 4-band (RGB + NIR) imagery in MRF format, in naip-source Amazon S3 bucket as 4-band (RGB + NIR) in uncompressed Raw GeoTiff format and naip-visualization as 3-band (RGB) Cloud Optimized GeotTiff format. NAIP data is delivered at the state level; every year, a number of states receive updates, with an overall update cycle of two or three years.

NAIP is administered by the U.S. Department of Agriculture’s Farm Service Agency (FSA) through the Aerial Photography Field Office (APFO) in Salt Lake City.

Data is available ranging from 2012 to 2017. The NAIP coverage map gives an idea on the area where is data is available for each year.

Accessing NAIP on AWS

Cloud-optimized NAIP imagery in MRF format on AWS, converted and managed by Esri, is located in the naip-analytic S3 bucket. The data is in a Requester Pays bucket, which means that you can access it freely within the us-west-2 region, but you will incur charges if you download it elsewhere.

The files are provided as digital ortho quarter quad tiles (DOQQs). Each individual tile covers a 3.75 x 3.75 minute quarter quadrangle plus a 300 meter buffer on all four sides. The DOQQs are provided as GeoTIFF, and each tile area corresponds to a USGS topographic quadrangle. All individual DOQQs are rectified in the UTM coordinate system, NAD 83. Some states, due to their size, use more than one UTM zone. Additionally, because the data is uncompressed, data volume for one state can be large. As an example, Texas includes over 17,000 files (over 3TB of 4-band imagery) for one of its acquisition years.

Data is grouped by state, year, resolution, data type and quadrangle.

For example, al/2017/100cm/rgbir/30085/m_3008501_ne_16_1_20171018.mrf is a MRF file for a quarter-quad in the state of Alabama, taken during the 2017 season, at 100cm resolution, and is the northeastern corner of the 30085 USGS quadrangle.

The corresponding FGDC metadata file for the above MRF is located under the /fgdc/ prefix, al/2017/100cm/fgdc/30085/m_3008501_ne_16_1_20171018.txt.

Data stored under the prefix /index/ are shapefiles that contain the file extents of each MRF.

Directory Structure

Imagery : Bucketname/state/YYYY/resolution*/bands/quad/.mrf (.idx, .lrc) files. (for naip-source and naip-visualization the file extension is .tif

Metadata: Bucketname/state/YYYY/resolution/fgdc/quad/.txt files

State/Year Index: Bucketname/state/YYYY/resolution/index/.shp (.dbf, .shx, .prj, .sbn) files

* Resolution will be defined in cm, either 100cm or 60cm

Image Info

naip-analytic: This bucket contains data processed by Esri. Files are formatted as MRF, which is a cloud-optimized open data format that provides fast access to data on S3. MRF can be accessed using GDAL and most applications build using GDAL, but also directly by web applications. Data is provided as 512x512 tiles, with reduced-resolution pyramids created using 2x sampling by averaging. Finally, data compression is efficient and lossless (meaning the data values here are the same as the source). These files were created using gdal_translate with the following command:

gdal_translate -of MRF -co OPTIONS=“LERC_PREC=0.5 V2=ON -co COMPRESS=LERC src_dataset dst_dataset

gdaladdo -r average src_dataset 2 4 8 16 32 64

Note: The data upto year 2015 was compressed using Lerc1 compression and for the year 2016 and 2017 Lerc2 compression was used. Click here for more info on Lerc compression.

We used Esri’s OptimizedRasters to convert this data.

naip-source: The GeoTIFF data in this bucket is provided by the APFO, with no post-processing and without pyramids. Instead of the raw data, we recommend using the optimized MRF data from the naip-analytic bucket.

naip-visualization: This bucket contains data converted and managed by Esri. This imagery is provided as Cloud Optimized GeoTIFFs, has been compressed using YCbCr JPEG with quality 85. It is provided as 512x512 tiles, with pyramids created using 2x sampling by averaging. It was created using gdal_translate with the following command:

gdal_translate -b 1 -b 2 -b 3 -of GTiff -co tiled=yes -co BLOCKXSIZE=512 -co BLOCKYSIZE=512 -co COMPRESS=DEFLATE -co PREDICTOR=2 src_dataset dst_dataset

gdaladdo -r average -ro src_dataset 2 4 8 16 32 64

gdal_translate -b 1 -b 2 -b 3 -of GTiff -co TILED=YES -co BLOCKXSIZE=512 -co BLOCKYSIZE=512 -co COMPRESS=JPEG -co JPEG_QUALITY=85 -co PHOTOMETRIC=YCBCR -co COPY_SRC_OVERVIEWS=YES –config GDAL_TIFF_OVR_BLOCKSIZE 512 src_dataset dst_dataset

Access Manifest

To see the full list of available files, you can access the bucket manifest with the aws-cli (version 15 and above) command below:

aws s3 cp s3://naip-analytic/manifest.txt manifest.txt –request-payer aws s3 cp s3://naip-source/manifest.txt manifest.txt –request-payer aws s3 cp s3://naip-visualization/manifest.txt manifest.txt –request-payer

or a command link

aws s3 ls s3://naip-analytic/ –request-payer

The S3 buckets are provided as a Requester Pays bucket, see here for more information on Requester Pays.