Python script to pull hosted features with attachments into a local file geodatabase. See ReadMe below.

ReadMe.md

1. Create local file geodatabase to hold data and attachments you want to download from ArcGIS Online (called data.gdb in script)

2. Create feature class (called myLayer in script), enable attachments, add globalID's

3. Add the following field to the feature class -GlobalID_str, text, length: 50

4. Create table called MatchTable (called MatchTable in script).

5. Add the following fields to the MatchTable table:

   • GlobalID_Str, text, length: 50
   • PhotoPath, text length: 255

6. Enable "sync" on hosted feature service (http://resources.arcgis.com/en/help/arcgisonline/index.html#//010q000000n0000000)

7. Open AGOL_pullFeatures script in text editor and modify the following: -ArcGIS Online username/password -REST url to feature service to pull from -path to and name of local file geodatabase -fields to pull from the hosted feature service (must match local feature class) -name of local feature class (this will hold the data from the hosted service and the attachments)

AGO_PullHostedFeatures.py

import os, urllib, urllib2, datetime, arcpy, json


## ============================================================================== ##
## function to update a field - basically converts longs to dates for date fields ##
## since json has dates as a long (milliseconds since unix epoch) and geodb wants ##
## a proper date, not a long.
## ============================================================================== ##
def updateValue(row,field_to_update,value):
    outputfield=next((f for f in fields if f.name ==field_to_update),None)  #find the output field
    if outputfield == None or value == None:        #exit if no field found or empty (null) value passed in
        return
    if outputfield.type == 'Date':
        if value > 0 :                                            # filter "zero" dates
            value = datetime.datetime.fromtimestamp(value/1000)   # convert to date - this is local time, to use utc time
            row.setValue(field_to_update,value)                   # change "fromtimestamp" to "utcfromtimestamp"
    else:
        row.setValue(field_to_update,value)
    return
## ============================================================================== ##

### Generate Token ###
gtUrl = 'https://www.arcgis.com/sharing/rest/generateToken'
gtValues = {'username' : 'agol_user',
'password' : 'agol_password',
'referer' : 'http://www.arcgis.com',
'f' : 'json' }
gtData = urllib.urlencode(gtValues)
gtRequest = urllib2.Request(gtUrl, gtData)
gtResponse = urllib2.urlopen(gtRequest)
gtJson = json.load(gtResponse)
token = gtJson['token']

### Create Replica ###
### Update service url HERE ###
crUrl = 'http://services.arcgis.com/xyz123456orgid/arcgis/rest/services/myFeatures/FeatureServer/CreateReplica'

crValues = {'f' : 'json',
'layers' : '0',
'returnAttachments' : 'true',
'token' : token }
crData = urllib.urlencode(crValues)
crRequest = urllib2.Request(crUrl, crData)
crResponse = urllib2.urlopen(crRequest)
crJson = json.load(crResponse)
replicaUrl = crJson['URL']
urllib.urlretrieve(replicaUrl, 'myLayer.json')

### Get Attachment ###
cwd = os.getcwd()
with open('myLayer.json') as data_file:
    data = json.load(data_file)

for x in data['layers'][0]['attachments']:
    gaUrl = x['url']
    gaFolder = cwd + '\\photos\\' + x['parentGlobalId']
    if not os.path.exists(gaFolder):
        os.makedirs(gaFolder)
    gaName = x['name']
    gaValues = {'token' : token }
    gaData = urllib.urlencode(gaValues)
    urllib.urlretrieve(url=gaUrl + '/' + gaName, filename=os.path.join(gaFolder, gaName),data=gaData)

### Create Features ###
rows = arcpy.InsertCursor(cwd + '/data.gdb/myLayer')
fields = arcpy.ListFields(cwd + '/data.gdb/myLayer')

for cfX in data['layers'][0]['features']:
    pnt = arcpy.Point()
    pnt.X = cfX['geometry']['x']
    pnt.Y = cfX['geometry']['y']
    row = rows.newRow()
    row.shape = pnt

    ### Set Attribute columns HERE ###
    ## makes use of the "updatevalue function to deal with dates ##

    updateValue(row,'Field1', cfX['attributes']['Field1'])
    updateValue(row,'Field2', cfX['attributes']['Field2'])
    updateValue(row,'Field3', cfX['attributes']['Field3'])
    updateValue(row,'Field4', cfX['attributes']['Field4'])
    # leave GlobalID out - you cannot edit this field in the destination geodb

    #comment out below fields if you don't have them in your online or destination geodb (editor tracking)
    updateValue(row,'CreationDate', cfX['attributes']['CreationDate'])
    updateValue(row,'Creator', cfX['attributes']['Creator'])
    updateValue(row,'EditDate', cfX['attributes']['EditDate'])
    updateValue(row,'Editor', cfX['attributes']['Editor'])

    updateValue(row,'GlobalID_str', cfX['attributes']['GlobalID'])

    rows.insertRow(row)

del row
del rows

### Add Attachments ###
### Create Match Table ###
rows = arcpy.InsertCursor(cwd + '/data.gdb/MatchTable')

for cmtX in data['layers'][0]['attachments']:
    row = rows.newRow()

    row.setValue('GlobalID_Str', cmtX['parentGlobalId'])
    row.setValue('PhotoPath', cwd + '\\photos\\' + cmtX['parentGlobalId'] + '\\' + cmtX['name'])

    rows.insertRow(row)

del row
del rows

### Add Attachments ###
arcpy.AddAttachments_management(cwd + '/data.gdb/myLayer', 'GlobalID_Str', cwd + '/data.gdb/MatchTable', 'GlobalID_Str', 'PhotoPath')

I am also getting the json decoding error. Where does ?token=' + token get added in the script?

Wondering if someone could help. A consultant built this code for a project and it was running fine until a couple of months ago. The error I am getting is:
Traceback (most recent call last):
File "C:\Users\gfausel\Desktop\NJTPA_ETL_Desktop.py", line 380, in
with zipfile.ZipFile(zip, "r") as z:
File "C:\Python27\ArcGIS10.3\lib\zipfile.py", line 770, in init
self._RealGetContents()
File "C:\Python27\ArcGIS10.3\lib\zipfile.py", line 811, in _RealGetContents
raise BadZipfile, "File is not a zip file"
BadZipfile: File is not a zip file.

Here is part of the code where it does the zip:
#Extract the zip file to the temp directory and get the GDB name
with zipfile.ZipFile(zip, "r") as z:
z.extractall(tempdir)
ziplist = z.namelist()

    zipname = ziplist[0]
    GDBName = zipname[0:(zipname.find(".gdb")+4)]

Any one getting this error? I do not know python so any help would be much appreciated.

If you are looking to sync hosted Feature Layer attachments on disk (so you can view images/documents/etc. outside of ArcGIS), you can check out the Downloading Feature Layer Attachments via the ArcGIS API for Python script on GeoNet.

@MKellyEsri thanks for your script! Should probably post this comment over on Esri Developer where you pushed it, but figured I'd put it here since the comment is related to attachments and I have a question about the JSON also. Any chance you're willing to modify your code to handle the establishment of local attachments to features? Your code is nice to download all the attachments, but I think it's a more common user need to download the attachments and associate them with features that you have downloaded. I think users commonly need to grab their features and associated attachments off of AGOL to do things with them in Desktop GIS and on PC. At least that's a really common need of mine. Also, I'm pretty sure my environment is configured correctly, and I had to comment everything about display and logger out to get your code to work.

@gEYEzer - how did you modify your JSON call? I've been tinkering and failing so far. Trying to avoid completely deconstructing the requests portion of the script to understand the issue...

It's now possible to grab a feature service and attachments and relationship class all in go from the server itself in a file-gdb, meaning I don't think this script is needed any more.

Point browser to http://services.arcgis.com/{xxx123456xxx}/arcgis/rest/services/{folder_name}/FeatureServer//createReplica

Set values to the below, unlisted items can just use default. Click on the resultant “statusURL”, and then “Result Url”, save zip file in wherever and extract a file-gdb with everything intact (except symbology).

Replica Name                  some_meaningful_name
Layers                        0,3 (select by index number)
Return Attachments            TRUE
Return Attachments by Url     TRUE
Create Replica Asynchronously TRUE
Sync                          None
Data Format                   FileGDB

Source: http://forums.arcgis.com/threads/60863-Export-Feature-Service-with-Attachments

Hello,

I did a lot of Searching on Internet, but did not find a proper documentation on replicas. I would like to know what happens to the Created Replicas. I wanted to export feature services to GDB on a daily basis(There are almost 40 feature services).
So can I create a replica everyday and then get the GDB?
After creating replicas with the options you mentioned, I'm able to download the GDB. But I cannot see the replicas. So, I do not know how to remove the replica because I don't find a replica ID.
My main concern is about creating replicas everyday which we don't use any other time except on the day which we are downloading GDBs.
Can you please help me with these questions.

My notebook does not work in ArcGIS Pro 3.0. I hope to know if anyone could help me how to export Feature Hosted Server Layer to SDE, I hope to back up the layer in SDE in python. Thank you very much.