Skip to content

Instantly share code, notes, and snippets.

@reedobrien
Created July 28, 2015 21:08
Show Gist options
  • Save reedobrien/a7cf32c3e4e6cad0deb6 to your computer and use it in GitHub Desktop.
Save reedobrien/a7cf32c3e4e6cad0deb6 to your computer and use it in GitHub Desktop.
Upload docs to AWS CloudSearch with boto3
import json
import boto3
settings = {
"aws.cloudsearch.doc":
"<doc-ep>",
"aws.cloudsearch.search":
"<search-ep>"
}
from avail.aws.cloudsearch import get_cloudsearch_endpoint_from_settings
doc = get_cloudsearch_endpoint_from_settings("doc", settings)
search = get_cloudsearch_endpoint_from_settings("search", settings)
one = {
"id": "201405140003HQ",
"type": "add",
"fields": {
"date_created": "2015-07-28T12:13:14Z", "description":
"""Expedition 39 Commander Koichi Wakata of the Japan Aerospace Exploration Agency (JAXA) is helped of the Soyuz Capsule just minutes after he and Soyuz Commander Mikhail Tyurin of Roscosmos, and Flight Engineer Rick Mastracchio of NASA, landed in their Soyuz TMA-11M spacecraft near the town of Zhezkazgan, Kazakhstan on Wednesday, May 14, 2014. Wakata, Tyurin and Mastracchio returned to Earth after more than six months onboard the International Space Station where they served as members of the Expedition 38 and 39 crews. Photo Credit: (NASA/Bill Ingalls)""",
"keywords": [
"Expedition 39", "Expedition 39 Landing", "Kazakhstan",
"Koichi Wakata", "Roscosmos (Russian Federal Space Agency)",
"Soyuz Capsule", "Zhezkazgan"
], "location": "Zhezkazgan, Kazakhstan", "media_type":
"image", "nasa_id": "201405140003HQ", "photographer":
"NASA/Bill Ingalls", "title": "Expedition 39 Soyuz TMA-11M Landing"
}
}
two = {
"id": "201401050001HQ",
"type": "add",
"fields": {
'date_created': '2014-01-05T16:38:57Z', 'description':
"""An Orbital Sciences Corporation Antares rocket is seen as it is rolled out to launch Pad-0A at NASA's Wallops Flight Facility, Sunday, January 5, 2014 in advance of a planned Wednesday, Jan. 8th, 1:32 p.m. EST launch, Wallops Island, VA. The Antares will launch a Cygnus spacecraft on a cargo resupply mission to the International Space Station. The Orbital-1 mission is Orbital Sciences' first contracted cargo delivery flight to the space station for NASA. Among the cargo aboard Cygnus set to launch to the space station are science experiments, crew provisions, spare parts and other hardware. Photo Credit: (NASA/Bill Ingalls)""",
'keywords': [
"Antares", "Cygnus", "Mid-Atlantic Regional Spaceport(MARS)",
"NASA Wallops Flight Facility (WFF)", "Rocket", "Virginia"
], 'location': 'Mid-Atlantic Regional Spaceport (MARS)', 'media_type':
'image', 'nasa_id': '201405140003HQ', 'photographer':
'NASA/Bill Ingalls', 'title': 'Antares Rocket Rollout'
}
}
three = {
"id": "201501280010HQ",
"type": "add",
"fields": {
"date_created": "2015-01-28T10:38:00Z", "description":
"""NASA Administrator Charles Bolden speaks to NASA personnel and others during a wreath laying ceremony as part of NASA's Day of Remembrance, Wednesday, Jan. 28, 2015, at Arlington National Cemetery in Arlington, Va. The wreaths were laid in memory of those men and women who lost their lives in the quest for space exploration. Photo Credit: (NASA/Joel Kowsky)""",
"keywords": ["Arlington", "Arlington National Cemetery",
"Challenger Memorial", "Charles Bolden",
"Columbia Memorial", "Day of Remembrance", "NASA", "USA",
"VA", "Virginia"], "location":
"Arlington National Cemetary", "media_type": "image", "nasa_id":
"201501280010HQ", "photographer": "NASA/Joel Kowsky", "title":
"Day of Remembrance"
}
}
four = {"id": "201405140003HQV1", "type": "delete"}
five = {"id": "201405140003HQV2", "type": "delete"}
doclist = json.dumps([one, two, three])
print(doc.upload_documents(documents=doclist, contentType="application/json"))
@huanganqing
Copy link

hello, where can i find avail.aws.cloudsearch ? thanks.

@dnm1977
Copy link

dnm1977 commented Mar 10, 2016

Second for knowing what "avail.aws.cloudsearch" is and where we can find it. Google gives no useful results (other than some other gists that don't tell us where to find it). Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment