Created
August 15, 2011 06:40
-
-
Save eightysteele/1145809 to your computer and use it in GitHub Desktop.
Bulkload post_import_function that shims in a repeated StructuredProperty.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def add_polygons(input_dict, instance, bulkload_state_copy): | |
| """Bulkload post_import_function that shims in a repeated StructuredProperty. | |
| This function allows us to bulkload a StructuredProperty model using the App | |
| Engine bulkloader by adding the following lines to the bulkload.yaml | |
| configuration file: | |
| - kind: LayerIndex | |
| connector: csv | |
| connector_options: | |
| encoding: utf_8 | |
| property_map: | |
| - property: __key__ | |
| external_name: id | |
| post_import_function: our_bulkloader_transforms.add_polygons | |
| Then, given our LayerPolygon model: | |
| class LayerPolygon(model.Model): | |
| polygonid = model.StringProperty(required=True) | |
| areaid = model.StringProperty(required=True) | |
| specieslatin = model.StringProperty(required=True) | |
| source = model.StringProperty(required=True) | |
| We create a StructuredProperty in our LayerIndex model: | |
| class LayerIndex(model.Expando): | |
| polygons = model.StructuredProperty(LayerPolygon, repeated=True) | |
| After the bulkload we can filter for the StructuredProperty field values as | |
| described in the Datastore Plus Tutorial, Part One (http://goo.gl/ky5sa). For | |
| example: | |
| qry = LayerIndex.query( | |
| LayerIndex.polygons == LayerPolygon( | |
| specieslatin='puma concolor', polygonid=2)) | |
| """ | |
| # Model the LayerPolygon properties as parallel arrays | |
| polygonid = [] | |
| areaid = [] | |
| specieslatin = [] | |
| source = [] | |
| # Populate the parallel arrays from values in the input_dict (a CSV row) | |
| polygons = simplejson.loads(input_dict['polygons']) | |
| for p in polygons: | |
| polygonid.append(p['polygonid']) | |
| areaid.append(p['areaid']) | |
| specieslatin.append(p['specieslatin']) | |
| source.append(p['source']) | |
| # Update entity instance with new property names that match the StructuredProperty | |
| instance['polygons.polygonid'] = polygonid | |
| instance['polygons.areaid'] = areaid | |
| instance['polygons.specieslatin'] = specieslatin | |
| instance['polygons.source'] = source | |
| return instance |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment