__init__.py
"""A mix-in handler for bulk loading data into an application.
For complete documentation, see the Tools and Libraries section of the
documentation.
To use this in your app, first write a script, e.g. bulkload.py, that
instantiates a Loader for each entity kind you want to import and call
bulkload.main(instance). For example:
person = bulkload.Loader(
'Person',
[('name', str),
('email', datastore_types.Email),
('birthdate', lambda x: datetime.datetime.fromtimestamp(float(x))),
])
if __name__ == '__main__':
bulkload.main(person)
See the Loader class for more information. Then, add a handler for it in your
app.yaml, e.g.:
urlmap:
- regex: /load
handler:
type: 1
path: bulkload.py
requires_login: true
admin_only: true
Finally, deploy your app and run bulkload_client.py. For example, to load the
file people.csv into a dev_appserver running on your local machine:
./bulkload_client.py --filename people.csv --kind Person --cookie ... \
--url http://localhost:8080/load
The kind parameter is used to look up the Loader instance that will be used.
The bulkload handler should usually be admin_only, so that non-admins can't use
the shell to modify your app's data. The bulkload client uses the cookie
parameter to piggyback its HTTP requests on your login session. A GET request
to the URL specified for your bulkload script will give you a cookie parameter
you can use (/load in the example above). If your bulkload handler is not
admin_only, you may omit the cookie parameter.
If you want to do extra processing before the entities are stored, you can
subclass Loader and override HandleEntity. HandleEntity is called once with
each entity that is imported from the CSV data. You can return one or more
entities from HandleEntity to be stored in its place, or None if nothing
should be stored.
For example, this loads calendar events and stores them as
datastore_entities.Event entities. It also populates their author field with a
reference to the corresponding datastore_entites.Contact entity. If no Contact
entity exists yet for the given author, it creates one and stores it first.
class EventLoader(bulkload.Loader):
def __init__(self):
EventLoader.__init__(self, 'Event',
[('title', str),
('creator', str),
('where', str),
('startTime', lambda x:
datetime.datetime.fromtimestamp(float(x))),
])
def HandleEntity(self, entity):
event = datastore_entities.Event(entity.title)
event.update(entity)
creator = event['creator']
if creator:
contact = datastore.Query('Contact', {'title': creator}).Get(1)
if not contact:
contact = [datastore_entities.Contact(creator)]
datastore.Put(contact[0])
event['author'] = contact[0].key()
return event
if __name__ == '__main__':
bulkload.main(EventLoader())
"""
---
bulkload.Loader.__init__(self, 'Greeting',
[
('author', datastore_types.users.User ),
('content', str ),
('curl1', str ),
('cmapinfo', datastore_types.Text),
('date', lambda x: datetime.datetime.strptime(x,'%Y-%m-%d %H:%M:%S')),
('dateJST', lambda x: datetime.datetime.strptime(x,'%Y-%m-%d %H:%M:%S')),
])
--
C:\google\bulkload>c:\google\bulkload\bulkload_client.py --filename greeting_test.csv --kind Greeting --url http://xxxx.appspot.com/load02
