Google App Engine SDK1.1.0

警告がでるようになったようだが、今のところ特に問題なし。

http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes
http://code.google.com/appengine/downloads.html

WARNING 2008-05-29 05:45:27,265 dev_appserver.py]
Could not initialize images API;
you are likely missing the Python "PIL" module. ImportError: No module named PIL

SearchableModel が__searchable_text_indexを生成

http://groups.google.com/group/google-appengine/browse_thread/thread/f64eacbd31629668
We've included a short-term full text search library in the
google.appengine.ext.search module. It's limited, so we don't discuss
it much in the documentation. We expect to provide a more robust, full
featured solution eventually.


http://groups.google.com/group/google-appengine/browse_thread/thread/4b912691a00eea59
Having created a SearchableModel entity, updating it doesn't seem to
update the full text index - Is that expected behaviour?

A little unit test to explain what I mean:

import sys, os, time
sys.path.append('/usr/local/google_appengine')
sys.path.append('/usr/local/google_appengine/lib/yaml/lib')

import unittest
from google.appengine.api import apiproxy_stub_map
from google.appengine.api import datastore_file_stub
from google.appengine.api import mail_stub
from google.appengine.api import urlfetch_stub
from google.appengine.api import user_service_stub
from google.appengine.ext import db, search

APP_ID = u'test'
AUTH_DOMAIN = 'gmail.com'
LOGGED_IN_USER = 't...@example.com'

class Searchable(search.SearchableModel):
searchtext = db.TextProperty()
.
.
.

Bulk Loader

http://code.google.com/appengine/articles/bulkload.html

1. c:\google\bulkload　フォルダを作成し、ここに以下のファイルを作成。
　myloader.py
　people.csv

2. c:/Program Files/Google/google_appengine/google/appengine/tools/bulkload_client.py
を c:\google\buloload に copy

3. c:\google> dev_appserver.py books/ 作成済みのプログラムを起動

起動まえに 以下のファイルに /load を追加

--- c:\google\books\app.yaml
- url: /
script: books.py

- url: /load
script: myloader.py
login: admin

実行
C:\google\bulkload>bulkload_client.py --filename people.csv
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　--kind Person
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　--url http://localhost:8080/load

3. ImportError: No module named google.appengine.ext.bulkload がでるので
3.1 C:\Program Files\Google\google_appengine\google\appengine\ext\bulkload\constants.py
を c:\google\buloload に copyし

3.2 -- c:\google\buloload\bulkload_client.py を一部改訂 --
try:
from google.appengine.ext.bulkload import constants
except:
import constants

3.3 再度実行したが login error となるので books/app.yaml を修正*1
C:\google\bulkload>bulkload_client.py --filename people.csv
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　--kind Person
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　--url http://localhost:8080/load
INFO 2008-05-28 21:37:29,276 bulkload_client.py]
　　　　　　　　　　　　Starting import; maximum 10　entities per post
INFO 2008-05-28 21:37:29,276 bulkload_client.py]
　　　　　　　　　　　　Importing 3 entities in 75 bytes
ERROR 2008-05-28 21:37:29,292 bulkload_client.py]
　　　　　　　　　　　　　An error occurred while importing: Received code 302:
　　　　　　　　　　　　　Requires login　ERROR
　　　　　2008-05-28 21:37:29,292 bulkload_client.py]
　　　　　　　　　　　　　Import failed

Success
C:\google\bulkload>
　　bulkload_client.py --filename people.csv --kind Person --url http://localhost:8080/load
INFO 2008-05-28 21:37:48,947 bulkload_client.py] Starting import;
　　　　　　　　　　 maximum 10 entities per post
INFO 2008-05-28 21:37:48,947 bulkload_client.py] Importing 3 entities in 75 bytes
INFO 2008-05-28 21:37:54,119 bulkload_client.py] Import succcessful

---
*1 bulkload.py は http で post しているので 安易に app.yaml の
　　　admin 部分のコメントアウトは危険。
　　　元のアプリケーションに　
　　　 'login_url': users.CreateLoginURL(self.request.uri),
　　　を追加して、ログインしておく。

注意メモ

db.StringProperty(multiline=True) のデータを
\n で split して処理するようなケースは注意
\n に限らず split する際は注意

for u in url.split('\n'):

この Propety のデータが表示されなくなる？　落とし穴がある様子。
どこかがわるいのか、なんらかの不具合のからみか？

回避策：
dev_appserver.py により再起動することで表示されるようになる。

Google にはサーバーの運用の実績はあるけれども、ユーザーの開発をサポートした実績は
まだあまりなく、したがって、こうした環境下でのデータの信頼性についても、実績はまだない。
また Gql の where 句の条件に <,> <=. >= などは one property per query

BadFilterError: BadFilterError:
invalid filter: Only one property per query may have inequality filters
 (<=, >=, <, >)..

しか使えないというのも気がつかなかった。
( 同じ Property には複数回つかえる 
　○   date >:1 and date  < :2
 ×   date >:1 and date2 < :2

工夫すればなんとかなることだけれども。　あと　select max(XX) 　などというのも、まだ(?)
ないので、最小値、最大値なども求めるのも、自分で処理しなければならない。
このあたりはやはり　RDBMS の歴史と実績を感じる。

elementtree.ElementTree

from google.appengine.api import urlfetch
from elementtree.ElementTree import *

url="http://api.flickr.com/services/rest/"
url += "?method=flickr.photos.search&text=test"
url += "&api_key=XXX"
url += "&per_page=3"
result = urlfetch.fetch(url)
#print result.content

tree = ElementTree(fromstring(result.content))

for item in tree.findall('.//photos'):
 print item[0].get('id')
 print item[0].get('title')
for d in item:
 print d.get('title')

http://python.matrix.jp/modules/ElementTree.html

There's no need to install anything, ElementTree is included in
Python 2.5

http://docs.python.org/lib/module-xml.etree.ElementTree.html

flickr.photos.search (Google App Engine)

from google.appengine.api import urlfetch
from xml.dom import minidom

url="http://api.flickr.com/services/rest/?method=flickr.photos.search&text=test&api_key=XXX"
result = urlfetch.fetch(url)

dom = minidom.parseString(result.content)

photos = []
for node in dom.getElementsByTagName('photo'):
photos.append({
  'server': node.getAttribute('server'),
  'id': node.getAttribute('id'),
  'secret': node.getAttribute('secret'),
  'owner': node.getAttribute('owner'),
  'title': node.getAttribute('title')
})

# print node.getAttribute('id'), \
#       node.getAttribute('title')

----------------------------
[ templates]
{% for e in photos %}
<a href="http://www.flickr.com/photos/{{ e.owner }}/{{ e.id }}/sizes/" >
<img src="http://static.flickr.com/{{ e.server }}/{{ e.id }}_{{ e.secret }}_s.jpg"
title="{{ e.title }}" border=0 /></a>
{% endfor %}

getElementsByTagName

<?xml version="1.0" encoding="utf-8" ?> <rsp stat="ok"> <method>flickr.test.echo</method> <name>value</name> <api_key>7c7ac3c40f14a809XXXX</api_key> </rsp>

から valueを取り出す。

from google.appengine.api import urlfetch
from xml.dom import minidom 
url="http://api.flickr.com/services/rest/?method=flickr.test.echo&name=value&api_key=XXX"
result = urlfetch.fetch(url)
print "---result"
print result
print "---result.content"
print result.content
dom = minidom.parseString(result.content)
r =dom.getElementsByTagName('name')[0].firstChild.data
print "---dom.getElementsByTagName"
print r

<?xml version="1.0" encoding="utf-8" ?> <rsp stat="fail"> <err code="100" msg="Invalid API Key (Key not found)" /> &l

からfail を取り出すにはdom.getElementsByTagName('rsp')[0].getAttribute('stat')

http://www.geocities.jp/xmlfirststep/pydom/pydom5.html

Google App Engine Interactive Console

urlfetch した結果の result ( object ) 　を
print result.content
とすると、　返信された xml の内容が確認できます。

from google.appengine.api import urlfetch
from xml.dom import minidom 
url="http://api.flickr.com/services/rest/?method=flickr.test.echo&name=value&api_key=XXX"
result = urlfetch.fetch(url)
print "---result"
print result
print "---result.content"
print result.content
dom = minidom.parseString(result.content)
r =dom.getElementsByTagName('name')[0].firstChild.data
print "---dom.getElementsByTagName"
print r

Request

---

Request Format には３つの書式があり、 REST の場合、URL に記述することだけで
リクエストを送信することができます。

http://api.flickr.com/services/rest/?method=flickr.test.echo&name=value

実際には以下のように必ず取得した api_key を追加しないとエラーとなります。

http://api.flickr.com/services/rest/?method=flickr.test.echo&name=value&api_key=XXXX

http://www.flickr.com/services/api/

A working XML parsing example!

http://groups.google.com/group/google-appengine/browse_thread/thread/85b7d03ff0d4ff2b/9fdfec112a4c051a


SimpleXMLTreeBuilder is no longer supported in the latest SDK. Returns
"object is unsubscriptable" error.


Update: I hope this is still useful for people searching for a working
example!

I can confirm that minidom now works in SDK 1.0.2. As I understand
it, this is because pyexpat is now in the GAE whitelist of C
libraries:

http://code.google.com/appengine/kb/libraries.html

I am using SDK 1.0.2 on Windows XP, therefore for this to work I
needed to patch urlfetch_stub.py because the development server was
discarding my URL parameters.

http://code.google.com/p/googleappengine/issues/detail?id=341

You no longer need to use upload a version of the SimpleXMLTreeBuilder
(although this still works) and the code using minidom now looks like
this:

-----

from google.appengine.api import urlfetch
from xml.dom import minidom

WEATHER_URL = 'http://xml.weather.yahoo.com/forecastrss?p=%s'
WEATHER_NS = 'http://xml.weather.yahoo.com/ns/rss/1.0'

def parse( url ) :
result = urlfetch.fetch(url)
if result.status_code == 200:
       return minidom.parseString(result.content)

def weather_for_zip(zip_code):
url = WEATHER_URL % zip_code
dom = parse(url)
forecasts = []
for node in dom.getElementsByTagNameNS(WEATHER_NS, 'forecast'):
    forecasts.append({
        'date': node.getAttribute('date'),
        'low': node.getAttribute('low'),
        'high': node.getAttribute('high'),
        'condition': node.getAttribute('text')
    })
return {
    'forecasts': forecasts,
    'title': dom.getElementsByTagName('title')[0].firstChild.data
}

print 'Content-Type: text/plain'
print ''
print weather_for_zip('94089')


{'title': u'Yahoo! Weather - Sunnyvale, CA', 'forecasts': [{'date': u'22 May 2008', 'high': u'75', 'low': u'53', 'condition': u'Sunny'}, {'date': u'23 May 2008', 'high': u'73', 'low': u'53', 'condition': u'Partly Cloudy'}]}

---

http://www.kharakawa.com/kh.log/archives/python/

Broken Datastore

IMProperty での例があったけれども、不正なデータが datatore に登録されて、
削除できなくなる ( fetch も) できなくなってしまう不具合がある。

http://groups.google.com/group/google-appengine/browse_thread/thread/3c3d2da5f7140b18/e80ac633370859db?hl=en&lnk=gst&q=can%27t+delete#e80ac633370859db


%Y ( 2008 ) を省略した日時データを datetime に登録してしまって、削除もできなくなってしまった。
結局 -cleare_datastore してしまったが...

value = datastore_types.FromPropertyPb(prop)
File "C:\Program Files\Google\google_appengine\google\appengine\api\datastore_types.py", line 1102, in FromPropertyPb
'Error converting pb: %s\nException was: %s' % (pb, msg))
BadValueError: Error converting pb: meaning: 7
name: "entry_date"
value <
int64Value: 0xfff831bf8bc87700
>
multiple: false

Exception was: timestamp out of range for platform time_t

http://code.google.com/appengine/docs/thedevwebserver.html

dev_appserver.py --datastore_path=/tmp/myapp_datastore myapp

Default (Windows XP)
C:\Documents and Settings\XXX\Local Settings\Temp\dev_appserver.datastore
C:\Documents and Settings\XXX\Local Settings\Temp\dev_appserver.datastore.history

Entity Groups, Ancestors and Paths

• Every entity belongs to an entity group, a set of one or more entities that can be manipulated in a single transaction.
• Entity group relationships tell App Engine to store several entities in the same part of the distributed network.
• A transaction sets up datastore operations for an entity group, and all of the operations are applied as a group, or not at all if the transaction fails.
• 
  
• When the application creates an entity, it can assign another entity as the parent of the new entity.
• Assigning a parent to a new entity puts the new entity in the same entity group as the parent entity.
• 
  
• An entity without a parent is a root entity.
  
• An entity that is a parent for another entity can also have a parent.
• A chain of parent entities from an entity up to the root is the path for the entity, and members of the path are the entity's ancestors.
  
• The parent of an entity is defined when the entity is created, and cannot be changed later.

Tips for using entity groups:

• Only use entity groups when they are needed for transactions. For other relationships between entities, use ReferenceProperty properties and Key values, which can be used in queries.
  
  グループはトランザクションのため
  データの関連づけには　ReferenceProperty と Key を使う
  

class ReferenceProperty(reference_class=None, verbose_name=None, collection_name=None, ...)

A reference to another model instance. For example, a reference may indicate a many-to-one relationship between the model with the property and the model referenced by the property.

reference_class is the model class of the model instance being referenced. If specified, only model instances of the class can be assigned to this property. If None, any model instance can be the value of this property.

collection_name is the name of the property to give to the referenced model class whose value is a Query for all entities that reference the entity. If no collection_name is set, then modelname_set (with the name of the model in lowercase letters and "_set" added) is used.

ReferenceProperty automatically references and dereferences model instances as property values: A model instance can be assigned to a ReferenceProperty directly, and its key will be used. The ReferenceProperty value can be used as if it were a model instance, and the datastore entity will be fetched and the model instance created when it is first used in this way. Untouched reference properties do not query for unneeded data.

class Author(db.Model):
name = db.StringProperty()

class Story(db.Model):
author = db.ReferenceProperty(Author)

story = db.get(story_key)
author_name = story.author.name

As with a Key value, it is possible for a reference property value to refer to a data entity that does not exist. If a referenced entity is deleted from the datastore, references to the entity are not updated. An application can explicitly db.get() the value of a ReferenceProperty (which is a Key) to test whether the referenced entity exists.

Deleting an entity does not delete entities referred to by a ReferenceProperty.

See also this introduction to reference properties.

Value type: db.Key (see above)

Introduction to reference properties.
http://code.google.com/appengine/docs/datastore/entitiesandmodels.html#References

References

A property value can contain the key of another entity. The value is a Key instance.

# To fetch and iterate over every SecondModel entity that refers to the
# FirstModel instance obj1:
for obj in obj1.firstmodel_set:
 # ...

The name of the back-reference property defaults to modelname_set (with the name of the model class in lowercase letters, and "_set" added to the end), and can be adjusted using the collection_name argument to the ReferenceProperty constructor.

Automatic referencing and dereferencing of model instances, type checking and back-references are only available using the ReferenceProperty model property class. Keys stored as values of Expando dynamic properties or ListProperty values do not have these features.

http://groups.google.com/group/google-appengine/browse_thread/thread/e9464ceb131c726f?hl=en
http://blog.arbingersys.com/2008/04/google-app-engine-better-many-to-many.html

http://cluebin.appspot.com/pasted/25

• The more entity groups your application has―that is, the more root entities there are―the more efficiently the datastore can distribute the entity groups across datastore nodes. Better distribution improves the performance of creating and updating data. Also, multiple users attempting to update entities in the same entity group at the same time will cause some users to retry their transactions, possibly causing some to fail to commit changes. Do not put all of the application's entities under one root.
• A good rule of thumb for entity groups is that they should be about the size of a single user's worth of data or smaller.
• Entity groups do not have a significant impact on the speed of queries.

テーブル 分解

a = "abc</tr>efg</tr>aaa" print a print a.split('</tr>')[0] print len(a.split('</tr>')) imgList = "<table>" imgList += "<tr>" imgList += "<td>abc1</td>" imgList += "<td>abc2</td>" imgList += "<td>abc3</td>" imgList += "</tr>" imgList += "<tr>" imgList += "<td>2abc1</td>" imgList += "<td>2abc2</td>" imgList += "<td>2abc3</td>" imgList += "</tr>" imgList += "<tr>" imgList += "<td>3abc1</td>" imgList += "<td>3abc2</td>" imgList += "<td>3abc3</td>" imgList += "</tr>" imgList += "</table>" print "imgList",imgList imgData = imgList[:-8][7:] print "imgData",imgData rr = range(0, len(imgData.split('</tr>'))-1 ) for r in rr: imgRow = imgData.split('</tr>')[r][8:][:-5] print "r,imgRow:", r,imgRow imgRow = imgRow.split('</td><td>') print "imgRow,len(imgRow)",imgRow,len(imgRow)

abc</tr>efg</tr>aaa abc 3 imgList <table><tr><td>abc1</td><td>abc2</td><td>abc3</td></tr><tr><td>2abc1</td><td>2abc2</td><td>2abc3</td></tr><tr><td>3abc1</td><td>3abc2</td><td>3abc3</td></tr></table> imgData <tr><td>abc1</td><td>abc2</td><td>abc3</td></tr><tr><td>2abc1</td><td>2abc2</td><td>2abc3</td></tr><tr><td>3abc1</td><td>3abc2</td><td>3abc3</td></tr> r,imgRow: 0 abc1</td><td>abc2</td><td>abc3 imgRow,len(imgRow) ['abc1', 'abc2', 'abc3'] 3 r,imgRow: 1 2abc1</td><td>2abc2</td><td>2abc3 imgRow,len(imgRow) ['2abc1', '2abc2', '2abc3'] 3 r,imgRow: 2 3abc1</td><td>3abc2</td><td>3abc3 imgRow,len(imgRow) ['3abc1', '3abc2', '3abc3'] 3

削除、一括登録

一括削除
http://code.google.com/appengine/docs/datastore/creatinggettinganddeletingdata.html

class editBlog(BaseRequestHandler):
def get(self):
cmd = self.request.get('cmd')
if cmd == 'delete':
  q = db.GqlQuery("SELECT * FROM Blog" )
  results = q.fetch(100)
#     db.delete(results)
  for result in results:
    result.delete()
  self.redirect('/_ah/admin/datastore?kind=Blog')

#   self.redirect('/helloworld/')

Deleting an entity does not change any Key values in the datastore that may have referred to the entity. If your application may attempt to de-reference a Key value for a deleted entity, the application should do so using db.get(), then test the return value before accessing properties.

get(keys)

Gets the entity or entities for the given key or keys, of any Model.

Arguments:

keys

A Key object or a list of Key objects.

If one Key is provided, the return value is an instance of the appropriate Model class, or None if no entity exists with the given Key. If a list of Keys is provided, the return value is a corresponding list of model instances, with None values when no entity exists for a corresponding Key.　　　　　　　　　　　　　　　　　　　　See also Model.get().

Deleting an entity that is an ancestor for other entities does not affect the other entities. As long as the application does not depend on the existence of the ancestor to build keys for the descendant entities, the application can still access the descendants.

ループによる登録が思うように動作しない
トランザクションの問題ではないようなのだが。
http://code.google.com/appengine/docs/datastore/transactions.html

データストア Property

http://code.google.com/appengine/docs/datastore/typesandpropertyclasses.html
http://code.google.com/appengine/docs/datastore/typesandpropertyclasses.html#StringProperty

class StringProperty(verbose_name=None, multiline=False, ...)

A short string property. Takes a Python str or unicode (basestring) value of 500 bytes or less.

500バイト以下

StringProperty property values are indexed, and can be used in filters and sort orders.

インデックスによるフィルタ、並び替え可能

If multiline is False, then the value cannot include linefeed characters. The djangoforms library uses this to enforce a difference between text fields and textarea fields in the data model, and others can use it for a similar purpose.

multiline=True にすると LF(Line Feed)を含めることができる

Value type: str or unicode

class TextProperty()

A long string.

Unlike StringProperty, a TextProperty value can be more than 500 bytes long. However, TextProperty values are not indexed, and cannot be used in filters or sort orders.

500 バイト以上対応

フィルタはつかえない
検索条件にするものには向かない

TextProperty values store text with a text encoding. For binary data, use BlobProperty.

Value type: Text

class DateTimeProperty(verbose_name=None, auto_now=False, auto_now_add=False, ...)

A date and time property.

If auto_now is True, the property value is set to the current time whenever the model instance is stored in the datastore, overwriting the property's previous value. This is useful for tracking a "last modified" date and time for a model instance.

If auto_now_add is True, the property value is set to the current time the first time the model instance is stored in the datastore, unless the property has already been assigned a value. This is useful for storing a "created" date and time for a model instance.

BadValueError: Property entry_date must be a datetime

Date-time values are stored as and returned using the UTC time zone. See datetime.datetime for a discussion of how to manage time zones.

Value type: datetime.datetime

datetime に文字列の日時をいきなり代入しようとすると以下のエラーとなる。

BadValueError: Property entry_date must be a datetime

strptime での対応が必要。

yymm = '2008-05-10 22:22:22'
ydate = datetime.datetime.strptime(yymm, '%Y-%m-%d %H:%M:%S')
print ydate

フォーマット文字	説明	出力例
a	'a.m.' または 'p.m.' (Associated Press に合わせるため，'.' が入っている点が PHP と違います)．	'a.m.'
A	'AM' または 'PM' です．	'AM'
B	実装されていません．
d	月の中の日． 2 桁のゼロ詰めです．	'01' から '31'
D	週の中の日． 3 文字のテキスト形式です．	'Fri'
f	12 時間表記の時と分．ただし，ゼロ分の場合には表示しません．独自の拡張です．	'1', '1:30'
F	月名を長いテキスト形式で表したものです．	'January'
g	12 時間表記の時．ゼロ詰めはしません．	'1' から '12'
G	24 時間表記の時．ゼロ詰めはしません．	'0' から '23'
h	12 時間表記の時です．	'01' から '12'
H	24 時間表記の時です．	'00' から '23'
i	分です．	'00' から '59'
I	実装されていません．
j	月の中の日．ゼロ詰めしません．	'1' から '31'
l	週の中の曜日．長いテキスト形式です．	'Friday'
L	閏年かどうかを表すブール値です．	True``または``False
m	月です．2 桁でゼロ詰めしたものです．	'01' から '12'
M	月です．3 文字のテキスト形式です．	'Jan'
n	月です．ゼロ詰めしません．	'1' から '12'
N	Associated Press スタイルの月の省略表記です．独自の拡張です．	'Jan.', 'Feb.', 'March', 'May'
O	グリニッジ標準時からの時差です．	'+0200'
P	時刻です．12 時間表記の時，分，そして 'a.m.'/'p.m.' です．分がゼロの場合には省略され，必要に応じて'midnight' または 'noon' になります．独自の拡張です．	'1 a.m.', '1:30 p.m.', 'midnight', 'noon', '12:30 p.m.'
r	RFC 822 に従ったフォーマットの日時です．	'Thu, 21 Dec 2000 16:01:07 +0200'
s	秒です． 2 桁のゼロ詰めです．	'00' から '59'
S	月の中の日につける 2 文字の序数接尾辞です．	'st', 'nd', 'rd' or 'th'
t	月の日数です．	28 から 31
T	計算機のタイムゾーン設定です．	'EST', 'MDT'
U	実装されていません．
w	週の中の曜日です．ゼロ詰めしません．	'0' (Sunday) to '6' (Saturday)
W	ISO-8601 に従った年の中の週番号です．週は月曜日から始まります．	1, 23
y	2 桁の年です．	'99'
Y	4 桁の年です．	'1999'
z	年の中の日	0 から 365
Z	タイムゾーンオフセットを秒であらわしたものです．UTC よりも西側のタイムゾーン値は全て負の値になり，東側の値は常に正になります．	-43200 から 43200

例:

It is {% now "jS F Y H:i" %}

フォーマット文字列中で普通の文字列を使いたければ，バックスラッシュでエスケープできます．下の例では，"f" が時刻を表すフォーマット指定子として解釈されないようにエスケープしています． "o" はフォーマット指定子ではないのでエスケープしていません:

It is the {% now "jS o\f F" %}

このテンプレートは "It is the 4th of September" になります．

http://ymasuda.jp/python/django/docs/templates.html

http://blog.livedoor.jp/matssaku/archives/cat_10009843.html

Calendar

calendar あるいは　dateutil　http://labix.org/python-dateutil
もあるけれども、 range を使うのがわかりやすいか。

y1 = 2007 y2 = 2008 yy = range(y1,y2) # (2001,2002 ..2008) mm = range(1,13) # (1,2 ... 12 ) print yy, mm for y in yy: for m in mm: m = str(m) if len(m) == 1: m = '0' + m ym = str(y) + '-' + str(m) print ym # 2008-01,2008-02 import datetime today = date.today() print today # 2008-05-13 t = datetime.timedelta(weeks=2) print t import calendar print calendar.setfirstweekday(calendar.SUNDAY) print calendar.calendar(2008)

[2007] [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12] 2007-01 2007-02 2007-03 2007-04 2007-05 2007-06 2007-07 2007-08 2007-09 2007-10 2007-11 2007-12 2008-05-13 14 days, 0:00:00 None 2008 January February March Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 1 2 1 6 7 8 9 10 11 12 3 4 5 6 7 8 9 2 3 4 5 6 7 8 13 14 15 16 17 18 19 10 11 12 13 14 15 16 9 10 11 12 13 14 15 20 21 22 23 24 25 26 17 18 19 20 21 22 23 16 17 18 19 20 21 22 27 28 29 30 31 24 25 26 27 28 29 23 24 25 26 27 28 29 30 31 April May June Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 1 2 3 1 2 3 4 5 6 7 6 7 8 9 10 11 12 4 5 6 7 8 9 10 8 9 10 11 12 13 14 13 14 15 16 17 18 19 11 12 13 14 15 16 17 15 16 17 18 19 20 21 20 21 22 23 24 25 26 18 19 20 21 22 23 24 22 23 24 25 26 27 28 27 28 29 30 25 26 27 28 29 30 31 29 30 July August September Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 1 2 1 2 3 4 5 6 6 7 8 9 10 11 12 3 4 5 6 7 8 9 7 8 9 10 11 12 13 13 14 15 16 17 18 19 10 11 12 13 14 15 16 14 15 16 17 18 19 20 20 21 22 23 24 25 26 17 18 19 20 21 22 23 21 22 23 24 25 26 27 27 28 29 30 31 24 25 26 27 28 29 30 28 29 30 31 October November December Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 1 1 2 3 4 5 6 5 6 7 8 9 10 11 2 3 4 5 6 7 8 7 8 9 10 11 12 13 12 13 14 15 16 17 18 9 10 11 12 13 14 15 14 15 16 17 18 19 20 19 20 21 22 23 24 25 16 17 18 19 20 21 22 21 22 23 24 25 26 27 26 27 28 29 30 31 23 24 25 26 27 28 29 28 29 30 31 30

urlfetch で rss.xml より img を検出

http://localhost:8080/_ah/admin/interactive　より実行

from google.appengine.api import urlfetch
from elementtree.ElementTree import *
import re

rss1 = 'http://movies.go.com/xml/rss/intheaters.xml'
url = rss1
rss = urlfetch.Fetch(url)
print rss
tree = ElementTree(fromstring(rss.content))

img_re = re.compile('<img src="([a-zA-Z0-9/:._\-]+)" ')
for item in tree.findall('.//item'):
title = item.find('title').text.strip()
description = item.find('description').text.strip()
imageMatch = img_re.search(description)
if (imageMatch):
picture_url = imageMatch.groups()[0]
print title
print picture_url

---

Speed Racer
http://movies.go.com/i/movies/895881/895881.jpg

--- 特定のページの src = .jpg 検出 ---

from google.appengine.api import urlfetch
from elementtree.ElementTree import *
import re
url = 'http://webdba.blogspot.com/2008/05/urlfetch-rss-img.html'
content = urlfetch.Fetch(url).content
img_re = re.compile('<img src="([a-zA-Z0-9/:._\-]+)" ',re.I)
img_re = re.compile('src="([a-zA-Z0-9/:._\-]+).jpg" ',re.I)
img_re = re.compile('src="([a-zA-Z0-9/:._\-]+[jpg|gif|png])" ',re.I)

imageMatch = img_re.search(content)
imageMatch = img_re.findall(content)
print imageMatch
for i in imageMatch:
print i

---

['http://code.google.com/appengine/images/appengine_lowres.jpg']
http://code.google.com/appengine/images/appengine_lowres.jpg

正規表現, 文字列の置換

文字列の置換 string.replace()

---
import string
str1 = "http://localhost"
str2 = string.replace(str1, 'http', 'file')
print str1
print str2
---
http://localhost
file://localhost

http://www.gaku.net/ukki/Python_e3_83_a1_e3_83_a2.html

-----------------------------------

リテラルのバックスラッシュにマッチさせるには、正規表現文字列として

'\\\\'

Python では　raw 文字列(raw string)表記を正規表現に利用
先頭に 　r 　を付加する

p = re.compile(r'([^"])\b((http|https)://[^ \t\n\r<>\(\)&"]+' \
r'[^ \t\n\r<>\(\)&"\.])')

http://www.python.jp/Zope/articles/tips/regex_howto/regex_howto_3
http://reddog.s35.xrea.com/wiki/Python%C8%F7%CB%BA%CF%BF.html　Python備忘録


改行文字を取り除くときは、 s[:-1] または s.rstrip('\n')

http://www.shido.info/py/python2.html

-------------------------------------------------------------
import re
p = re.compile("ab.", re.I)
result = p.findall("AbdABCAAbb")
print result

p = re.compile("ab.")
print p.sub("xxx", "abcdeaabcde")
print p.sub("xxx", "Abcdeaabcde")

p = re.compile("ab.", re.I)
print p.sub("xxx", "Abcdeaabcde")
print "------------"

url =  "aa  https://aa.com/ bb"

p = re.compile(r'([^"])\b((http|https)://[^ \t\n\r<>\(\)&"]+' \
                      r'[^ \t\n\r<>\(\)&"\.])')

m = p.match('aa  http://aa.com/ bb' + url + url )
print m
m = p.sub('XXX','aa  http://aa.com/ bb' + url + url )
print m
print "------------"

# print m.group(0)

print "-- " + url + " --"
print "->"
print p.sub("xxx", url )

p = re.compile('[a-z]+')
m = p.match('tempo')
print m
print m.group(0)

-----

from google.appengine.api import urlfetch
from elementtree.ElementTree import *
import re

url = 'http://blog.goo.ne.jp/xxxxxx/m/200803'
#content = urlfetch.Fetch(url).content.decode("euc-jp")
content = url.split('/')

print content[0]
print content[1]
print content[2]
print content[3]

content = url.split('/')[2]
print content

google.appengine.api.datastore

cccwiki サンプルではPage オブジェクトを作成している
　
　この内容は　Datastore Viewerでも参照することができる。
























　その基本動作の確認

c:/Program Files/Google/google_appengine/google/appengine/api/datastore.py

import os
from google.appengine.api import datastore
from google.appengine.api import datastore_types

name = 'test'
entity = datastore.Entity('Page')
entity['name'] = name

query = datastore.Query('Page')
query['name ='] = name
entities = query.Get(1)
print >> sys.stdout, len(entities)
print >> sys.stdout, entities[0]
print >> sys.stdout, entities[0]['user']
print >> sys.stdout, entities[0].key()
print >> sys.stdout, entities[0].key().id()
------
1
{u'content': u'<h1>test</h1>', u'user': users.User(email='test@example.com'), u'modified': datetime.datetime(2008, 5, 9, 14, 41, 30, 62000), u'name': u'test', u'created': datetime.datetime(2008, 5, 9, 14, 41, 30, 62000)}
test@example.com
agpoZWxsb3dvcmxkcgsLEgRQYWdlGJQBDA
148

DatePropertyにはやはり不具合(bug)あり

date = db.DateTimeProperty(auto_now_add=True) の値が
対象の datastore のデータを更新する度に、時刻が進んでいってしまう現象は
以下の Patch により直った。 ( SDK 1.0.1)

DateProperty does not work properly in dev environment
http://code.google.com/p/googleappengine/issues/detail?id=131

c:/Program Files/Google/google_appengine/google/appengine/api/datastore_types.py

---

I tried a patch to module datastore_types.py, which seems to work:

change line 1033
from
lambda val: datetime.datetime.fromtimestamp(float(val) / 1000000.0),
to
lambda val: datetime.datetime.utcfromtimestamp(float(val) / 1000000.0),

Per the Python doc, fromtimestamp() converts to the local timezone.

---

Key names and IDs cannot be used like property values

　　 key と key_name と ID があり、これを理解していないいけない。
　　　
　　　GQL で key は select できるが、 key_name や ID は select できない。
　　　・　select してきた key から　key_name や ID を知ることはできる。
　　　　　　　　　　　　・　検索条件に ID や key_name を指定することができる。
　　　　　　　　　　　　・　検索条件に key を指定することは GQL ではできない。

注意: datastore には２つある　　　
from google.appengine.ext import db　　　　　　 これが　GQL　 api の解説あり
from google.appengine.api import datastore これは　cccwiki のサンプルなどで使用されている


key_name 指定して put すると













ID は空欄で、Key_Name に値が保存される。

http://daily.profeth.de/2008/04/er-modeling-with-google-app-engine.html

import sys #import sys module for printing to stdout
from google.appengine.ext import db

class Car(db.Model):
brand  =  db.StringProperty(required=True)
wheels =  db.ListProperty(db.Key)

mercedes    = Car(brand="Mercedes", key_name ="b")
# mercedes.put()
print >> sys.stdout, mercedes.brand

select = mercedes.get_by_id( 130 )
print >> sys.stdout, select.brand

select = mercedes.get_by_key_name( "b" )
print >> sys.stdout, select.brand

id = 130
select = Car.get(db.Key.from_path('Car', id))
print >> sys.stdout, select.brand

key_name = "b"
select = Car.get(db.Key.from_path('Car', key_name))
print >> sys.stdout, select.brand

print >> sys.stdout, select.key()
print >> sys.stdout, select.key().id()
print >> sys.stdout, select.key().name()

select * from Greeting where key = "xxxx"
select * from Greeting where id = xxx
のような検索はできない。

select user_name from Greeting where id = xxxx
update Greeting set user_name = 'new name' where id = xxxx
のかわりに
greetings = db.GqlQuery("SELECT * FROM Greeting")

id = greeting.key().id()

id = int(self.request.get('id'))
greeting = Greeting.get(db.Key.from_path('Greeting', id))

row id から再度検索する必要なない。
get(db.Key.from_path('Greeting', id)　これが再検索と同じ

key さえわかれれば db.get() で　OK
from google.appengine.ext import db
r = db.get("agpoZWxsb3dvcmxkcgsLEgRCbG9nGNQBDA")
print r.title

Tip: Key names and IDs cannot be used like property values in queries. However, you can use a named key, then store the name as a property. You could do something similar with numeric IDs by storing the object to assign the ID, getting the ID value using obj.key().id(), setting the property with the ID, then storing the object again.

http://code.google.com/appengine/docs/datastore/keysandentitygroups.html#Kinds_Names_and_IDs

http://code.google.com/appengine/articles/djangoforms.html

追加仕様検討
Issue 178: Add unique=True/False to db.property constructor to ensure uniqueness
http://code.google.com/p/googleappengine/issues/detail?id=178

Trunsaction / run_in_transaction
http://code.google.com/appengine/docs/datastore/functions.html#run_in_transaction

Google App Engine / ワイルドカード検索

　部分一致検索はできないようだが、擬似的に行うことができる。
ただ、これも unicode がからむと少々注意が必要

下記を参考に helloworld.py にてテストしたが、
　#!-*- coding:utf-8 -*-　がきいているためか、

　ＯＫ
 greetings = db.GqlQuery("SELECT * FROM Greeting where content >= :1 and content < :2 ", 
            search_key, search_key + u"\uFFFD" )
　ＮＧ
# greetings = db.GqlQuery("SELECT * FROM Greeting where content >= :1 and content < :2 ",     #             search_key, urllib.unquote(search_key).decode("utf8") + u"\uFFFD" ) 

でした。 http://code.google.com/appengine/docs/datastore/queriesandindexes.html

Tip: Query filters do not have an explicit way to match just part of a string value, but you can fake a prefix match using inequality filters:

db.GqlQuery("SELECT * FROM MyModel WHERE prop >= :1 AND prop < :2", "abc", "abc" + "\xEF\xBF\xBD")

This matches every MyModel entity with a string property prop that begins with the characters abc. The byte string "\xEF\xBF\xBD" represents the largest possible Unicode character. When the property values are sorted in an index, the values that fall in this range are all of the values that begin with the given prefix.

http://groups.google.com/group/google-appengine/browse_thread/thread/47abf4b8842ab865?hl=en

Next problem is that I can't search the datastore with a parameter
from the query string, something like this:

import urllib

q = "Ambj%C3%B6" # Url encoded u"Ambjö"

fsgs = db.GqlQuery("SELECT * FROM Fsg WHERE fsg >= :1 AND fsg < :2", urllib.unquote(q), urllib.unquote(q) + u'\uFFFD')

Works perfect if I only have ascii characters. Is urllib.unquote wrong
utility for this?

> q = "Ambj%C3%B6" # Url encoded u"Ambjö"
> fsgs = db.GqlQuery("SELECT * FROM Fsg WHERE fsg >= :1 AND fsg < :2", > urllib.unquote(q), urllib.unquote(q) + u'\uFFFD')

You've got a similar problem as before. The string urllib.unquote()
returns isn't a Unicode string either, so you need to convert it to
Unicode. Eg:
urllib.unquote(q).decode("utf8") + u"\uFFFD"