I need to be able to search my users faster. A search without memcache takes 8 seconds. Now my code utilizes memcache with the help of Not Dot Net which reduces the search time to 4 sec. My question now is, how can I make it faster?
qUserSearch = Utilities.deserialize_entities(memcache.get("qUserSearch"))
if not qUserSearch:
qUserSearch = db.GqlQuery("SELECT * FROM User ORDER BY created DESC").fetch(100000)
memcache.add("qUserSearch", Utilities.serialize_entities(qUserSearch))
searchLength = len(searchData)
hits = []
gotHits = False
for u in qUserSearch:
searchEmail = u.email[0:searchLength]
if searchEmail == searchData:
hits.append( u.key() )
else:
# Since I only search for the first x chars
# will there never be hits after it's had hits
if gotHits:
break
return hits
My first ideas:
- convert to ndb and SELECT only, needed, data FROM User
- SQL way, create a table with less data. Just usernames and key to metadata
- The new full text search
- The old searchable models by 3rd party devs
- Memcache only needed properties and traverese that instead of everything
Or do you have other ideas? Which once do you think I will save the most time on?