write table cell real-time python -

i loop trough database, find appropriate values , insert them in appropriate cell in separate file. maybe csv, or other human-readable format. in pseudo-code:

for item in huge_db:    list_of_objects_to_match:       if itemmatch():          if there_arent_three_matches_yet_in_list():             matches++             result=performoperationonitem()             write_in_file(result, row=object_to_match_id, col=matches)          if matches 3:             remove_this_object_from_object_to_match_list() 

can think of way other going every time through outputfile line line? don't know search for... better, there better ways find 3 matching objects in db , have results in real-time? (the operation take while, i'd see results popping out rt)

assuming itemmatch() reasonably simple function, think want better pseudocode:

for match_obj in list_of_objects_to_match:   db_objects = query_db_for_matches(match_obj)   if len(db_objects) >= 3:       result=performoperationonitem()       write_in_file(result, row=match_obj.id, col=matches)   else:       write_blank_line(row=match_obj.id)  # if want 

then trick becomes writing query_db_for_matches() function. without detail, i'll assume you're looking objects match in 1 particular field, call type. in pymongo such query like:

def query_db_for_matches(match_obj):     return pymongo_collection.find({"type":match_obj.type}) 

to run efficiently, make sure database has index on field(s) you're querying on first calling:

pymongo_collection.ensure_index({"type":1}) 

the first time call ensure_index take long time huge collection. each time after fast -- fast enough put query_db_for_matches before find , fine.