python - Remove Duplicates from Text File -

i want remove duplicate word text file.

i have text file contain such following:

none_none  confighandler_56663624 confighandler_56663624 confighandler_56663624 confighandler_56663624  none_none  columnconverter_56963312 columnconverter_56963312  predicatesfactory_56963424 predicatesfactory_56963424  predicateconverter_56963648 predicateconverter_56963648  confighandler_80134888 confighandler_80134888 confighandler_80134888 confighandler_80134888 

the resulted output needs be:

none_none  confighandler_56663624  columnconverter_56963312  predicatesfactory_56963424  predicateconverter_56963648  confighandler_80134888 

i have used command: en=set(open('file.txt') not work.

could me how extract unique set file

thank

here's option preserves order (unlike set), still has same behaviour (note eol character deliberately stripped , blank lines ignored)...

from collections import ordereddict  open('/home/jon/testdata.txt') fin:     lines = (line.rstrip() line in fin)     unique_lines = ordereddict.fromkeys( (line line in lines if line) )  print unique_lines.keys() # ['none_none', 'confighandler_56663624', 'columnconverter_56963312',predicatesfactory_56963424', 'predicateconverter_56963648', 'confighandler_80134888'] 

then need write above output file.