~grubng-dev/grubng/tools-python

« back to all changes in this revision

Viewing changes to robots.py

  • Committer: thindil
  • Date: 2010-04-23 08:22:45 UTC
  • Revision ID: thindil2@gmail.com-20100423082245-xoh1sbxppltd78qr
fixed bug with crash script on invalid URL

Show diffs side-by-side

added added

removed removed

Lines of Context:
60
60
                        robot.read()
61
61
                    except IOError:
62
62
                        continue
 
63
                    except InvalidURL:
 
64
                        continue
63
65
                    oldroboturl = roboturl
64
66
                #if we can visit this link, add it to file
65
67
                try: