java - HtmlUnitDriver causes problems while getting an url -

i have page crawler developed in java using selenium libraries. crawler goes through website launches through javascript 3 applications displayed html in popup windows.

the crawler has no issues when launching 2 of applications, on 3rd 1 crawler freezes forever.

the code i'm using similar to

public void applicationselect() {   ...   //obtain url parsing tag href attributed   ...    this.driver = new htmlunitdriver(browserversion.internet_explorer_8);   this.driver.sejavascriptenabled(true);   this.driver.get(url); //the code not execute after point 3rd app   ... } 

i have tried clicking on web element through following code

public void applicationselect() {   ...   webelement element = this.driver.findelementbylinktext("linktext");   element.click(); //the code not execute after point 3rd app   ... } 

clicking on produces same result. above code, i've made sure getting right element.

can tell me problem i'm having?

on application side, cannot disclose information html code. know makes things harder trying solve problem , apologize in advance.

=== update 2013-04-10 ===

so, added sources crawlers , saw in this.driver.get(url) getting stuck on.

basically, driver gets lost in infinite refresh loop. within webclient object instantiated htmlunitdriver, htmlpage loaded continually refreshes seemingly without end.

here code waitingrefreshhandler, contained in com.gargoylesoftware.htmlunit:

public void handlerefresh(final page page, final url url, final int requestedwait) throws ioexception {   int seconds = requestedwait;   if (seconds > maxwait_ && maxwait_ > 0) {     seconds = maxwait_;   }   try {     thread.sleep(seconds * 1000);   }   catch (final interruptedexception e) {     /* can happen when refresh happening navigation started      * settimeout or setinterval. navigation cause threads      * interrupted, including current thread in case. should safe      * ignore since thread doing navigation. should      * refactor force navigation happen on main thread.      */     if (log.isdebugenabled()) {       log.debug("waiting thread interrupted. ignoring interruption continue navigation.");     }   }   final webwindow window = page.getenclosingwindow();   if (window == null) {     return;   }   final webclient client = window.getwebclient();   client.getpage(window, new webrequest(url)); } 

the instruction "client.getpage(window, new webrequest(url))" calls webclient once again reload page, once more call same refresh method. seems go on indefinetly, not filling memory because of "thread.sleep(seconds * 1000)", forces 3m wait before trying again.

does have suggestion on how can work around issue? got suggestion create 2 new htmlunitdriver , webclient classes extend original ones. override relevant methods in order avoid problem.

thanks again.

i solved eternal refresh problem creating nothing refreshhandler class:

public class refreshhandler implements com.gargoylesoftware.htmlunit.refreshhandler {      public refreshhandler() { }   public void handlerefresh(final page page, final url url, final int secods) { } } 

in addition, extended htmlunitdriver class , overriding method modifywebclient, set new refreshhandler:

public class htmlunitdriverext extends htmlunitdriver {    public htmlunitdriverext(browserversion version) {     super(version);   }   @override   protected webclient modifywebclient(webclient client) {     client.setrefreshhandler(new refreshhandler());     return client;   } } 

the method modifywebclient nothing method created in htmlunitdriver purpose.

cheers.