Trouble with https via the proxy

[matthewbretherton]

I am testing a web application ( www.tradingpost.com.au ) using Cucumber and Celerity 0.9.0 and the test sites for the application are on a lcoal intranet. I am a strong fan of Celerity . . . its a great product for testing web apps

I need to test using https as login and credit card collection is an essential component of system behaviour and I also need a proxy as parts of the system reference external sites. I can run without but miss javascript loads and also suffer excessive times outs

I am finding that the HtmlUnit proxy bypass does not consistently work with https. It seems that only the very first request after browser instantiantion for https is served correctly

This is more an HTMLUnit issue than a Celerity issue but all my code and examples are Celerity

If I set up the browser with this method

def new_browser( host_to_bypass )
browser = Celerity::Browser.new(
:proxy => "tpg-proxy.corp1.local:8080",
:resynchronize => true,
:log_level => :severe)
browser.webclient.setUseInsecureSSL(true)
browser.webclient.java_script_enabled = false
if host_to_bypass
proxyconfig=browser.webclient.getProxyConfig()
proxyconfig.addHostsToProxyBypass(host_to_bypass)
end
browser
end

When if I do some tests

require 'rubygems'
require 'Celerity'
#1. Just https

browser=new_browser ('10.169.142.33')

browser.goto "https://10.169.142.33/Home"
puts browser.status_code
200
browser.goto "https://10.169.142.33/Home"
puts browser.status_code
504
#2. Just http: in same session ( and aslo after restarting jruby and loading gems )

browser=new_browser ('10.169.142.33')
browser.goto "http://10.169.142.33/Home"
puts browser.status_code
200
browser.goto "http://10.169.142.33/Home"
puts browser.status_code
200

. . . and so on as often as I please

#3 just http: then https:

browser=new_browser ('10.169.142.33')
browser.goto "http://10.169.142.33/Home"
puts browser.status_code
200
browser.goto "https://10.169.142.33/Home"
puts browser.status_code
504
#4 https: external then internal

browser=new_browser ('10.169.142.33')
browser.goto "https://www.google.com.au"
puts browser.status_code
200
browser.goto "https://10.169.142.33/Home"
puts browser.status_code
504
#5 https https: extenal then internal

browser=new_browser ('10.169.142.33')
browser.goto "https://10.169.142.33/Home"
puts browser.status_code
200
browser.goto "https://www.google.com.au"
puts browser.status_code
200

This is quite a problem as it only allows a smal part of the system to be tested

Any advice on how where when to correct, any advice on where better to seek advice or even pointers to the correct code in html unit appreciated

thanks in advance
Matthew B

[matthewbretherton]

browsing into the hTMLUnit code and lookin to see some internal state I found this series of calls

browser.page.getWebResponse.getWebRequest.getProxyHost

#4 https: external then internal

browser=new_browser ('10.169.142.33')
browser.goto "https://www.google.com.au"
puts browser.status_code
200

browser.goto "https://10.169.142.33/Home"
puts browser.status_code
504

browser.page.getWebResponse.getWebRequest.getProxyHost
nil

Just becomes more baffling

[matthewbretherton]

dsmiley on sourceforge HTmlUnit

answered my query on connected issues

"
My patch is not "all based around isSocksProxy" -- that is one feature.
It fixes accidental re-use issues with how certain settings are set on the
httpclient object. I suggest you apply the patch and see if it works for
you.
"

https://sourceforge.net/tracker/index.php?func=detail&aid=3402274&group_id=47038&atid=448266

and I try that as he sugegsts and report back

[matthewbretherton]

Early testing shows that the dsmiley ptach does solve the problem

Is a pity we lost a lot of time and focus fighting a problem that was already detected and resolved

What is the best way to go about ensuring this fix gets back into the HTMLUnit release