java - Set user-agent property in https connection header -

i unable set user-agent property https connection. i've gathered, http-header properties can set through either -dhttp.agent vm option or through urlconnection.setrequestproperty(). however, setting user-agent through vm option causes " java/[version]" appended whatever value of http.agent is. @ same time setrequestproperty() works http connections, not https (at least when tried it).

java.net.url url = new java.net.url( "https://www.google.com" ); java.net.urlconnection conn = url.openconnection(); conn.setrequestproperty("user-agent","mozilla/5.0 (windows nt 5.1; rv:19.0) gecko/20100101 firefox/19.0"); conn.connect(); java.io.bufferedreader serverresponse = new java.io.bufferedreader(new java.io.inputstreamreader(conn.getinputstream())); system.out.println(serverresponse.readline()); serverresponse.close(); 

i've found/verified problem inspecting http communictions using wireshark. there way around this?

update: addition info

it seems didn't deep enough communication. code running behind proxy communication observed against proxy, set through -dhttps.proxyhost, , not target website (google.com). anyway, during https connection, method connect, not get. here wireshark capture of https communication attempt. mentioned above, user-agent set through -dhttp.agent because urlconnection.setrequestproperty() has no effect (user-agent = java/1.7.0). in case, notice appended java/1.7.0. question remains same, why happening , how around it?

connect www.google.com:443 http/1.1 user-agent: mozilla/5.0 (windows nt 5.1; rv:19.0) gecko/20100101 firefox/19.0 java/1.7.0 host: www.google.com accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 proxy-connection: keep-alive  http/1.1 403 forbidden x-bst-request-id: mwpwwh:m7d:39175 x-bst-info: ch=req,t=1366218861,h=14g,p=4037_7213:1_156,f=pefilter,r=peblockcatchallrule,c=1905,v=7.8.14771.200 1363881886 content-type: text/html; charset=utf-8 pragma: no-cache content-language: en cache-control: no-cache content-length: 2491 

by way, request forbidden because proxy filters user-agent, java/1.7.0 causing rejection. i've appended java/1.7.0 user-agent of http connection , proxy refuses connection too. hope i'm not going crazy :).

i've found/verified problem inspecting http communictions using wireshark. there way around this

this not possible. communication on ssl socket obscured casual observation encryption protocol. using packet capture software able view initiation of ssl connection , exchange of encrypted packets, content of packets can extracted @ other end of connection (the server). if not case https protocol whole broken, whole point of secure http communications man-in-the-middle type attacks (where in case mitm packet sniffer).

example capture of https request (partial):

.n....e... .........../..5..3..9..2..8.. ..............@........................ql.{...b....osr..!.4.$.t...-.-.t....q...m..ql.{...lm..l...um.m...........s. ...n...p^0}..i..g4.hk.n......8y...............e...a..>...0...0......... ).s.......0 ..*.h.. .....0f1.0...u....us1.0...u. . google inc1"0 ..u....google internet authority0.. 130327132822z. 131231155850z0h1.0...u....us1.0...u... california1.0...u... mountain view1.0...u. . google inc1.0...u....www.google.com0..0

theoretically, way know if user-agent header being excluded if have access google servers, in actuality there nothing in either https specification or java's implementation of excludes headers have been sent on http.

example capture of http request:

get / http/1.1
user-agent: mozilla/5.0 (windows nt 5.1; rv:19.0) gecko/20100101 firefox/19.0
host: www.google.com
accept: text/html, image/gif, image/jpeg, *; q=.2, /; q=.2
connection: keep-alive

both example captures generated exact same code:

url url = new url(target); urlconnection conn = url.openconnection(); conn.setrequestproperty("user-agent",         "mozilla/5.0 (windows nt 5.1; rv:19.0) gecko/20100101 firefox/19.0"); conn.connect(); bufferedreader serverresponse = new bufferedreader(         new inputstreamreader(conn.getinputstream())); system.out.println(serverresponse.readline()); serverresponse.close(); 

except https target "https://www.google.com", , http "http://www.google.com".

---

edit 1:

based off updated question, using -dhttp.agent property does indeed append 'java/version' user agent header, described following documentation:

http.agent (default: “java/<version>”)
defines string sent in user-agent request header in http requests. note string “java/<version>” appended 1 provided in property (e.g. if -dhttp.agent=”foobar” used, user-agent header contain “foobar java/1.5.0” if version of vm 1.5.0). property checked once @ startup.

the 'offending' code in static block initializer of sun.net.www.protocol.http.httpurlconnection:

static {     // ...     string agent = java.security.accesscontroller             .doprivileged(new sun.security.action.getpropertyaction(                     "http.agent"));     if (agent == null) {         agent = "java/" + version;     } else {         agent = agent + " java/" + version;     }     useragent = agent;      // ... } 

an obscene way around 'problem' snippet of code, 1000% recommend not use:

protected void forceagentheader(final string header) throws exception {     final class<?> clazz = class             .forname("sun.net.www.protocol.http.httpurlconnection");      final field field = clazz.getfield("useragent");     field.setaccessible(true);     field modifiersfield = field.class.getdeclaredfield("modifiers");     modifiersfield.setaccessible(true);     modifiersfield.setint(field, field.getmodifiers() & ~modifier.final);     field.set(null, header); } 

using override https.proxyhost, https.proxyport , http.agent set gives desired result:

connect www.google.com:443 http/1.1
user-agent: mozilla/5.0 (windows nt 5.1; rv:19.0) gecko/20100101 firefox/19.0
host: www.google.com
accept: text/html, image/gif, image/jpeg, *; q=.2, /; q=.2
proxy-connection: keep-alive

but yea, don't that. safer use apache httpcomponents:

final defaulthttpclient client = new defaulthttpclient(); httphost proxy = new httphost("127.0.0.1", 8888, "http"); httphost target = new httphost("www.google.com", 443, "https"); client.getparams().setparameter(connroutepnames.default_proxy, proxy); httpprotocolparams         .setuseragent(client.getparams(),                 "mozilla/5.0 (windows nt 5.1; rv:19.0) gecko/20100101 firefox/19.0"); final httpget = new httpget("/");  httpresponse response = client.execute(target, get);