Man in the middle proxy - Not working

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Man in the middle proxy - Not working

Raj-66
Hi All
    Requirement:- I want to build a man in the middle proxy application.
    I have experimented so many methods to achieve this. But my application is failing when I tried some https url's from the browser
    (IE 8 and Firefox 3.7). 
    I have configured my browser proxy settings to '4433' port. My application is listening on this port, when I connect to this port from my browser,         with the URL https://localhost:4433 it is  working, only a certificate warning is there.
    When I try to connect to another secured site, the SSL_accept function is returning -1 and my error code is as follows.
    "2572:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy request:.ssls23_srvr.c:391:"
    I am not able to trace out the problem for many days.
    Anybody please help me to trace out this issue, or send me some sample application. What could be reasons for failure
 
Platforms I am using are :
 
MS Windows XP service pack 2
MS Visual Studio 2008 , VC++

Thanks,
Raj
Rajmohan SK
Reply | Threaded
Open this post in threaded view
|

RE: Man in the middle proxy - Not working

Alexey Drozdov
Hi!

When your setup proxy setting for browsers, they using HTTP CONNECT method for establish pure tcp-connection via proxy (not for local resources).
It's seems like:

Client send HTTP-request to proxy
        CONNECT remotehost:port HTTP/1.1
        Host: remotehost:port

And begin wait HTTP-response like:
        HTTP/1.1 200 Connection established

Then browser send initiate ssl handshake over this pure tcp-channel.

Your proxy get HTTP-request instead ssl-handshake and fail:
2572:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy request:.ssls23_srvr.c:391

---
/ Alexey Drozdov


From: [hidden email] [mailto:[hidden email]] On Behalf Of Raj
Sent: Monday, July 26, 2010 12:59 PM
To: [hidden email]
Subject: Man in the middle proxy - Not working

Hi All
    Requirement:- I want to build a man in the middle proxy application.
    I have experimented so many methods to achieve this. But my application is failing when I tried some https url's from the browser
    (IE 8 and Firefox 3.7). 
    I have configured my browser proxy settings to '4433' port. My application is listening on this port, when I connect to this port from my browser,         with the URL https://localhost:4433 it is  working, only a certificate warning is there.
    When I try to connect to another secured site, the SSL_accept function is returning -1 and my error code is as follows.
    "2572:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy request:.ssls23_srvr.c:391:"
    I am not able to trace out the problem for many days.
    Anybody please help me to trace out this issue, or send me some sample application. What could be reasons for failure
 
Platforms I am using are :
 
MS Windows XP service pack 2
MS Visual Studio 2008 , VC++

Thanks,
Raj
Rajmohan SK
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Man in the middle proxy - Not working

JoelKatz
Alexey Drozdov wrote:

> Hi!
>
> When your setup proxy setting for browsers, they using HTTP CONNECT
> method for establish pure tcp-connection via proxy (not for local
> resources).
> It's seems like:
>
> Client send HTTP-request to proxy
> CONNECT remotehost:port HTTP/1.1
> Host: remotehost:port
>
> And begin wait HTTP-response like:
> HTTP/1.1 200 Connection established
>
> Then browser send initiate ssl handshake over this pure tcp-channel.
>
> Your proxy get HTTP-request instead ssl-handshake and fail:
> 2572:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy
> request:.ssls23_srvr.c:391
>
> ---
> / Alexey Drozdov

In other words, you switched to SSL too early. The way you did it, how would
you know what host and port you were supposed to proxy a connection to?! You
have to wait and get the CONNECT request from the client to know what host
and port they want a connection to. Then send an HTTP 200 reply, and then
begin proxying.

DS

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Man in the middle proxy - Not working

Rene Hollan-2
RE: Man in the middle proxy - Not working

Oh! I totally misunderstood this. I thought OP wanted to MITM SSL sessions (which is possible, if (a) the traffic is decrypted, (b) certs are reissued and resigned, and (c) the client TRUSTS the modified cert chain (typically its root cert)).

This is just HTTPS Proxy. In which case other answers about terminating the HTTP connection first are correct.


-----Original Message-----
From: [hidden email] on behalf of David Schwartz
Sent: Tue 7/27/2010 4:19 AM
To: [hidden email]
Subject: RE: Man in the middle proxy - Not working

Alexey Drozdov wrote:

> Hi!
>
> When your setup proxy setting for browsers, they using HTTP CONNECT
> method for establish pure tcp-connection via proxy (not for local
> resources).
> It's seems like:
>
> Client send HTTP-request to proxy
>       CONNECT remotehost:port HTTP/1.1
>       Host: remotehost:port
>
> And begin wait HTTP-response like:
>       HTTP/1.1 200 Connection established
>
> Then browser send initiate ssl handshake over this pure tcp-channel.
>
> Your proxy get HTTP-request instead ssl-handshake and fail:
> 2572:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy
> request:.ssls23_srvr.c:391
>
> ---
> / Alexey Drozdov

In other words, you switched to SSL too early. The way you did it, how would
you know what host and port you were supposed to proxy a connection to?! You
have to wait and get the CONNECT request from the client to know what host
and port they want a connection to. Then send an HTTP 200 reply, and then
begin proxying.

DS

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Man in the middle proxy - Not working

JoelKatz

Rene Hollan:

> Oh! I totally misunderstood this.
> I thought OP wanted to MITM SSL sessions (which is possible, if
> (a) the traffic is decrypted, (b) certs are reissued and resigned,
> and (c) the client TRUSTS the modified cert chain (typically its
> root cert)).

> This is just HTTPS Proxy. In which case other answers about
> terminating the HTTP connection first are correct.

No, you were correct. He does want to MITM SSL sessions.

A MITM and a normal proxy operate precisely the same way up until the actual
proxying part starts. His problem is earlier, when he establishes the
connection to the client, determines what host and port the client wants to
talk to, and then switches to his SSL proxy/MITM capability.

All those steps are the same.

1) Accept plaintext connection.

2) Wait for client to send request.

3) Confirm CONNECT request, host and port valid.

4) Send 200 reply.

5) Make connection to host and port requested by client.

6) If normal proxying, begin proxying (copy ciphertext between client and
server). If MITMing, begin MITMing (do SSL negotiation with both client and
plaintext, copy plaintext between client and server).

DS

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Man in the middle proxy - Not working

Raj-66

Hi All
    Thank you so much for all the response.
    I have one more doubt. If we do normal proxying on https connection, is
it possible to read the https content data, at least the URL

Thanks, Raj Rajmohan SK
----- Original Message -----
From: "David Schwartz" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, July 28, 2010 1:07 AM
Subject: RE: Man in the middle proxy - Not working


>
> Rene Hollan:
>
>> Oh! I totally misunderstood this.
>> I thought OP wanted to MITM SSL sessions (which is possible, if
>> (a) the traffic is decrypted, (b) certs are reissued and resigned,
>> and (c) the client TRUSTS the modified cert chain (typically its
>> root cert)).
>
>> This is just HTTPS Proxy. In which case other answers about
>> terminating the HTTP connection first are correct.
>
> No, you were correct. He does want to MITM SSL sessions.
>
> A MITM and a normal proxy operate precisely the same way up until the
> actual
> proxying part starts. His problem is earlier, when he establishes the
> connection to the client, determines what host and port the client wants
> to
> talk to, and then switches to his SSL proxy/MITM capability.
>
> All those steps are the same.
>
> 1) Accept plaintext connection.
>
> 2) Wait for client to send request.
>
> 3) Confirm CONNECT request, host and port valid.
>
> 4) Send 200 reply.
>
> 5) Make connection to host and port requested by client.
>
> 6) If normal proxying, begin proxying (copy ciphertext between client and
> server). If MITMing, begin MITMing (do SSL negotiation with both client
> and
> plaintext, copy plaintext between client and server).
>
> DS
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Man in the middle proxy - Not working

Raj-66
In reply to this post by JoelKatz
Thanks for all the response
        1. I was able to do the handshaking successfully with the browser.
On receiving the request from the browser I will send "HTTP OK "  response
back to the browser, I was able to do the handshaking and read the actual
GET request.
        2. Then I create a new socket to establish the connection with
server. The connection was successful.
            Sends the request to the server
            Reads the request from the server

When I read the response from the server it always return empty. I don't
know what went wrong here. I am reading the data from the socket using
'recv' function. Can anybody tell me what went wrong

Thanks,
Raj
Rajmohan SK

----- Original Message -----
From: "David Schwartz" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, July 28, 2010 1:07 AM
Subject: RE: Man in the middle proxy - Not working


>
> Rene Hollan:
>
>> Oh! I totally misunderstood this.
>> I thought OP wanted to MITM SSL sessions (which is possible, if
>> (a) the traffic is decrypted, (b) certs are reissued and resigned,
>> and (c) the client TRUSTS the modified cert chain (typically its
>> root cert)).
>
>> This is just HTTPS Proxy. In which case other answers about
>> terminating the HTTP connection first are correct.
>
> No, you were correct. He does want to MITM SSL sessions.
>
> A MITM and a normal proxy operate precisely the same way up until the
> actual
> proxying part starts. His problem is earlier, when he establishes the
> connection to the client, determines what host and port the client wants
> to
> talk to, and then switches to his SSL proxy/MITM capability.
>
> All those steps are the same.
>
> 1) Accept plaintext connection.
>
> 2) Wait for client to send request.
>
> 3) Confirm CONNECT request, host and port valid.
>
> 4) Send 200 reply.
>
> 5) Make connection to host and port requested by client.
>
> 6) If normal proxying, begin proxying (copy ciphertext between client and
> server). If MITMing, begin MITMing (do SSL negotiation with both client
> and
> plaintext, copy plaintext between client and server).
>
> DS
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Man in the middle proxy - Not working

JoelKatz

Raj wrote:

> Thanks for all the response
>         1. I was able to do the handshaking successfully with the
> browser.
> On receiving the request from the browser I will send "HTTP OK "
> response
> back to the browser, I was able to do the handshaking and read the
> actual
> GET request.
>         2. Then I create a new socket to establish the connection with
> server. The connection was successful.
>             Sends the request to the server
>             Reads the request from the server
>
> When I read the response from the server it always return empty.

What does that mean? Are you doing a blocking read or a non-blocking read?
If 'read' returns zero, then the connection was closed by the server. If
'read' returns a number less than zero, there is an error -- tell us what
error you are getting. If 'read' returns a number greater than zero, then
that is the first part of the response.

> I
> don't
> know what went wrong here. I am reading the data from the socket using
> 'recv' function. Can anybody tell me what went wrong

So, what return value do you get from 'recv'?

DS

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Man in the middle proxy - Not working

Dave Thompson-5
In reply to this post by Raj-66
> From: [hidden email] On Behalf Of Raj
> Sent: Wednesday, 04 August, 2010 01:09

> Thanks for all the response
>         1. I was able to do the handshaking successfully with
> the browser.
> On receiving the request from the browser I will send "HTTP
> OK "  response
> back to the browser, I was able to do the handshaking and
> read the actual
> GET request.

To be clear: I interpret you received CONNECT, sent OK,
did SSL handshake between browser and you (SSL_accept),
then SSL_read (data which is a) GET request.

>         2. Then I create a new socket to establish the
> connection with
> server. The connection was successful.
>             Sends the request to the server
>             Reads the request from the server
>
(Obviously you mean read response.)

> When I read the response from the server it always return
> empty. I don't
> know what went wrong here. I am reading the data from the
> socket using
> 'recv' function. Can anybody tell me what went wrong
>
Is the connection to the server clear, or SSL?

If SSL, you must use SSL_{connect,write,read,etc} throughout,
with a different SSL* pointer than the one for the client side.
And check for errors and report them etc.

If clear, either:

- you did the send and/or recv wrong; we'd have to look at
your code, which you should simplify/trim as much as possible.

- the server didn't like the request you sent, or you,
strongly enough it just closed the connection. For HTTP
this should be rare; most issues with the actual request
(such as bad method or resource, unauthorized, bad or
prohibited or required body, etc.) have defined HTTP
error responses. Something like a firewall or frontend
that works at the TCP level might just disconnect you,
although in my experience they usually block or reject
the initial connection (SYN) or break abruptly (RST),
either of which appears to your program as an error
return (canonically -1, not 0).

Can you contact the people operating the server, and
can they check their logs around the time of your attempt?

Can you connect to the server from a browser on the machine
running your proxy, or at one nearby on the same subnet?
In clear, SSL, or both? And do a GET like the one you are
(receiving and) forwarding from your client? Successfully?

Can you run a monitor like tcpdump or wireshark while running
your program, to see what was actually sent to the server
and confirm if any data or what flags came back?



______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Man in the middle proxy - Not working

Raj-66
In reply to this post by JoelKatz
Hi
    recv function is returning 0

Thanks, Raj Rajmohan SK

----- Original Message -----
From: "David Schwartz" <[hidden email]>
To: <[hidden email]>
Sent: Thursday, August 05, 2010 6:05 AM
Subject: RE: Man in the middle proxy - Not working


>
> Raj wrote:
>
>> Thanks for all the response
>>         1. I was able to do the handshaking successfully with the
>> browser.
>> On receiving the request from the browser I will send "HTTP OK "
>> response
>> back to the browser, I was able to do the handshaking and read the
>> actual
>> GET request.
>>         2. Then I create a new socket to establish the connection with
>> server. The connection was successful.
>>             Sends the request to the server
>>             Reads the request from the server
>>
>> When I read the response from the server it always return empty.
>
> What does that mean? Are you doing a blocking read or a non-blocking read?
> If 'read' returns zero, then the connection was closed by the server. If
> 'read' returns a number less than zero, there is an error -- tell us what
> error you are getting. If 'read' returns a number greater than zero, then
> that is the first part of the response.
>
>> I
>> don't
>> know what went wrong here. I am reading the data from the socket using
>> 'recv' function. Can anybody tell me what went wrong
>
> So, what return value do you get from 'recv'?
>
> DS
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Man in the middle proxy - Not working

Raj-66
In reply to this post by Dave Thompson-5
Hi
    I will describe my code snippet below

    The module for connecting to server

         SOCKET RequestSock;
         SOCKADDR_IN ClientAddr;
         RequestSock =
WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED);
         pHost = gethostbyname(pcTargetURL);
         memset(&ClientAddr,0,sizeof(ClientAddr));
         int iAddrLen = sizeof(ClientAddr);
         ClientAddr.sin_family = AF_INET;
         memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length);
         ClientAddr.sin_port = htons(atoi(pcPort));
         if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr,
sizeof(SOCKADDR_IN)))
         {
              closesocket(RequestSock); // Connection failed
              return false;
         }

         WSAOVERLAPPED SendOverlapped;
         DWORD dwSendDataLen = 0;
         WSABUF ClientRequestBuf;
         WSAEVENT SendEvent[1];
         ClientRequestBuf.buf = pcData;
         ClientRequestBuf.len = strlen(pcData);
         SendEvent[0] = WSACreateEvent();
         SendOverlapped.hEvent = SendEvent[0];
         iRes =
WSASend(RequestSock,&ClientRequestBuf,1,&dwSendDataLen,dwFlag,&SendOverlapped,NULL);
            // Sending data to the server

    FYI
        pcPort = 443
        pcTargetURL = L"www.facebook.com";
       pcData = "GET https://www.facebook.com HTTP/1.0\r\n\r\n"

Thanks, Raj Rajmohan SK
----- Original Message -----
From: "Dave Thompson" <[hidden email]>
To: <[hidden email]>
Sent: Thursday, August 05, 2010 7:48 AM
Subject: RE: Man in the middle proxy - Not working


>> From: [hidden email] On Behalf Of Raj
>> Sent: Wednesday, 04 August, 2010 01:09
>
>> Thanks for all the response
>>         1. I was able to do the handshaking successfully with
>> the browser.
>> On receiving the request from the browser I will send "HTTP
>> OK "  response
>> back to the browser, I was able to do the handshaking and
>> read the actual
>> GET request.
>
> To be clear: I interpret you received CONNECT, sent OK,
> did SSL handshake between browser and you (SSL_accept),
> then SSL_read (data which is a) GET request.
>
>>         2. Then I create a new socket to establish the
>> connection with
>> server. The connection was successful.
>>             Sends the request to the server
>>             Reads the request from the server
>>
> (Obviously you mean read response.)
>
>> When I read the response from the server it always return
>> empty. I don't
>> know what went wrong here. I am reading the data from the
>> socket using
>> 'recv' function. Can anybody tell me what went wrong
>>
> Is the connection to the server clear, or SSL?
>
> If SSL, you must use SSL_{connect,write,read,etc} throughout,
> with a different SSL* pointer than the one for the client side.
> And check for errors and report them etc.
>
> If clear, either:
>
> - you did the send and/or recv wrong; we'd have to look at
> your code, which you should simplify/trim as much as possible.
>
> - the server didn't like the request you sent, or you,
> strongly enough it just closed the connection. For HTTP
> this should be rare; most issues with the actual request
> (such as bad method or resource, unauthorized, bad or
> prohibited or required body, etc.) have defined HTTP
> error responses. Something like a firewall or frontend
> that works at the TCP level might just disconnect you,
> although in my experience they usually block or reject
> the initial connection (SYN) or break abruptly (RST),
> either of which appears to your program as an error
> return (canonically -1, not 0).
>
> Can you contact the people operating the server, and
> can they check their logs around the time of your attempt?
>
> Can you connect to the server from a browser on the machine
> running your proxy, or at one nearby on the same subnet?
> In clear, SSL, or both? And do a GET like the one you are
> (receiving and) forwarding from your client? Successfully?
>
> Can you run a monitor like tcpdump or wireshark while running
> your program, to see what was actually sent to the server
> and confirm if any data or what flags came back?
>
>
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Man in the middle proxy - Not working

Dave Thompson-5
> From: [hidden email] On Behalf Of Raj
> Sent: Thursday, 05 August, 2010 01:06

>     I will describe my code snippet below
>
>     The module for connecting to server
>
>          SOCKET RequestSock;
>          SOCKADDR_IN ClientAddr;
>          RequestSock =
> WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED);

I don't know much about 'OVERLAPPED' in Windows, but I think
it's something like 'nonblocking' in Unix.

>          pHost = gethostbyname(pcTargetURL);
>          memset(&ClientAddr,0,sizeof(ClientAddr));
>          int iAddrLen = sizeof(ClientAddr);
>          ClientAddr.sin_family = AF_INET;
>          memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length);
>          ClientAddr.sin_port = htons(atoi(pcPort));
>          if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr,
> sizeof(SOCKADDR_IN)))
>          {
>               closesocket(RequestSock); // Connection failed
>               return false;
>          }
>
>          WSAOVERLAPPED SendOverlapped;
>          DWORD dwSendDataLen = 0;
>          WSABUF ClientRequestBuf;
>          WSAEVENT SendEvent[1];
>          ClientRequestBuf.buf = pcData;
>          ClientRequestBuf.len = strlen(pcData);
>          SendEvent[0] = WSACreateEvent();
>          SendOverlapped.hEvent = SendEvent[0];
>          iRes =
> WSASend(RequestSock,&ClientRequestBuf,1,&dwSendDataLen,dwFlag,
> &SendOverlapped,NULL);
>             // Sending data to the server
>
At this point, the send probably hasn't actually happened.
And if you call [WSA]Recv and it returns, it almost certainly
hasn't actually been done either. You probably have to do
some kind of synchronization with the .hEvent, following
whatever Windows rules are applicable.

>     FYI
>         pcPort = 443
>         pcTargetURL = L"www.facebook.com";
>        pcData = "GET https://www.facebook.com HTTP/1.0\r\n\r\n"
>


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Man in the middle proxy - Not working

Jakob Bohm-7
On 05-08-2010 22:49, Dave Thompson wrote:
 >> From: [hidden email] On Behalf Of Raj
 >> Sent: Thursday, 05 August, 2010 01:06
 >
 >>      I will describe my code snippet below
 >>
 >>      The module for connecting to server
 >>
 >>           SOCKET RequestSock;
 >>           SOCKADDR_IN ClientAddr;
 >>           RequestSock =
 >> WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED);
 >
 > I don't know much about 'OVERLAPPED' in Windows, but I think
 > it's something like 'nonblocking' in Unix.

Actually, this is the Win32 level way to do Asynchronous IO (AIO),
meaning that the operation will happen immediately or in the
background, without the cost of extra threads or buffering.
The OS will hold on to your buffer pointer and other arguments
and then signal completion by signaling the provided or event
or (if no event is given), the socket itself.  Once signaled,
the buffers are yours again and the success/failure, transmitted
length etc. are available.

The actual transmission logic will usually happen in DMA,
interrupt handlers etc.

 >
 >>           pHost = gethostbyname(pcTargetURL);
 >>           memset(&ClientAddr,0,sizeof(ClientAddr));
 >>           int iAddrLen = sizeof(ClientAddr);
 >>           ClientAddr.sin_family = AF_INET;
 >>           memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length);
 >>           ClientAddr.sin_port = htons(atoi(pcPort));
 >>           if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr,
 >> sizeof(SOCKADDR_IN)))
 >>           {
 >>                closesocket(RequestSock); // Connection failed
 >>                return false;
 >>           }
 >>
 >>           WSAOVERLAPPED SendOverlapped;
 >>           DWORD dwSendDataLen = 0;
 >>           WSABUF ClientRequestBuf;
 >>           WSAEVENT SendEvent[1];
 >>           ClientRequestBuf.buf = pcData;
 >>           ClientRequestBuf.len = strlen(pcData);
 >>           SendEvent[0] = WSACreateEvent();
 >>           SendOverlapped.hEvent = SendEvent[0];
 >>           iRes =
 >> WSASend(RequestSock,&ClientRequestBuf,1,&dwSendDataLen,dwFlag,
 >> &SendOverlapped,NULL);
 >>              // Sending data to the server
 >>
 > At this point, the send probably hasn't actually happened.
 > And if you call [WSA]Recv and it returns, it almost certainly
 > hasn't actually been done either. You probably have to do
 > some kind of synchronization with the .hEvent, following
 > whatever Windows rules are applicable.
 >
Yep

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Man in the middle proxy - Not working

Raj-66
In reply to this post by Dave Thompson-5
Hi

    Can you send me some code snippet which shows how to commutate with
webserver and read the content data

Thanks,
Raj
Rajmohan SK

----- Original Message -----
From: "Dave Thompson" <[hidden email]>
To: <[hidden email]>
Sent: Friday, August 06, 2010 2:19 AM
Subject: RE: Man in the middle proxy - Not working


>> From: [hidden email] On Behalf Of Raj
>> Sent: Thursday, 05 August, 2010 01:06
>
>>     I will describe my code snippet below
>>
>>     The module for connecting to server
>>
>>          SOCKET RequestSock;
>>          SOCKADDR_IN ClientAddr;
>>          RequestSock =
>> WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED);
>
> I don't know much about 'OVERLAPPED' in Windows, but I think
> it's something like 'nonblocking' in Unix.
>
>>          pHost = gethostbyname(pcTargetURL);
>>          memset(&ClientAddr,0,sizeof(ClientAddr));
>>          int iAddrLen = sizeof(ClientAddr);
>>          ClientAddr.sin_family = AF_INET;
>>          memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length);
>>          ClientAddr.sin_port = htons(atoi(pcPort));
>>          if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr,
>> sizeof(SOCKADDR_IN)))
>>          {
>>               closesocket(RequestSock); // Connection failed
>>               return false;
>>          }
>>
>>          WSAOVERLAPPED SendOverlapped;
>>          DWORD dwSendDataLen = 0;
>>          WSABUF ClientRequestBuf;
>>          WSAEVENT SendEvent[1];
>>          ClientRequestBuf.buf = pcData;
>>          ClientRequestBuf.len = strlen(pcData);
>>          SendEvent[0] = WSACreateEvent();
>>          SendOverlapped.hEvent = SendEvent[0];
>>          iRes =
>> WSASend(RequestSock,&ClientRequestBuf,1,&dwSendDataLen,dwFlag,
>> &SendOverlapped,NULL);
>>             // Sending data to the server
>>
> At this point, the send probably hasn't actually happened.
> And if you call [WSA]Recv and it returns, it almost certainly
> hasn't actually been done either. You probably have to do
> some kind of synchronization with the .hEvent, following
> whatever Windows rules are applicable.
>
>>     FYI
>>         pcPort = 443
>>         pcTargetURL = L"www.facebook.com";
>>        pcData = "GET https://www.facebook.com HTTP/1.0\r\n\r\n"
>>
>
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Man in the middle proxy - Not working

Raj-66
Hi
       I was able to read the content data from the server using SSL_read
and put back to the browser by using SSL_write. I don't know whether is a
right approach or not. I have done the experiment in these two urls

1. https://s-static.ak.facebook.com/rsrc.php/z9Q0Q/hash/8yhim1ep.ico

2. https://s-static.ak.facebook.com/rsrc.php/z8OGI/hash/41j5eq4v.png

For the first try I got the response as follows and I was able to see the
icon in my browser
    HTTP/1.1 200 OK
    Cache-Control: public, max-age=31536000
    Content-Length: 318
    Content-Type: image/x-icon
    Expires: Sat, 06 Aug 2011 06:58:14 -0700
    Last-Modified: Sat, 01 Jan 2000 00:00:00 GMT
    P3P: CP="DSP LAW"
    Pragma:
    X-Cnection: close
    Date: Fri, 06 Aug 2010 13:58:14 GMT

But for the second link, which is  42,565 bytes long, I am receiving the
following output. I understood that there is more to do inorder to read the
content data, which I am not sure about
    HTTP/1.1 200 OK
    Cache-Control: public, max-age=31536000
    Content-Length: 42565
    Content-Type: image/png
    Expires: Sat, 06 Aug 2011 07:04:17 -0700
    Last-Modified: Sat, 01 Jan 2000 00:00:00 GMT
    P3P: CP="DSP LAW"
    Pragma:
    X-Cnection: close
    Date: Fri, 06 Aug 2010 14:04:17 GMT

    Can anybody tell me what else should I do inorder to read the content
and show it the browser. The following are sending some code snippets


            RequestSock =
WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED);
             pHost = gethostbyname(pcTargetURL);
             memset(&ClientAddr,0,sizeof(ClientAddr));
             ClientAddr.sin_family = AF_INET;
             memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length);
             ClientAddr.sin_port = htons(atoi(pcPort));
              if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr,
sizeof(SOCKADDR_IN)))
             {
                  closesocket(RequestSock); // Connection failed
                  return false;
              }

             SSL *Serverssl;
             Serverssl = SSL_new(m_pSSLCtx);
             SSL_set_fd(Serverssl, RequestSock);
            iRes = SSL_connect(Serverssl);
             if(iRes <= 0 )
             {
                      ERR_print_errors_fp(stderr);
                      cout << " connect Failed " << endl;
              }
               iRes = SSL_write(Serverssl,pcData, strlen(pcData));
             SSL_accept(Serverssl);
             do
             {
                      dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize);
                       SSL_write(SourceSsl,pBuff,dwReadDataLen);
                       cout << "Read buffer \n" << pBuff << endl;
               } while(SSL_pending(Serverssl));


Thanks,
Raj
Rajmohan SK

----- Original Message -----
From: "Raj" <[hidden email]>
To: <[hidden email]>
Sent: Friday, August 06, 2010 10:12 AM
Subject: Re: Man in the middle proxy - Not working


> Hi
>
>    Can you send me some code snippet which shows how to commutate with
> webserver and read the content data
>
> Thanks,
> Raj
> Rajmohan SK
>
> ----- Original Message -----
> From: "Dave Thompson" <[hidden email]>
> To: <[hidden email]>
> Sent: Friday, August 06, 2010 2:19 AM
> Subject: RE: Man in the middle proxy - Not working
>
>
>>> From: [hidden email] On Behalf Of Raj
>>> Sent: Thursday, 05 August, 2010 01:06
>>
>>>     I will describe my code snippet below
>>>
>>>     The module for connecting to server
>>>
>>>          SOCKET RequestSock;
>>>          SOCKADDR_IN ClientAddr;
>>>          RequestSock =
>>> WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED);
>>
>> I don't know much about 'OVERLAPPED' in Windows, but I think
>> it's something like 'nonblocking' in Unix.
>>
>>>          pHost = gethostbyname(pcTargetURL);
>>>          memset(&ClientAddr,0,sizeof(ClientAddr));
>>>          int iAddrLen = sizeof(ClientAddr);
>>>          ClientAddr.sin_family = AF_INET;
>>>          memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length);
>>>          ClientAddr.sin_port = htons(atoi(pcPort));
>>>          if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr,
>>> sizeof(SOCKADDR_IN)))
>>>          {
>>>               closesocket(RequestSock); // Connection failed
>>>               return false;
>>>          }
>>>
>>>          WSAOVERLAPPED SendOverlapped;
>>>          DWORD dwSendDataLen = 0;
>>>          WSABUF ClientRequestBuf;
>>>          WSAEVENT SendEvent[1];
>>>          ClientRequestBuf.buf = pcData;
>>>          ClientRequestBuf.len = strlen(pcData);
>>>          SendEvent[0] = WSACreateEvent();
>>>          SendOverlapped.hEvent = SendEvent[0];
>>>          iRes =
>>> WSASend(RequestSock,&ClientRequestBuf,1,&dwSendDataLen,dwFlag,
>>> &SendOverlapped,NULL);
>>>             // Sending data to the server
>>>
>> At this point, the send probably hasn't actually happened.
>> And if you call [WSA]Recv and it returns, it almost certainly
>> hasn't actually been done either. You probably have to do
>> some kind of synchronization with the .hEvent, following
>> whatever Windows rules are applicable.
>>
>>>     FYI
>>>         pcPort = 443
>>>         pcTargetURL = L"www.facebook.com";
>>>        pcData = "GET https://www.facebook.com HTTP/1.0\r\n\r\n"
>>>
>>
>>
>> ______________________________________________________________________
>> OpenSSL Project                                 http://www.openssl.org
>> User Support Mailing List                    [hidden email]
>> Automated List Manager                           [hidden email]
>>
>
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Man in the middle proxy - Not working

Dave Thompson-5
> From: [hidden email] On Behalf Of Raj
> Sent: Friday, 06 August, 2010 10:14

>        I was able to read the content data from the server
> using SSL_read
> and put back to the browser by using SSL_write. I don't know
> whether is a
> right approach or not.

If you are doing an SSL connection to the server then
SSL_write to and SSL_read from the server are correct.
(And you should since the client is requesting SSL.)
SSL_read from and SSL_write back to the client are
correct if the client is SSL, and you said it is.

> For [an .ico] I got the response as follows and I was
> able to see the
> icon in my browser <snip>
> But for [a .png], which is  42,565 bytes long, I am
> receiving the
> following output. I understood that there is more to do
> inorder to read the
> content data, which I am not sure about <snip>
>     Can anybody tell me what else should I do inorder to read
> the content
> and show it the browser. The following are sending some code snippets
>
>
>             RequestSock =
> WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED);

For a socket used with openssl directly, I believe OVERLAPPED
will be ignored and is of no use. I think you would have to do
your own 'physical' level either as your own BIO type or as
a BIO_pair looping back to your code (the more usual way).
Frankly I don't think you're anywhere near ready for that.

>              pHost = gethostbyname(pcTargetURL);
>              memset(&ClientAddr,0,sizeof(ClientAddr));
>              ClientAddr.sin_family = AF_INET;
>              memcpy(&ClientAddr.sin_addr,pHost->h_addr,
> pHost->h_length);
>              ClientAddr.sin_port = htons(atoi(pcPort));
>               if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr,
> sizeof(SOCKADDR_IN)))
>              {
>                   closesocket(RequestSock); // Connection failed
>                   return false;
>               }
>
>              SSL *Serverssl;
>              Serverssl = SSL_new(m_pSSLCtx);
>              SSL_set_fd(Serverssl, RequestSock);
>             iRes = SSL_connect(Serverssl);
>              if(iRes <= 0 )
>              {
>                       ERR_print_errors_fp(stderr);
>                       cout << " connect Failed " << endl;
>               }
>                iRes = SSL_write(Serverssl,pcData, strlen(pcData));

You should check for error (<=0) and report/handle it. Error on _write
especially initial is not common, but if it ever happens, proceeding
with other operations will likely cause much greater confusion.

>              SSL_accept(Serverssl);

This is useless. SSL_accept _creates_ a server-side endpoint;
it is not applicable to a client-side endpoint.

>              do
>              {
>                       dwReadDataLen =
> SSL_read(Serverssl,pBuff,iBufferSize);
>                        SSL_write(SourceSsl,pBuff,dwReadDataLen);
>                        cout << "Read buffer \n" << pBuff << endl;

Again check for errors. Especially on the _read side,
they are actually quite possible.

Also, the data read by SSL_read (like POSIX read or C fread)
does not get a null terminator byte added, so outputting
pBuff as a C-style string is likely to append garbage,
especially on the second or more time through the loop.

>                } while(SSL_pending(Serverssl));
>
That's your problem. SSL_pending only indicates data _already
received and buffered_ by OpenSSL but not yet read by the app.
For responses more than one SSL record (max 32kbytes if I recall
correctly, and server may choose less) AND (probably) more than
the TCP window (varies but typically 2 MTU = about 3kbytes to start)
there will be some time delay between receiving the first chunk
of the data and the next, and the next and so on.

For a waited/blocking socket, which is the default as you have here,
you need to keep reading from the server (and in your case writing
back to the client) until you've done all the data in the response.
If you require, or the server chooses, HTTP/1.0 style conn-per-txn
(also known as connection: close or not-keepalive or not-pipelined,
and also not-chunked) you can just loop until you receive "EOF" (0)
from SSL_read, caused by the server closing the connection.

If you allow and the server uses HTTP/1.1 keep-alive (or pipelining)
and/or chunked data, the situation can get quite a bit more
complicated. See RFC 2616.

If you use a nonblocking socket (which is supported on Windows
as far as I know but is apparently not the same as OVERLAPPED)
you can also do your own timeout -- that is, read until EOF
or optionally calculated end of the response body, *or* timeout.
Since HTTP servers will normally send a complete response within
a short time (like at most a few seconds), and if one doesn't
a person at a browser usually doesn't want to wait anyway,
this can be a good simple compromise.



______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Man in the middle proxy - Not working

Raj-66
Hi

       Thanks for your valuable suggestion. I didn't understand some points
which you described in the previous posting, may because of my lack of
exposure to the socket technology.

    I have tried one more method to read the data from the socket, which was
partially successful  it is defined as follows
do
 {
      dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize);  // Gets the
data from the server side
      SSL_write(SourceSsl,pBuff,dwReadDataLen); // Writes the data back to
the SSL
 } while(dwReadDataLen > 0 );

By using this method I am able to read the content data from the server and
put it back to my browser. But this method is not consistent though,
Sometimes browse request will not get completed and also it takes lot of
time complete one browse request
-------------------------
Replies and quires to the previous posting

> For a socket used with openssl directly, I believe OVERLAPPED
> will be ignored and is of no use. I think you would have to do
> your own 'physical' level either as your own BIO type or as
> a BIO_pair looping back to your code (the more usual way).
> Frankly I don't think you're anywhere near ready for that.

I didn't understand about this, Can you describe this in more detail, Sorry
for that I am new to this technology

> You should check for error (<=0) and report/handle it. Error on _write
> especially initial is not common, but if it ever happens, proceeding
> with other operations will likely cause much greater confusion.
>

I have checked all the error codes of SSL functions in my application, I
have posted only some code snippet to avoid junk data

>>              SSL_accept(Serverssl);
> This is useless. SSL_accept _creates_ a server-side endpoint;
> it is not applicable to a client-side endpoint.

I have removed this from my application

> Also, the data read by SSL_read (like POSIX read or C fread)
> does not get a null terminator byte added, so outputting
> pBuff as a C-style string is likely to append garbage,
> especially on the second or more time through the loop.

I have outputted the buffer only for indicative purpose. I have removed the
code for outputting the buffer

> That's your problem. SSL_pending only indicates data _already
> received and buffered_ by OpenSSL but not yet read by the app.
> For responses more than one SSL record (max 32kbytes if I recall
> correctly, and server may choose less) AND (probably) more than
> the TCP window (varies but typically 2 MTU = about 3kbytes to start)
> there will be some time delay between receiving the first chunk
> of the data and the next, and the next and so on.

    } while(SSL_pending(Serverssl));

Instead of using the above condition I have opted for

    while(dwReadDataLen > 0 );

By using this I was able to read the content data.


> For a waited/blocking socket, which is the default as you have here,
> you need to keep reading from the server (and in your case writing
> back to the client) until you've done all the data in the response.
> If you require, or the server chooses, HTTP/1.0 style conn-per-txn
> (also known as connection: close or not-keepalive or not-pipelined,
> and also not-chunked) you can just loop until you receive "EOF" (0)
> from SSL_read, caused by the server closing the connection.

"EOF" (0)

 I am not sure about EOF(0), is that some thing similar to End Of File in
C++;

> If you allow and the server uses HTTP/1.1 keep-alive (or pipelining)
> and/or chunked data, the situation can get quite a bit more
> complicated. See RFC 2616.
>
> If you use a nonblocking socket (which is supported on Windows
> as far as I know but is apparently not the same as OVERLAPPED)
> you can also do your own timeout -- that is, read until EOF
> or optionally calculated end of the response body, *or* timeout.
> Since HTTP servers will normally send a complete response within
> a short time (like at most a few seconds), and if one doesn't
> a person at a browser usually doesn't want to wait anyway,
> this can be a good simple compromise.

Could you send me some code snippet using 'bio' in SSL, I have seen using
'bio' is some sample applications instead of Sockets

Thanks,
Raj
Rajmohan SK

----- Original Message -----
From: "Dave Thompson" <[hidden email]>
To: <[hidden email]>
Sent: Saturday, August 07, 2010 9:06 AM
Subject: RE: Man in the middle proxy - Not working


>> From: [hidden email] On Behalf Of Raj
>> Sent: Friday, 06 August, 2010 10:14
>
>>        I was able to read the content data from the server
>> using SSL_read
>> and put back to the browser by using SSL_write. I don't know
>> whether is a
>> right approach or not.
>
> If you are doing an SSL connection to the server then
> SSL_write to and SSL_read from the server are correct.
> (And you should since the client is requesting SSL.)
> SSL_read from and SSL_write back to the client are
> correct if the client is SSL, and you said it is.
>
>> For [an .ico] I got the response as follows and I was
>> able to see the
>> icon in my browser <snip>
>> But for [a .png], which is  42,565 bytes long, I am
>> receiving the
>> following output. I understood that there is more to do
>> inorder to read the
>> content data, which I am not sure about <snip>
>>     Can anybody tell me what else should I do inorder to read
>> the content
>> and show it the browser. The following are sending some code snippets
>>
>>
>>             RequestSock =
>> WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED);
>
> For a socket used with openssl directly, I believe OVERLAPPED
> will be ignored and is of no use. I think you would have to do
> your own 'physical' level either as your own BIO type or as
> a BIO_pair looping back to your code (the more usual way).
> Frankly I don't think you're anywhere near ready for that.
>
>>              pHost = gethostbyname(pcTargetURL);
>>              memset(&ClientAddr,0,sizeof(ClientAddr));
>>              ClientAddr.sin_family = AF_INET;
>>              memcpy(&ClientAddr.sin_addr,pHost->h_addr,
>> pHost->h_length);
>>              ClientAddr.sin_port = htons(atoi(pcPort));
>>               if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr,
>> sizeof(SOCKADDR_IN)))
>>              {
>>                   closesocket(RequestSock); // Connection failed
>>                   return false;
>>               }
>>
>>              SSL *Serverssl;
>>              Serverssl = SSL_new(m_pSSLCtx);
>>              SSL_set_fd(Serverssl, RequestSock);
>>             iRes = SSL_connect(Serverssl);
>>              if(iRes <= 0 )
>>              {
>>                       ERR_print_errors_fp(stderr);
>>                       cout << " connect Failed " << endl;
>>               }
>>                iRes = SSL_write(Serverssl,pcData, strlen(pcData));
>
> You should check for error (<=0) and report/handle it. Error on _write
> especially initial is not common, but if it ever happens, proceeding
> with other operations will likely cause much greater confusion.
>
>>              SSL_accept(Serverssl);
>
> This is useless. SSL_accept _creates_ a server-side endpoint;
> it is not applicable to a client-side endpoint.
>
>>              do
>>              {
>>                       dwReadDataLen =
>> SSL_read(Serverssl,pBuff,iBufferSize);
>>                        SSL_write(SourceSsl,pBuff,dwReadDataLen);
>>                        cout << "Read buffer \n" << pBuff << endl;
>
> Again check for errors. Especially on the _read side,
> they are actually quite possible.
>
> Also, the data read by SSL_read (like POSIX read or C fread)
> does not get a null terminator byte added, so outputting
> pBuff as a C-style string is likely to append garbage,
> especially on the second or more time through the loop.
>
>>                } while(SSL_pending(Serverssl));
>>
> That's your problem. SSL_pending only indicates data _already
> received and buffered_ by OpenSSL but not yet read by the app.
> For responses more than one SSL record (max 32kbytes if I recall
> correctly, and server may choose less) AND (probably) more than
> the TCP window (varies but typically 2 MTU = about 3kbytes to start)
> there will be some time delay between receiving the first chunk
> of the data and the next, and the next and so on.
>
> For a waited/blocking socket, which is the default as you have here,
> you need to keep reading from the server (and in your case writing
> back to the client) until you've done all the data in the response.
> If you require, or the server chooses, HTTP/1.0 style conn-per-txn
> (also known as connection: close or not-keepalive or not-pipelined,
> and also not-chunked) you can just loop until you receive "EOF" (0)
> from SSL_read, caused by the server closing the connection.
>
> If you allow and the server uses HTTP/1.1 keep-alive (or pipelining)
> and/or chunked data, the situation can get quite a bit more
> complicated. See RFC 2616.
>
> If you use a nonblocking socket (which is supported on Windows
> as far as I know but is apparently not the same as OVERLAPPED)
> you can also do your own timeout -- that is, read until EOF
> or optionally calculated end of the response body, *or* timeout.
> Since HTTP servers will normally send a complete response within
> a short time (like at most a few seconds), and if one doesn't
> a person at a browser usually doesn't want to wait anyway,
> this can be a good simple compromise.
>
>
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Man in the middle proxy - Not working

JoelKatz

Raj wrote:

>     I have tried one more method to read the data from the socket,
> which was
> partially successful  it is defined as follows
> do
>  {
>       dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize);  // Gets
> the
> data from the server side
>       SSL_write(SourceSsl,pBuff,dwReadDataLen); // Writes the data back
> to
> the SSL
>  } while(dwReadDataLen > 0 );

        This is the basic idea of how you proxy, but it can't work for a
general HTTP proxy. For one thing, it assumes the end of a reply is marked
by the close of a connection. This is true for some HTTP requests, but it's
not true in general.

        You can write a proxy two different ways:

        1) You can understand the protocol you are parsing and know when it
changes directions. Based on this understanding, you can switch from
proxying in one direction to proxying in the other.

        2) You can avoid having to understand the protocol you are parsing.
But in this case, you will not know which side is supposed to send data
next, so you must always be ready to proxy in either direction.

        It seems you do neither of these two things. You try to proxy in
only one direction at a time but you don't track the protocol. How do you
even know when you've sent the entire request and can even enter this loop?
How do you know when you've read the entire reply and can begin reading the
next request?

        Your test condition, 'dwReadDataLen>0' will be true so long as the
connection is healthy. It will typically remain healthy even when the reply
has been fully sent.

        DS
 

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Man in the middle proxy - Not working

Raj-66

Hi
      I have created multiple threads for processing the multiple socket
request. On each thread I am waiting on a processing a single socket request
only
      May I attach my sample application along with my next posting so that
you will get more idea about what I am doing and you can instruct me as well
what went wrong in my application.

Thanks,
Raj
Rajmohan SK

----- Original Message -----
From: "David Schwartz" <[hidden email]>
To: <[hidden email]>
Sent: Thursday, August 19, 2010 5:51 AM
Subject: RE: Man in the middle proxy - Not working


>
> Raj wrote:
>
>>     I have tried one more method to read the data from the socket,
>> which was
>> partially successful  it is defined as follows
>> do
>>  {
>>       dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize);  // Gets
>> the
>> data from the server side
>>       SSL_write(SourceSsl,pBuff,dwReadDataLen); // Writes the data back
>> to
>> the SSL
>>  } while(dwReadDataLen > 0 );
>
> This is the basic idea of how you proxy, but it can't work for a
> general HTTP proxy. For one thing, it assumes the end of a reply is marked
> by the close of a connection. This is true for some HTTP requests, but
> it's
> not true in general.
>
> You can write a proxy two different ways:
>
> 1) You can understand the protocol you are parsing and know when it
> changes directions. Based on this understanding, you can switch from
> proxying in one direction to proxying in the other.
>
> 2) You can avoid having to understand the protocol you are parsing.
> But in this case, you will not know which side is supposed to send data
> next, so you must always be ready to proxy in either direction.
>
> It seems you do neither of these two things. You try to proxy in
> only one direction at a time but you don't track the protocol. How do you
> even know when you've sent the entire request and can even enter this
> loop?
> How do you know when you've read the entire reply and can begin reading
> the
> next request?
>
> Your test condition, 'dwReadDataLen>0' will be true so long as the
> connection is healthy. It will typically remain healthy even when the
> reply
> has been fully sent.
>
> DS
>
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Man in the middle proxy - Not working

Dave Thompson-5
In reply to this post by Raj-66
> From: [hidden email] On Behalf Of Raj
> Sent: Wednesday, 18 August, 2010 06:49

>     I have tried one more method to read the data from the
> socket, which was partially successful  it is defined as follows
> do
>  {
>       dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize);  
> // Gets the
> data from the server side
>       SSL_write(SourceSsl,pBuff,dwReadDataLen); // Writes the
> data back to
> the SSL
>  } while(dwReadDataLen > 0 );
>
That is my simple until-EOF version, see below.

> By using this method I am able to read the content data from
> the server and
> put it back to my browser. But this method is not consistent though,
> Sometimes browse request will not get completed and also it
> takes lot of
> time complete one browse request

That's pretty vague. One off-the-cuff guess:

As I alluded to, this method has the limitation that it will
only (exit and) close when the server does, so if the request
allows keepalive and the server chooses it, you never turn around
and see if the client=browser is trying again. Maybe it is. If so,
*that* request will never go anywhere. Maybe your browser is
timing out that request and retrying on a new (good) connection.
Maybe this depends on your browser/version/config or request(s),
or even the contents of the response page e.g. script or ActiveX.

You could look at the response data (just the headers is enough)
to see if keepalive is enabled, and check whether your loop
actually exits (i.e. you got EOF, which you *probably* won't
*if* the server chose keepalive). Or you could look at both
responses and requests on the local side with (I think) ssldump.
Or you could use a client which tells you (much) more about
the requests it is making; I guess wget might be persuaded.

Remember there is a big difference between a webpage and a
browser=client request and server response. Typical webpages
have CSS, scripts, images, and sometimes frames and objects.
One webpage may be 10 or 50 or 200 requests and responses.
Often a browser won't show you all of the page, and sometimes
even any of it, until all the requests/responses are complete.

If this is the problem, you need to either:

- get the server to do one response per connection (and close).
I know downgrading the request to 1.0 works, and I'm pretty sure
replacing or adding as applicable Connection: close on 1.1 does.
There may also be server-dependent ways.

- recognize the end of the response and close downward
(and upward also, since this connection is now orphan).
If the browser quickly tries a second request it will
get an error, but (much) faster, and more certainly retry
(since server async close is a more 'expected' error).

- recognize the end of the response and turn around
to handle another request (and response etc.).

> -------------------------
> Replies and quires to the previous posting
>
> > For a socket used with openssl directly, I believe OVERLAPPED
> > will be ignored and is of no use. I think you would have to do
> > your own 'physical' level either as your own BIO type or as
> > a BIO_pair looping back to your code (the more usual way).
> > Frankly I don't think you're anywhere near ready for that.
>
> I didn't understand about this, Can you describe this in more
> detail, Sorry
> for that I am new to this technology
>
See below for the detail I have, which is not very much.

> > You should check for error (<=0) and report/handle it. <snip>
> I have checked all the error codes of SSL functions in my
> application, I have posted only some code snippet to avoid junk data
>
Good. Mention that with your posted code, to avoid getting sidetracked.

> >>              SSL_accept(Serverssl);
> > This is useless. SSL_accept _creates_ a server-side endpoint;
> > it is not applicable to a client-side endpoint.
>
> I have removed this from my application
>
> > Also, the data read by SSL_read (like POSIX read or C fread)
> > does not get a null terminator byte added, <snip>
> I have outputted the buffer only for indicative purpose. I
> have removed the
> code for outputting the buffer
>
> > That's your problem. SSL_pending only indicates data _already
> > received and buffered_ by OpenSSL but not yet read by the app. <snip>
>     } while(SSL_pending(Serverssl));
>
> Instead of using the above condition I have opted for
>
>     while(dwReadDataLen > 0 );
>
> By using this I was able to read the content data.
>
See next.

>
> > For a waited/blocking socket, which is the default as you have here,
> > you need to keep reading from the server (and in your case writing
> > back to the client) until you've done all the data in the response.
> > If you require, or the server chooses, HTTP/1.0 style conn-per-txn
> > (also known as connection: close or not-keepalive or not-pipelined,
> > and also not-chunked) you can just loop until you receive "EOF" (0)
> > from SSL_read, caused by the server closing the connection.
>
> "EOF" (0)
>
>  I am not sure about EOF(0), is that some thing similar to
> End Of File in
> C++;
>
It's the same idea as C/C++ but actually more like POSIX/Unix.
C stdio (getchar, scanf, etc.) mostly returns -1* and also sets
a flag that can be tested with feof(fp); except fgets returns null
and fread returns a short count (which can be 0 or >0 but <N).
C++ AFAIK generally just sets a flag which can be tested several
ways, including istream<>::operator bool() (I think that's the
right syntax, I haven't used it for a while, it's at least close).
POSIX/Unix read and recv return 0, and can also return short count
>0 and <N without EOF; OpenSSL follows this convention. Thus
countread==0 means EOF and countread>0 means not-EOF, so that
change is exactly "loop until EOF" as I said *for 1.0 style*.

(*Note: technically the C standard only requires a negative int
value for EOF. In practice everybody always uses -1.)

> > If you allow and the server uses HTTP/1.1 keep-alive (or pipelining)
> > and/or chunked data, the situation can get quite a bit more
> > complicated. See RFC 2616.
> >
> > If you use a nonblocking socket (which is supported on Windows
> > as far as I know but is apparently not the same as OVERLAPPED)
> > you can also do your own timeout -- that is, read until EOF
> > or optionally calculated end of the response body, *or* timeout.
> > Since HTTP servers will normally send a complete response within
> > a short time (like at most a few seconds), and if one doesn't
> > a person at a browser usually doesn't want to wait anyway,
> > this can be a good simple compromise.
>
> Could you send me some code snippet using 'bio' in SSL, I
> have seen using
> 'bio' is some sample applications instead of Sockets
>
First, there's no connection at all between using BIOs and
handling 1.1 (keepalive and chunked). BIOs are 'below' OpenSSL
and HTTP is 'above'. BIOs could be useful for something like
OVERLAPPED sockets; they are of no use at all for 1.1.

Second, you* don't use BIOs instead of sockets (for SSL);
you use them in addition. (You can also use BIOs in OpenSSL
for disk files, like keys and certs, but that's unrelated.)
OpenSSL BIO is a generalized structure that does many things,
but there is one BIO type that wraps a socket namely BIO_sock.
If you just give libssl a socket, it actually creates and
internally uses a BIO_sock, your code just doesn't see it.
You can create the BIO_sock yourself, but there's no point.
(* To be clear, I use 'you' in the US sense of 'anybody',
not necessarily 'Raj' in particular. I understand UK usage
is 'one' and 'one's' etc., but that's unnatural to me.)

What you can do that may be useful, but not easy, is substitute
another BIO that isn't BIO_sock but still does the socket I/O.
To expand what I said before, there are two approaches:

- actually write your own BIO type, which does anything you want;
this will duplicate most of BIO_sock but (presumably) with changes
(such as OVERLAPPED, although I doubt that would actually get you
any benefit over default thread scheduling).

- use BIO_pair, which is a builtin type rather like a Unix pipe;
OpenSSL writes one 'end' and your code gets the data at the other
and must actually send it; conversely your code must actually
receive remote data and pass it in for OpenSSL to read. Since
your code cannot predict when to send and receive, you must
effectively do both 'all the time', either by threads that run
'continuously' (but may block) or by polling logic (which can be
pretty complicated) to respond to whatever happens whenever.

This is all from reading the documentation and apps/*
I don't have code that actually does this; my applications
are fine with the much simpler nonblocking-sockets + select.
(Including a proxy for a protocol much simpler than HTTP.)

<snip history>


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
12