Attention, this method is NOT pipelining as described in comments, and even might break if the http connection is too fast with httplib.ResponseNotReady. I’ll update this post when i’ll found a real and simple way to achieve pipelining, because one possible way to do it with httplib is really ugly
I wanted to do HTTP Pipelining using urllib2. But, first of all, what is pipelining ?
HTTP pipelining is a technique in which multiple HTTP requests are written out to a single socket without waiting for the corresponding responses.
What is the benefit of pipelining ? Less network load, speedup processing !
I was searching a way to do it with urllib2… But solution are complicated, and not fit well to my needs.
But way, why stay on urllib2 ? Use httplib !
Reusing the connection :
First, reusing the same connection
import httplib
server = httplib.HTTPConnection('yourserver.com')
server.request('GET', '/index.html')
print 'RESPONSE1:', server.getresponse().read()
server.request('GET', '/index2.html')
print 'RESPONSE2:', server.getresponse().read()
server.request('GET', '/index3.html')
print 'RESPONSE3:', server.getresponse().read()
Second, try pipelining !
import httplib
server = httplib.HTTPConnection('yourserver.com')
server.request('GET', '/index.html')
res1 = server.getresponse()
server.request('GET', '/index2.html')
res2 = server.getresponse()
server.request('GET', '/index3.html')
res3 = server.getresponse()
print 'RESPONSE1:', res1.read()
print 'RESPONSE2:', res2.read()
print 'RESPONSE3:', res3.read()