Summer is here, need a new theme for vim !
Attention, this method is NOT pipelining as described in comments, and even might break if the http connection is too fast with httplib.ResponseNotReady. I’ll update this post when i’ll found a real and simple way to achieve pipelining, because one possible way to do it with httplib is really ugly
I wanted to do HTTP Pipelining using urllib2. But, first of all, what is pipelining ?
HTTP pipelining is a technique in which multiple HTTP requests are written out to a single socket without waiting for the corresponding responses.
What is the benefit of pipelining ? Less network load, speedup processing !
I was searching a way to do it with urllib2… But solution are complicated, and not fit well to my needs.
But way, why stay on urllib2 ? Use httplib !
Reusing the connection :
First, reusing the same connection
import httplib server = httplib.HTTPConnection('yourserver.com') server.request('GET', '/index.html') print 'RESPONSE1:', server.getresponse().read() server.request('GET', '/index2.html') print 'RESPONSE2:', server.getresponse().read() server.request('GET', '/index3.html') print 'RESPONSE3:', server.getresponse().read()
Second, try pipelining !
import httplib server = httplib.HTTPConnection('yourserver.com') server.request('GET', '/index.html') res1 = server.getresponse() server.request('GET', '/index2.html') res2 = server.getresponse() server.request('GET', '/index3.html') res3 = server.getresponse() print 'RESPONSE1:', res1.read() print 'RESPONSE2:', res2.read() print 'RESPONSE3:', res3.read()
After a long time, c++, not c++, benchmarks, testing, bug-fixes, documentation fixes, website, releases notes and wiki… PyMT 0.4 is released !
Thanks to all of you, that’s was fun 🙂
For PyMT, Sharath need BGR support into PyMT, while his graphic card don’t support GL_BGR.
Well, after adapting swap code from Pyglet sourcecode, he say: “< patali> it works but very slow”
The goal is to swap a string from ‘bgrbgrbgr’ to ‘rgbrgbrgb’. Just let’s do a rapid benchmark.
import sys import re import time swap1_pattern = re.compile('(.)(.)(.)', re.DOTALL) swap1_repl = r'\3\2\1' def swap1(bytes): return swap1_pattern.sub(swap1_repl, bytes) def swap2(bytes): out = '' for i in xrange(0, len(bytes), 3): b, g, r = bytes[i:i+3] out += r out += g out += b return out def swap3(bytes): bytes = list(bytes) for i in xrange(0, len(bytes), 3): b, g, r = bytes[i:i+3] bytes[i:i+3] = r, g, b return ''.join(bytes) def swap4(bytes): blues = bytes[0::3] greens = bytes[1::3] reds = bytes[2::3] return ''.join(''.join(x) for x in zip(reds, greens, blues)) def swap5(bytes): from array import array a = array('b', bytes) a[0::3], a[2::3] = a[2::3], a[0::3] return a.tostring() def swap6(bytes): import numpy a = numpy.array(bytes, 'c') b = a[...,::-1] return b.tostring() def swap7(bytes): a = list(bytes) a[0::3], a[2::3] = a[2::3], a[0::3] return ''.join(a) def bench(func, bytes): sys.stderr.write('Bench %s: ' % str(func)) start = time.time() for i in xrange(20): sys.stderr.write('.') ret = func(bytes) if ret[:3] != 'rgb' or ret[-3:] != 'rgb': sys.stderr.write('INVALID DATA, start=%s, end=%s' % (ret[:3], ret[-3:])) break end = time.time() - start sys.stderr.write('| Finished in %.4f ' % end) return ret bytes = 'bgr' * 256 * 256 bench(swap1, bytes) bench(swap2, bytes) bench(swap3, bytes) bench(swap4, bytes) bench(swap5, bytes) bench(swap6, bytes) bench(swap7, bytes)
Swap1() is actually the way of pyglet. swap2/swap3 is from me, and others is found on some threads in pygame mailing list.
So, who win ?
21:09 tito@ashaka ~ $ python benchmark.py Bench <function swap1 at 0x7f1d154632a8>: ....................| Finished in 3.2713 Bench <function swap2 at 0x7f1d15463230>: ....................| Finished in 1.2860 Bench <function swap3 at 0x7f1d15470cf8>: ....................| Finished in 0.6535 Bench <function swap4 at 0x7f1d15470e60>: ....................| Finished in 0.6475 Bench <function swap5 at 0x7f1d15470ed8>: ....................| Finished in 0.0367 Bench <function swap6 at 0x7f1d15470f50>: ....................| Finished in 0.1959 Bench <function swap7 at 0x7f1d15473050>: ....................| Finished in 0.1482
The array solution is a lot faster than any other implementation ! And it’s python standard 🙂 Let’s take this…
Edit: thanks for Milan to come with another awesome solution !
def swap0(bytes): return bytes[::-1]
Result is: 0.0059 !
Tonight, i’ve tryed to analyse more the header part, and understand how data part work. And… found something !
I’ve been some week without blogging, but lot of things happened ! Finally, i’ve been able to see Thomas in IRL at Google Mentor Summit in San Francisco. It was an awesome week-end, funny, creative and geeky ! Lot of ideas, and goal : making PyMT more stable and well designed : the roadmap of PyMT 0.4 is finally available. We’ve also done a little session too at summit.
Thomas is now 3 days to ITS 2009 in Banff Canada, for a conference about PyMT. The paper will be available for everyone soon 🙂 And this week-end, i’m going to Edinburgh with Lucie to meet Mark, and see the awesome new multitouch room of Napier University. Sharath is also going next week to a FOSS in India to spread PyMT too 🙂
Theses days are really cool 🙂
Here is a very good news, Christian, Pawel and Nuigroup support me to go on the Google Mentor Summit, in 23/24 October ! I’ll finally meet all of them, and specially Thomas, the creator of awesome PyMT framework 🙂
This is the result to take care of Sharath during his GSOC 🙂
I’ll stay 2 days more with Thomas, to hack the future PyMT :p (well, work isn’t the real word… i want to see California a little :p)
Thanks Guys !
PyMT provide 2 way to make animations :
- property animation with Animation() class, can be created with start_animation() / stop_animation(),
- inner animation, that watch changes on a property and start automatically an animation.
They are things we cannot do actually :
- Fade in/out a widget and their children
- Scale the widget content
So, i was thinking to use Fbo to make more complex animation on a widget.
Before starting the how-to, here a demo-video.
PyMT – New desktop with menu animation
Ancien disque dur SEAGATE ST3750630AS 750Go :
$ sudo hdparm -tT /dev/sda /dev/sda: Timing cached reads: 7224 MB in 2.00 seconds = 3616.79 MB/sec Timing buffered disk reads: 304 MB in 3.00 seconds = 101.17 MB/sec $ sudo ~/seeker /dev/sda1 Seeker v2.0, http://www.linuxinsight.com/how_fast_is_your_disk.html Benchmarking /dev/sda1 (14456MB), wait 30 seconds....... Results: 114 seeks/second, 8.77 ms random access time
Nouveau disque dur SSD G.SKILL Falcon 64Go + 64Mo cache :
$ sudo hdparm -tT /dev/sdb /dev/sdb: Timing cached reads: 7056 MB in 2.00 seconds = 3532.79 MB/sec Timing buffered disk reads: 528 MB in 3.00 seconds = 175.71 MB/sec $ sudo ~/seeker /dev/sdb2 Seeker v2.0, http://www.linuxinsight.com/how_fast_is_your_disk.html Benchmarking /dev/sdb2 (51513MB), wait 30 seconds...... Results: 5452 seeks/second, 0.18 ms random access time
J’ai déplacé / et /home sur le nouveau disque, l’ancien reste là pour le stockage de donnée sur /usr/local 🙂
Ca marche du feu de dieu !