PyMT 0.5 out, what’s next ?
One week ago, we’ve released PyMT 0.5 (release notes). I’m very glad to see this release. We’ve working so much to make it stable, and we’ll continue.
The good part of this release is the availability on next Ubuntu Maverick. If you don’t know, Ubuntu Maverick will be multitouch. Before making the announcement public, we’ve joined the HCI group, and they have done documentation about Multitouch in Ubuntu, and PyMT in Ubuntu.
That’s a very good news, and i hope that we’ll be able to enhance our collaboration with Ubuntu, and share experience.
So, what’s next ?
Since people are asking always that “we should do a roadmap”, PyMT Roadmap is updated.
Next version will be the PyMT 0.5.1, and release is scheduled for the 6 September. This will include another set of fixes, found during the 0.5 release. And this will be the version shipped on next Ubuntu too. Auto detection of MT Hardware, fixes text rendering outside of the text input, gstreamer appsink missing, more unit test…
Starting September, the “PyMT Week Report” will back. Since the community is growing, it will be a good thing to know what’s new on the toolkit every weeks, and explain if we are hitting trouble, or if all the developments is ok 
Then… I’ve lot of idea on my mind, and will focus on :
- Spatialization: be able to handle 1 millions of widgets, added on an infinite plane.
- Graphics: finish and use graphics library (new from 0.5) for all of our widgets.
- App subsystem: replace the old MTPlugin with a new Application subsystem, that permit the use to browse and launch any PyMT application available on his system
- Enhance CSS writing (less or sass-lang)
Then in november, i’ll try to port PyMT on Javascript… using Pyjamas and HTML5.
PyMT is really going to be… awesome !
Joojoo, funny part :)
joojoo
description: Desktop Computer
product: PM235
vendor: nVidia
version: To Be Filled By O.E.M.
serial: To Be Filled By O.E.M.
width: 32 bits
capabilities: smbios-2.6 dmi-2.6
configuration: boot=normal chassis=desktop uuid=00020003-0004-0005-0006-000700080009
*-core
description: Motherboard
product: To be filled by O.E.M.
vendor: To be filled by O.E.M.
physical id: 0
version: To be filled by O.E.M.
serial: To be filled by O.E.M.
slot: To Be Filled By O.E.M.
Multitouch experience on Joojoo
I’m writing this blog post just to remember what i’ve done, and how. Don’t want to forget it for the next time It’s incomplete, and done for developer.
Last week, i’ve got a Joojoo, the old named “Crunchpad”. The hardware look good: Intel ATOM N270 + NVidia ION (GeForce 9400M) + 4Go SSD + 1Go Ram + 1.3M Webcam + USB + eGalax dualtouch screen + Wifi. At least, it look like a good hardware to hack on. Before you ask, i’ve buyed this tablet cause of NVidia ION, it support OpenGL 3.0, enough to run PyMT ! I’ll not say anything about the default OS installed on Joojoo… but will start directly how to install custom OS on it.
How to install custom OS on JooJoo
- You can use any usb device on the joojoo (CD or USB). I’ve used Unetbootin to create a custom USB boot key for Lucid Ubuntu. Don’t select net install, wifi driver is not included in the setup.
- Plug a usb switch + usb keyboard + usb key on the usb port of Joojoo
- Stay pressed on the DEL key + press twice the power button (once to make Joojoo logo appear, and a second longer to power the screen)
- In the boot menu, change the order of Joojoo SSD and your USB key, to make your USB key primary.
- Reboot by pressing twice the power button as described.
If you see nothing on the screen, reset the joojoo with the tool, and press the power button twice again.
Twice is really important, without that, the screen will stay black. On the second time, you must see the screen power up.
About drivers :
- The default wifi driver is not working, so go on http://www.realtek.com/downloads/ , search for 8192SE, and take RTL8192SE drivers for linux 2.6.
- For Nvidia ION, just download latest drivers from http://nvidia.com/.
- For touchscreen… that’s the whole problem. It’s not working by default. You can use the constructor driver available http://home.eeti.com.tw/web20/eGalaxTouchDriver/linuxDriver.htm, but dual touch is not working.
So, i’ve buyed this tablet for Dual Touch + OpenGL 3.0… Let’s check if windows is working with dual touch.
How to install window 7 on JooJoo
On this part, i would like to thank a lot Fabien for his experience on this domain.
The problem is, with 4Go SSD, you can’t install Normal Windows Seven. You must build a custom Window. To do that :
- go on Microsoft Windows 7 Embedded
- download the Standard 7 Toolkit
- launch the image configurator
- select all the things you want on windows… Like Window desktop, USB Boot, Power Management… I’ll upload my “answer” file tomorrow.
- your image must be < 2Go.
- burn your answer on usb (Tool menu). You’ll have a USB boot key ready to install Window Seven Embedded !
Then, it’s the same step as before, plug, reboot, install.
After installation of eGalax windows driver, the touchscreen work with dual touch !
How to capture USB on windows seven ?
Since wireshark is not able to capture USB, and no free toolkit is able to do it at this moment, i’ve used HHD Usb monitor, in trial version. Just run it, select Raw capture, Start, play with the screen, reboot the screen, play again with the screen (1 finger, then 2). And export the whole capture into HTML.
And here is the Joojoo eGalax USB Capture.
And that’s a bingo. We have bulk message with a 0×6 len, and other with 0xc len (double of 0×6.) Look really like we got 2 touch ! I don’t know what is the setup message, but look like theses messages activate the screen ! Now… go back on linux to test
Check eGalax dual touch support with python-usb
Enable usb_debug (optionnal)
Ok, this was really easy, cause of the old experience of Hack of Dell SX2210T screen. Even if the experience was not a success, learning how USB work was really funny. First thing, activate USB sniffing to check if we are doing it right :
$ sudo modprobe usb_debug
$ sudo cat /sys/kernel/debug/usb/usbmon/0u (0u is where my device is connected)
You’ll have a bunch of message, as soon as you play with the device. You can read usbmon/usb_debug documentation about the format of the message.
Search your device with python-usb
When you’re searching for device on dmesg, you’ll have :
generic-usb 0003:0EEF:720C.0001: input,hiddev96,hidraw0: USB HID v2.10 Pointer [eGalax Inc. USB TouchController] on usb-0000:00:06.0-1/input0
You can also use lsusb :
Bus 004 Device 002: ID 0eef:720c D-WAV Scientific Co., Ltd
The 0EEF is your vendor ID, and 720C is your product ID. Let’s search our device in USB busses:
def get_egalax_device(vendor=0x0eef, product=0x720c):
busses = usb.busses()
for bus in busses:
for device in bus.devices:
if device.idVendor != vendor:
continue
if device.idProduct != product:
continue
return device
return None
Send SETUP configuration
To understand more how USB work, you can check USB nut shell, the first part is talking about Setup packet.
This is exactly what we want, since on the capture we have :
000006: Control Transfer (UP), 02.08.2010 01:50:04.275 +0.0. Status: 0x00000000
Pipe Handle: 0x855dbe94
0A 01 41 0A 01 41
Setup Packet
40 00 00 00 00 00 06 00
Recipient: Device
Request Type: Vendor
Direction: Host->Device
Request: 0x0 (Unknown)
Value: 0x0
Index: 0x0
Length: 0x6
We have exactly 2 setup packet to transfer to the device. The most complicated is to understand python-usb.
- Recipient -> usb.RECIP_DEVICE
- Request Type -> usb.TYPE_VENDOR
- Direction -> usb.ENDPOINT_OUT
So, let’s send the first setup packet. This packet activate the device to talk with us (we’re receiving lot of packet from him).
conf = '\x40\x00\x00\x00\x00\x00\x06\x00' reqType = usb.ENDPOINT_OUT | usb.TYPE_VENDOR | usb.RECIP_DEVICE handle.controlMsg(reqType, usb.REQ_SET_CONFIGURATION, conf)
And the second request activate dual touch support (without him, we’re receiving only one touch) :
conf = '\x0a\x01\x41\x0a\x01\x41' handle.controlMsg(reqType, usb.REQ_SET_CONFIGURATION, conf)
And, that’s done ! When we read the device, we are receiving packet !
The format is :
- Flags : 8 bits (161 = ALIVE, 160 = UP)
- X : 16 bits
- Y : 16 bits
- ID : 8 bits (65 = touch 1, 66 = touch 2)
The flags and id is not really flags/id… because the value is not common. But it’s not changing on pressure, and not in the time… Dunno why.
Still, here is the full source code of the eGalax Joojoo dual touchscreen with python-usb.
And after ?
Normally, 2.6.35 include eGalax driver, but it’s not working. Will check another day why the kernel driver don’t support dual touch for this screen !
FITG 2010 finished, debriefing.
Another awesome event, organized by Damien Marchal and Nicolas Roussel. And for the first time, the PyMT Core team was in the same place. Thanks to all my friends in Lille for coming during the show, it was really nice to see you too !
We’ve done one conference, using the “presemt” application. For all who wanted to use our presentation application, you can get it on PyMT Apps repository. The slides will be available in PDF soon. A lot of peoples was interested, and impressed by PyMT itself. I’ve been very glad to meet some PyMT users, specially Jay !
On the barcamp, we’ve animated 3 sessions:
- Write your first PyMT Application (mathieu)
- Make a game with PyMT + PyMunk (christopher + thomas)
- Advanced OpenGL optimization + Python optimization
(Content will be available soon)
During the event, some work have been done :
- finish blobtracker in Movid
- ability to record / save PyMT events (event-recording branch)
- ability to share event over network (event-share branch)
- using PyMT + Ogre 3D with Anthony Martinet
And we’ve think a lot about the future of PyMT, and where we want to go for 1.0. More information soon
Also, a intern in the MINT team is actually researched to work on a project with PyMT. If any people is interested, don’t hesitate to send our CV to Nicolas.
It look like the future of PyMT is very good, and it’s just a start….
PyMT/Movid @FITG2010
That’s going to be a very nice event, again. A big UP to ircica team which have work with PyMT nearly from the start, and special thank to Damien Marechal for his contribution !
Now, they are doing a event on 3 days… and they are more.
Thomas Hansen and Christopher Denter from PyMT/Movid will be here ! For the very first time, our core team will be at the same place, IRL. That’s will change a lot from IRC
Be ready for our workshop, we are scheduled in Friday, 13h30, for 15 minutes + 5 minutes for discussion.
We have some ideas about how we can be really wonderful during only 15 little minutes
I’ll animate a session around coding with PyMT in Saturday. If you want to join, be ready with your laptop. It would be better if you already have PyMT installed.
For more information, you can read the FITG2010 website !
THSF 2010 finished, debriefing.
First of all, thanks to every awesome guys i’ve meet on this event. In particular to Marc Bruyère who give me a nice seat in his appartment. And every other guys in the tetalab !
I’ve been in Mix’art – Myrys, where living lot of artists, very nice. Creativity and OpenMind spirit is the keyword of the place.
The event involved Corpus Media, and they made fantastic show, based on the body, with video visualization and voice/audio transformation. I’ve take lot of pictures of mix’art, myrys, just browse them a little bit to feel the spirit
I’ve made a conference on PyMT on saturday. Unfortunatly, the streaming was cutted due to some guys using Free Internet connectivity for downloading torrents… Thanks guys ): Part of the streaming is available, but audio is desync. Specially for this event, you can see the Presemt application, made with Thomas and I. It’s available on PyMT-Apps : http://github.com/tito/pymt-apps/tree/master/presemt/. Presemt is a application to make dynamic presentation, based on PyMT.
I’ll record a little video, and push it on the website !
I think people have really appreciated the conference, and we already got new users
During the workshop day, Jimmy ask me if we can do an SMS Wall with PyMT, and play with it… I’ve spend more than 1 hour to check how i can get SMS from my HTC Hero. And i found Remote Notifier. Once a message is received on my Phone, it broadcast a TCP Packet using Wifi. 30 minutes later, we’ve got our first version of the SMSWall.
With a nice girl from tetalab (i don’t remember his name :/), we’ve started to make a V2 on saturday, with some other vizualization. But i’ve not been able to finish the code…
During the last day, i wanted to play with their LedWall in construction. An awesome work made by Lionel and Fabrice. This was my first attempt to use Arduino… and success ! After fixing some issues in the Arduino code, they have been able to use a sketch: Camera -> Pixelize -> Gray scale -> Send to arduino.
Now, you can check the video :
THSF – PyMT SMSWall
THSF – PyMT SMSWall + Deaf people
THSF – Led Wall !
And the pictures : http://picasaweb.google.fr/txprog/THSF2010
PyMT/Movid @THSP
THSP is a Festival in Toulouse, with lot of events and hackers, from the 28 to the 30 May.
Saturday, i’ll do a talk/introduction about PyMT and Movid. Jimi is doing a workshop during the whole event, to build another Wall, and some other Multitouch Table. It will be a nice playground for both software! Thanks to Marc Bruyere and Jimi Hertz for giving me the possibility to talk here
Read more about the event with the THSP Program May 2010.
Multitouch Linux Support for PyMT
Since the last week, i’ve done some jobs about supporting nativly multitouch screens in PyMT. Linux was missing in the native support, next to Windows and MacOSX, already available.
2 supports have been realized : HID Input Support, and MPX/XI2 Support.
1. Kernel Part
HID Input is the Input part of Linux Kernel. Since 2.6.34, lot of multitouch screens got their drivers integrated in the source code. Most of the drivers have been done by the Stephane Chatty from the ENAC Laboratory, inside the ShareIT project.
HID Input is reading the Event generated by the kernel, using /dev/input/eventX.
The Event are documented in the input.h. New event type have been added for Multitouch support: all the ABS_MT_. As today, we support :
- ABS_MT_POSITION_X : X position
- ABS_MT_POSITION_Y: Y position
- ABS_MT_PRESSURE : pressure
- ABS_MT_TOUCH_MAJOR : width of the object
- ABS_MT_TOUCH_MINOR : height of the object
Other event are ignored, but would be interested to add the support, like ABS_MT_ORIENTATION. But right now, no device support them.
This work is available on the latest version of PyMT. Check our PyMT github repository. If you want to known how to use it, you can read the mail sent to the pymt-dev mailing list.
The result of my work : PyMT + HIDInput support on Linux
2. Xorg
Since Xorg 7.5, MPX is available. MPX is the support of Multi Pointer inside X. Benjamin (from the Enac too), have provided a modified evdev module for Xorg, to read Multitouch Event from kernel, and pass them to Xorg.
This work require a running daemon, actually called multitouchd (on the same page as evdev). The daemon detect all the devices with Multitouch support, and create by default 5 fake mouse device, attached to the main multitouch device. With this way, each touch control one fake mouse device.
Unfortunaly, this is not stable, and with a stantum screen, once i get more that 4 or 5 touch, Xorg crash.
Anyway, i’ve start the support in the PyMT branch input-xi2. This rely on python-xlib, but this library required some change to make our work run.
Since it’s really more complicated than HID Input, i’ll not explain all the work right now. Maybe in another blog post.
3. So, which screen i could buy to play with ?
That’s pretty simple. ENAC (again) maintain a full list of multitouch screen working on linux, and the status of the support in linux kernel.
For my part, i’m playing with an Acer T230H. A little bit bugged, and support only 2 points. But enough to play and debug with right now.
That’s all for the moment, enjoy !
Blog updated !
My website is now part of the blog. The correct url is now :
http://txzone.net/
Please also update your rss to the new url :
http://txzone.net/?feed=rss2