Txzone

Using AI for refactoring Vue 2 to Vue 3

Wed, 08 Mar 2023 17:07:42 +0000

2022 has been an incredible year for AI-generated content. I see that as a tool, the same way you listen to music and create music afterwards. The same way my developer journey has been from reading open-source code, and contributing to the field.

I have a project made with Vue 2 using Typescript, Vue Property Decorator and Quasar. My goal was to refactor to Vue 3 using Typescript with Composition API. My time is limited on the subject and not a big priority, and the amount of work seems too much for my bare hand that, 2 years ago, that i dropped it.

We are in 2023, could AI help me for a such task ?

TDLR

Gist: https://gist.github.com/tito/230863f714ab43fc92ad55510de2ca4e

The test case

Here is my “test” source, where I’ll experiment the refactoring. The actual file is actually longer as it include template and style tag, but it doesn’t matter for our example.

<script lang="ts">
import { Component, Prop, Vue, Watch } from 'vue-property-decorator';

@Component
export default class Login extends Vue {
 public $store: any;
 public $q: any;
 public $route: any;
 public loading: boolean = false;
 public error: any = null;
 public email: string = '';
 public password: string = '';

 public async created() {
 const msgkey = this.$route.params.msg;
 if (msgkey === 'LOGOUT_SUCCESS') {
 this.$q.notify('Vous êtes déconnecté');
 }
 if (this.$route.query.email !== undefined) {
 this.email = this.$route.query.email;
 }
 }

 public handleSubmit() {
 this.error = null;
 if (this.password.length > 0) {
 this.loading = true;
 this.$store.dispatch('login', {
 email: this.email,
 password: this.password,
 }).catch((error: any) => {
 this.error = 'Email ou mot de passe invalide';
 this.loading = false;
 });
 }
 }
}
</script>

Copilot

I use copilot day-to-day as my advanced auto completion. While i don’t use it for creating a new file from scratch, it is a very handy tool for auto completion for repetitive tasks, from variable declarations, object copying, filter/map…

But that’s all it is. I cannot really take a whole file, and give instruction to copilot for converting from one syntax to another…

ChatGPT

Using OpenAI ChatGPT API, i got a prompt that fairly give me the output i wanted.

You are an assistant design to help developper for migrating their code from Vue 2 to Vue 3 using Typescript with Composition API. Here is a set of rules you must absolutely follow:

1. Rewrite the <script lang="ts"> to <script setup lang="ts">
2. The content of the script tag must be a valid Typescript code
3. The component must be flattened into the script setup
4. Remove any "export default".
5. Use the `onMounted` hook instead of the `created` lifecycle hook if necessary
6. Use the `useRoute` approach instead of $route. Same for $router.
7. Store is not using vuex but pinia.
8. Auth related function is accessible in stores/auth.ts, using useAuthStore.
9. Do not use Ref is the type can be infered from the value pass into ref()
10. Do not put all the methods and properties into a global const object
11. Prefer using global "const router = useRouter()" instead of live instanciation when needed

My problem is, result are not consistent, and do follow my rules.

First generation

Not as perfect as wanted. It choose to not use onMounted(), and flattened the code, which is allowed by the instruction. However, this.$q is here, which end up failing.

<script setup lang="ts">
import { useRouter } from 'vue-router';
import { useAuthStore } from '@/stores/auth';
import { ref } from 'vue';

const router = useRouter();
const authStore = useAuthStore();

const loading = ref(false);
const error = ref(null);
const email = ref('');
const password = ref('');

const handleSubmit = () => {
 error.value = null;
 if (password.value.length > 0) {
 loading.value = true;
 authStore.login({ email: email.value, password: password.value})
 .catch((error: any) => {
 error.value = 'Email ou mot de passe invalide';
 loading.value = false;
 });
 }
};

const msgkey = router.params.msg;
if (msgkey === 'LOGOUT_SUCCESS') {
 this.$q.notify('Vous êtes déconnecté');
}
if (router.query.email !== undefined) {
 email.value = router.query.email as string;
}

</script>

Second generation

That’s what bother me. By giving the same input, i have a now a different output. It recognized Quasar, and put the code into the onMounted(). Works for me. BUT, now the login function don’t have the same input as before, it decided to flatten the argument. OK.

<script setup lang="ts">
import { useRoute } from 'vue-router';
import { onMounted, ref } from 'vue';
import { useQuasar } from 'quasar';
import { useAuthStore } from '@/stores/auth';

const loading = ref(false);
const error = ref(null);
const email = ref('');
const password = ref('');

const { notify } = useQuasar();
const route = useRoute();
const authStore = useAuthStore();

onMounted(() => {
 const msgkey = route.params.msg;
 if (msgkey === 'LOGOUT_SUCCESS') {
 notify('Vous êtes déconnecté');
 }
 if (route.query.email !== undefined) {
 email.value = route.query.email as string;
 }
});

const handleSubmit = () => {
 error.value = null;
 if (password.value.length > 0) {
 loading.value = true;
 authStore.login(email.value, password.value)
 .catch((error: any) => {
 error.value = 'Email ou mot de passe invalide';
 loading.value = false;
 });
 }
};
</script>

When it went wrong

I tried to play with temperature, top_n, but from time to time, it generate something against the rules.

<script setup lang="ts">
...
const Login = () => {
 const store = useAuthStore();
 const router = useRouter();
 const loading = ref(false);
 const error = ref(null);
 const email = ref('');
 const password = ref('');

 onMounted(() => {
 ...
 });

 const handleSubmit = () => {
 ...
 }

 return {
 loading,
 error,
 email,
 password,
 handleSubmit
 }
}
</script>

<script setup lang="ts">
...
export default defineComponent({
 components: {
 GaugeChart,
 },
 setup() {
 const id = ref<number>();
 const loading = ref(false);
 const mapZoom = 8;
 const resources = ref([]);
...

Conclusion

I can go again and give another set of rules to use for the migration, especially moving from a single store using Vuex to many stores using Pinia.

Out of 92 files needed to convert, totally over 22997 lines to convert, around 10 needed a refactoring by hand as i was not able to generate a correct approach. Nevertheless, it was much faster to go other it with the tool and review rather than manually pass through everything.

For today, i take that as a win.

Kivy Programming Contest 2014

Sun, 16 Mar 2014 19:16:41 +0000

Kivy is organizing its second application development contest!. This is a great chance for users both new and experienced to show off their skills and compete for prizes. Entries will be judged on a range of criteria accessible to both new and experienced programmers, so don’t be afraid to dive in!

A big big thanks to the Python Software Foundation for being our first and lovely sponsor :)

2048, in Python / Kivy

Sun, 16 Mar 2014 19:14:15 +0000

If you don’t know the 2048 puzzle yet, you should. But be careful, it’s time consuming. I won’t go on the whole clone history of the game, but will just say that if you want to play it on the Web, play to the Gabriel Cirulli 2048. The author said it wont make an official iOS/Android app, so… let’s give Kivy a try :)

Only 2 hours of coding where needed for the first published version of the game. Then i learned on the Google Play Game services APIs for integrating a leaderboard and achievements. And after a couple of hours during the night, what i can say is: Be patient! Even if you follow the documentation carefully, you need to wait few hours before your app_id works on their server, even in test mode. There never warn us about it.

When the setup is done and it start working, it is easy to login:

 PythonActivity = autoclass('org.renpy.android.PythonActivity')
 GameHelper = autoclass('com.google.example.games.basegameutils.GameHelper')

 gh_instance = GameHelper(PythonActivity.mActivity, GameHelper.CLIENT_ALL)
 gh_instance_listener = GameHelperListener()
 gh_instance.setup(gh_instance_listener)
 gh_instance.onStart(PythonActivity.mActivity)
 android.activity.unbind(on_activity_result=_on_activity_result)
 android.activity.bind(on_activity_result=_on_activity_result)

That’s how you can unlock achievement:

# uid is the Google UID for the achievement you want
if gh_instance.isSignedIn():
 Games.Achievements.unlock(gh_instance.getApiClient(), uid)

And put the user score on the leaderboard:

# uid is the Google UID for the leaderboard you've created.
# You can have multiple leaderboard.
if gh_instance.isSignedIn():
 Games.Leaderboards.submitScore(gh_instance.getApiClient(), uid, score)

There is a little more code around that, but globally, using the new Play Games services APIs is now easy. I hope this little piece of code will help peoples to integrate it into their app. Somebody need to start a Python library for managing Google Play API and Game Center for iOS.

Have a look at the source code for an example of the integration with the Play Games services APIs :)

Using Facebook SDK with Python-for-Android / Kivy

Thu, 08 Aug 2013 09:50:26 +0000

For another museum project, Arnaud asked me to see if we could integrate Facebook in an app on Android. The usual libraries are made for desktop, and manually open a webbrowser to handle the user authorization. But that’s not really nice for us, it would be nicer to have a native integration instead. Let’s see how we can use the official Facebook Android SDK.

Prerequisites

You’ll need to have:

Kivy’s python-for-android
A Facebook account (obviously)
Facebook SDK 3.0

Get the example

Clone the Kivy/Facebook example:

$ git clone https://github.com/tito/p4a-facebook-example

Register a Facebook application

Go to the Facebook developers interface
Click on
Fill the information “App Name”. Don’t put “FB” or “Facebook” in the title, or your application name will be considered as Invalid :)
Pass the captcha
Write down your application id somewhere
Now, you need to activate Native Android App. The package name is the same name as the one we will pass in --package when building the APK. The Class Name is the packagename/activity. Activity class is always the same for all the applications compiled with python-for-android.
Right now, the Facebook Android SDK will not authorize our android application to be used, because Facebook want to know the signature hash that will be used by our generated APK. To generate the key hashes, you need to use the keytool. Here is an example for the android “debug” key:
```
$ keytool -exportcert -alias androiddebugkey \
 -keystore ~/.android/debug.keystore | \
 openssl sha1 -binary | openssl base64
Enter keystore password: android
u+bzQmG87L298C4KGM8yODi3W/4=
```
Copy the generated key hash, and paste it to the field:
Setup the our application
Go into p4a-facebook-example, and edit the main.py
Search for FACEBOOK_APP_ID, and replace with your own Application Id
Go into python-for-android, and create a distribution:
```
$ ./distribute.sh -m 'kivy'
```
Go into the dist/default

Generate the APK one time (for generating all the file we need to customize). Replace /path/to/p4a-facebook-example:

./build.py --name "Facebook test" --package org.test.facebook \
 --version 1 --private /path/to/p4a-facebook-example/ \
 --window --permission INTERNET debug

Add a new reference to project.properties to include the Facebook SDK. The path to the Facebook SDK MUST be a relative path, otherwise you’ll get issues during the next build.
```
android.library.reference.1=../../facebook-android-sdk-3.0.2/facebook
```
Edit the templates/AndroidManifest.tmpl.xml to include the Facebook login activity, just under the <application>:
Rebuild your application… and you’re ready to test :)

Install the application on your device:

adb install -r bin/Facebooktest-1-debug.apk

Test your application

The Facebook SDK 3.0 require to separate read and publish permissions. To the user, it means you’ll have 2 dialog to accept. Even if you just want to publish to the application stream, you need to have the basic permission (read) accepted by the user before publish permissions. This is already implemented in the example app.

When you start the application the first time, you’ll have:

Depending if you have the Native Facebook application or not, you’ll have 2 kinds of authentification boxes.

Without Facebook installed on the device:

And with Facebook installed on the device:

After authentication, the user will need to accept our permissions:

It’s done!

The application have now the right to post :) The example post in the user stream as the application itself, not as the user. It’s called “publish_actions”.

Getting further

Obviously, when you use the Facebook SDK itself, you feel the pain of an API designed for Java. For every callback that Facebook want to call, you need to implement a Java class and define the callback method. Python is really simpler and fun to use.

See for yourself: we want to make a call of “[Request.newStatusUpdateRequest](https://developers.facebook.com/docs/reference/android/current/Request#newStatusUpdateRequest(Session, String, Callback))”. Se weed first to implement a GraphUserCallback class, in Java. Thanks to Pyjnius, we can do it directly in Python:

class _FacebookGraphUserCallback(PythonJavaClass):

 __javainterfaces__ = ['com.facebook.Request$GraphUserCallback']
 __javacontext__ = 'app'

 def __init__(self, callback):
 self.callback = callback
 super(_FacebookGraphUserCallback, self).__init__()

 @java_method('(Lcom/facebook/model/GraphUser;Lcom/facebook/Response;)V')
 def onCompleted(self, user, response):
 self.callback(user, response)

This Python/Java class will call our own python callback when the Java callback onCompleted will be called. Then:

@run_on_ui_thread
def post(self, text, callback):
 req = Request.newStatusUpdateRequest(
 self._session, text, _FacebookRequestCallback(callback))
 req.executeAsync()

All you have to do at the end, is to call the post method:

def fb_post(self, text):
 def callback(*args):
 from time import time
 self.post_status = 'message posted at {}'.format(time())
 self.facebook.post(text, callback=callback)

I don’t provide a wrapper around all the possible Request method you can do with Facebook. I just have an example for “post” and “me”. Both wrap “[Request.newStatusUpdateRequest](https://developers.facebook.com/docs/reference/android/current/Request#newStatusUpdateRequest(Session, String, Callback))” and “[Request.newMeRequest](https://developers.facebook.com/docs/reference/android/current/Request#newMeRequest(Session, GraphUserCallback))”. Please note that every request call must happen in the UI thread. Use the python-for-android/runnable module for that, with @run_on_ui_thread decorator.

The end

It was not easy to get it right, and it still complex to make all the pieces together. I’ve tried to use their own LoginButton (because they said that’s how they want it, everywhere), but i’ve not be able to use it in our app. Mostly because the LoginButton is a pure android widget, and because it doesn’t want to bring back the Activity after login. I was stuck too much time on it, and preferred to go in another way. Please note that you should not share an application with a fake Facebook button, the design / look-and-feel must be the same as the original one.

I hope that will help some of you to get started !

Kivy 1.7.0 is out

Sun, 19 May 2013 07:48:23 +0000

If you missed it, Kivy 1.7.0 is out last Monday. It’s quite a big release that include some big fixes we did during the :

Android Emulator is now supported
ScrollView has been re-factored with new physics calculations. To be exact, the scrollview doesn’t calculate anything now, it just pass the touch position to an Scroll effect. This class calculate the movement’s velocity, and the over-scroll’s distance (means how far you scrolled out of the bounds). Then, we implemented 2 visual effects that use the over-scroll to make the scrollview act as a [Damped spring, and/or to fade out the scrollview if you over-scroll too much.](http://kivy.org/docs/api-kivy.effects.dampedscroll.html)
Garden: a separated organization for centralize user’s widgets and addons. You can create garden packages very easily, and import it in the source code with just “kivy.garden.packagename”. Few garden packages are already available!
And more, Check the announcement :)

Crowdfunding Kivy on Raspberry Pi

Tue, 26 Feb 2013 17:06:37 +0000

Kivy doesn’t run yet on the Raspberry Pi, and we need your help to crowdfund it! Even is Kivy is built to be compatible for GLES 2.0, we lack of a python EGL support.

The plan is to add a new EGL Window provider, in addition to the detection of non-multitouch devices such as keyboard and mouse.

All the informations are available at: https://www.bountysource.com/#fundraisers/143-kivy-on-raspberry-pi

Pyjnius: Accessing Java classes from Python

Mon, 20 Aug 2012 13:43:43 +0000

We’re pleased to announce the Pyjnius project. It’s a Python library to access Java classes from Python, on the desktop or on Android.

Source code: github.com/kivy/pyjnius
Documentation: pyjnius.readthedocs.org

We found others libraries such as JPype or Py4j, but we weren’t happy with the design and the usability. Using Jython wasn’t an option either, because we wanted to use the library within our python for android project.

Let me show you how simple it is to use Pyjnius:

>>> from jnius import autoclass
>>> Stack = autoclass('java.util.Stack')
>>> stack = Stack()
>>> stack.push('hello')
>>> stack.push('world')
>>> stack.pop()
'world'
>>> stack.pop()
'hello'

What we demonstrate here is the autoclass function that create kind-of proxy that reflect all the methods and fields available from the Java class java.util.Stack.

Ok, maybe you want an Android-related example? Just have a look:

from jnius import autoclass
from time import sleep

MediaRecorder = autoclass('android.media.MediaRecorder')
AudioSource = autoclass('android.media.MediaRecorder$AudioSource')
OutputFormat = autoclass('android.media.MediaRecorder$OutputFormat')
AudioEncoder = autoclass('android.media.MediaRecorder$AudioEncoder')

# Record the Microphone with a 3GP recorder
mRecorder = MediaRecorder()
mRecorder.setAudioSource(AudioSource.MIC)
mRecorder.setOutputFormat(OutputFormat.THREE_GPP)
mRecorder.setOutputFile('/sdcard/testrecorder.3gp')
mRecorder.setAudioEncoder(AudioEncoder.ARM_NB)
mRecorder.prepare()

# Record 5 seconds
mRecorder.start()
sleep(5)
mRecorder.stop()
mRecorder.release()

More examples are available in the documentation. We have a mapping between Java/Python type, native arrays, support for methods with multiple signatures, and a lot more. We are using Cython + JNI internally. The performance cost is minimal.

The library is already available for Python for android.

The library have been done by Gabriel Pettier and myself, under the Kivy organization.

IOS Support for Kivy

Sun, 11 Mar 2012 22:45:42 +0000

Kivy has now added official support for IOS platform. You can now package your Kivy application for the Ipad (and other iOS devices, testers needed).

The current instructions are available here: http://kivy.org/docs/guide/packaging-ios.html

For the full story on the multiple Apple limitations we are working with, i want to share the hardest part for Python integration: Apple’s prohibition on using dlopen() to load dynamic libraries. In a typical case, a compiled Python library’s extension is a “.so”, and the “.so” is dlopen() at import.

That being said, as we did for the python-for-android project published in January, we are redirecting the compilation object to create static libraries. Theses libraries are included in the final application binary. But it’s not enough: we must also tell to Python to look for the library entry point in the application binary, instead of dlopen(). So in the python dynload loader:

return (dl_funcptr) dlsym(RTLD_MAIN_ONLY, funcname)

This way, Python will always look at the application binary, and never need to use dlopen().

If you are worried that Apple would reject a Python-based application, or even using Kivy altogether, we have tested it for you: the game that won the Kivy contest has been packaged for IOS, submitted to Apple… and accepted. You can found Deflectouch on iTunes (source code).

Anyway, Kivy is now officially supporting 5 platforms: Windows, Linux, MacOSX, Android and IOS!

Enjoy :)

Kivy programming contest #1 is over!

Mon, 06 Feb 2012 10:34:54 +0000

During January, we hosted a contest for promoting the Kivy framework. The goal was simple: create a game using Kivy, without external non-pure python dependencies. All the entries have been submitted on our sponsor Github (and thanks to NotionInk), under a compatible OSS licence. The contest registered 21 entries, and 11 submissions was valid.

Our winners are:

Deflectouch, from Cyril Stoller
FishLife, from Zogg
memoryKivy, from Niavlys
Centripetal, from Dilon Cower
Flingy, from Andy Wilson

In term of numbers, everything is growing:

19919 unique visitors for the website (vs 9772 in December)
+56 subscribers on the mailing list (192 in total)
229 messages on kivy-users (vs 94 in December)
23 pull requests (vs 6 in December)

That was a great event so far, with a perfect timing: Kivy have now 1 year old! (first release was 1st February 2011).

Introducing "Python for Android"

Sun, 08 Jan 2012 16:20:06 +0000

I’m glad to share a new project called Python for Android. The goal of this project is to package your python application into an APK.

https://github.com/kivy/python-for-android

The project is under the umbrella of Kivy organization, but is not designed to be limited to Kivy only. Read the documentation to correctly install the NDK/SDK Android, and set the needed environment variables.

The packaging is done in 4 steps:

Ensure you have Android SDK/NDK downloaded and correctly installed
Ensure you have some environment set
Create a Python distribution containing the selected modules
Use that distribution to build an APK of your Python application

Creating the python distribution is as simple as that:

# create a simple distribution with python + PIL + Kivy
./distribute.sh -m "pil kivy"

# create a distribution with python + openssl + pil + kivy
./distribute.sh -m "openssl pil kivy"

A directory dist/default will be created, including the result of the whole arm compilation. Available libraries as for today: jpeg pil png sdl sqlite3 pygame kivy android libxml2 libxslt lxml ffmpeg openssl.

The second step is a little bit harder, since you need to provide more information for Android:

cd dist/default
./build.py --package org.test.touchtracer --name touchtracer \
--version 1.0 --dir ~/code/kivy/examples/demo/touchtracer debug installd

# --package: java name of your application
# --name: title of your application
# --version: version of your application
# --dir: location of your application containing the main.py

Then you’ll get a nicely bin/touchtracer-1.0-debug.apk

Pro:

A blacklist.txt file that can be used to exclude files in the final APK
Reusable distribution for other applications
Modular recipes architecture
Be able to build independents python distributions

Cons:

You need a main.py file that will be used for starting your application
Only one java bootstrap available, using OpenGL ES 2.0.
Only Kivy toolkit is working. I’m sure that other people can enhance it to add other toolkit recipes. But for example, pygame is not gonna to work because the android project is OpenGL ES 2.0: pygame drawing will not work.

I hope you’ll like it :)

We would like to thank Renpy / PGS4A for its initial pygame for android project

Kivy programming contest #1

Mon, 26 Dec 2011 10:29:55 +0000

Kivy, an open source project for building cross platform applications and user interfaces, is announcing its 1st programming contest. The contest will start with the publication of official rules on January 1st. Registrations will be accepted through January 25th, and submissions will be accepted through January 31st. Prizes include Nvidia Tegra 2 android tablets from NotionInk, One-year “bronze” plans and t-shirts.

The goal of the contest is to encourage interested developers to learn more about Kivy, which is a cross platform Application and User Interface software library for the python programming language. Kivy applications run on Linux, Windows, OSX, Android (and iOS). The framework supports most inputs protocols and devices like WM_Touch, WM_Pen, Mac OS X Trackpad and Magic Mouse, single/multi touch screens, Mtdev, Linux Kernel HID, TUIO, and uses pure hardware accelerated rendering to enable developers to really take user interfaces to a new level.

The contest is sponsored by NotionInk, an indian company that designs tablets PCs, and Github, a web-based hosting service for software developments projects.

Join the contest at : http://kivy.org/#contest

Kivy 1.0.8 released ! What's next ?

Wed, 26 Oct 2011 23:09:07 +0000

101 days after 1.0.7, Kivy 1.0.8 have been released (changelog).

This release include:

New widgets: Virtual Keyboard, Accordion, ProgressBar
Video support on android - Python bindings for ffmpeg have been done and are available on github. Theses are focused to be usable on android within Kivy project, but any help to clean and make the code more crossplatform is welcome !
Scroll wheel support in scrollview
- 1.6x more performance on Kivy langage
Support for GIF image + set of images in Zip files
Bezier and stipple lines in graphics instructions
… and lot of improvements and fixes !

A documentation translation project have been started (thanks for Sphinx for generating gettext translations automatically from python autodoc). Help us at: http://translation.kivy.org/.

And now, where to go ?

The roadmap is not written, since what we do is mostly depending of our interests / jobs / contributions. But community is growing fast ! I see $N multistroke recognizer, GSOC integration (better macosx integration), iOS port, better android packaging, unit test, better desktop integration, widget theming support, 3D graphics instructions…

If you still hesitating about using Kivy… just look at the frontpage for the current awesome projects made by the community. And they are lot of others projects going to be released during the next month ! We are also glad to see some big companies to use Kivy as the base for building their products. If you have also cool project going on, don’t hesitate to share :)

To finish, here is a video from Cyril that playing on Icarus Touch, an instrument that use Kivy for his UI. Awesome work !

▶ Watch the video on YouTube

Fixing perl errors when using gitolite + ssh + custom locales

Tue, 18 Oct 2011 09:44:09 +0000

If you have locale issues on a server, and your server is english only, this kind of issues can happen:

$ git push origin master
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_MESSAGES = "fr_FR.UTF-8",
	LC_COLLATE = "fr_FR.UTF-8",
	LC_CTYPE = "fr_FR.UTF-8",
	LANG = "en_US.utf8"
 are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Everything up-to-date

You might want to check your /etc/ssh/sshd_config and look at the AcceptEnv token. On this ssh server, it was:

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

My locale environment was passed to the server, but the server doesn’t know about fr. Since i don’t want to install fr locales, just comment that line:

# Allow client to pass locale environment variables
# AcceptEnv LANG LC_*

Done.

Texture compression, why does it matter ?

Sun, 04 Sep 2011 10:44:22 +0000

We all care about cash, time, life, love, and if you’re doing computer graphics, you might care about the memory consumption of your graphics card. Why ? For the same reason when you running out of cash :)

I’ll explain why does it matter to compress texture, and compare available possibilities. My personnal goal is to be able to load a lot of FULL HD pictures on tablet, for a museum project. The analysis is focused on DXT1 compression and size. I’m looking forward to ETC1 and will update that blog post with the result in the future.

What are we dealing with

If you are doing an application that display lot of hd pictures, that’s matter. We’ll start from this simple math statement: a full HD picture is 1980x1020 with 4 channels (RGBA). Whatever if your pictures is in PNG, or JPEG, your graphics card is not able to understand it, and will store it in its memory decompressed. So this image will eat:

1920 x 1080 x 4 = 8294400 bytes = 7.91MB 1920 x 1080 x 4 + mipmaps = 10769252 bytes = 10.27MB

In theory. Because it might be more if your graphics card doesn’t support NPOT texture. If not, usually, the texture will be resized to the closest POT texture available, mean for us: 2048 x 2048. Then the size for POT will be:

2048 x 2048 x 4 = 16777216 bytes = 16MB 2048 x 2048 x 4 + mipmaps = 22369620 bytes = 21MB

Compressions types

They are plenty types of compression availables. The most common are S3TC (including DXT1, DXT3, DXT5) from Savage3 originally, LATC from Nvidia, PVRTC from PowerVR, ETC1 from Ericsson…

Not all of them are available everywhere, and it’s depending a lot from your hardware. Here is a list of tablet / vendor / texture compression available. (only tablet, not desktop computer.) Thanks to this stackoverflow thread about supported OpenGL ES 2.0 extensions on Android devices

Tablette	Vendor	DXT1	S3TC	LATC	PVRTC	ETC1	3DC	ATC
(desktop computer) GeForce GTX 560	NVIDIA	X	X	X
Nexus One	Qualcom					X	X	X
Toshiba Folio	NVIDIA	X	X	X		X
LGE Tablet	NVIDIA	X	X	X		X
Galaxy Tab	PowerVR				X	X
Acer Stream	Qualcomm					X	X	X
Desire Z	Qualcomm					X	X	X
Spica	Samsumg	X	X
HTC Desire	Qualcomm					X	X	X
VegaTab	NVIDIA	X	X	X		X
Nexus S	PowerVR				X	X
HTC Desire HD	Qualcomm					X	X	X
HTC Legend	Qualcomm					X	X	X
Samsung Corby	Qualcomm					X	X	X
Droid 2	PowerVR				X	X
Galaxy S	PowerVR				X	X
Milestone	PowerVR				X	X

We are seeing that ETC1 is standard compression for OpenGL ES 2, unfortunately, it will not work on desktop. PVRTC is specific to PowerVR device: it’s a standard on Ipad/Iphone.

Using DXT1

If you use DXT1, you need a POT image. DXT1 doesn’t work on NPOT.

To convert any image to DXT1 without any tool, you must know that your graphics card is capable of doing it, using specials opengl functions. But i wanted to precalculate them. Nvidia texture tools contains tools for converting them, but you need an Nvidia card. For all others, you might want to look at Libsquish. It’s designed to compress in software a raw image to DXTn. The result will be not a DXT1 “file”, because DXT1 is the compression format. The result will be stored in a DDS file, that we’ll see later about it.

If you want to be able to use libssquish in Python, you might want to apply my patch available on their issue tracker

For DXT1, the size of the file is not depending of the image content:

DXT1 2048x2048 RGBA = 2097152 bytes = 2MB

That’s already a big improvement. DXTn is able to store mipmaps of the texture too. For this size, the calculation is:

DXT1 2048x2048 RGBA + mipmap = 2795520 bytes = 2.73MB

Comparaison table

Type	Resolution	File size	GPU size	Images in a 256MB GPU	Images in a 512MB GPU
Raw RGBA image (POT)	2048 x 2048	-	16384KB	16	32
PNG image (NPOT)	1920 x 1080	4373KB	8040KB	32	65
PNG Image in reduced POT resolution	1024 x 1024	1268KB	4096KB	64	128
DXT1 without mipmap	2048 x 2048	2048KB	2048KB	128	256
DXT1 without mipmap, reduced	1024 x 1024	512KB	512KB	512	1024
DXT1 with mipmap	2048 x 2048	2730KB	2730KB	96	192
DXT1 with mipmap, reduced	1024 x 1024	682KB	682KB	384	768

As soon as we use compression, what we see is:

The file size is the same as GPU size
Even with POT texture compared to NPOT dxt1, we can still store 4x more images in GPU

And with Kivy ?

DXT1 itself is the compression format, but you cannot actually use it like that. You need to store the result is a formatted file. DDS.

Kivy is already able to read DDS files. But you must ensure that your graphics card is supporting DXT1 or S3TC. Look at gl_has_capability() function in Kivy then.

Kivy at EuroPython - Lightning explanation

Thu, 23 Jun 2011 13:46:27 +0000

For all the guys that was interested about Kivy lightning talk at EuroPython 2011, due to some incomprehension about what is Kivy, here is a lightning explanation to make it clear.

Kivy is a Python framework designed for creating of Natural Users Interfaces. The framework containing abstraction for loading image, video, audio. It have a complete new approach about input events, and widgets. For example, you can use lot of widgets at the _same_ time, something not really possible in classical framework (qt, gtk…): try to touch on a button while selecting something in a list. This is not only about multitouch for one user, but also for multi users. Kivy graphics engine is in OpenGL ES2, and all the widgets are using it.

If you write an application in top of Kivy, you can deploy it on Linux, MacOSX, Windows and Android. Without changing anything in your code. Because it’s in Python.

The presentation tool i’ve used is PreseMT. It have been made by Christopher and Me. And it’s an application built using Kivy. A version of this tool is already published on Android Market. PreseMT have been written in one week, and use lot of Kivy features. But it’s still not finished, and we are missing lot of features, like the ability to export the presentation in a “good” format. We have plan to make an export in HTML5, that will support animation too.

Feel free to follow me on @mathieuvirbel !

Recording OpenGL output to H264 video

Fri, 17 Jun 2011 10:48:48 +0000

Apitrace is a tool for recording all the gl commands in a trace file. The trace file can be replay in later time, and they got a nice gui for checking all the gl call every frame, with introspection. They have a glretrace software that replay a trace file. We can use it to get the output of everyframe and push it in a gstreamer pipeline to make a video.

Why not using gtkRecordMyDesktop or other screen capture ? Sometime, the overhead of capturing and encoding video on live take too much CPU. And the application start to slow down. I didn’t see any slowdown using apitrace, and the trace file is very small compared to video output or raw video output.

So first, compile apitrace with stdout support:

$ git clone git://github.com/tito/apitrace.git
$ cd apitrace
$ git checkout snapshot-stdout
$ mkdir build
$ cd build
$ cmake ..
$ make

Take any opengl application, and make a trace file. The trace file will have the name of the binary. In my case, python is an alias to python2.7: the trace file will be python2.7.trace.

$ LD_PRELOAD=./glxtrace.so python ~/code/kivy/examples/demo/pictures/main.py
# replay for fun now
$ ./glretrace python2.7.trace

To be able to make a video from the trace file, you need to know the size of the window, and the initial framerate. Here, my example is running at 800x600, 60fps:

$ ./glretrace -sr python2.7-trace | \
 gst-launch fdsrc ! \
 videoparse width=800 height=600 format=rgbx framerate=60 ! \
 videoflip method=5 ! videorate ! ffmpegcolorspace ! \
 video/x-raw-yuv,width=800,height=600,framerate=\(fraction\)30/1 \
 x264enc pass=quant ! avimux ! filesink location=output.avi

The final video will be saved in output.avi. You can check the video output here :

▶ Watch the video on YouTube

Designing Configuration and Settings UI for Kivy

Thu, 09 Jun 2011 12:19:29 +0000

From 3 weeks now, i’m working on packaging kivy application, to create a installer/bundle/deb of a Kivy application. The reason is simple: as soon as you are doing an application, the user should not care about installing Kivy itself. In the same time, i’ve work on other projects that require to have their own configuration. From a long time, we always wanted to have some in-app settings for configure Kivy. Even Android have a “settings” button, we wanted to use it. :)

This is now possible.

Yes, it look like the honeycomb settings panel. Kind of. Well.

The configuration is automatically handled by the App class, and you can put your own token on it. The settings UI (that you’re seeing on the screen) is created from a JSON definitions. You can press F1 or the settings key on android to bring the settings panel, hook the on_config_change to know when a configuration token is changed from the settings ui, etc.

This is available in master, and will be published on next 1.0.7 version. If you are interested, please read and give feedback about the App documentation and the Settings documentation.

NPOT textures support in OpenGL

Sun, 05 Jun 2011 21:20:21 +0000

If you already done OpenGL development, you should be aware of POT (Power of two) texture. Because of very old conventions, the texture size must be a power of two size. Not necessarily the same for width and height though : 256x256 is valid as 128x512.

The usual thing to do when you want to load an NPOT texture (like 23x61) is to:

take his closed POT size: 32x64
depending of the book you’re reading: blit/strech the 23x61 to the 32x64 texture
OR blit without stretch, and adjust texture coordinate (this is what kivy does right now.)

The downside part of this approach is that you’re lost a part of memory. Bad.

While ago, i remember to found the Rectangle texture support from NVidia. Aaah, finally, is it what we was waiting from a long time ? Erm, no. Their implementation have lot of downsides:

The usage of a specific texture target: GL_TEXTURE_RECTANGLE_NV
No mipmap support
The texture coordinates are not normalized from 0-1… but from 0-width/height of the image
Some wrap mode are not supported (GL_REPEAT for eg.)

But today… i discover that most graphics card are supporting rectangle texture. If the extension GL_ARB_texture_non_power_of_two (OES_texture_npot for OpenGL ES platform), you can finally ensure that loading NPOT texture will… just work as expected :

You can still use GL_TEXTURE_2D
Mipmapping are supported
Texture coordinates are from 0-1
All wrap mode are supported

A little note here, in OpenGL ES 2, they have a native support for NPOT texture, but with somes limitations related to mipmapping.

If you want to just load NPOT texture safely without using rectangle texture, just check the availability of theses extensions :

extensions = glGetString(GL_EXTENSIONS).split()
npot_support = ('OES_texture_npot' in extensions or \
 'GL_ARB_texture_non_power_of_two' in extensions)

Using microphone peak as input

Tue, 24 May 2011 12:01:21 +0000

I’m currently on a project that involve disabled peoples, audio and kinect. Theses boys and girls are doing lot of loud sounds, so the idea is to use their sound as a trigger. We can use gstreamer to make that work quite easily, cause it have everything we need: a audio source, and level calculator.

import pygst
pygst.require('0.10')
import gst, gobject
gobject.threads_init()

pipeline = gst.parse_launch(
 'pulsesrc ! audioconvert ! '
 'audio/x-raw-int,channels=2,rate=44100,endianness=1234,'
 'width=32,depth=32,signed=(bool)True !'
 'level name=level interval=10000000 !'
 'fakesink')

level = pipeline.get_by_name('level')
bus = pipeline.get_bus()
bus.add_signal_watch()

def show_peak(bus, message):
 # filter only on level messages
 if message.src is not level:
 return
 if not message.structure.has_key('peak'):
 return
 # read peak
 print 'peak', message.structure['peak'][0]

# connect the callback
bus.connect('message', show_peak)

# start the pipeline
pipeline.set_state(gst.STATE_PLAYING)

ctx = gobject.gobject.main_context_default()
while ctx:
 ctx.iteration()

The output could be something like this :

peak -35.2370719856
peak -35.0252114393
peak -10.8591229158
peak -4.6007387433
peak -4.85102463679
peak -6.45292575653
peak -6.83102903668
peak -7.39486319074
peak -13.9852340247
peak -17.423901433
peak -35.0852178272
peak -35.8725208237

Next, we can use that information to record their sound, and use it on some scenario. So, instead of use the fakesink, we can use appsink. This module allow you to read the buffer pushed by the previous module. So we can put theses buffers into a list, and use them when needed :)

The state machine will handle the 3 phases :

Wait for a peak > -30db
Recording the sound, stop when the peak is < -32db
Replay the last sound

Note: The -30 / -32 are taken from my tests. If you have more noise, you need to adjust theses triggers.

And here is the final example:

import pygst
pygst.require('0.10')
import gst, gobject
gobject.threads_init()

pipeline_play = None
pipeline = gst.parse_launch(
 'pulsesrc ! audioconvert ! '
 'audio/x-raw-int,channels=2,rate=44100,endianness=1234,'
 'width=32,depth=32,signed=(bool)True !'
 'level name=level interval=10000000 !'
 'appsink name=app emit-signals=True')

state = 'wait'
peak = -99
buffers = []
level = pipeline.get_by_name('level')
app = pipeline.get_by_name('app')
bus = pipeline.get_bus()
bus.add_signal_watch()

def show_peak(bus, message):
 global peak
 # filter only on level messages
 if message.src is not level:
 return
 if not message.structure.has_key('peak'):
 return
 # read peak
 peak = message.structure['peak'][0]

def enqueue_audio_buffer(app):
 buffers.append(str(app.emit('pull-buffer')))

def play_sample(sample):
 global pipeline_play
 with open('audio.dat', 'wb') as fd:
 fd.write(sample)
 if pipeline_play is None:
 pipeline_play = gst.parse_launch(
 'filesrc location=audio.dat !'
 'audio/x-raw-int,channels=2,rate=44100,endianness=1234,'
 'width=32,depth=32,signed=(bool)True !'
 'audioamplify amplification=2 ! autoaudiosink')
 pipeline_play.set_state(gst.STATE_NULL)
 pipeline_play.set_state(gst.STATE_PLAYING)

# connect the callback
bus.connect('message', show_peak)
app.connect('new-buffer', enqueue_audio_buffer)

# start the pipeline
pipeline.set_state(gst.STATE_PLAYING)

# main loop
ctx = gobject.gobject.main_context_default()
while ctx:
 ctx.iteration()
 print state, peak

 # wait for somebody to make a sound
 if state == 'wait':
 if peak > -30:
 state = 'record'
 continue
 # discard any buffer
 buffers = []

 # record the current audio, until the peak is going down
 elif state == 'record':
 if peak < -32:
 state = 'replay'
 continue

 # replay last sound
 elif state == 'replay':
 play_sample(''.join(buffers))
 state = 'wait'

Usage: Just make a sound… and it will replay just after.

Kivy on Android, part 2

Tue, 12 Apr 2011 20:56:09 +0000

Hi guys,

Look like people are following my blog and waiting for android version of Kivy. We have a launcher that you can already use. Check the :

Maybe during the next release, or a little bit after, i’ll release a software to create an Android package of a Kivy application. The code is already on launchpad, but it’s still a work in progress. As soon as i have finished, i’ll publish it on kivy-dev mailing list. If you didn’t subscribe yet, do it now ! :)

More to come by the end of that week… !