I got hold of a Kindle Fire via a re-seller on eBay UK, so we could make sure that Mixtikl worked fine on there.
This being Android, Mixtikl worked first time! I installed Mixtikl using "adb install", having first configured adb to recognise the Kindle Fire.
I changed just a couple of things to make it "Kindle Fire friendly". Those being:
- I had to increase the audio latency a bit to prevent audio break-up; that took just a few minutes...
- I had to account for Kindle Fire's taking-up the bottom 20 pixels to display the soft menu bar, which was also very easy...
I must say that Mixtikl works *really* well on the Kindle Fire. The Mixtikl icon doesn't display at the moment; but I have read that is because the icon is downloaded only when you purchase the app from the Amazon Kindle App Store.
I should say that I'm really very impressed with the Kindle Fire! It is a really nice size, fabulous for reading books (thanks to the touch screen, which isn't on the older Kindle of course). It seems very fast, with a lovely clear display and it is very responsive. Will I like it more than my iPad 1? Well, I prefer the size of the Kindle Fire, and like the stereo speakers; and I like that it was so easy to get our app running on it. :) The iPad has the advantage of having lots of good quality games for it... but it is *very* much more expensive ... if the Kindle Fire had more storage space, it'd make quite a nice MP3 player.
It's a pain that the Amazon App store is US-only at the moment, but I'm sure they'll roll that out worldwide soon enough. I think they're going to sell huge numbers of Kindle Fires!
I also want to mention that so far, the Amazon developer support team have been *incredibly* responsive to the questions I've had; I'm genuinely impressed!
Pete
Saturday, 24 December 2011
Monday, 19 December 2011
Mixtikl on Android - reprise
The port of Mixtikl to Android is now complete.
I thought I'd share how I ended-up porting Mixtikl to Android, as this has been a long road!
When I first started looking at porting Mixtikl to Android, there were a couple of blocking issues;
- no support for C++ template library (though eventually STLPort arrived!)
- the original Mixtikl made heavy use of Modal code for dialog handling
Both of those two items made a port with Android a non-starter.
However, when Airplay (now renamed to Marmalade) added Android support, we were able to attempt a port of Mixtikl; as Airplay contained both C++ template library support, and supported modal code.
Ultimately however, the audio had too many problems (very, very high latency, mono only etc.) and this made the Airplay-based port too low quality to allow us to release.
Time moves on however, and a while back, Google added full support for the C++ standard template library to the Android NDK. And, critically, they added support for both OpenSL (to Android 2.3), and 32-bit ARM code generation (via armeabi-v7a). Given those changes, we spend a month or so removing all the modal code from Mixtikl (not an easy undertaking: you have to bear in mind that Mixtikl is > 500K lines of C++ code!).
So, given those changes in both the Android NDK and Mixtikl's own internal architecture, we were able to start and complete a full native port of Mixtikl to Android.
The bulk of Mixtikl remains in C++; all the graphics are drawn via the same C++-based graphics engine we use on all platforms. The interaction with Java is just a thin JNI layer that looks after drawing primitives and some native dialog interfacing. The OpenSL support is just a thin adaptor layer between our audio engine and OpenSL.
Mixtikl performs brilliantly on Android in my opinion; though as noted elsewhere the Android OpenSL implementation is missing a big trick in regards latency management; and we have to configure Mixtikl to run with slightly higher latency that we would like in order to avoid any potential audio break-up.
I should note that debugging C++ code on Android remains a pain, you have to rely on trace statements, though it is at least possible now to use gdb and the bt command to trap crashes when they happen and see where the code died in the call stack. The good news is that Mixtikl is very solid as 99% of the code is cross-platform and very well tested, so I've hardly had to touch gdb!
I thought I'd share how I ended-up porting Mixtikl to Android, as this has been a long road!
When I first started looking at porting Mixtikl to Android, there were a couple of blocking issues;
- no support for C++ template library (though eventually STLPort arrived!)
- the original Mixtikl made heavy use of Modal code for dialog handling
Both of those two items made a port with Android a non-starter.
However, when Airplay (now renamed to Marmalade) added Android support, we were able to attempt a port of Mixtikl; as Airplay contained both C++ template library support, and supported modal code.
Ultimately however, the audio had too many problems (very, very high latency, mono only etc.) and this made the Airplay-based port too low quality to allow us to release.
Time moves on however, and a while back, Google added full support for the C++ standard template library to the Android NDK. And, critically, they added support for both OpenSL (to Android 2.3), and 32-bit ARM code generation (via armeabi-v7a). Given those changes, we spend a month or so removing all the modal code from Mixtikl (not an easy undertaking: you have to bear in mind that Mixtikl is > 500K lines of C++ code!).
So, given those changes in both the Android NDK and Mixtikl's own internal architecture, we were able to start and complete a full native port of Mixtikl to Android.
The bulk of Mixtikl remains in C++; all the graphics are drawn via the same C++-based graphics engine we use on all platforms. The interaction with Java is just a thin JNI layer that looks after drawing primitives and some native dialog interfacing. The OpenSL support is just a thin adaptor layer between our audio engine and OpenSL.
Mixtikl performs brilliantly on Android in my opinion; though as noted elsewhere the Android OpenSL implementation is missing a big trick in regards latency management; and we have to configure Mixtikl to run with slightly higher latency that we would like in order to avoid any potential audio break-up.
I should note that debugging C++ code on Android remains a pain, you have to rely on trace statements, though it is at least possible now to use gdb and the bt command to trap crashes when they happen and see where the code died in the call stack. The good news is that Mixtikl is very solid as 99% of the code is cross-platform and very well tested, so I've hardly had to touch gdb!
Tuesday, 6 December 2011
Android - how to reduce the audio latency
I been approached a few times recently about how to write low-latency audio latency apps in Android, and I point such queries to OpenSL and say "look how much better it is that it used to be" (provided you use C++ of course)...
But the bottom line is that the Android audio APIs - and that includes OpenSL - still miss the basic trick, which is to be able to offer a programmatic contract that is as simple as this:
App : what sets of audio rates and formats do you support?
(API responds, maybe only 22Khz stereo, for sake of simplicity here)
App asks: how many audio blocks do I need to keep prepared and primed in the queue to guarantee no break-up, at this rate/format, with minimal latency?
(API responds, maybe 2 blocks of 512 sample frames each).
App prepares the first 2 blocks, submits, and gets "block delivery" callbacks; it runs a separate thread to keep internal queues topped-up ready to deliver to the callbacks.
With that sort of contract, every app can run at minimal latency dictated by the underlying audio device (through the driver layer), on any device. Without that sort of contract, every app developer is kept guessing, and has to assume a worst case that works on all devices available for testing. :)
I implemented this scheme nearly a decade ago in the intent Sound System (I had the luxury of designing and implementing a complete audio architecture for mobile devices from scratch!). It is a piece of cake to do, and IMO audio developers for Android are screaming for it. Intent was focused on ultra-low audio latency for games / multimedia and musical instruments...
I should note, Apple have also missed the same trick - they've been able to get away with it however, as the offer a fixed range of hardware....
But the bottom line is that the Android audio APIs - and that includes OpenSL - still miss the basic trick, which is to be able to offer a programmatic contract that is as simple as this:
App : what sets of audio rates and formats do you support?
(API responds, maybe only 22Khz stereo, for sake of simplicity here)
App asks: how many audio blocks do I need to keep prepared and primed in the queue to guarantee no break-up, at this rate/format, with minimal latency?
(API responds, maybe 2 blocks of 512 sample frames each).
App prepares the first 2 blocks, submits, and gets "block delivery" callbacks; it runs a separate thread to keep internal queues topped-up ready to deliver to the callbacks.
With that sort of contract, every app can run at minimal latency dictated by the underlying audio device (through the driver layer), on any device. Without that sort of contract, every app developer is kept guessing, and has to assume a worst case that works on all devices available for testing. :)
I implemented this scheme nearly a decade ago in the intent Sound System (I had the luxury of designing and implementing a complete audio architecture for mobile devices from scratch!). It is a piece of cake to do, and IMO audio developers for Android are screaming for it. Intent was focused on ultra-low audio latency for games / multimedia and musical instruments...
I should note, Apple have also missed the same trick - they've been able to get away with it however, as the offer a fixed range of hardware....
Sunday, 4 December 2011
Android - high performance audio - how to do it
I've just been through a very interesting period of work, sorting-out a high-performance audio interface for the Android port of Mixtikl. I've learned quite a few things - here are the highlights.
Firstly, target Android 2.3 or later. This allows you to use OpenSL ES, which is the only realistic approach for low-latency audio on Android. The audio allows you to delivery audio with pretty low latency, and totally avoids any problems of garbage collection blocking you. This of course assumes that all your audio code is written in C++!
As you're using OpenSL ES, and assuming you have some very heavy audio DSP going-on (like in Mixtikl!), you'll need to use a separate Posix thread to keep your audio callbacks pumped-up. Basically, if your OpenSL ES audio block callbacks take much time at all, then your audio will break-up. So, use a worker thread to keep sufficient audio pre-processed and ready to be picked-up and delivered via your callbacks!
Finally, and believe me this is important (!): make sure you target armeabi-v7a, and that you use the right compiler flags to generate 32-bit ARM7 code. If you on the other hand use the default settings, you'll generate Thumb code for armeabi - and your code will run staggeringly slower (!!), and audio break-up is inevitable if you're doing anything serious. So: don't bother targeting armeabi devices... and Thumb code is a no-no.
Follow my advice, and you can create sophisticated audio software. Don't follow it, and you're going to find things tough! ;)
I should note that support for armeabi-v7a *emulator* targets arrived in the Android 4.0 SDK... which makes things a lot easier as well... something worth knowing!
Finally... here is a useful resource on the background to OpenSL ES on Android...:
http://mobilepearls.com/labs/native-android-api/opensles/index.html
Firstly, target Android 2.3 or later. This allows you to use OpenSL ES, which is the only realistic approach for low-latency audio on Android. The audio allows you to delivery audio with pretty low latency, and totally avoids any problems of garbage collection blocking you. This of course assumes that all your audio code is written in C++!
As you're using OpenSL ES, and assuming you have some very heavy audio DSP going-on (like in Mixtikl!), you'll need to use a separate Posix thread to keep your audio callbacks pumped-up. Basically, if your OpenSL ES audio block callbacks take much time at all, then your audio will break-up. So, use a worker thread to keep sufficient audio pre-processed and ready to be picked-up and delivered via your callbacks!
Finally, and believe me this is important (!): make sure you target armeabi-v7a, and that you use the right compiler flags to generate 32-bit ARM7 code. If you on the other hand use the default settings, you'll generate Thumb code for armeabi - and your code will run staggeringly slower (!!), and audio break-up is inevitable if you're doing anything serious. So: don't bother targeting armeabi devices... and Thumb code is a no-no.
Follow my advice, and you can create sophisticated audio software. Don't follow it, and you're going to find things tough! ;)
I should note that support for armeabi-v7a *emulator* targets arrived in the Android 4.0 SDK... which makes things a lot easier as well... something worth knowing!
Finally... here is a useful resource on the background to OpenSL ES on Android...:
http://mobilepearls.com/labs/native-android-api/opensles/index.html
Friday, 21 October 2011
gdb with the Android NDK - now easy with ddd wrapping ndk-gdb!
You might already know this, but android debugging from gdb is now really, really easy - if you can stomach gdb!
See Android/ndk6/docs/NDK-GDB.html in your NDK documentation.
Basically, the app's libs/armeabi folder must have a gdbserver in it... which is put there automatically by the ndk-build command (remember not to ship your app with gdbserver in it!):
So...: build your app, install with e.g.:
adb install -r myfile.apk
Start your app directly from the Android UI, and in a terminal, change to your product's jni folder, and you're ready to debug!
Using command-line gdb
If you prefer command-line gdb, type this:
ndk-gdb com.example.myproject
You'll find that ndk-gdb will break as your app attaches. Set a breakpoint if you want (optional!), e.g.:
b Java_com_example_hellojni_HelloJni_stringFromJNI
Enter "cont" to continue...
And the debugger will now hit your breakpoints - use normal gdb commands to control it!
Using the graphical ddd debugger
If, like me, you prefer to use a graphical debugger, it is really easy to use ddd as a wrapper around gdb!
Many thanks to a blog reader for this suggestion!
It is really easy to use ddd instead... here are the steps!
Preparation:
1. Install ddd ...!
2. Modify your ndk-gdb shells script, to simply comment-out the "exit 1" call when an unexpected argument is passed
Running ddd
Start your app from Eclipse
In a terminal window, change to your project's jni folder (if you're not there already!)
Run ddd like this - and that is all there is to it!
ddd --debugger ndk-gdb
Eclipse
Using gdb for JNI/NDK from Eclipse is way too painful - I wouldn't bother. :)
Run-time breakpoints
On a final note, you can use this code to force a breakpoint at runtime...
#define BREAKPOINT __asm__ ("bkpt 0")
printf ("Hitting breakpoint!");
BREAKPOINT;
printf ("Gone past breakpoint!");
See Android/ndk6/docs/NDK-GDB.html in your NDK documentation.
Basically, the app's libs/armeabi folder must have a gdbserver in it... which is put there automatically by the ndk-build command (remember not to ship your app with gdbserver in it!):
So...: build your app, install with e.g.:
adb install -r myfile.apk
Start your app directly from the Android UI, and in a terminal, change to your product's jni folder, and you're ready to debug!
Using command-line gdb
If you prefer command-line gdb, type this:
ndk-gdb com.example.myproject
You'll find that ndk-gdb will break as your app attaches. Set a breakpoint if you want (optional!), e.g.:
b Java_com_example_hellojni_HelloJni_stringFromJNI
Enter "cont" to continue...
And the debugger will now hit your breakpoints - use normal gdb commands to control it!
Using the graphical ddd debugger
If, like me, you prefer to use a graphical debugger, it is really easy to use ddd as a wrapper around gdb!
Many thanks to a blog reader for this suggestion!
It is really easy to use ddd instead... here are the steps!
Preparation:
1. Install ddd ...!
2. Modify your ndk-gdb shells script, to simply comment-out the "exit 1" call when an unexpected argument is passed
Running ddd
Start your app from Eclipse
In a terminal window, change to your project's jni folder (if you're not there already!)
Run ddd like this - and that is all there is to it!
ddd --debugger ndk-gdb
Eclipse
Using gdb for JNI/NDK from Eclipse is way too painful - I wouldn't bother. :)
Run-time breakpoints
On a final note, you can use this code to force a breakpoint at runtime...
#define BREAKPOINT __asm__ ("bkpt 0")
printf ("Hitting breakpoint!");
BREAKPOINT;
printf ("Gone past breakpoint!");
Friday, 23 September 2011
Mixtikl 5... iOS/Mac/Windows
Work on Mixtikl 5 is keeping me very busy... we've decided that Mixtikl 5 will be the first version we release for Android. However, we'll focus first on finishing-off the iOS / Mac /Windows versions...
Friday, 2 September 2011
Android - using findbugs from outside of Eclipse
I've been using findbugs to help track-down issues in Java code. A great tool, and very easy to use. However, the Eclipse-based plug-in keeps crashing for me with a NullPointerException on certain projects.
The work-around to this is to run the findbugs UI directly.
To use findbugs on Windows, get the latest findbugs distribution from http://findbugs.sourceforge.net/downloads.html, and then create a shortcut along the following lines (note the -X argument, as the JVM might require quite a bit of memory to avoid failing for big projects!):
On Mac, simply run something like this directly:
It is really use to use - just select File -> New Project, add your class folders to the top pane, your source folders to the bottom pane, and let it run!
The work-around to this is to run the findbugs UI directly.
To use findbugs on Windows, get the latest findbugs distribution from http://findbugs.sourceforge.net/downloads.html, and then create a shortcut along the following lines (note the -X argument, as the JVM might require quite a bit of memory to avoid failing for big projects!):
\java.exe -Xmx1024m -jar C:\findbugs-1.3.9\lib\findbugs.jar -gui
On Mac, simply run something like this directly:
java -Xmx1024m -jar ~/findbugs-1.3.9/lib/findbugs.jar -gui &
It is really use to use - just select File -> New Project, add your class folders to the top pane, your source folders to the bottom pane, and let it run!
Tuesday, 30 August 2011
Upgrade to XCode 4.1 - problems with VALID_ARCHS for iOS Simulator builds
Now that I've taken the plunge and moved to XCode 4.x from XCode 3.x ... I had a really weird problem trying to build my iOS projects for the simulator.
The builds failed with an error related to the i386 architecture. I simply couldn't figure-out how to solve this directly with the XCode IDE.
Luckily, I did manage to figure-out a fix: my solution was to edit the project.pbxproj files by hand with vim, and remove a handul of fixed definitions of VALID_ARCHS ... which presumably were legacies from previous XCode versions. This is not the first time that I've been forced to fix XCode projects by hand!
The builds failed with an error related to the i386 architecture. I simply couldn't figure-out how to solve this directly with the XCode IDE.
Luckily, I did manage to figure-out a fix: my solution was to edit the project.pbxproj files by hand with vim, and remove a handul of fixed definitions of VALID_ARCHS ... which presumably were legacies from previous XCode versions. This is not the first time that I've been forced to fix XCode projects by hand!
Monday, 29 August 2011
Mac - how to build installs the easy way
I was having some issues getting my Mac installers (using packagemaker) working with the latest XCode.
Surfing-around, I found an amazing free tool called Packages. The link is here:
http://s.sudre.free.fr/Software/Packages/about.html
I've changed our own installers to use this, and all my Mac install problems have disappeared with an hours work. :)
Pete
Surfing-around, I found an amazing free tool called Packages. The link is here:
http://s.sudre.free.fr/Software/Packages/about.html
I've changed our own installers to use this, and all my Mac install problems have disappeared with an hours work. :)
Pete
Wednesday, 10 August 2011
Audio and MIDI Visualization in iOS and Android
I've spent the past month focused mainly on Intermorphic's Audio/MIDI visualizer framework, dusting-down the work I did (in the main) several years ago, and bringing it all up-to-date.
I've recently experimented with lots of stuff, 2D and 3D, on iOS (UIKit ,Quartz, Core Animation and OpenGL ES) and Android (Canvas and OpenGL ES), and have learned a lot about performance differences between these approaches. I've also learned that OpenGL ES is a truly horrible API in which to get things done. :)
Anyways - things are coming on nicely: the next update for Mixtikl should be another good one!
I've recently experimented with lots of stuff, 2D and 3D, on iOS (UIKit ,Quartz, Core Animation and OpenGL ES) and Android (Canvas and OpenGL ES), and have learned a lot about performance differences between these approaches. I've also learned that OpenGL ES is a truly horrible API in which to get things done. :)
Anyways - things are coming on nicely: the next update for Mixtikl should be another good one!
Wednesday, 3 August 2011
Android - turn key events into UTF8
As I've been asked a few times how to turn Android key events into UTF8 data, I figured it might be a good idea to share this info in my blog! It is actually really easy:
@Override
public boolean onKeyDown(int keyCode, KeyEvent event) {
int [] unicodePoints = new int[1];
unicodePoints[0] = event.getUnicodeChar();
String x = new String(unicodePoints, 0, 1);
byte utf8Bytes[];
try {
utf8Bytes = x.getBytes("UTF-8");
// Do something with your UTF-8 data here!
// ....
// ...
} catch (UnsupportedEncodingException e) {
Log.e(TAG, "Failed to key event to UTF8");
Throw(e);
}
return true;
}
@Override
public boolean onKeyDown(int keyCode, KeyEvent event) {
int [] unicodePoints = new int[1];
unicodePoints[0] = event.getUnicodeChar();
String x = new String(unicodePoints, 0, 1);
byte utf8Bytes[];
try {
utf8Bytes = x.getBytes("UTF-8");
// Do something with your UTF-8 data here!
// ....
// ...
} catch (UnsupportedEncodingException e) {
Log.e(TAG, "Failed to key event to UTF8");
Throw(e);
}
return true;
}
Tuesday, 26 July 2011
Mixtikl on Android - with the NDK and C++
Mixtikl is now around 95% complete on Android.
I've actually ported directly with the NDK/C++, without using Juce, as my 2d portability layer for Mixtikl is so highly developed. The port only took around a week (having already done a lot of background work in the previous few months laying the ground for this and possibly other ports!).
The Audio drivers have caused some fun; I've ended-up implementing support for AudioTrack and AudioRecord in native code with pthreads. Audio support in Android really is very poor...
The biggest headache is the basic problem of debugging C++ code on Android; I'm reduced to using trace statements. It helps that 99% of the code in Mixtikl is cross-platform and fully debugged on other platforms!
That said, the latest NDK (R6) is much improved with STL support and what have you; there is no longer any need to use Crystax.
I'm still very likely to use Juce for the Liptikl and Noatikl ports to Android, of course!
I've actually ported directly with the NDK/C++, without using Juce, as my 2d portability layer for Mixtikl is so highly developed. The port only took around a week (having already done a lot of background work in the previous few months laying the ground for this and possibly other ports!).
The Audio drivers have caused some fun; I've ended-up implementing support for AudioTrack and AudioRecord in native code with pthreads. Audio support in Android really is very poor...
The biggest headache is the basic problem of debugging C++ code on Android; I'm reduced to using trace statements. It helps that 99% of the code in Mixtikl is cross-platform and fully debugged on other platforms!
That said, the latest NDK (R6) is much improved with STL support and what have you; there is no longer any need to use Crystax.
I'm still very likely to use Juce for the Liptikl and Noatikl ports to Android, of course!
Thursday, 30 June 2011
Juce, C++ and Android
I've been experimenting with using the fabulous Juce to create programs for Android using C++.
You can find out about Juce here: http://www.rawmaterialsoftware.com/juce.php
If you haven't used Juce yet - give it a go! I've been using it for years now to create cross-platform programs for Mac and Windows...
You can find out about Juce here: http://www.rawmaterialsoftware.com/juce.php
If you haven't used Juce yet - give it a go! I've been using it for years now to create cross-platform programs for Mac and Windows...
Monday, 31 January 2011
SourceTree - git app for Mac
I've started using SourceTree for painlessly reviewing historical changes in my Mac's git repository.
Steve (the SourceTree developer) has just added a feature which easily allows you to visualise changes between arbitrary versions of any given file, which is far easier than trying to do with the git command-line tools!
Give it a try - its a really nice tool!
Steve (the SourceTree developer) has just added a feature which easily allows you to visualise changes between arbitrary versions of any given file, which is far easier than trying to do with the git command-line tools!
Give it a try - its a really nice tool!
Subscribe to:
Posts (Atom)
