Monday, 12 March 2012

Debugging JNI/C++ code under Android/Eclipse is now easy!

At long last: debugging JNI/C++ code under Android/Eclipse is now easy!

All I had to do was to get this amazing software for Eclipse - thank you ARM ! ...:

http://www.arm.com/products/tools/software-tools/ds-5/community-edition/ds-5-community-edition-debug.php

Because I'm using a Mac, I had to install under a Linux VM on my Mac...

The only obscure thing I found when following the install/configure instructions, was that I in order for the new DB-5 Debugger Debug Configuration to find my device, I first to do this:

sudo ln -s /usr/bin/adb

Brilliant!!

Pete

Wednesday, 29 February 2012

Android - handling rotation/orientation in an Activity - the easy way!

As I've been asked a few times about the easy way to handle rotation in an Android Activity, I thought I'd outline what you need to do here. It is really easy when you know how!

Firstly, you need to ensure that you create your layouts such that they resize easily. See this post for more details:
http://sseyod.blogspot.com/2010/09/art-of-scalable-android-layouts.html

Secondly, if you find that you need a different layout for your Activity (say in Landscape mode), create it in a parallel folder to your portait layout like this...:

res/layout/fred.xml
res/layout-land/fred.xml


In your mainfest xml activity... block, make sure you put this:

android:configChanges="orientation|keyboardHidden"


In your Activity's onCreate() method implementation, pretty much all you need to do is call this method:

  myConfigureUI();


Create the following override method in your Activity (this is called when the orientation changes as your Activity is rotated):

@Override
public void onConfigurationChanged(Configuration newConfig) {

  // Copy current item values from all your UI gadgets...!
  String myvalue = myitem.toString();
  // etc...

  // Call the method that re-orients and re-creates the gadget bindings
  myConfigureUI();

  // Restore the gadget values to the post-rotation UI:
  myitem.setText(myvalue);
}


Finally, you can create your new method, which makes your Activity use the correct layout to suit your orientation, and which looks-up all the gadgets, prepares their data and sets-up their event handlers (remember: this is called both at creation time, and whenever a rotation event occurs!):


private void myConfigureUI() {

  /*
  // Can check the orientation of the screen...
  if (getResources().getConfiguration().orientation == Configuration.ORIENTATION_LANDSCAPE) {
    setContentView(R.layout.main_landscape);
  } else {
    setContentView(R.layout.main);
  }
  */

  // This will automatically pick-up the landscape layout or portrait layout...
  setContentView(R.layout.main);

  // Look-up new items etc. ... mySpinner is an instance variable in this example...
  mySpinner = (Spinner) findViewById(R.id.spinnerx);

  mySpinner .setOnItemSelectedListener(new OnItemSelectedListener() { ...
  ...}
  // etc. ...
}

Monday, 23 January 2012

Noatikl 2

We've been working very hard on Noatikl 2, and it is going to be a huge improvement on Noatikl 1.x!

Noatikl 2 will include built-in support for the Partikl synth that has already been featured in Mixtikl (in addition to all of Noatikl's traditional MIDI I/O options).

When Noatikl 2 launches later this year, it will be available directly from the Mac App Store in standalone form (Windows versions and Mac Plug-in variants will be on the Intermorphic web store as usual). We're also planning to bring it to both iOS and Android at some point this year. :)

Pete

Saturday, 24 December 2011

Mixtikl on Kindle Fire!

I got hold of a Kindle Fire via a re-seller on eBay UK, so we could make sure that Mixtikl worked fine on there.

This being Android, Mixtikl worked first time! I installed Mixtikl using "adb install", having first configured adb to recognise the Kindle Fire.

I changed just a couple of things to make it "Kindle Fire friendly". Those being:
- I had to increase the audio latency a bit to prevent audio break-up; that took just a few minutes...
- I had to account for Kindle Fire's taking-up the bottom 20 pixels to display the soft menu bar, which was also very easy...

I must say that Mixtikl works *really* well on the Kindle Fire. The Mixtikl icon doesn't display at the moment; but I have read that is because the icon is downloaded only when you purchase the app from the Amazon Kindle App Store.

I should say that I'm really very impressed with the Kindle Fire! It is a really nice size, fabulous for reading books (thanks to the touch screen, which isn't on the older Kindle of course). It seems very fast, with a lovely clear display and it is very responsive. Will I like it more than my iPad 1? Well, I prefer the size of the Kindle Fire, and like the stereo speakers; and I like that it was so easy to get our app running on it. :) The iPad has the advantage of having lots of good quality games for it... but it is *very* much more expensive ... if the Kindle Fire had more storage space, it'd make quite a nice MP3 player.

It's a pain that the Amazon App store is US-only at the moment, but I'm sure they'll roll that out worldwide soon enough. I think they're going to sell huge numbers of Kindle Fires!

I also want to mention that so far, the Amazon developer support team have been *incredibly* responsive to the questions I've had; I'm genuinely impressed!

Pete

Monday, 19 December 2011

Mixtikl on Android - reprise

The port of Mixtikl to Android is now complete.

I thought I'd share how I ended-up porting Mixtikl to Android, as this has been a long road!

When I first started looking at porting Mixtikl to Android, there were a couple of blocking issues;
- no support for C++ template library (though eventually STLPort arrived!)
- the original Mixtikl made heavy use of Modal code for dialog handling
Both of those two items made a port with Android a non-starter.

However, when Airplay (now renamed to Marmalade) added Android support, we were able to attempt a port of Mixtikl; as Airplay contained both C++ template library support, and supported modal code.
Ultimately however, the audio had too many problems (very, very high latency, mono only etc.) and this made the Airplay-based port too low quality to allow us to release.

Time moves on however, and a while back, Google added full support for the C++ standard template library to the Android NDK. And, critically, they added support for both OpenSL (to Android 2.3), and 32-bit ARM code generation (via armeabi-v7a). Given those changes, we spend a month or so removing all the modal code from Mixtikl (not an easy undertaking: you have to bear in mind that Mixtikl is > 500K lines of C++ code!).

So, given those changes in both the Android NDK and Mixtikl's own internal architecture, we were able to start and complete a full native port of Mixtikl to Android.

The bulk of Mixtikl remains in C++; all the graphics are drawn via the same C++-based graphics engine we use on all platforms. The interaction with Java is just a thin JNI layer that looks after drawing primitives and some native dialog interfacing. The OpenSL support is just a thin adaptor layer between our audio engine and OpenSL.

Mixtikl performs brilliantly on Android in my opinion; though as noted elsewhere the Android OpenSL implementation is missing a big trick in regards latency management; and we have to configure Mixtikl to run with slightly higher latency that we would like in order to avoid any potential audio break-up.

I should note that debugging C++ code on Android remains a pain, you have to rely on trace statements, though it is at least possible now to use gdb and the bt command to trap crashes when they happen and see where the code died in the call stack.  The good news is that Mixtikl is very solid as 99% of the code is cross-platform and very well tested, so I've hardly had to touch gdb!

Tuesday, 6 December 2011

Android - how to reduce the audio latency

I been approached a few times recently about how to write low-latency audio latency apps in Android, and I point such queries to OpenSL and say "look how much better it is that it used to be" (provided you use C++ of course)...

But the bottom line is that the Android audio APIs - and that includes OpenSL - still miss the basic trick, which is to be able to offer a programmatic contract that is as simple as this:

App : what sets of audio rates and formats do you support?
(API responds, maybe only 22Khz stereo, for sake of simplicity here)
App asks: how  many audio blocks do I need to keep prepared and primed in the queue to guarantee no break-up, at this rate/format, with minimal latency?
(API responds, maybe 2 blocks of 512 sample frames each).
App prepares the first 2 blocks, submits, and gets "block delivery" callbacks; it runs a separate thread to keep internal queues topped-up ready to deliver to the callbacks.

With that sort of contract, every app can run at minimal latency dictated by the underlying audio device (through the driver layer), on any device.  Without that sort of contract, every app developer is kept guessing, and has to assume a worst case that works on all devices available for testing. :)

I implemented this scheme nearly a decade ago in the intent Sound System (I had the luxury of designing and implementing a complete audio architecture for mobile devices from scratch!). It is a piece of cake to do, and IMO audio developers for Android are screaming for it. Intent was focused on ultra-low audio latency for games / multimedia and musical instruments...

I should note, Apple have also missed the same trick - they've been able to get away with it however, as the offer a fixed range of hardware....

Sunday, 4 December 2011

Android - high performance audio - how to do it

I've just been through a very interesting period of work, sorting-out a high-performance audio interface for the Android port of Mixtikl. I've learned quite a few things - here are the highlights.

Firstly, target Android 2.3 or later. This allows you to use OpenSL ES, which is the only realistic approach for low-latency audio on Android. The audio allows you to delivery audio with pretty low latency, and totally avoids any problems of garbage collection blocking you. This of course assumes that all your audio code is written in C++!

As you're using OpenSL ES, and assuming you have some very heavy audio DSP going-on (like in Mixtikl!), you'll need to use a separate Posix thread to keep your audio callbacks pumped-up. Basically, if your OpenSL ES audio block callbacks take much time at all, then your audio will break-up. So, use a worker thread to keep sufficient audio pre-processed and ready to be picked-up and delivered via your callbacks!

Finally, and believe me this is important (!): make sure you target armeabi-v7a, and that you use the right compiler flags to generate 32-bit ARM7 code. If you on the other hand use the default settings, you'll generate Thumb code for armeabi - and your code will run staggeringly slower  (!!), and audio break-up is inevitable if you're doing anything serious. So: don't bother targeting armeabi devices... and Thumb code is a no-no.

Follow my advice, and you can create sophisticated audio software. Don't follow it, and you're going to find things tough! ;)

I should note that support for  armeabi-v7a *emulator* targets arrived in the Android 4.0 SDK... which makes things a lot easier as well... something worth knowing!

Finally... here is a useful resource on the background to OpenSL ES on Android...:
http://mobilepearls.com/labs/native-android-api/opensles/index.html

Friday, 21 October 2011

gdb with the Android NDK - now easy with ddd wrapping ndk-gdb!

You might already know this, but android debugging from gdb is now really, really easy - if you can stomach gdb!

See Android/ndk6/docs/NDK-GDB.html in your NDK documentation.

Basically, the app's libs/armeabi folder must have a gdbserver in it... which is put there automatically by the ndk-build command (remember not to ship your app with gdbserver in it!):

So...: build your app, install with e.g.:

adb install -r myfile.apk

Start your app directly from the Android UI, and in a terminal, change to your product's jni folder, and you're ready to debug!



Using command-line gdb
 

If you prefer command-line gdb, type this:

ndk-gdb com.example.myproject

You'll find that ndk-gdb will break as your app attaches. Set a breakpoint if you want (optional!), e.g.:

b Java_com_example_hellojni_HelloJni_stringFromJNI

Enter "cont" to continue...

And the debugger will now hit your breakpoints - use normal gdb commands to control it!







Using the graphical ddd debugger


If, like me, you prefer to use a graphical debugger, it is really easy to use ddd as a wrapper around gdb!

Many thanks to a blog reader for this suggestion!

It is really easy to use ddd instead... here are the steps!




Preparation:

1. Install ddd ...!
2. Modify your ndk-gdb shells script, to simply comment-out the "exit 1" call when an unexpected argument is passed



Running ddd 



Start your app from Eclipse

In a terminal window, change to your project's jni folder (if you're not there already!)
Run ddd like this - and that is all there is to it!

ddd --debugger ndk-gdb



Eclipse

Using gdb for JNI/NDK from Eclipse is way too painful - I wouldn't bother. :)



Run-time breakpoints


On a final note, you can use this code to force a breakpoint at runtime...


    #define BREAKPOINT __asm__ ("bkpt 0")
    printf ("Hitting breakpoint!");

    BREAKPOINT;
    printf ("Gone past breakpoint!");

Friday, 23 September 2011

Mixtikl 5... iOS/Mac/Windows

Work on Mixtikl 5 is keeping me very busy... we've decided that Mixtikl 5 will be the first version we release for Android. However, we'll focus first on finishing-off the iOS / Mac /Windows versions...

Friday, 2 September 2011

Android - using findbugs from outside of Eclipse

I've been using findbugs to help track-down issues in Java code.  A great tool, and very easy to use. However, the Eclipse-based plug-in keeps crashing for me with a NullPointerException on certain projects.

The work-around to this is to run the findbugs UI directly.

To use findbugs on Windows, get the latest findbugs distribution from http://findbugs.sourceforge.net/downloads.html, and then create a shortcut along the following lines (note the -X argument, as the JVM might require quite a bit of memory to avoid failing for big projects!):
\java.exe -Xmx1024m -jar C:\findbugs-1.3.9\lib\findbugs.jar -gui

On Mac, simply run something like this directly:
java -Xmx1024m -jar ~/findbugs-1.3.9/lib/findbugs.jar -gui &

It is really use to use - just select File -> New Project, add your class folders to the top pane, your source folders to the bottom pane, and let it run!






Tuesday, 30 August 2011

Upgrade to XCode 4.1 - problems with VALID_ARCHS for iOS Simulator builds

Now that I've taken the plunge and moved to XCode 4.x from XCode 3.x ... I had a really weird problem trying to build my iOS projects for the simulator.

The builds failed with an error related to the i386 architecture. I simply couldn't figure-out how to solve this directly with the XCode IDE.

Luckily, I did manage to figure-out a fix: my solution was to edit the project.pbxproj files by hand with vim, and remove a handul of fixed definitions of VALID_ARCHS ... which presumably were legacies from previous XCode versions. This is not the first time that I've been forced to fix XCode projects by hand!

Monday, 29 August 2011

Mac - how to build installs the easy way

I was having some issues getting my Mac installers (using packagemaker) working with the latest XCode.

Surfing-around, I found an amazing free tool called Packages. The link is here:

http://s.sudre.free.fr/Software/Packages/about.html

I've changed our own installers to use this, and all my Mac install problems have disappeared with an hours work. :)

Pete

Wednesday, 10 August 2011

Audio and MIDI Visualization in iOS and Android

I've spent the past month focused mainly on Intermorphic's Audio/MIDI visualizer framework, dusting-down the work I did (in the main) several years ago, and bringing it all up-to-date.

I've recently experimented with lots of stuff, 2D and 3D, on iOS (UIKit ,Quartz, Core Animation and OpenGL ES) and Android (Canvas and OpenGL ES), and have learned a lot about performance differences between these approaches. I've also learned that OpenGL ES is a truly horrible API in which to get things done. :)

Anyways - things are coming on nicely: the next update for Mixtikl should be another good one!


Wednesday, 3 August 2011

Android - turn key events into UTF8

As I've been asked a few times how to turn Android key events into UTF8 data, I figured it might be a good idea to share this info in my blog! It is actually really easy:

  @Override
  public boolean onKeyDown(int keyCode, KeyEvent event) {
    int [] unicodePoints = new int[1];
    unicodePoints[0] = event.getUnicodeChar();
    String x = new String(unicodePoints, 0, 1);
    byte utf8Bytes[];
    try {
      utf8Bytes = x.getBytes("UTF-8");
      // Do something with your UTF-8 data here!
      // ....
      // ...
    } catch (UnsupportedEncodingException e) {
      Log.e(TAG, "Failed to key event to UTF8");
      Throw(e);
    }
    return true;
  }

Tuesday, 26 July 2011

Mixtikl on Android - with the NDK and C++

Mixtikl is now around 95% complete on Android.

I've actually ported directly with the NDK/C++, without using Juce, as my 2d portability layer for Mixtikl is so highly developed. The port only took around a week (having already done a lot of background work in the previous few months laying the ground for this and possibly other ports!).

The Audio drivers have caused some fun; I've ended-up implementing support for AudioTrack and AudioRecord in native code with pthreads. Audio support in Android really is very poor...

The biggest headache is the basic problem of debugging C++ code on Android; I'm reduced to using trace statements. It helps that 99% of the code in Mixtikl is cross-platform and fully debugged on other platforms!

That said, the latest NDK (R6) is much improved with STL support and what have you; there is no longer any need to use Crystax.

I'm still very likely to use Juce for the Liptikl and Noatikl ports to Android, of course!

Thursday, 30 June 2011

Juce, C++ and Android

I've been experimenting with using the fabulous Juce to create programs for Android using C++.

You can find out about Juce here: http://www.rawmaterialsoftware.com/juce.php

If you haven't used Juce yet - give it a go! I've been using it for years now to create cross-platform programs for Mac and Windows...

Monday, 31 January 2011

SourceTree - git app for Mac

I've started using SourceTree for painlessly reviewing historical changes in my Mac's git repository.

Steve (the SourceTree developer) has just added a feature which easily allows you to visualise changes between arbitrary versions of any given file, which is far easier than trying to do with the git command-line tools!

Give it a try - its a really nice tool!

Thursday, 7 October 2010

Android: creating pop-up menus at specific screen locations

If you need to create a pop-up menu in Android, anchored to a specific View on the screen you could use this simple code as a template:
http://www.londatiga.net/it/how-to-create-quickaction-dialog-in-android/

However, what if you want your pop-up to appear at a particular position on screen, where you don't have a View to anchor to (maybe to overlay some other element on-screen that you don't have ownership of)?

My solution was as follows:
- Create a new Linear Layout to use as a "canvas" - call it my_canvas - and set to fill parent.
- Create a single ImageView (with empty drawable - it'll remain invisible!) within that layout, set to wrap content; call it my_canvas_image.
- We then create dynamically a view based on the layout, with left and right margin properties to pixel-position to the correct place on screen; use the image in that view as the anchor for our pop-up; and remove the view when done.

When your activity is created, grab the ViewGroup used to create the main activity layout like this:

    LayoutInflater inflater = (LayoutInflater)getApplicationContext().getSystemService(Context.LAYOUT_INFLATER_SERVICE);
    ViewGroup appViewGroup = (ViewGroup) inflater.inflate(R.layout.myApp, null);
    setContentView(appView);

When you need to create the pop-up at a given screen location, you can use code like this:


    WindowManager windowManager = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE);
    int screenWidth = windowManager.getDefaultDisplay().getWidth();
    int screenHeight = windowManager.getDefaultDisplay().getHeight();
   
    LinearLayout.LayoutParams paramsx = new LinearLayout.LayoutParams(screenWidth, screenHeight);
    paramsx.leftMargin = putPopUpHereX;
    paramsx.topMargin = putPopUpHereY;
       
    LayoutInflater inflater = (LayoutInflater)context.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
    View canvasView = inflater.inflate(R.layout.my_canvas, null);
   
    appViewGroup.addView(canvasView);

    ImageView imageViewUseAsAnchor = (ImageView) canvasView.findViewById(R.id.my_canvas_image);
   
... the imageViewUseAsAnchor item can then be used as the "anchor" to which to attach the pop-up menu at your required screen location!

When you've finished, you can then remove the "canvas" view with:

    appViewGroup.removeView(canvasView);

Tuesday, 28 September 2010

The art of scalable Android Layouts

One of the harder things to figure-out in Android, is how to do nice layouts that work well on different screen sizes. And, as always, I found it difficult to track-down that information!

The key for me in many circumstances is to position elements in percentage terms, generally using fill_parent; this is the opposite of most of the examples I have seen, which tend to use wrap_content! My approach is to ensure that size is defined in relative terms from the top-down using layout_weight; rather than allowing children to dictate their own size (and thereby mess-up the overall layout) via wrap_content...!

Here is a simple example of my approach.

Imagine you have a screen which you want to split into two panels, aligned horizontally, where you want the right-most panel to take-up around 20% of the screen size.

Create a new layout which is a frame layout. Set layout_width and layout_height to fill_parent. We'll use this for most other elements.

Create a LinearLayout within the frame parent. Set orientation to horizontal. Set layout_width and layout_height to fill_parent.

Create two elements (whatever type you need for your panels, maybe ImageView with a drawable background picture if you're just experimenting) as children of your new LinearLayout. Set both to have layout_width of 0dp and layout_weight of 50. If you want one layout to be wider than the other, simple adjust the weights; the element with the larger weight will be bigger proportionately. Make sure you sent layout_height to be fill_parent; otherwise the panels won't properly fill the screen in different configurations!

Want to split-up your right-hand panel into two vertical elements? Replace your right-hand element with a new LinearLayout. Set layout_width to 0dp, and layout_weight to whatever your previous right-hand element was (e.g. 50). Set orientation to vertical. Set layout_height to fill_parent. Put two new elements (again, maybe ImageViews for example...) in the new LinearLayout; set both to have layout_height of 0dp and layout_weight of 50. If you want one layout to be taller than the other, simple adjust the weights. Make sure you sent layout_width to be fill_parent; otherwise (as before!) the panels won't properly fill the screen in different configurations.

And so on! If you need to use an invisible spacer, you can simply follow the above approach and use an empty LinearLayout with the appropriate layout_weight.

This makes sense once you've tried it out; remember to test the layout you create with a variety of AVD sizes!