Thursday, 16 September 2010

Android - building the PDK using the SDK, and adding a system service

Just how difficult can it be to:
- download the Android source code
- figure-out how to add a system service
- call that service from an activity installed to the system after the event
- test this all out with the Android emulator.

The answer is: a lot harder than you might think!

Because this was a complete pain, involving a lot of fiddling-around and tracking-down and experimentation, I thought it would be a good to set-down the steps I had to follow. I really hope it helps somebody else out at some point!

I'm assuming that you're using Windows XP, Vista or 7 as your base machine.

Downloading the Android Source Code (Platform Development Kit - PDK)

The Android Platform Development Kit (PDK) download instructions are here.

http://source.android.com/source/download.html
First things first: install VirtualBox and install Ubuntu 32-bit as a Virtual Machine under Virtual Box.

Then install Android under this Virtual Machine.

You’ll require the java5 jdk.
The 64-bit build does not always work properly... it depends on which version of Android you’re using...:
   
http://groups.google.com/group/android-building/browse_thread/thread/193332fd6850a2a
All the open-source codelines that mirror Google's internal froyo
branches (i.e. froyo and the various android-2.2) use 1.5 like
Google's matching internal codelines do, since that is the version
that Google developed and tested with through the entire froyo
development cycle.
We backported the changes to switch to version 1.6 (64-bit) from our
internal master branch to the open-source master just as we switched
our internal master branch to 1.6, since from that point there was no
requirement for contributions to the open-source master branch to
build under 1.5 any more.
If you need to use the 64-bit version, then in order to complete the sudo apt-get install git-core ... command, you'll need to do this to get the correct JDK configured:
# Tell the system where to get the JDK...
sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
# Update the source list:
sudo apt-get update
# Install sun-java6-jdk;
sudo apt-get install sun-java6-jdk
The build problems otherwise are very simiilar to those for the 32-bit system.
However, various blocker issues arose. I gave up, and reverted to 32-bit building...

IMPORTANT NOTE: use the following as your repo init command... otherwise, you’ll get “work-in-progress” which might not build (if you want a different Android source code tag other than froyo, then feel free to use one):

repo init -u git://android.git.kernel.org/platform/manifest.git -b  froyo

If you have problems with Java, try doing this:

sudo update-java-alternatives -s java-1.5.0-sun

Building the Android Source Code

The Android PDK build instructions are here:

http://pdk.android.com/online-pdk/guide/build_system.html

Here is what I did.
cd ~/mydroid
lunch
# Choose 2. simulator
make
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=2.2
TARGET_PRODUCT=sim
TARGET_BUILD_VARIANT=eng
TARGET_SIMULATOR=true
TARGET_BUILD_TYPE=debug
TARGET_BUILD_APPS=
TARGET_ARCH=x86
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=MASTER
============================================

Note: the alternative of trying choosecombo (selecting: simulator, debug, eng) isn’t recommended by Google heads; they recommend the use of lunch.

To fix this error...:
usr/include/bits/fcntl2.h:51: error: call to ‘__open_missing_mode’ declared with attribute error: open with O_CREAT in second argument needs 3 arguments

... I had to edit frameworks/base/core/jni/android_server_Watchdog.cpp
... and add this new argument at the end of the function call where the reference to O_CREAT:
    , 0644);

To fix this error...:
frameworks/base/opengl/libagl/egl.cpp: In member function ‘virtual EGLBoolean android::egl_window_surface_v2_t::swapBuffers()’: frameworks/base/opengl/libagl/egl.cpp:554: internal compiler error: in add_phi_arg, at tree-phinodes.c:391

... I had to remove the const on line 554 in frameworks/base/opengl/libagl/egl.cpp

If you get problems reported when building, related to a missing file called asoundlib.h, you’ll need to install these packages and then re-attempt the make:
libasound2-dev
lib32asound2-dev

If you get problems reported when building, related to a missing files called something similar to wx/wxprec.h, you’ll need to install this package (WxWidgets) and then re-attempt the make:
sudo  apt-get install libwxgtk2.6-0 libwxbase2.6-0 libwxbase2.6-dev

If you get problems reported when building like this:
|~/mydroid/development/simulator/app/DeviceWindow.cpp:175: undefined reference to `wxImage::HasAlpha() const'

You’ll have to edit:

development/simulator/app/DeviceWindow.cpp - comment-out the lines to HasAlpha()
development/simulator/app/MainFrame.cpp - comment-out the lines to AppendSeparator()

Modify external/stlport/stl/_num_put.c in line 351. to use the first form of the #ifdef block (the second one, which would otherwise be chosen, causes compilation to fail).

Remove external/gtest/Android.mk, external/gtest/test/src/Android.mk and external/gtest/test/Android.mk (which had STL-related problems in my system!)


Installing the Android SDK

Download the Android 2.2 SDK from here:
Expand the download file to e.g. ~/Downloads/android-sdk-linux_x86

Add this to your PATH, e.g. (but: make sure this comes before any other Android-related paths...):

export PATH=$PATH:~/Downloads/android-sdk-linux_x86
Restart your shell!

cd ~/Downloads/android-sdk-linux_x86/tools
export ANDROID_SWT=/home/mycompany/Downloads/android-sdk-linux_x86/tools/lib/x86
android
... Select and install the SDK Platform Android 2.2, API 8, revision 2
Further notes on installing the Android SDK 2.2
Now, copy the Platform SDK so that the PDK can find it!

    # Create a new terminal, and then...
cd ~/mydroid
export PATH=/home/mycompany/mydroid/out/host/linux-x86/bin:$PATH
# Note that when you first do this, ~/mydroid/out/host/linux-x86/platforms and ~/mydroid/out/host/linux-x86/add-ons do NOT exist!
cp -pR \
~/Downloads/android-sdk-linux_x86/platforms \
~/Downloads/android-sdk-linux_x86/platforms \
out/host/linux-x86

Running the emulator with the newly built image

# Create a new terminal, and then...
cd ~/mydroid
export PATH=~/mydroid/out/host/linux-x86/bin:$PATH
export ANDROID_SWT="~/mydroid/out/host/linux-x86/framework"
# Now-up the emulator with the correct image!
./out/host/linux-x86/bin/emulator -system out/target/product/generic/system.img -sysdir out/target/product/generic -kernel prebuilt/android-arm/kernel/kernel-qemu -data out/target/product/generic/userdata.img

Adding a new service to the System

With reference to:
http://groups.google.com/group/android-porting/browse_thread/thread/f9a383ce949d1557?pli=1

Create a new service as frameworks/base/packages/IMyTest, as a very rough clone of frameworks/base/packages/SettingsProvider

Edit build/target/product/core.mk, to add the following line after the line showing DefaultContainerService :

        MyTestService \

Create various files:

mydroid/frameworks/base/packages/MyTestService/AndroidManifest.xml

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.mycompany">
       <application android:label="@string/service_name">
          <service android:name=".MyTestService"
                   android:enabled="true"
                   android:exported="true"/>
       </application>
</manifest>
   
mydroid/frameworks/base/packages/MyTestService/Android.mk


LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE_TAGS := optional
LOCAL_SRC_FILES := $(call all-subdir-java-files)
LOCAL_PACKAGE_NAME := MyTestService
LOCAL_CERTIFICATE := platform
include $(BUILD_PACKAGE)

mydroid/frameworks/base/packages/MyTestService/src/com/mycompany/MyTestService.java

package com.mycompany;
import com.mycompany.IMyTestService;
import com.android.internal.content.PackageHelper;
import android.content.Intent;
import android.content.pm.IPackageManager;
import android.content.pm.PackageInfo;
import android.content.pm.PackageInfoLite;
import android.content.pm.PackageManager;
import android.content.pm.PackageParser;
import android.net.Uri;
import android.os.Environment;
import android.os.IBinder;
import android.os.ParcelFileDescriptor;
import android.os.Process;
import android.os.RemoteException;
import android.os.ServiceManager;
import android.os.StatFs;
import android.app.IntentService;
import android.util.DisplayMetrics;
import android.util.Log;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import android.os.FileUtils;
import android.provider.Settings;
//
// Based on src/com/android/defcontainer/DefaultContainerService.java
// Also see:
//   frameworks/base/packages/MyTestService/AndroidManifest.mk
//   frameworks/base/packages/MyTestService/Android.mk
//   frameworks/base/core/java/com/mycompany/IMyTestService.aidl
//   frameworks/base/Android.mk
//
// Note that the .aidl file can be used by external activity projects.
//
public class MyTestService extends IntentService {
    private static final String TAG = "MyTestService";
    private IMyTestService.Stub mBinder = new IMyTestService.Stub()
    {
          public void setValue(int val)
          {
                  Log.i(TAG, "In setValue! ");
          }
    };
    public MyTestService () {
          super("MyTestService");
          setIntentRedelivery(true);
    }
    @Override
    protected void onHandleIntent(Intent intent)
    {
          Log.i(TAG, "onHandleIntent!");
    }
    public IBinder onBind(Intent intent) {
          Log.i(TAG, "onBind!");
          return mBinder;
    }
}

mydroid/frameworks/base/packages/MyTestService/res/values/strings.xml

<?xml version="1.0" encoding="utf-8"?>
<resources xmlns:xliff="urn:oasis:names:tc:xliff:document:1.2">
       <!-- service name  -->
       <string name="service_name">MyTestService</string>
</resources>

mydroid/frameworks/base/core/java/com/mycompany/IMyTestService.aidl

package com.mycompany;
interface IMyTestService {
/**
* {@hide}
*/
void setValue(int val);
}
The interface file will need to be added to the build system, so edit frameworks/base/Android.mk
... and add the following at the end of the list of LOCAL_SRC_FILES:

core/java/com/mycompany/IMyTestService.aidl \

Type this to make... if you don’t do the make clobber, your new MyTestService.apk file won’t be built!

make clobber
make

You will now see this error reported:

(unknown): error 3: Added class IMyTestService to package android.os
(unknown): error 3: Added class IMyTestService.Stub to package android.os
******************************
You have tried to change the API from what has been previously approved.
To make these errors go away, you have two choices:
 1) You can add "@hide" javadoc comments to the methods, etc. listed in the
    errors above.
 2) You can update current.xml by executing the following command:
       make update-api
    To submit the revised current.xml to the main Android repository,
    you will need approval.
******************************

Type this to work-around this:

    make update-api

Run this to verify that your new service is now in the build images...:
ls -ltr ~/mydroid/out/target/product/generic/system/app/MyTestService.apk
find . -name "*.img" | xargs -- grep "MyTestService.apk"

... This should show that the code you’ve created has appeared in ./userdata.img ...
Edit this to look for IMyTestService,  to verify that it has been added!

~/mydroid/frameworks/base/api/current.xml

5. Test the service APk is picked-up by the emulator...

Start up the emulator (using the previous long command line!)
Using adb logcat, you will find the message saying that the package has changed.
I/PackageManager(   68): /system/app/MyTestService.apk changed; collecting cert

6. Test Program

To test the service, create a "Hello World" activity using the project wizard under Eclipse on Windows.

MyServiceTest.java:
package com.mycompany.MyServiceTest;
import android.app.Activity;
import android.content.ComponentName;
import android.content.Context;
import android.content.Intent;
import android.content.ServiceConnection;
import android.os.Bundle;
import android.os.IBinder;
import android.util.Log;
import com.mycompany.IMyTestService;
public class MyServiceTest extends Activity {
  /** Called when the activity is first created. */
  IMyTestService imts;
  private ServiceConnection mConnection = new ServiceConnection()
  {
      @Override
      public void onServiceConnected(ComponentName className, IBinder service)
      {
          Log.d("MyServiceTest", "onServiceConnected");
          imts= IMyTestService.Stub.asInterface(service);
          try
          {
              // Try a dummy call to the service!
              imts.setValue(7); // Any value would do, this is just a test!
          }
          catch (Exception e)
          {
              e.printStackTrace();
          }
      }
      @Override
      public void onServiceDisconnected(ComponentName arg0) {
          Log.d("MyServiceTest", "onServiceDisconnected");
          imts = null;
      }
  };
  @Override
  public void onCreate(Bundle savedInstanceState) {
      super.onCreate(savedInstanceState);
      Log.d("MyServiceTest", "onCreate");
      setContentView(R.layout.main);
      // Bind-in to our service!
      Log.d("MyServiceTest", "try to bind service");
      Intent intent = new Intent();
      intent.setClassName("com.mycompany", "com.mycompany.MyTestService");
      bindService(intent, mConnection, Context.BIND_AUTO_CREATE);
      Log.d("MyServiceTest", "done try to bind service");
  }
}

Copy the IMyTestService.aidl file from your Linux box, in your Windows/Eclipse project’s src/com/mycompany sub-folder (you’ll first have to create this folder in your activity’s project folders).

Refresh your project files from the Eclipse menu and rebuild/debug.

This should run benignly under Windows, warning that it was unable to find the service.

Copy the apk file to your Linux VM, and install using:

adb install MyServiceTest.apk

Run the app using the emulator’s launcher; you should see the service used!
I/MyTestService(  360): onBind!
D/MyServiceTest(  354): onServiceConnected
I/MyTestService(  360): In setValue!

Easy, wasn't it? :-D

Friday, 2 July 2010

Mixtikl - added clipboard support

The new version of Mixtikl adds Intua Clipboard support for the iPhone/iPod/iPad version, amongst many other updates! This Mixtikl update is a big step forward for us, we're really looking forward to releasing it in a few weeks!

Wednesday, 26 May 2010

Android version of Mixtikl coming along

The tidy-up phase of the Android port of Mixtikl is coming on nicely. My shortlist of things to do is getting very short. :)

Thursday, 20 May 2010

Customized Android NDK r3 with enabled support of C++ exceptions, RTTI and Standard C++ Library

This looks very useful... (thanks to Chris for the pointer!)

Customized Android NDK r3 with enabled support of C++ exceptions, RTTI and Standard C++ Library...


http://www.crystax.net/android/ndk-r3.php

Sunday, 9 May 2010

Mixtikl on Android

The Mixtikl port to Android has been making further good progress; I think we're now only a couple of weeks from first release... it is working pretty well on my HTC Hero device, as I type this.

Hopefully we'll be able to follow-suit with Symbian once the Android version is out of the door...!

Thursday, 6 May 2010

MeeGo - how to download the source code using Cygwin / Windows

You can get the very latest MeeGo source code from here:

http://repo.meego.com/MeeGo/devel/trunk/repo/source/

I downloaded all the rpm files easily enough using wget (under cygwin, on my Windows machine).

To extract the contents of the downloaded rpm files, do this for each rpm file (assuming you have the rpm tools installed with cygwin!):

rpm2cpio filexyz.rpm | cpio -idmv --no-absolute-filenames

Then...

For each extracted myfile.tar.gz file, do this:
tar -xvf myfile.tar.gz

And...

For each extracted myfile.tar.bz2 file, do this:
tar -jxvf myfile.tar.bz2

You will then have all the latest MeeGo source code extracted to your computer!

Friday, 30 April 2010

Mixtikl on Airplay - audio now working too!

Good progress - Mixtikl's Audio is now working under the Airplay ARM simulator, so I now need to get it all up-and-running on an actual Android device. Fingers crossed. :)

Edit: Mixtikl is running under the Android simulator using Airplay - amazing. :-D

Many congratulations to Ideaworks!!

Thursday, 29 April 2010

Mixtikl Airplay port - to Android/Symbian

I've been porting Mixtikl to Qt and Airplay DK in parallel.

Now that the Qt port is done, I've also got the Airplay SDK port running in the ARM debugger emulator.

I've yet to add the audio hooks, and there are a couple of rough edges, but I'm looking forward to trying it all out in a few days on my HTC Hero Android phone!

Due to a weird problem with the Airplay SDK, I'm unable to link for X86 target (I'm waiting for a solution at http://airplaysdk.com/node/761). The same code builds fine however for the ARM Debugger - so at least I'm able to make progress. :)

The documentation has been pretty frustrating, so here are two useful tips to save you time!
- to blend-in an image, use Iw2DSetAlphaMode() to set a colour with the appropriate alpha channel setting before calling Iw2DDrawImage(); the alpha colour setting is used to blend the bitmap! Also, set the rgb values to other than 255, to scale the colour; useful to draw a white coloured bitmap in different colours, for example.
- if you want to clip, then call IwGxSetScissorScreenSpace(...) to define a clip window before any calls to Iw2D*(...).

The routine to query for pointer support didn't work for me - maybe I made the call wrong - I'm having to ignore the result from that call, I need to raise that in the forum once my build issue is resolved. :)

Anyways, Mixtikl now runs on the ARM emulator, so the next step is to complete the audio hooks and try on Android and Symbian devices!

Note: lack of support in Airplay for threads and clipboard will cut-back on some of the Mixtikl functionality for now on Android/Symbian... until those items are exposed in future Airplay updates!

Porting Mixtikl to Qt

It didn't take me long to port Mixtikl to Qt: just a few days to get it all running. Of course, it helps that Mixtikl was designed to be portable. :)

The biggest gotcha was shown-up by a weird linker error relating to missing vtable for a class.

It turns out that this was for a few reasons!

1. If your class implements slots, the declaration *must* be put in a header file; put it in a .cpp file, and qmake won't won't find it and you get the vtable link error.
2. That header file MUST be in your .pro file's list of header files; forget this, qmake won't won't find it and you get the vtable link error.
3. Make sure your class has a special line added. You must have the Q_OBJECT line as the first line in your class prototype, as shown below - no punctuation at the end of the line!

class myclass : public QObject
{
  Q_OBJECT
  // ... rest of class follows
};


4. The class must be derived from a class that ultimately derives from QObject
5. Your slot implementation method must have a "slots:" line before it. e.g.:

public slots:
  void setValue(int value);

6. You must then run qmake from the console/terminal, while in the folder that contains your .pro file (that will rebuild the makefiles to pull-in the neceesary bits).

One more thing to mention: I found using QTimer to be problematic if using threads; use QThread instead to roll your own timers if you have problems. I had similar issues with NSTimer on iPhone...

Once you've figured that out, Qt is a piece of cake to use. A very easy to use, powerful, flexible C++ framework.

Wednesday, 21 April 2010

Airplay SDK - building C++ apps as Android .apk files - first impressions

I noticed the other day that the Airplay SDK now uses modern C++ with STL - nice, and essential for my code. I already knew that the Airplay SDK supported Android, and having taking a look at my Android porting options I really didn't want to directly port my own C++ app code to Android using STL and JNI - there would be too much pain involved. :-0

So, having just installed the latest Android SDK on Windows 7 and Mac, I figured it was time - finally - to download and install the Airplay SDK evaluation.

In installed the Windows version on my Mac under Boot camp. This requires Visual Studio 2008 (the Express edition should work, if you're interested...).

Installation on Windows was easy, the eval license registered properly and I was soon following the instructions to build my first basic demo project. [I actually started with the Mac version - still in Beta - but it failed to register my eval license! Serves me right for starting with Beta software :)]

I double-clicked the example project file for a 3d demo game, which automatically constructed and opened a suitably configured Visual Studio project. Building this for the x86 target was as easy as building any other Visual Studio project - very simple.

Running-up the project under Visual Studio debugger for Windows, ran the app through a device emulator, which worked smoothly.

I next trivially rebuilt the app for ARM Debug target, ran under Visual Studio under ARM emulator on Windows - again this was easy and worked fine - it runs the app through an arm device Emulator, with a graphical version of GDB as the debugger.

Having rebuilt the app for ARM Release target, I next used a simple utility that comes with the Airplay SDK, to bundle-up the app an a .apk file (the installer format for Android). This was really easy. Note, however, that I first had to rename the Android SDK 1.6 folder on my PC (see the Airplay SDK forum for details on how/why!), and had to install a couple of other tools that are clearly documented in the Airplay documentation). Anyways: once your system is configured, building the .apk file is very easy. :)

I next installed the .apk to the Android SDK's emulator using the "adb install" command, and ran-up the application directly from Android SDK emulator. This worked OK, albeit with a slow frame rate I've been told that on devices the frame rate is very, very much faster than the emulator).

So, a really good experience so far. I'm looking forward to finding out more in due course. I was impressed enough to get a 99 dollar "indie" license; the "iPhone" license is actually free, which is very cool.

I'll need to find out how well it all works on a real Android device, and try some complex 2D graphics and heavy-duty audio, but the Airplay SDK so far looks very promising. It is powerful enough that it even looks like it'd be relatively straightforward to port frameworks like Qt to the Airplay SDK.

I found the online documentation to be a pain to navigate, but the documentation in Windows CHM file format (in the SDK) is pretty good.

The bottom line for me is how easy it was to get from generic C++ code, to a .apk file for Android - very impressive indeed. Not only that, the SDK supports buildling Symbian .sis files, Window Mobile .cab files ... and various other platforms. Plenty here to keep me busy and productive!

Edit: I've just discovered that Airplay doesn't support threads at the moment... which is problematic, to say the least!

Edit: I've heard via the Airplay forums that Thread support is coming in a few months ... which is very good news. :) Meanwhile, the port of Mixtikl to Android/Symbian via Airplay (and also a port of Mixtikl via Qt...) is coming along well, so maybe we'll have something to report in a few weeks...