2010-03-31

meeting Android OS and Dalvik VM

3.26: summary:

. android as a subtopic of addx
is concerned with how androids ideas
could contribute to or be reused by addx .
. of immediate interest is their vm,
which is significantly more efficient than the java vm .

. some are upset by the changes it made to linux
while others call it a refreshingly clean architecture;
the body language of the google staff who were
defending the changes indicates that a lot of
compromises had to be made in order to appease
the various business-related needs
of the other oha (Open Handset Alliance) members .
. it's still an open platform,
but users can only modify their system as root users
-- a line crossed that can void their contract .
. unlike the Apple app'store model,
there is nothing in the Android model
that would conflict with addx`goals .

3.19: web.addx/android:
. wiki's android .
. some app dev's call android a gutted linux .

thoughts of an android system engineer:

Matt Porter was heavily involved in the MIPS and PPC ports of Android:
"(Android is not Linux in the strict sense of the word
because important userspace components are missing,
thereby making Android comparatively inaccessible and inflexible.
Android uses, for example, its own mount system
that works with MMC subsystems out-of-box
rather than with USB devices.
Support is missing for udev, glibc, and
SysV process communication,
but are replaced by a somewhat hard to change,
hard-coded implementation from the Open Handset Alliance.
. Android makes no use of tslib for touchscreen support
and lacks effective Ethernet support.
[More arguments are included in his set of slides.]
Second, the Android community is
lagging behind other Linux and open source communities,
partly because the platform is commonly developed
outside the Android Open Source Project (AOSP) tree
and given less priority in the open repository.) .
Harald Welte summarizes Matt Porter's critique:
"(. google has simply thrown 5-10 years
of Linux userspace evolution into the trashcan
and re-implemented it partially for no reason.
Things like hard-coded device lists/permissions in object code
rather than config files,
the lack of support for hot-plugging devices (udev),
the lack of kernel headers.
A libc that throws away System V IPC
that every unix/Linux software developer takes for granted.
The lack of complete POSIX threads.
Just one more practical example:
You cannot even plug a USB drive to an android system,
since /dev/sd* is not an expected device name in their
hardcoded hotplug management.)
. Android's design is defended:
"(Android's specialized and inflexible character
was a result of performance and resource reasons.)
. the android userspace sounds pretty tight;
what's it like for the dev'ers? [3.20:
. those concerns were by app dev's, not users!
they are crying about not being able to use
the usual linux tool chains to dev' for linux .
. it's likely that dev'ers are not accustomed to
having big brother defending the consumer
by creating so much burden for the sake of efficiency
and security -- the sad fact is
that linux is a malware magnet just like windows
exactly because
those platforms let dev's have their way .]

. Android is now owned by the Open Handset Alliance
where google is just one of many members;
the oha has an interesting page showing
all the roles in the communications industry,
and all the players in controlling Android .

lecture of why Android is based on a new vm:

. libraries written in C and other languages
can be compiled to ARM native code and installed
by using the Android Native Development Kit (NDK) .
The NDK converts c to bytecode not the cpu's native code
Simply re-coding a method to run in C
usually does not result in a large performance increase
but does always increase application complexity.
The NDK can, however, be an effective way to
reuse a large corpus of existing C/C++ code.
--[. if c is not your favorite lang,
it would apparently be best to adapt your
translator: adda -> android's java
rather than: adda -> c for android's ndk .
3.20:
. but how much "(complexity) are we talking about here?
creating an adda compiler for a completely new language
is itself a significantly complex problem .
]
. as per the usual security measures
android distinguishes the app dev'
from the platform (system) dev' .
. only the system dev's can make good use of
Java Native Interface (JNI),
a programming framework that allows
byte code running in a Java VM
to call and to be called by cpu's native code .
--[. this is the same trade-off
that the iphone os had to make:
you could allow app' dev'ers to write native code
but then require them to
submit their code to human inspection
and risk having it rejected by hooded judges;
or,
you could do it the android way:
let anyone come without inspection
but then keep them tied up in a rather slow vm .]

Native classes can be called from Java code
running under the Dalvik VM
using java's System.loadLibrary call .
Unlike most Java VMs that are
stack-based, running Sun`java bytecode,
and having problems with concurrency,
the Dalvik VM, with a new byte code set,
is register-based, and is designed to
efficiently support concurrency .

. the new vm is optimized for platforms having
small ram, no hard drive, and a slow cpu .
. total ram on many platforms
is projected to be limited to 64mb;
and, app's typically have only 20 mb to play in .
. the system lib is 10mb --[ steep learning curve
but allows reuse of a lot of code .]
. by having a large library of native code
there's reduced need for jit,
because each bytecode instruction
represents many lines of highly-optimized native code .

. java's jar.files arrange all info within
the class it belongs to;
whereas [Dalvik EXecutable] files (dex.files)
contain the same info but arranged for
maximal sharing, with segments for vari'data,
and sharing of constant's in these segments:
string, type, proto[function signature], field, method .
. using that organization,
dex.files, even when uncompressed, take up
less space than jar files, even when compressed .

. cpu speeds in phones are expected to be
250 .. 500 mhz, bus speed is 100 mhz;
with a 16..32k cache .
types of mem:
( clean: unmodified
. dirty: modified -- app's structs and heaps
. private: accessed by only one process
. shared: accessed by multiple proceses .
)
. {clean vs dirty} mark bits are kept separate
instead of being embedded with each obj .
advantages: reduces cache misses,
avoids unsharing and padding needs;

. an android app's install-time work
is the same as ada's compile-time work:
time for a lot of verifications and optimizations .
[3.26:
. this is done with the desktop dev'tools,
not on the mobile unit itself .
. the toolchain converts java lang'
to java vm byte code (.jar)
and then converts .jar's to android vm bytecode .]

. being register-based avoids instruction dispatch:
30% fewer instructions and code units,
35% more bytes per instruction
but it can fetch 2 bytes at a time;
eg, compare jar's vs dex's for this code:
sumarray(arr#.int).long:
( sum.long=0
; for (i.int in arr) sum`+ i
; return sum
)
. java bytecode does this in
(25 bytes 14 dispatches 45 reads, 16 writes)
. android bytecode does it in
(18 bytes 6 dispatches 19 reads, 6 writes) .

. humans can interact at 10..30 reactions/s
and expect video at 25..30 frames/s,
audio sync'd at 10 times/s .
. calling those times the human scale,
contrast that with computer scale:
just handling human interaction takes
far less than 1% of cpu's capacity .
. mobile devices run on tiny batteries;
so, your app should be going back to sleep
rather than staying in a loop,
and when it does loop,
some loop`heads are more efficient than others;
they show you a list of loop head coding styles
and their impact on cpu time (battery life).

. why base app dev'ing on the java lang?
much of openware work is done in that;
and, there are compilers for many other lang's
that will translate to java classes (jar.files);
android has a tool that can convert these jar's
to android's bytecode (dex's)
-- but they don't do that dynamically on the handheld,
so the jython idea is out .
[3.26:
. a twist of the language question would be
why use the java lang but then toss the java vm idea ?
. the really progressive idea is llvm:
a tool that converts llvm code to your vm's bytecode .
--. Apple's clang project translates obj'c to llvm;
the llvm community is then translating llvm to
a variety of cpu's, including mac's x86 .]

[3.26: llvm`backend for android`Dalvik
Patrick Walton
Date: Fri, 09 Jan 2009 22:04:16 -0800
Subject: Type safety of the Dalvik VM?
. I'm interested in writing an LLVM backend for Dalvik
(allowing C code to be compiled to Dalvik),
Certainly, the apparent lack of type safety
makes it easy to write a C-to-Dalvik compiler,
but I want to make sure that my methods won't depend on
exploiting bugs that are going to be fixed.
"Andrew Stadler (Google)"
Date: Fri, 9 Jan 2009 22:11:09 -0800
. make yourself very familiar with the bytecode verifier,
which runs over all java bytecode
before it gets to the actual Dalvik VM,
and provides expected the safety checks .
fadden
Date: Sun, 11 Jan 2009 10:34:41 -0800 (PST)
. type safety is a goal, and in some cases
Dalvik is safe to a greater extent than other VMs
(e.g. it's more picky about the width of sub-32-bit integers).
Some details are available in dalvik/docs/verifier.html.
Identifying type-safety issues during verification
allows us to avoid making those checks at runtime,
improving performance. Because DEX files are read-only
to everything but a couple of system processes,
we can (usually) do the verification
at app`install-time, and not have to repeat it
at app`startup-time.

--. it sounds like the best bet is to find llvm ->java vm .

David Roberts 2009
JVM Backend
I've written a backend for LLVM
that allows LLVM IR to be transformed to a Java/JVM class file

2008: only “foreign” language to run on Android is Scala .
Invoke JNI based methods (Bridging C/C++ -> Java)

Developing using C++
StephC
Date: Tue, 13 Nov 2007 03:53:17 -0000
Local: Mon, Nov 12 2007 8:53 pm
J2ME is a real nightmare for mobile games developers,
it's even considered by many as the reason why
mobile games aren't really suceeding so far.
John Carmack, a game developer:
"(It turns out that I'm a lot less fond of Java
for resource-constrained work.
I remember all the little gripes with Java's language,
like no unsigned bytes,
and the consequences of strong typing, like no memset,
and the inability to read resources into
anything but a char array,
...
Even when compiled to completely native code,
performance is hobbled by Java semantic requirements
like range checking on every array access .)
]