2010-06-30

addx guarding

6.26: addx/IPC monitoring:
. how to assure that IPC ability is fair?
need to keep a rough count of each process's
use of message-sends,
if things are getting bogged down,
and someone is out-mailing a lot,
then need to alert user to the app
for ideas about why that app might need to out-mail so much .

6.29: addx/malloc/privacy assurance:
. when doing mem alloc,
privacy could be done with minimal erasing
if each sensitive domain
did its own recycling .
. also if the next domain is trusted (the os)
then erasing would not be needed,
suggesting that erasure not be done until
the block is reallocated .
. on most practical systems though,
addx will be a system on an untrusted system,
so then you'd need to erase as soon as possible,
and still not be sure of privacy!

much robotics can be done via simulation

6.24: addx/robotics/much can be done via simulation:

. when video is recognized,
ie, tells us what's happening in a world,
it's because we are
parsing a matrix of pixels
into a theatre of actors and props,
as we understand them to exist from experience .
. the list of actors and their positions
is what the robotic intelligence algorithms
can work with;
so, having a lot of practice with
programming this symbol-binding transform:
pixel matrix -> actor set configuration
is how a desk engineer can
contribute to robotics .

. the robot has a working model or world
like the database of a game
(a 3-D simulated world) .
. the main test needed
is to have a game that makes
vid's of your 3-D world,
and then see how fast and completely
your transform can recreate that 3-D world
from that vid .
. before thinking that is ok
we need to deal with how
real vid's are fuzzy and smeared .
. this poor quality of real-time vid
is why the algorithms need to find
the edges of fuzzy masses,
and identify joint points
among the shape's changes .
. this can't be done with
scanning one frame at a time,
there must be a quick way to show how
2 frames differ from eachother:
only movement can reveal movable masses
among the camoflage of noise .
. if the quality of the vid is very high,
then instead of having to
analyze an entire stream differential,
it could study just one snapshot .

. once the id is bound to a database obj,
then most analysis time can be spent on
reaffirming the actor's position and posture:
eg, which way is it facing,
what vector is the movement following, etc .

. just as a pixel mass is bound to an actor,
joint points are bound via outline of mass,
changes in joint points are easily tracked that way
and are key to identifying
potential and current actions .
. another analysis
-- that can be done concurrently --
is finding new objects .
. if the robot's scan of area
can't be persistent
or an obj's trail is lost
the actors have to be sortable by
spheres of possible location .

mvc and observer notifications

6.21: addx/mvc and observer notifications:

. if there is a std graphics api like opengl,
shouldn't scripted app's have access to it ?
[6.27: how to keep mvc separation?
the model is using graphics to
make data components;
then the view can still have control in display
(graphics can be thumb-nailed,
displayed in a box of the intended size
but with descriptive text inside, etc ) .]

. the core system needs to build from
mac's idea of encouraging
all app's to provide an interface
that other app's and scripts can use
for using the services of that app .
[6.27:
. if the model does have some
new way of letting the user generate
data that is based on graphics,
then in a strict mvc model,
the model programmer will need to
contribute to the controller's code
through a plug-in architecture .
eg, the basic adde will have only
the sort of gui dynamics needed for
generating text and pictures .
. these behaviors will be bound to
the relevant datatypes;
a model can extend adde to do games
by binding a plug-in to game types
that explains how to display interaction .
. the proof of mvc separation will be
that adde, or the user's chosen controller
can show robots or networked users
playing the same game .
(game architecture should be based on
layers of activity
so as to minimize redrawing
and instead rely on compositing;
when something affects a model
all it's observers (view controlers and robots)
need to be notified .) ]

mvc with client-server architecture

6.20: addx/mvc with client-server architecture:

. strict mvc means model lets
controller do the image writing to screen .
. if part of the model includes
generating images,
then these can be composited
by the controller
to wherever the chosen view is .

. the x server allows remote servers
to be treated similar to local servers .
. the addx server model is not
concerned with such networking;
the main point is mvc modularity:
. the servers providing an mvc`model
need to be working for mvc`controllers
not for the human users directly .
. the model can then be used either by
human views or robotic views;
most typically, the point is to provide
more choice for the human
by letting a robot filter what the
model is intending for the human,
similar to the way bodyguards work .
[6.26:
. this isn't meant to escape from
contracts like google's
where the ad's must be human-visible
and the inquiries human-initiated;
it simply means that all models
are capable of robot accessability .
. one way this can help while
fulfilling the google contract
is to have the robot help with
making wise keyword choices:
it presents a list of suggestions
and the human can merely press enter .
. robots can also be reading along
to help humans spot the best leads .
]
. in addition to that mvc architecture,
the model may want to divide itself
into server and client parts
in order to efficiently modify the model
from afar .
[my intuitive assumption:]
. the x window system works on the idea that
it would be most efficient if
the os could provide a generic client part
that many servers could use,
since servers often need to make similar moves .

api:
. don't I need to have a
carefully selected set of api's
in order to know how to do the
x thing (where the model has
separate server and client parts) ?
. one could take it
one example at a time:
cloud computing depends heavily on x:
. some things like controlling the cursor
can't be done quickly eno' by remote .
. if the model does some
major modifications
the data needs to be with the server;
that's where it originates from,
so no problem .
. letting the user quickly scan
large portions of data
means that the data should be
mirrored there too
unless the platform is tiny .

review of higher view:
. mvc works naturally with x,
since the Server = Model
and the Controller along with its
choice of View = Client .
. the View consists of
the Controllers way of arranging
the data in a form that is
optimal for viewing .
. remote does complicate things a bit:
the non-remote controller has
instant access to the model,
but now with the model so far away,
the controller is having to think about
how much to ask for in advance
depending on the size of the local system .


6.26: addx/x protocol/server client terminology:
The term server and client have different meanings.
XServer runs on your graphically rich desktop
(it serves up the view)
X11-client (like app is client of OS)
may run anywhere on the connected network
lwn.net:
The program which provides access to
system resources (display, keyboard, mouse)
is the server.
The shorter-lived applications
which connect to the server
to make use of these shared resources
are the clients.
This is perfectly consistent with
every other use of "(client/server) .
What throws people is that
we're used to thinking of servers as
a particular type of *hardware*,
steadily ticking away the CPU-hours
in some climate-controlled back room,
out of sight, whereas "clients"
are the visible workstation PCs
at everyone's desk.
However, the X client/server terminology
is perfectly reasonable
from a *software* point-of-view,
and X is software, not hardware.

NetVM and cap'based security

6.19: addx/security/vmnet and cap'based:
Joanna Rutkowska Jun 17, 2010 at 4:53 AM
qubes-devel@googlegroups.com
"(you might not have VT-d working,
in which case you would not benefit from
additional security provided by sandboxed NetVM, ...)
. how could this NetVM idea
merge with cap-based security?
. with very fast switching
every app could have virtual access to
every other component .
. of these connections, you could
watch or log them, stop them,
or undo them by allowing only
virtual access (not modifying the
original object, but some mirror)
. the reason we need L4 [hypervisor]
is to reuse untrusted wares
and to have cooperation among
the mutually distrusting .
. ok L4 is a great platform because
it can act as a microkernel too .
. need to see how that means the
face changes for app dev's
-- is it just good news for sys' eng's?
. even without having L4 as an os,
the process-isolating architectures
like seL4 and Microsoft's Singularity
can still be useful as part of an
app framework that can give app dev's
more confidence in their own wares .
[6.24: ie, the basic stack has been:
(basic OS providing common lib's,
between hardware, and apps)
and then in addition to the basic OS,
there could be a security layer
provided by an app framework OS .]
[6.30: L4 vs seL4:
seL4 adds cap'based security to L4 .]

6.20: addx/seeing the light:
[6.30: todo: prove this!]
. it can seem rather pointless
trying to develope a secure app
on an insecure OS;
but for personal use, at least,
there can be security if
all the apps use that framework .
. even if you can't find hardware
that is designed for okL4 (open seL4)
it could help to have an app framework
that is based on an seL4 .

addx/security/what's needed on secure platform:
6.21:
. after seeing this googlecode blog
I was thinking about how to include
cap'based security .
6.27:
. chrome os already has good
separation between app's,
but what is that allows a browser
to rifle through your files
after hitting the go-away button?
has chrome browser really fixed that?
[6.30:
I've not had this problem on Chrome yet;
but what is "(sandboxed) mean
if it's so easy to steal my permission? ]
. if the guard is finally here,
then the only needed
cap'based permissions would be
the design of an app like addx
that is a scripting environment .

viewing changing amounts

6.11: adde/mem display:
. the system should be showing
how much mem it's using;
but, if the mem is changing rapidly,
then then display will be unreadable:
in that case,
the minimal display has both the amount,
and the rate of change;
so, then if you see the rate of change
is non-zero
then you know the amount will change soon,
it is only frozen temporarily for readability .

library mgt integrated with torrent client

6.1: adde/library mgt integrated with torrent client:
. I had this idea that all components
should be torrented by default
with the torrent client built into the apt-get service
but a problem with torrents generally,
is that when getting service from an isp,
the isp is a bottleneck;
because,
you're a subnode of the isp;
the isp can efficiently flow only when
not everyone is streaming at once;
whereas, torrenting expects the opposite:
if any streaming occurs,
all nodes should be streaming until the job is done .
. nevertheless, if it's good eno' for large downloads,
it should also be helpful with
masses of small downloads, [right?]
. when you open the torrent client,
it's listing all the modular packages
you got from the debian library,
having them in original form .
todo:
. do the packages get modified during installation?
if so, this would double the amount space
needed for this to work .

. another problem is that with smaller loads,
it may not be easy to convince people to give
when they're not getting as much .
. I got the idea from
hearing that skype is a torrent;
when you sign in to take calls,
you are also volunteering to torrent calls .
. the apt-get client could at least torrent
for the time it was downloading .
. the client could explain
this is one way you could help with
paying for the cost of
streaming packages and updates .
. there could be an adjustment for
stopping when the fan got noisy, etc .

journaling . elder care helps dev's too

6.1: adde/journaling/elder care helps dev's too:
. so many computer problems and clues;
trouble trusting my mem now ...
. computers for elders definitely aren't here yet!
we need a modular architecture
where no app does anything
-- inputting our outputting --
without a log recording it .
. you would think this is
the first thing designers would do;
they are always fielding calls like:
"( I don't know what I did .
how can reproduce the problem?)
and always giving advice like:
"( please be aware
of every little keystroke you make;
so that you can recreate the problem,
noting (eg, logging)
what conditions are triggering the behavior
-- the platform, its config', and your input).

gooey gui portability

6.28: pos.adda/dev.gui/stay with the mainstream tools:

. sdl (simple directmedia layer)
reminds me that being low level
could get you into trouble;
the original adda plan
to simply write a native code compiler
for each system, might be
not only cleaner, but also safer,
since a platform's own code and tools
are likely to take better care of you
if not with a safer language,
then at least with documentation
that warns you where the landmines are .

. a browse through sdl-style portability
points out that c is not multi-core .
. the goal of portability
-- as in maximal c coding --
could interfere with some
very smart c concurrency extensions .
. if concurrency should be pervasive,
your code will be riddled with non-std c
(don't even bother trying for portable).
[6.30: but the mvc pattern could help with
turning concurrency into a module? ]

adda exceptions performance

6.27: web.adda/exceptions/performance:
. I'm wondering how c++ exceptions
were inherently slowing c programs:
the addx vision of exceptions
is that there is always a scheduler
pre-empting functions for multitasking;
and, this scheduler is
gathering all the exceptions in a library;
... but it's not just routines
that handle exceptions:
every sub.block with a routine
can provide a unique handler for an exception;
and, these "(catch blocks) can be nested .
impl' idea#1:
. catch.blocks need to push their address
onto an exception stack;
then after an exception,
the scheduler can look on this exception stack
to the addresses of handlers .
. a program that has no handlers
would have an exception stack with
only one item:
the system's set of handlers .

web: Exceptions performance penalty:
In C++ Programming Language 3rd edition section 14.8
Stroustrup writes that it's possible to
implement exception handling
in such a way that there is
no run time overhead
when no exception is thrown
-- but it's not easy .
There is generally some overhead
as we have to keep track of the local objects
whose constructors have run,
so when exception is thrown
their destructors are called.
see Scott Meyer's "More effective C++" .

. g++ creates exception tables.
From the value of program counter
you can know which function/scope you're in.
For each scope you then know
which destructors have to be called
and using the stack address
you delete the apropriate objects.

We suppose that you can get
{ __builtin_return_address (LEVEL)
, __builtin_frame_address (LEVEL)
} for the ESP and EPC
of the caller of level LEVEL.

plain old c ... minus

6.25: pos.adda/plain old c ... minus:
[6.30:
. google's native-code sandboxes
in Android and in Chromium OS
are an important platform,
and should be considered before
deciding what "std" C is .] [6.28:
. many platforms will restrict the std libraries
in order to further enhance security
or adapt to smaller, energy-wise situations .]

. keep in mind what's available in
the android ndk:
libc (the C library), -- as Bionic, not glibc.
JNI interface, -- controller-view of your mvc;
OpenGL ES {1.1, 2.0} (3D graphics)
libm (Math),
libz (Zlib compression)
liblog (Android logging)
libjnigraphics (JNI Pixel buffer access)

. the API is included with the sdk;
so, get that installed on ubuntu
(eclipse runs better there than mac?) .

details of android's custom libc:
. customizing libc (as Bionic)
was necessary for three reasons:
* License:
keep GPL out of user-space.
Bionic code uses the BSD license.
* Size:
Bionic is half the size of glibc
* Speed:
custom pthread implementation.
Bionic has built-in support for
important Android-specific services
such as system properties and logging.

It doesn't support certain POSIX features,
like wide chars (wchar_t)
-- The world has moved on to Unicode
with its various fixed and variable
width encodings,
which the wide character type
is not particularly useful for.

. use ICU for internationalization support,
(International Components for Unicode)
instead of LOCALE .
ICU is a mature, widely used set of
C/C++ and Java libraries providing
Unicode and Globalization support
for software applications.
ICU is widely portable
and gives applications the same results on all platforms
and between C/C++ and Java software.
threads
The bionic C library has it's own thread API,
not the same as either original LinuxThreads or NPTL.
It implements only support for Dalvik JVM,

Mutexes, rwlocks, condvars, etc
are all implemented using kernel futexes,
-- see Ulrich Drepper's futex whitepaper.
There is no pthread_cancel().
Threads can exit,
but can not be killed by another thread.
There is no pthread_atfork().
Thread local storage is implemented,
with up to 64 keys handled.
Android reserves several of these for its own use:
the per-thread id and errno,
as well as two variables related to OpenGL
whose function I do not understand.

stdio, stdlib, string, unistd:
. removed the LOCALE support from strtod()
(i.e., is the decimal point a period or a comma?
In the Bionic library it is always a period).

{openlog(), syslog()} are replaced with
__libc_android_log_print()

Bionic uses Doug Lea's malloc, dlmalloc.
Bionic also provides a hash table
to track allocations
looking for leaks, in malloc_leak.c.

no asynchronous AIO routines
aio_read() or aio_write().

no crypt()
-- uses OpenSSL for that .
. dispenses with most file-based Unix administration.
no getfsent,
because no /etc/fstab.
. there is a /var/run/utmp,
and so there is a getutent()
Android implements its own account management,
and does not use /etc/passwd.

. no getpwent(),
and getpwnam()/getpwuid() are implemented as
wrappers around an Android ID service.
At present, the Android ID service consists of
25 hard-coded accounts in android_filesystem_config.h

. no termios support (good riddance).

Bionic is a BSD-based libc with support for
Linux system calls and interfaces.
If limitations prove untenable,
the syscall and pthread mutex implementation
could be repurposed into the heavier
FreeBSD/NetBSD/OpenBSD libc,
though handling thread cancellation
while using the Bionic mutexes
could require additional work.

oop theories, designs, implementation notes

6.1: adda/oop/based on vari-sized records:

. the 3 sorts of inheritance:
# subtyping:
. inheritors are assignment compatible .
# symbolic description:
. describes one type in terms of
another (incompatible) type .
# generics:
. an object's function is selected by
the function's name (or hash of that)
-- rather than referring to methods by
index, pointer, or offset .
-- this is the obj'c meaning of inheritance; [6.7:
obj'c also supports subtyping . 6.30:
... actually,
any system that supports generics,
implicitely supports subtyping .]
. after review oop ideas,
I wondered if the the trailer idea
would be useful for oop subclassing .

. parameters that vary in size
are accommodated by replacing value with
a mini-pointer (an index into
an associated trailer array)
that then leads to the value .
. a vari-sized record is organized as
a head (the mini-pointers)
and body (the trailer).
. in the vari-length record,
the number of fields is fixed,
whereas in multi-inheriting oop,
not only does the body grow
but the head too is vari-sized .
. provide a mini-filesystem (fs)...
[6.30:
. each class expects to find
it's own instance variables
arranged in the usual way
even when it's one of many ancestors
of the obj's class .
. in a mini-fs, the directory
doesn't have to be on the obj;
just containing a type.tag,
will be eno', since that is a
key to using the lib mgr's map:
(instance's class, ancestor.class) ->
offset into instance .
-- this map shows what offset to use
into the instance
to get to an inherited class's
embedded instance record .]

6.30: todo.adda/oop/obj'c dynamic linking:
. when there is multi-inheritance,
you have an instance variable where
there are separate sections
for each subclass's privates .
. it would be efficient if type-mgr is
passed ptr's to the operated obj's;
and generally,
how is each type-mgr supervised
so as to insure that they
don't affect any surrounding bits?
. you trust your compiler,
but then in obj'c,
you can relink modules dynamically;
and, the new module might be
code from another compiler .
. need to review obj'c
for how to best use,
and not be used by, that subsystem .

6.5: adda/oop/how does c do obj'c dynamic bindings:
. the generic-oop functionality of obj'c
comes from being able to launch a subrogram
when given a function`name and a type.tag;
this could be done in unix by calling to the os
to execute a shell command;
. without that ability,
each addition to the class library
would require editing the obj'c run-time engine
and re-linking it with the expanded class library .

6.7: adda/oop/subclasses as subtypes:
. what is a new type, vs a subtype, in adda?
subtypes can't remove or ignore
inherited functions,
but new types can do a reuse but remove;
how does Ada do this ?
inheritors are new types (meaning it's
not assignment-compatible
with the inherited type);
if you want a variable to have subtype polymorphism
(ie, be assignment-compatible with
all inheritors of Baseclass),
then you need to declare the var's type to be
Baseclass'class .

6.23: pos.adda/obj'c and cocoa bindings:
. the reason to not interface cocoa
is that it's very inefficient with its
ref'counting and its (typical) way
of implementing oop .
. to the extent there does need to be
a binding to it (for the sake of
getting the usual services)
the binding will not be direct:
the mac gives all the power to the developer,
whereas addx will give the user
most of the gui design .
. for example, in nu,
"( locally-scoped assignments are
instances of NSMutableDictionary; )
this might look like an
excellent chance to reuse,
but you might be using thousands of those,
which means juggling a lot of within
both your subsystem's heap
and the mac's system heap .
. it might be ok for prototyping,
but eventually, you want basic services
to be based on your own mem'mgt .

adda with mac, lisp, & nu

6.23: pos.adda/lisp/reuse of nu:
. can adda reuse nu by mapping to nu trees?
the primary job of adda is to
translate my lang to the platform's;
nu may not be a direct translation,
esp'ly given that it's completely oop
and maps to obj'c rather than
trying to stay in ansi c .
. the way nu could be quite handy
is to study how it implements
the lisp-like behaviors on the mac .
holy grail:
. here in this neontology example
he's got code for everything that is
usually done by the gui tool"InterfaceBuilder .
. replacing that is holy grail of addx .

6.23: adda/lisp/including s-expressions:
. the users of nu and lisp
like to remind us that
using a lang based on s-expressions
is not just being lazy:
it's just much easier to understand
when designing code that reads code .
. therefore, adda must have both modes:
english-math,
and s-expressions .
. english-math actually follows
syntax rules convenient to humans;
whereas, s-expression is simply
converting nested parentheticals
directly to nested trees .

adda's enumerated types

6.22: adda/type"enum:
. for the sake of readability,
adda`enums should be translated
directly to c`enums;
however, c`enums do need a lot of
helper functions in order to provide
ada-style functionality .
. it should also add -- in addition to Ada's way --
the same functionality for enums
that c has for pointers:
eg, if the ptr points at 4-byte ints,
then ptr+1 actually means ptr+4;
likewise for enums:
(e+1) means find the enum value
of next enum;
. if the enum is complicated by
non-consecutive values:
eg, { off, green:10, yellow:50, red:100 }
then it needs to generate a table
that enumerates the enum values:
eg, if the value will be in {0, 10, 50, 100}
then the table: value -> ordinal
is {0:0, 10:1, 50:2, 100:3};
with the ordinal value,
the encoded value is found from
an array lookup: {0, 10, 50, 100};
so if e=green+1; then
{ off, green:10, yellow:50, red:100
}(green) -> 10:
e= val#10 + ord#1 then
{0:0, 10:1, 50:2, 100:3}(10) -> 1:
e= ord#1 + ord#1 = ord#2 then
{0, 10, 50, 100}(+2) -> 50:
yellow
--. if there are only a few var's
it would be more efficient to
store both the ordinal and value in e .
. if values are consecutive
then after possibly shifting for a non-zero start,
the ordinal equals the value;
so, the enum`arithmetic functions need only
check for [out of range] .
. there should be unique type`names for
the 3 classes of enum types:
* subtype"simple-enum
has consecutive values that are
bijecting to names;
* subtype"encoding-enum has
non-consecutive bijections;
* subtype"entyping-enum is non-bijective:
there is no functional mapping
from code to name;
rationale for the name "(entyping):
traversing the values of the entyping-enum
then returns a sort of typing or
partitioning into equivalence classes:
eg, a mapping of enum`names to {0,1}
{ off:0, green:1, yellow:1, red:1 }
would have a traversal that returns
sets:
the first set is all names encoded as 0:
{ off }
the next set is names encoded as 1:
{ green, yellow, red } .

pointer syntax depends on typing

6.21: adda/pointer/syntax depends on typing:
. my current ideas of adopting folder syntax
as the syntax for pointers dereferences
seems to go against how folders are
normally used: (dir/) is same as (dir) .
. my semantics have said
(dir/) is body of folder
whereas
(dir) is ptr to folder .
. nevertheless,
the semantics are the same if
considering the type expected:
(eg,
a (copy) or (move) operation
is not expecting to operate on
pointers implementing the container
) .
a var' expecting a pointer
converts either case to a pointer,
whereas, a var' expecting subfolder
converts either case to a subfolder .

adda formatted strings

6.5: adda/formatted strings:
. the same escape code ( a "!" in [brackets] )
is used for adde's markup language for
indicating a command rather than literal text;
it could also work like a c`formatting string:
when the escape code is embedded in string
it means the following expression
is replaced with the eval of same
(the value returned by the eval'd expression
will have an image attribute that is applied
to merge the value with surrounding string).
. in the general case
(not knowing the string until run.time),
having escape codes embedded in strings
requires a virtual machine (like addm)
for run-time compile and eval's .
. the eval could be static by
doing it c's way:
limit the string var's syntax to
just being a param`index number, etc,
but that would be hard to read
like c's formatting string;
it can be both readable and not needing addm
if it limited the escaped expressions
to just symbols (ie, the names of active var's);
eg, "(var= [ ! ]var's-name)
. any subprogram that accepts a formatted string
also uses another parameter
for accepting its caller's environ pointer
which gives access to caller's locals .
. this environ is the same the symbol table
used by the compiler's parser:
as it finds a name in the source code,
it checks the environ else adds it;
and then replaces the string string name
with the symbol table index
until the entire string
is a tree of mini-pointers to symbols
(ie, symbol table indices).
. the symbol table then includes
stack offsets for finding the value at run.time .
. these same facilities can be used by
subprograms wanting a value given a var`name:
(namestring, environ) -> val .
. the efficient way to do that
(without such reflection facilities)
is to allow only constant literal strings
to be used as format strings;
then the compiler can replace it with
a concatenation expression;
eg, "(var = [ ! ]var`name)
becomes: "(var=) & var`image .

adda keeping things documented

6.4: adda/syntax decl's for freedom with readability:
. as much as possible, the code should be
readable from just knowing english and math;
that means,
that if you want to do strange things like
,() = (,,,)
then you need to describe that in a
syntax declaration, something at the top of
every source file [6.30:
though it could by pointer,
and then the code reader would auto'ly
expand all such pointers .]

6.20: adda/monkey patch vs freedom:
. after learning of "(monkey patch),
the place to be is full documentation:
if there are any ways that relinking
can modify a given package,
the state of current linkage needs to be
readable as part of the source,
so that linkage changes show up as
a modification to the package .