Dr.Barbara Liskov`the software crisis averted with modularity based on data abstraction

21.6.10:  7.12: news.adda/lang/

Dr.Barbara Liskov`the software crisis averted with modularity based on data abstraction:

. the key to programming in the large 

is modularity based on data abstraction

and that was not obvious before her work,

but now her work is mainstream, so it now seems obvious.

#robotics beware offensive autonomous weapons

8.1: news.cs/robotics/beware offensive autonomous weapons:
8.3: summary:
. we are getting close to robotics that are
so talented they may serve as soldiers;
but scientists warn we should avoid
such a robotics arms race.
. supercomputers will be used to design
robotic soldiers we don't fully understand.
. robots designed for mutually assured destruction
could cause the extinction of the human race
much more effectively than nuclear weapons.

democratized hardware design

9.17: news.adds/openware/democratized hardware design:
co.yt#ARMdevices.net Sep 15, 2014:
Google Project Ara Keynote: 
“What if hardware was more like software?
Google’s Project Ara and the
democratization of the hardware ecosystem.”

logging to help debug and user experience

4.24: adda/architecture/logging to help debug and user experience:
12.20: summary:
. logging of calls can tell you where an error occurs;
when errors are out of your control .

securing the virtual machine #addm

addm/sec/securing the virtual machine:
4.20: 12.20: summary:
. a virtual machine can provide security
along with some user freedom to develope code
by using a microkernel architecture
and learning from the Chrome browser team:
you can't build a secure app unless you
secure the underlying OS and firmware too .

design by autocrat

11.20:  addx/design by autocrat:
. what if you are deluded, and need others' help
to show you that your design is fundamentally flawed;
but you are drugged by the freedom of working alone?
well, I've wanted to reuse some other work
but I've often found the documentation unreadable,
so this is about building an understanding
through finding that my design was flawed .
. at least I will be on the step toward
both designing a program I like,
and writing some documentation I can read
(likely others can read it too,
because I'm very forgetful,
so I will need to be thorough).

MS`Portable Class Libraries

10.2: news.adda/MS`Portable Class Libraries:

Windows 8 programming model is like Xamarin,
[ where you write your apps entirely in C#,
and can run the resulting code on many platforms:
iOS, Android, Windows and Mac.]-12.8
. MS does this with Portable Class Libraries (PCL's)
which are subprograms that can be used anywhere
by attaching to whatever gui is on the current device
(supporting device-specific user interfaces).
PCLs let you wrap common code into a library,
ready for use by various gui's .

12.8: adda gui access is based on similar idea:

"( subprograms should not be controlling a gui interface,
rather they should be designed primarily for
use by other subprograms,
just as unix tools are meant to be .
(well, unix takes a shortcuts by
making its tool language a character string
that could also be readable by humans
but that was a security blunder because of parsing errors
that confuse critical datatypes like
{filenames, command names}).
. so, anyway, back to how humans fit in:
in order for a human to interact with
a subprogram that speaks type-tagged binary,
the human needs an agent who will
convert this robot speak into a gui .
. this agent is the Controller of the Model subprogram,
and it creates a View the human can interact with .)

10.2: 12.8:
. rather than having a subprogram export gui interactions
they are generating output like html
meant for describing what to display
but instead of being text a human can understand
it is encoded for quick access by other subprograms
including the human user's interaction agent
which builds a gui like html does .
. all native data types,
from which all other data types are composed,
have a gui presentation .
for instance, a tuple data type
is naturally represented as a dialog window,
and dialogs are then nestable like tuples can be .

extending the const-var architecture

3.31: addm/extending the const-var architecture:
. what I'm calling the constant-var architecture
makes use of hardware isolation mechanisms
by safely dividing the system into
constant code, and variable data .
. there is another segment to consider:
# write-once-read-many:
. what's variable across process instances
can be constant during execution .
# special permissions:
. the implied permission is that the
data in the process belongs to that process .
. the activation record's resource display
is an example of special permissions:
the process has permission only to
read the resource display, not write to it;
but the supervisor can modify it .

virtual machine for obj'c services

1.23: addm/
simulates obj'c when obj'c is not available:

managing self-modifying code

1.31: adda/managing self-modifying code:
(inspired by python unpickle vulnerability)
. the safe pickle is built by the system .
. it can be compared to the decompile,
how is it extensible? that is to ask
how are objects built in the first place?

#SOA -- where is #security today?

1.29: web.cs/soa/where is security today?:
. I was looking for updates on
how SOA is preventing malware;
instead, I find this claim about SOA's vulnerabilities:

"( Modern buffer overflows are more difficult to exploit
than Aleph One's
smashing the stack for fun and profit.
You should look into modern bypasses to ASLR
such as heap spraying or heap feng shui.
Attacks like jmp2reg (jmp2esp jump2ebx ect...)
are also interesting for bypasses for ASLR.
Attacking ActiveX components is fun.
I used H.D. Moore's AxMan with great success.
Here is the remote code execution exploit I found
using AxMan .
. Here are more exploits that I have written .
. The best fuzzer is by far PeachFuzz,
and writing some pit files for it
can be very fruitful research.
Buffer overflows and sql injection
are the most talked about,
but there are a couple hundred categories for vulnerabilities
and they are identified by CWE numbers.
Its worth exploring, I think it will surprise you
what NIST thinks a vulnerability is.

. that had nothing to do with SOA;
it should have been subtitled
"exploits you can avoid by using SOA" .

dream theory's tech perfection plan

12.8: adds/relig/dream theory's tech perfection plan:
. welcome to my wealth4all religion .
. you don't even have to believe in it to be saved by it,
but if everyone did believe,
it would save a lot of grief .
. the universe's god has no way of
not choosing bad experiences
(all that's left for the god to do
is making sure the bad comes first
and that some good experiences
are in power to keep civilization alive
during this boot camp of evolution);
but we've already seen
every sort of pain and inequality,
so, any time now,
we certainly could move on to
experiencing pure wealth4all .
. the emotional pains the god gives us
are put there just to provoke the wars
that promote evolution of the technology
that ensures eternal survival beyond sun death;
but, if everyone were to realize
god needs tech not war,
then we could simply volunteer
to put more money into tech .
. of course that would mean less money for
the popular entertaining games such as:
my family is bigger than your family ...
so, it's not a belief without costs;
but non-believers are welcomed to be
dragged through war:
wealth4all is optional
until tech is perfected .
. it's not like god is punishing us for
being uncooperative with the needed
Tech Perfection Plan
but if you did believe that,
the obvious data certainly could
support that view;
however, at this point in the
universe's experience distribution
war is no longer needed for
pinning the pains to the beginning;
but, war is still a backup plan
in case we don't feel it necessary to
divert population expansion funds
towards the Tech Perfection Plan .
. wealth4all: I'm a believer .

model-view and composite tech

[thought I blogged this 9.2, but found as draft;
then found a note that the reason it was draft
was a concern that the body was still too buggy;
nevertheless, the first part is still interesting .]
7.17: web.adda/architecture/MVC vs MVVM:
8.31: summary:
. in trying to find an explanation of
the essential differences between
all the variants of the MVC architecture,
it was most helpful to see Alexy Shelest's
2009`MVC vs MVP and MVVM:
first he reminds us of MVC's original definition:
"( the “Gang of Four” book
doesn't refer to MVC as a design pattern
but as a “set of classes to build a user interface”
that uses design patterns such as
Observer, Strategy, and Composite.
It also uses Factory Method and Decorator,
but the main MVC relationship is defined by
the Observer and Strategy patterns. ).
[8.31:
. then, Shelest had an interesting summary of
the historical evolution of MV-architectures, eg:

"( Potel questioned the need for the MVC's Controller;
He noticed that modern user interfaces
already provide most of the Controller functionality
within the View class,
and therefore the Controller seems a bit redundant.).

. after reading that, I wrote this:

. I didn't get mvc either, and promptly redefined it:
subrograms should not be controlling a gui interface,
rather they should be designed primarily for
use by other subprograms,
just as unix tools are meant to be .
(well, unix takes a shortcuts by
making its tool language a character string
that could also be readable by humans
but that was a security blunder because of parsing errors
that confuse critical datatypes like
{filenames, command names}).
. so, anyway, back to how humans fit in:
in order for a human to interact with
a subprogram that speaks type-tagged binary,
the human needs an agent who will
convert this robot speak into a gui .
. this agent is the Controller of the Model subprogram,
and it creates a View the human can interact with .]

API-GUI equivalence

8.7: adda/api-gui equivalence:
intro:
. every aspect of a subprogram's gui
should be mapping to some feature of
the subprogram's interface (API);
so how is the API specifying
an array of menus with submenus?
sometimes there is menu-izing naturally formed by
an app inheriting from a service type,
like the file menu is,
for apps that use the file system .
. a datatype's operations are going under the Edit.menu;
because, that's the general term for the current datatype .
. a View.menu would belong to the human's agent
that was providing various ways to
format the display of data; [11.11:
but, a subprogram's API might have multiple views too .]

explorations of virtual memory

7.1: adda/vmem'mgt/intro:

. in our virtual memory stack system
we are replacing each of the stack's
subprogram activation records (act'rec's)
with a pointer to a resizable object
(ie, it points to an expandable array in the heap );
thus, the stack [8.29:
-- if we didn't have a stackless architecture -- ]
becomes an array of pairs:
( return address
, pointer to act'rec
) . if our allotted ram is getting full,
we can file the obj's attached to earlier parts of the stack
or even file earlier segments of a very long stack .

dynamic linking to video driver

7.11: bk.addm/Gordon Letwin`Inside OS#2:
[8.13: intro:
os/2 was microsoft's next big thing for 1998,
but it never happened despite being
a leap forward in security .
. in this reading session,
I was wondering how it dealt with the issue of
unstable drivers . intro's dynamic linking .]
p89: the familiar static linking:
. the linker handles static links by
noting which symbols are marked external
and hooking those up with similar symbols
to be found in accompanying .obj files .
p29: the new dynamic linking:
. dynamic linking is how operating systems can be
extended or patched by the user or the apps;
just like hardware can accept new circuit boards .
p13:
. because drivers needed protected mode
there would need to be a mode transition
with every write to the display
but we avoid this by having apps
not access device drivers directly;
rather they do so through dynamic linking .
[. how does that explain it?
is dyna'linking facilitating our ability to
run device drivers in user mode?]
p109:
. dyna'linking to the video display driver
is possible because it doesn't require
hardware interrupts;
drivers are generally located in the kernel
only because some are needing access to
hardware interrupts .

architectures that prevent freezing #mac

5.9: sci.cyb/mac/architectures that prevent freezing:
to: cocoa-dev@lists.apple.com
. in a pre-emptive OS there should be no freezing;
given the new concurrency model
that includes the use of the graphics processor GPU
to do the system's non-graphics processing,
my current guess is that the freezes happen when
something goes wrong in the GPU,
and the CPU is just waiting forever .
. the CPU needs to have some way of getting control back,
and sending an exception message to
any of the processes that were affected by the hung-up GPU .
. could any of Apple's developers
correct this theory or comment on it ?

microsoft's changes for an insecure internet

5.18:  news.cyb/dev.net/silverlight killed by html5:
subodhnpushpak 2011:

Microsoft Silverlight is an application framework
for writing and running rich Internet applications,
with features and purposes similar to those of Adobe Flash.
. it's being replaced by html5 .

. the cool thing about it was
programming the browser with ironpython
-- or any other .net language --
instead of javascript .

6.10: why:

. the .NET system was designed for competing with java;
the cool idea at that time was mobile code;
but since internet security issues have gotten so thick,
the cool idea now is either minimizing code installs
by using web services with an html interface,
or controlling app quality
with something like Apple's App Store .

. now that html5 offers a multi-media experience,
the .NET and SilverLight plug-in's
are being discouraged as unnecessary installs .
. some apps may seem slow in a browser,
and those will need native code
that should be policed by an App Store .

. to replace .NET's multi-lang feature,
here's how your favorite languages
can be turned into javascript:
# for Python code:
. use Pyjamas, a port of Google Web Toolkit.
# for C, C++, Objective-C:
. use the Emscripten LLVM-to-JavaScript compiler
(eg, if your translator emits Objective-C,
then it can be converted to LLVM byte code,
which can finally be converted to JavaScript ).

. given the move away from mobile code,
Microsoft has evolved from .NET
and its new language C#,
back to a popular language, C++
with Component Extensions, C++/CX .
. its syntax borrows from C++/CLI
but targets native instead of managed code.
. Apple went through such a phase also
when it dropped their new lang, Dylan,
for the popular lang, C,
with oop extensions, Objective-C .
. the .NET developer framework was part of
WPF (Win' Presentation Foundation),
and Silverlight was a .NET-powered flash competitor;
the new platform using c++/CX
is called WinRT (Win' Run Time).

okL4 & Minix3 vs Xen #microkernel #security

4.23: web: okl4 vs xen:
what is okL4? a microkernel impl'ing a hypervisor:

A hypervisor is a virtual-machine monitor,
designed for running de-privileged “guest” OS's .
. it contains a kernel (ie, a part that is running in
the most privileged mode of the hardware).
A microkernel contains the minimal amount of code
needed to run in the most privileged mode of the hardware
in order to build arbitrary (yet secure) systems.
So the primary difference between the two is purpose,
and that has implications on structure and APIs.
By the microkernel's generality requirement,
it can be used to implement a hypervisor.
This is what OKL4 labs is doing .
In fact, the 1997 SOSP paper by Härtig et al
. the AIM benchmarks for L4-Linux
report a maximum throughput which is
only 5% lower than that of native Linux;
therefore, (well-designed) microkernels are
quite usable as hypervisors.

How about the other way round?
Can a hypervisor be used to implement a microkernel?
While a hypervisor is less powerful in the sense that
it doesn't have the generality of a microkernel,
it typically has a much larger TCB
(trusted computing base) .
It contains all the virtualization logic,
and all the physical device drivers .
For example, the Xen hypervisor itself is about
5–10 times the size of the OKL4 microkernel
(in kLOC [1000's of Lines Of Code]).
In addition, it has the privileged
special virtual machine “Dom0”,
which contains a complete Linux system,
all part of the TCB (which is therefore
of the order of a MLOC [1000 kLOC]).
Compare this 1000 kLOC hypervisor
to the OKL4's 15 kLOC TCB .
A small TCB is important for
safety, security and reliability
-- by enforcing the principle of least authority --
and as such, it's especially important to
mission-critical embedded systems.

4.23: web: minix and okL4 are complementary:

4.29: web.cyb/sec/minix vs okL4/minix on L4/
searching the minix3 group:
(vs the group for the old minix)
[minix3] Re: minix as hypervisors
jayesh     11/5/09

. we are thinking of using the
microkernel feature of minix
to implement hypervisors,
can anybody suggest on where to start ...

Tomas Hruby     11/5/09

Is your goal to make Minix kernel a hypervisor
or you want to use it without touching?
You would need to change the kernel a lot
to be able to use it as a hypervisor
as it is fairly tight with the userspace.
Changing the kernel (and userspace to be compliant)
so that the kernel could host another personality
would be very valuable work.
We already work on certain features
which should get us closer to a true hypervisor
although it is not our high priority.

Unlike in Linux where kernel is what makes difference,
in Minix the userspace system is what makes it unique.
In theory, there is not much difference between a
hypervisor and a micro-kernel,
Minix kernel would need substantial changes though.
As I mentioned, we sort of follow that direction.
It's going to take some time and effort.
The biggest obstacle I see is how the new VM works.
There is a strong coupling between the kernel and VM.
Right now you cannot have
multiple VM servers on top of Minix kernel,
therefore you cannot have multiple
independent virtual machines.
I can imagine a stage between the current state
and Minix-true-hypervisor
when the machines share the one VM server.
They would not be cleanly isolated though.
On the other hand, it would be a great improvement.

Possibly an interesting project would be to
port the Minix system (server, driver, etc,
except the kernel) to some variant of L4.
L4 is a high performing micro-kernel
used primarily for virtualization
that lacks a server-based system like Minix.
This would be an interesting, valuable
and much simpler task.
In contrast to Minix,
some L4 clones run on many architectures
which would make Minix OS
immediately available for them too.

Ben Leslie (ok-labs.com) Mon Jul 9 09:24:46 EST 2007

On Wed Jun 27, 2007 at 12:45:39 +0200, Martin Christian wrote:
>> As for Minix3, it grows fast since last year due to
>> good organization and open strategy
>> that attracts open-source programmers.
>> And I think maybe one day, it will become
>> more influential than L4 if things progress as now.
>That's a good point! I was also wondering
> what OKL4's understanding of Open Source is?
> More precisly these are my questions:
>1.) Why is OKL4 developed in a closed source repository?
> It would add much more confidence over
> OKLs commitment to Open Source
> if they used an open repository
> like the Linux kernel does.

Some of our work is subject to NDAs [non-disclosure agreements]
which have been signed with the relevant clients.
As such this work cannot be made public
and we cannot even 'talk around' the work being done.
We therefore made the conservative decision to
not make our repositories open to the public
but instead to release the publicly releasable code
in the form of a tarball that we can verify
contains nothing that would put us in breach of any NDA.
We take our customers' privacy concerns very seriously.
At the same time, we also want to keep our code open
for use by the community.
The solution we have arrived at allows us to
keep the released source very much 'open'.
...
We are quite happy with how Minix is going
and think there is enough room for both of us
out there in the development community.
In fact there have been thesis topics at UNSW
about reusing Minix component on top of OKL4[1].
It seems that there has also been other interest
in this in the past such as the L4/Minix project[2],
although this was based on Minix 2, not Minix 3.
Ben
[1] www.cse.unsw.edu.au ... KJE13.html
[2] http://research.nii.ac.jp/~kazuya/L4.Minix/

historical moment linux is announced #minix #microkernel #security

4.23: news.cyb/sec/minix/
historical moment linux is announced to minix list:
. Linus is asking comp.os.minix
what they would like to see featured
in his 86-specific rewrite of minix .
. the first thread ends with an ignored call to
not have to compile device drivers into the kernel .
. decades later I would find a youtube lecture
complaining that linux really needs to have
modular device drivers
so that you don't have to reinstall them
every time a kernel upgrade comes out .
Adam David     8/26/91

One of the things that really bugs me about minix
is the way device drivers have to be compiled into the kernel.
So, how about doing some sensible
installable device driver code
(same goes for minix 2.0 whenever).
(adamd@rhi.hi.is)

Samuel S. Paik     6/26/92

User Level Drivers!  User Level Drivers!
Why should device drivers per se
be part of the kernel? (Well, performance purposes...)
I've liked Domain OS where you could
map a device's registers into your process memory.
If you also include a way of bouncing interrupts
from a particular device to a process,
then we could have user level device drivers.
Then, for performance reasons,
after everything is debugged
there should be a way to move device drivers
into the kernel--but only if we want to...
Samuel Paik
d65y@vax5.cit.cornell.edu

Frans Meulenbroeks     6/29/92

Nice idea, but there are a lot of hardware configurations
where you cannot simply give [a process]
access to one device register .

microkernel vs a monolithic
may not be the real issue:
. much of what Linus objects to in minix
is not that it's a microkernel vs a monolithic;
rather, minix in 1992 was also
high-level coded to be portable,
whereas linux is tailor-fit to the 80386 cpu
using very-low level assembler code .
. making the best use of a particular processor
requires a lot of assembly language programming;
so, couldn't you have a tailor-fit microkernel ?

. the primary intent of the microkernel
is to take advantage of the fact that
a processor has 2 privilege levels;
and, your job as a microkernel designer
is to minimize the amount of code
that is running within supervisor mode;
such that even the drivers are in user mode .

. Linus stated "( porting things to linux
is generally /much/ easier
than porting them to minix );
now, I'm not sure of the particulars,
but it seems that this ease of porting
would come from the fact that most programs
are making full use of unix-usable C language,
and apparently the problem with minix was
requiring security-related modifications to source code
such that the minix-usable C
looked much different than unix-usable C .

. the key to popular security, it would seem,
is creating a special compiler that
transforms unix-C source code
into binaries that respect our boundaries .
. if the C code is also including assembler,
then only virtualization can save us .

the multi-threaded issue:
. when writing a unix file system
a monolithic os is naturally multithreaded
whereas the minix microkernel
required hacks of its message queues?

. perhaps this means that the filesystem
was considered to be one object,
such that if multiple processes want to access files,
they have to request jobs through one interface;
whereas, linux is not using one interface,
rather, each process is capable of locking a file
and so, any time a process is given some cpu time,
it has direct access to its file buffer
(in both cases the disk drive would be seen as one object;
because, it queues the disk access requests
and orders them to minimize disk arm movement;
then it buffers some files in ram,
and processes have instant access to their file's buffer).

Frans Meulenbroeks  and Linus debate multi-threading:
Linus>

I'd also suggest adding threading support:
the fs and mm processes need to be multithreaded
(or page faults etc are very difficult indeed to handle,
as a page-fault can happen in the fs process
and often needs the fs process to be handled).

Frans>

My thoughts about multithreading are mixed.
On the one side I like the performance gain.
On the other hand this complicates things,
so it does not really fit into the minix scope.

Linus Jun 26 1992>

Multi-threading isn't a question of performance:
you generally get better performance too,
but the most important part is that,
without multithreading, some things are
impossible or much more complicated .
I already mentioned demand-paging and virtual memory
that effectively /need/ multithreading,
but some other quite mundane things are simply
not possible to do without it.

The filesystem simply /has/ to be multithreaded
or you need a lot of ugly hacks.
Look at the tty code in the minix fs:
it's not exactly logical or nice.
As a tty request can take a long time,
minix has to do small ugly hacks
to free the fs process as fast as possible
so that it can do some other request while the tty is hanging.
It does a messy kind of message redirection,
but the redirection isn't a kernel primitive,
but an ugly hack to get this particular problem solved.

Not having multithreading
also results in the fact that the system tasks
cannot send each other messages freely:
you have to be very careful that there aren't
dead-locks where different system calls try to
send each other messages.  Ugly.
Having multithreaded system tasks
would make a lot of things cleaner
(I don't think user tasks need to multi-thread,
but if the kernel supports it for system tasks,
it might as well work for user tasks also).
...
[. a hacked single-process message-passing fs]
removes a lot of the good points of messages.
What's the idea in using minix as a teaching tool
if it does some fundamentally buggy things?

Frans Jun 29 1992>

Sorry, but I do not understand why I cannot get
paging or virtual memory without multithreaded systems.
Of course there are essential parts of the system
that must remain in memory permanently.
But why can't the core kernel do
demand paging or virtual memory
(or dispatch the work to another tasks).
What other mundane things are not possible??

I don't think multithreadedness is needed. Not even for fs.
What is needed is a message buffering ipc mechanism
and a version of fs which does not do a sendrec, but only send,
and which has a global receives
which gets all the result messages.
Then a single threaded fs does work.
--
Frans Meulenbroeks        (meulenbr@prl.philips.nl)
        Philips Research Laboratories

[5.14: TODO: back to the main point:
. I'm still wondering what it is about Linus's
definition of "(microkernel)
that precludes it having a high degree of parallelism
regardless of whether you're being multi-threaded
or using message queues .]

Linus Benedict Torvalds     1/29/92

>1. MICROKERNEL VS MONOLITHIC SYSTEM
True, linux is monolithic,
and I agree that microkernels are nicer. ... .
From a theoretical (and aesthetical) standpoint
linux loses.
If the GNU kernel had been ready last spring,
I'd not have bothered to even start my project:
the fact is that it wasn't and still isn't.
Linux wins heavily on points of being available now.

Ken Thompson     2/4/92
...

I would generally agree that microkernels are
probably the wave of the future.
However, it is in my opinion easier to
implement a monolithic kernel.
It is also easier for it to turn into a mess in a hurry
as it is modified.
                                Regards,
                                        Ken
-- "Rowe's Rule: The odds are five to six
that the light at the end of the tunnel
is the headlight of an oncoming train."       -- Paul Dickson

5.14:
. the new minix3 is much like
Linus was wishing minix 1 would have been;
plus, it's a microkernel .