Intel's lack of documentation stalled MINIX

11.21: news.cyb/adds/doc/Intel's lack of documentation stalled MINIX:
. in an IEEE Computer interview,
[ieeeComputerSociety 2014]
Tanenbaum revealed he designed MINIX when
AT&T closed the source for Unix Version 7
and they could no longer use it
when teaching about the design of an OS.
. but it kept crashing until he heard a rumor
(nothing in the documentation mentioned it)
that the Intel chip when overheating
was causing an interrupt 15.
If his student Robert hadn't revealed the interrupt,
"there would have been no MINIX" he said.

helenOS built from scratch #microkernel

5.13: web.cyb/sec/microkernels/
helenOS/built from scratch:
5.15: summary:
. why is the helenOS crew constructing
yet another microkernel, when okL4 is here?
it's an academic project .
. anyway, we need microkernels,
and good competition is a good thing!
. that may be why Google Summer of Code is
funding some of it -- along with genode .
. see helen's wiki
and design papers .

okL4 & Minix3 vs Xen #microkernel #security

4.23: web: okl4 vs xen:
what is okL4? a microkernel impl'ing a hypervisor:

A hypervisor is a virtual-machine monitor,
designed for running de-privileged “guest” OS's .
. it contains a kernel (ie, a part that is running in
the most privileged mode of the hardware).
A microkernel contains the minimal amount of code
needed to run in the most privileged mode of the hardware
in order to build arbitrary (yet secure) systems.
So the primary difference between the two is purpose,
and that has implications on structure and APIs.
By the microkernel's generality requirement,
it can be used to implement a hypervisor.
This is what OKL4 labs is doing .
In fact, the 1997 SOSP paper by Härtig et al
. the AIM benchmarks for L4-Linux
report a maximum throughput which is
only 5% lower than that of native Linux;
therefore, (well-designed) microkernels are
quite usable as hypervisors.

How about the other way round?
Can a hypervisor be used to implement a microkernel?
While a hypervisor is less powerful in the sense that
it doesn't have the generality of a microkernel,
it typically has a much larger TCB
(trusted computing base) .
It contains all the virtualization logic,
and all the physical device drivers .
For example, the Xen hypervisor itself is about
5–10 times the size of the OKL4 microkernel
(in kLOC [1000's of Lines Of Code]).
In addition, it has the privileged
special virtual machine “Dom0”,
which contains a complete Linux system,
all part of the TCB (which is therefore
of the order of a MLOC [1000 kLOC]).
Compare this 1000 kLOC hypervisor
to the OKL4's 15 kLOC TCB .
A small TCB is important for
safety, security and reliability
-- by enforcing the principle of least authority --
and as such, it's especially important to
mission-critical embedded systems.

4.23: web: minix and okL4 are complementary:

4.29: web.cyb/sec/minix vs okL4/minix on L4/
searching the minix3 group:
(vs the group for the old minix)
[minix3] Re: minix as hypervisors
jayesh     11/5/09

. we are thinking of using the
microkernel feature of minix
to implement hypervisors,
can anybody suggest on where to start ...

Tomas Hruby     11/5/09

Is your goal to make Minix kernel a hypervisor
or you want to use it without touching?
You would need to change the kernel a lot
to be able to use it as a hypervisor
as it is fairly tight with the userspace.
Changing the kernel (and userspace to be compliant)
so that the kernel could host another personality
would be very valuable work.
We already work on certain features
which should get us closer to a true hypervisor
although it is not our high priority.

Unlike in Linux where kernel is what makes difference,
in Minix the userspace system is what makes it unique.
In theory, there is not much difference between a
hypervisor and a micro-kernel,
Minix kernel would need substantial changes though.
As I mentioned, we sort of follow that direction.
It's going to take some time and effort.
The biggest obstacle I see is how the new VM works.
There is a strong coupling between the kernel and VM.
Right now you cannot have
multiple VM servers on top of Minix kernel,
therefore you cannot have multiple
independent virtual machines.
I can imagine a stage between the current state
and Minix-true-hypervisor
when the machines share the one VM server.
They would not be cleanly isolated though.
On the other hand, it would be a great improvement.

Possibly an interesting project would be to
port the Minix system (server, driver, etc,
except the kernel) to some variant of L4.
L4 is a high performing micro-kernel
used primarily for virtualization
that lacks a server-based system like Minix.
This would be an interesting, valuable
and much simpler task.
In contrast to Minix,
some L4 clones run on many architectures
which would make Minix OS
immediately available for them too.

Ben Leslie (ok-labs.com) Mon Jul 9 09:24:46 EST 2007

On Wed Jun 27, 2007 at 12:45:39 +0200, Martin Christian wrote:
>> As for Minix3, it grows fast since last year due to
>> good organization and open strategy
>> that attracts open-source programmers.
>> And I think maybe one day, it will become
>> more influential than L4 if things progress as now.
>That's a good point! I was also wondering
> what OKL4's understanding of Open Source is?
> More precisly these are my questions:
>1.) Why is OKL4 developed in a closed source repository?
> It would add much more confidence over
> OKLs commitment to Open Source
> if they used an open repository
> like the Linux kernel does.

Some of our work is subject to NDAs [non-disclosure agreements]
which have been signed with the relevant clients.
As such this work cannot be made public
and we cannot even 'talk around' the work being done.
We therefore made the conservative decision to
not make our repositories open to the public
but instead to release the publicly releasable code
in the form of a tarball that we can verify
contains nothing that would put us in breach of any NDA.
We take our customers' privacy concerns very seriously.
At the same time, we also want to keep our code open
for use by the community.
The solution we have arrived at allows us to
keep the released source very much 'open'.
...
We are quite happy with how Minix is going
and think there is enough room for both of us
out there in the development community.
In fact there have been thesis topics at UNSW
about reusing Minix component on top of OKL4[1].
It seems that there has also been other interest
in this in the past such as the L4/Minix project[2],
although this was based on Minix 2, not Minix 3.
Ben
[1] www.cse.unsw.edu.au ... KJE13.html
[2] http://research.nii.ac.jp/~kazuya/L4.Minix/

historical moment linux is announced #minix #microkernel #security

4.23: news.cyb/sec/minix/
historical moment linux is announced to minix list:
. Linus is asking comp.os.minix
what they would like to see featured
in his 86-specific rewrite of minix .
. the first thread ends with an ignored call to
not have to compile device drivers into the kernel .
. decades later I would find a youtube lecture
complaining that linux really needs to have
modular device drivers
so that you don't have to reinstall them
every time a kernel upgrade comes out .
Adam David     8/26/91

One of the things that really bugs me about minix
is the way device drivers have to be compiled into the kernel.
So, how about doing some sensible
installable device driver code
(same goes for minix 2.0 whenever).
(adamd@rhi.hi.is)

Samuel S. Paik     6/26/92

User Level Drivers!  User Level Drivers!
Why should device drivers per se
be part of the kernel? (Well, performance purposes...)
I've liked Domain OS where you could
map a device's registers into your process memory.
If you also include a way of bouncing interrupts
from a particular device to a process,
then we could have user level device drivers.
Then, for performance reasons,
after everything is debugged
there should be a way to move device drivers
into the kernel--but only if we want to...
Samuel Paik
d65y@vax5.cit.cornell.edu

Frans Meulenbroeks     6/29/92

Nice idea, but there are a lot of hardware configurations
where you cannot simply give [a process]
access to one device register .

microkernel vs a monolithic
may not be the real issue:
. much of what Linus objects to in minix
is not that it's a microkernel vs a monolithic;
rather, minix in 1992 was also
high-level coded to be portable,
whereas linux is tailor-fit to the 80386 cpu
using very-low level assembler code .
. making the best use of a particular processor
requires a lot of assembly language programming;
so, couldn't you have a tailor-fit microkernel ?

. the primary intent of the microkernel
is to take advantage of the fact that
a processor has 2 privilege levels;
and, your job as a microkernel designer
is to minimize the amount of code
that is running within supervisor mode;
such that even the drivers are in user mode .

. Linus stated "( porting things to linux
is generally /much/ easier
than porting them to minix );
now, I'm not sure of the particulars,
but it seems that this ease of porting
would come from the fact that most programs
are making full use of unix-usable C language,
and apparently the problem with minix was
requiring security-related modifications to source code
such that the minix-usable C
looked much different than unix-usable C .

. the key to popular security, it would seem,
is creating a special compiler that
transforms unix-C source code
into binaries that respect our boundaries .
. if the C code is also including assembler,
then only virtualization can save us .

the multi-threaded issue:
. when writing a unix file system
a monolithic os is naturally multithreaded
whereas the minix microkernel
required hacks of its message queues?

. perhaps this means that the filesystem
was considered to be one object,
such that if multiple processes want to access files,
they have to request jobs through one interface;
whereas, linux is not using one interface,
rather, each process is capable of locking a file
and so, any time a process is given some cpu time,
it has direct access to its file buffer
(in both cases the disk drive would be seen as one object;
because, it queues the disk access requests
and orders them to minimize disk arm movement;
then it buffers some files in ram,
and processes have instant access to their file's buffer).

Frans Meulenbroeks  and Linus debate multi-threading:
Linus>

I'd also suggest adding threading support:
the fs and mm processes need to be multithreaded
(or page faults etc are very difficult indeed to handle,
as a page-fault can happen in the fs process
and often needs the fs process to be handled).

Frans>

My thoughts about multithreading are mixed.
On the one side I like the performance gain.
On the other hand this complicates things,
so it does not really fit into the minix scope.

Linus Jun 26 1992>

Multi-threading isn't a question of performance:
you generally get better performance too,
but the most important part is that,
without multithreading, some things are
impossible or much more complicated .
I already mentioned demand-paging and virtual memory
that effectively /need/ multithreading,
but some other quite mundane things are simply
not possible to do without it.

The filesystem simply /has/ to be multithreaded
or you need a lot of ugly hacks.
Look at the tty code in the minix fs:
it's not exactly logical or nice.
As a tty request can take a long time,
minix has to do small ugly hacks
to free the fs process as fast as possible
so that it can do some other request while the tty is hanging.
It does a messy kind of message redirection,
but the redirection isn't a kernel primitive,
but an ugly hack to get this particular problem solved.

Not having multithreading
also results in the fact that the system tasks
cannot send each other messages freely:
you have to be very careful that there aren't
dead-locks where different system calls try to
send each other messages.  Ugly.
Having multithreaded system tasks
would make a lot of things cleaner
(I don't think user tasks need to multi-thread,
but if the kernel supports it for system tasks,
it might as well work for user tasks also).
...
[. a hacked single-process message-passing fs]
removes a lot of the good points of messages.
What's the idea in using minix as a teaching tool
if it does some fundamentally buggy things?

Frans Jun 29 1992>

Sorry, but I do not understand why I cannot get
paging or virtual memory without multithreaded systems.
Of course there are essential parts of the system
that must remain in memory permanently.
But why can't the core kernel do
demand paging or virtual memory
(or dispatch the work to another tasks).
What other mundane things are not possible??

I don't think multithreadedness is needed. Not even for fs.
What is needed is a message buffering ipc mechanism
and a version of fs which does not do a sendrec, but only send,
and which has a global receives
which gets all the result messages.
Then a single threaded fs does work.
--
Frans Meulenbroeks        (meulenbr@prl.philips.nl)
        Philips Research Laboratories

[5.14: TODO: back to the main point:
. I'm still wondering what it is about Linus's
definition of "(microkernel)
that precludes it having a high degree of parallelism
regardless of whether you're being multi-threaded
or using message queues .]

Linus Benedict Torvalds     1/29/92

>1. MICROKERNEL VS MONOLITHIC SYSTEM
True, linux is monolithic,
and I agree that microkernels are nicer. ... .
From a theoretical (and aesthetical) standpoint
linux loses.
If the GNU kernel had been ready last spring,
I'd not have bothered to even start my project:
the fact is that it wasn't and still isn't.
Linux wins heavily on points of being available now.

Ken Thompson     2/4/92
...

I would generally agree that microkernels are
probably the wave of the future.
However, it is in my opinion easier to
implement a monolithic kernel.
It is also easier for it to turn into a mess in a hurry
as it is modified.
                                Regards,
                                        Ken
-- "Rowe's Rule: The odds are five to six
that the light at the end of the tunnel
is the headlight of an oncoming train."       -- Paul Dickson

5.14:
. the new minix3 is much like
Linus was wishing minix 1 would have been;
plus, it's a microkernel .