golang's issue#9 -- clashes with go! -- revisited

10.7.30: news.adda/lang"go!/clash with go

. I was reminded of a name dispute
between Google and a language designer,
and couldn't remember the details;
I didn't know how it compared to
Microsoft denying Linux PC's
the name Lindows (mocking their name ?)
so I revisited the matter .

wiki's take needs an update:

"( On the day of the general release,
Francis McCabe, developer of the
Go! programming language,
requested a name change of Google's language
to prevent confusion with his language.)

. that is not quite accurate:
he claimed, point blank, that "(Go)
was exactly the name of his language,
and if you look on his personal filesharing website,
indeed, "(go) is the nickname;
but,
in both his whitepaper (pdf) and his book intro,
the published name is "(go!) not "(go):

Go!
– A Multi-paradigm Programming Language
for Implementing Multi-threaded Agents
K.L. Clark Dept. of Computing Imperial College London, UK
F.G. McCabe Fujistu Labs of America Sunnyvale CA, USA

. reading that paper,
you soon find why the name is "(go!)
-- and not go --
. the main purpose of go! is to
reverse some of prolog's faults;
. cleverly,
the reverse of "(go!) is (!og)
as in the "(log) of "(prolog)
which stands for (logic).

. the one thing he wishes most
would go away in prolog
is the "(!) operator (pronounced "(cut))
hence the name "(go!)
as in "(go away, Cut operator):

"( Go! has many features in common with Prolog,
particularly multi-threaded Prolog’s,
there are significant differences
related to transparency of code
and security.
Features of Prolog that mitigate against
transparency,
such as the infamous cut (!) primitive,
are absent from Go!.)

. meanwhile, the name of
google's go lang'
could be short for "(GOogle),
and is said to be a reference to
the design being motivated by
a need for compilers to go faster:

"( "In Google we spent so long
literally waiting for compilations,
even though we have parallelism
in all of these tools to help;
even incremental builds can be slow.
And we looked at this and realized
many of the reasons for that
are just fundamental to C and C++,
as were the tools that everybody used .
So we wanted to start from scratch .)

. wikipedia has this reference:

Google 'Go' Name Brings Accusations Of 'Evil'
InformationWeek Thomas Claburn November 11, 2009

"( McCabe's Go! programming language
is described in a 2007 book he published
and in a research paper published in 2004 (pdf)) .

McCabe:

"( It is in the tradition of
languages like Prolog.
In particular, my motivation was
bringing some of the discipline
of software engineering
to logic programming.)

. and *that*
is exactly the origin of the "(!)
-- an essential part of the name's genius --
so it's bewildering why he would
want to also claim the "(go) name
-- in addition to the "(go!) name .
some wikipedia authors speculate
that there could be some confusion,

"(McCabe requested a name change
to prevent confusion with his language, Go! .)

indeed, some have used "(yahoo!) as an example:

"( Comment 77 by wrinkles, Nov 11, 2009
Go and Go! are not the same? Great,
I just started a new company called Yahoo.)

you can't use "(Yahoo) as a noun
for something other than Yahoo!
without giving some context .
eg, hearing "(go!) might provoke
the question:
"( did you mean "(go faster),
or "(go away cut-operator) ?
) .]
. there's a link to to go's "(Issue 9) debate
where Frank McCabe wrote up an issue with Go:
"I have already used the name for
*MY* programming language."

all of McCabe's entries in Issue 9 to date:
"(

by fmccabe, Nov 10, 2009
I have been working on a programming language,
also called Go,
for the last 10 years.
There have been papers published on this
and I have a book.
I would appreciate it if
google changed the name of this language;
as I do not want to have to
change my language!

Comment 2 by fmccabe, Nov 10, 2009
If you google (sic) francis mccabe go
you will find some references.
I published the book on lulu.com

Comment 5 by fmccabe, Nov 10, 2009
My language is called Go!.
The book is called Let's Go!.
The issue is not whether or not Google's go
will be well known. It is one of fairness.

[7.31:
. if google throws a party for "(go)
they should invite "(go!),
to make sure in the future
people finding (go!) don't assume it's
some extension of google's (go) ? ]

Comment 300 by fmccabe, Nov 11, 2009
. I am very grateful for the
support I have received on this thread.
It seems to have hit a nerve.
. I want to make one particular point,
some people have suggested that
"I should be grateful"
for the extra advertising.
My response to that is that
I was not actively looking for this advertising.

[7.31:
. doesn't clarification require advertising
when there is such disagreement on
whether (go) clashes with (go!) ?
]

It was not me who picked a clashing name.
. I fully understand that it is possible that
insufficient search was done before hand.
However, when I picked the name Go!
I did try to find out if anyone else was using it.
In fact, I was kind of surprised that no one was!;
since it was clearly a great name.
. For those interested, Go! is a bi-lingual pun.

[7.30:
-- in Japan, where Prolog is very popular .
]

My previous work focused on a language called April.
In Japanese, the literal back-translation of April
is "4th Month".
Go is Japanese for 5.)

responses to McCabe's issue (verifying go! exists):

Comment 81 by yarkot1, Nov 11, 2009 [paraphrased]
'Go!' is available on sourceforge.net (networkagent),
and was developed jointly with McCabe and Clark
with commits back in 2000:
"A group of systems for building network-oriented intelligent agents,
consisting an agent communications infrastructure,
April - an agent construction programming language,
Go! - a logic programming language
and DialoX - an XML-based user interface engine".
[7.30:
. go! can be pronounced "(networkagent go)? ]

Comment 243 by andy.arvid, Nov 11, 2009
The Go! Source: homepage.mac.com
[redirects to nk11r10-homepage.mac.com]

april-9-30-07.tgz 2.3 MB
go-9-30-07.tgz 3.8 MB
InstallingGo.rtf 6 KB
ooio-9-30-07.tgz 821 KB

[7.30:
. see the problem here?
in some situ's (like unix) it's not wise*
to use the "(!) character in the name .
... could name it gol to look like (go!) ?
*: Characters you should not use in filenames:
| ; , ! @ # $ ( ) < > / \ " ' ` ~ { } [ ] = + & ^
. ]

Comment 454 by abraham.estrada, Nov 11, 2009
http://en.wikipedia.org/wiki/Go!_%28programming_language%29
[7.30:
. that link includes these resources:]

References
Clark, K.L.; McCabe, F.G. (2003).
"Go! for multi-threaded deliberative agents" .
International Conference on Autonomous Agents (AAMAS'03): 964–965.
Clark, K.L.; McCabe, F.G. (2006).
"Ontology oriented programming in go!" .
Applied Intelligence 24 (3): 189–204.
Further reading
Clark, K.L.; McCabe, F.G. (2003).
Ontology Oriented Programming in Go! .
Clark, K.L.; McCabe, F.G. (2004).
"Go!—A Multi-Paradigm Programming Language
for Implementing Multi-Threaded Agents" .
Annals of Mathematics and Artificial Intelligence 41 (2-4): 171–206. .

Comment 222 by keithsthompson, Nov 11, 2009
There's a Go! program on 99-bottles-of-beer.net;
it's been there since 2005.
That site is an excellent place to check for existing language names.

responses to McCabe's issue (resolution attempts):

Comment 6 by zhenshe41, Nov 10, 2009
In Go! , can the IDE know the differences between
Go! and go ?

Comment 40 by patla073, Nov 11, 2009
Why not just name it Golang?
Erlang - "Ericsson Language"
Golang - "Google Language"

Comment 64 by david.kitchen, Nov 11, 2009
@40 Golang looks like a winner...
they're already using the domain golang.org
and it looks like no-one else is using
that as a language name.

Comment 45 by tuxthelinuxdood, Nov 11, 2009 [paraphrased]
It is obvious that "do no evil" Google employees
did not research the name
in terms of existing languages before release.
[7.30:
. how is it obvious?
they could have easily seen C# vs C
as a precident for go vs go! .]

Comment 54 by pierrevm, Nov 11, 2009
First Closure (name-squatting Clojure)
now Go stopping Go! in its tracks.
Just another week in the life of a giant company. [...]

Comment 66 by yless42, Nov 11, 2009
[...]
. as you all rightly pointed out,
one is called "go"
and the other is called "Go!".
Am I the only person seeing the similarities
between that and "C" == "C#"?
If you think that having an extra character
is a problem,
you should go speak to Microsoft first.
[...]
. I suggest you see these lists:
wikipedia.org, esolangs.org .
. See how many languages there are on those lists
where one name is only separated from another name
by one character? [...]

Comment 70 by pygy79, Nov 11, 2009
@66 the spelling may be different,
but in both cases (Clojure and Go!),
the pronounciation of the
Google newly introduced products
is identical.
[7.30:
not identical in both cases:
. just as C# is pronounced c-sharp;
go! should be pronounced go-bang
or go-cut (prolog`cut-operator) .]

Comment 79 by rnmboon, Nov 11, 2009
They should change the name to "god".
Why hold back on the level of ambition here.

Comment 80 by j...@ww.com, Nov 11, 2009
Google should do the right thing
and change their name, be gracious about it.
. Do no evil, remember ?

Comment 89 by jamesda...@gmail.com, Nov 11, 2009
#77: Why not start a company called "Google!" ?

Comment 90 by AxelSanner, Nov 11, 2009
[wikipedia's don't be evil]
then ... don't
[7.30:
. that page tells about "(don't be evil) being
Google's "(informal corporate motto (or slogan));
the page states that in January 2010,

"(Apple CEO Steve Jobs strongly criticized the slogan,
saying: "We did not enter the search business.
They[Google] entered the phone business.
Make no mistake they want to kill the iPhone.)

-- and then he scoffs at "(don't be evil);
but, he might be just showing off
to shareholders . look:
. Google is in the phone business because
the phone is where much of the googling
could be coming from;
so, they just want to make sure
a mobile platform is out there .
. Apple still has a competitive product:
a phone running on the secure
mac microkernel with solid software
(I don't have a smartphone,
but my mac desktop is a lot more solid
than my linux laptop
-- the linux world scoffs at microkernels
for losing energy, but Ubuntu is more likely
to lose my data ... I need to
take the Android hint
and do more cloud computing!) .]

Comment 157 by mich...@sun-sol.com, Nov 11, 2009
They should name the language "Evil--"
(do no evil...).

Comment 161 by ropers, Nov 11, 2009
[...] appreciating the difference between
terms that merely sound
and terms that actually *are* identical. [...]

Comment 170 by tomhaste, Nov 11, 2009
Some of these comments are plain silly. Heres another to join in;
If Go isnt the same as Go!;
Then Google isnt the same as Google!.

Comment 212 by Mo6eeeB, Nov 11, 2009
[...]
"Go!" wouldn't work, because
the only way I can think
of audibly pronouncing the "!"
is "bang" and "Go bang"
just sounds silly.
[...]

Comment 219 by lozeno1982, Nov 11, 2009
To those who say "it's like C and C#"
or "it's like C and C++" etc...
No, it's not the same case.
. First, C# and C++ are called this way
because they inherited their {sintax, syntax}
from C
and wanted to express that kind of legacy;
Second, they are actually pronounced DIFFERENTLY:
it's "See" (C), "See-plus-plus" (C++)
and "See-Sharp" (C#).
. Now, how do you pronounce Go and Go! ?
I read them both the same way: "Go".
I can't read a "!" to make a difference.
[7.30:
. this reminds that the reason yahoo
spells their name with a "(!)
is to pronounce the word with enthusiasm;
literally to step up the octave
of the last syllable .]

Comment 255 by insomniac8400, Nov 11, 2009
Go and Go Bang
are as different as C and C Sharp.

Comment 274 by THM...@gmail.com, Nov 11, 2009
[...]
(C vs. C++, vs. C# anyone)).

Comment 312 by merkey88, Nov 11, 2009
This reminds me of an old court case
where Yahoo! had to change their name to the previously stated
from Yahoo (with no exclamation).
By contrast, your programming language
already has the exclamation point,
so as long as Google does not use
an exclamation after Go (or go),
there should be no reason to change.

Comment 382 by samterrell, Nov 11, 2009
[...]
If Wikipedia can deal with the issue,
[... GML, Go, Go!, GOAL, ...]

Comment 521 by darkhorn, Nov 12, 2009
[...] write a new language called "Google!".'
Comment 525 by ashish.afriend, Nov 12, 2009
@521: There are differences
in a language and a company.

Comment 563 by kikito, Nov 12, 2009
You can read the ! symbol in English.
It is commonly pronounced as "Bang".
"Go Bang" vs "Go"
is exactly the same difference as in
"C Sharp" vs "C", pronuntiation-wise.

Comment 584 by malonsosanchez, Nov 12, 2009
[...]
Go and Go! are very similar.

Comment 610 by davidsinger0, Nov 12, 2009
[...]
. you yourself said you used the term
"Let's Go!" for a book
which if you google is the name of a
very popular book series.
Are you then infringing on their rights?

Comment 654 by roman.go...@gmail.com, Nov 12, 2009
. Nothing wrong with google using Go,
just like nothing wrong with
C++ using the C,
nor using it in C# by Microsoft.
There is no confusion,
and there is no real difficulty in finding
help for those languages.
. I think this is a non-issue
and is just an excuse for people to
get into the town-hall mentality
of yelling out absurdities
and not looking at the facts.
[...]
. Go and Go! can co-exist,
even if !Go comes out.

Comment 741 by victor.petrov, Nov 12, 2009
[...]
For those of you who
brought the C -> C++ example:
your example isn't valid
in this situation.
C++ was actually a set of
extensions to the C language.
Google's Go is NOT
an extension to Go!,
nor is Go! an extension to Go.
[7.30:
. but the point is that
they are distinguishable names;
also, C# is not an extension
of either {C, C++},
and people thought that was a great name,
easily pronouncing it c-sharp;
just as we'll easily pronounce
go! as go-Cut .]

Comment 744 by coolboygreatone, Nov 12, 2009
@741: What about C++ and C#
P.S. : Google should not change the name

Comment 828 by julian.notfound, Nov 13, 2009
Forgot to put a link to slashdot discussion .

Comment 843 by davidsarah.hopwood, Nov 13, 2009
[7.30:
. one David-Sarah Hopwood
has been active in the E lang'
and cap'based security
where she's had many pro' conversations
with several google employees:
Ben Laurie [@] benl@google.com (2009)
Mike Stay [@] stay@google.com (2009)
Mark S. Miller [@] erights@google.com (2008...2010)
. ]

. "Go" is a really terrible name
for a programming language.
Besides the two languages,
it has two common meanings in English
(the verb and the name of the board game),
and
it also means "naked" in Croatian,

[7.30:
-- naked, as in barefoot-fast?
]

and "him" in Polish.

[7.30:
-- him, as in speed & strength vs longevity?
]

This many existing meanings
is what you would expect
for such a short word.
. Anyway, who chooses a name for
a programming language these days
without googling "FOO programming language"
to see if it's already taken?
That's impolite at best;
even downright negligent.

[7.30:
but why assume they didn't see "(go!) ?
would go# be ok in a C# world ?
.]
. The paper on the original Go! language,
incidentally, is at
http://www.doc.ic.ac.uk/~klc/annals.pdf
. It's a concurrent-logic dialect of Prolog
with asynchronous message passing
and Hindley-Milner type inference. [link]

Comment 878 by Lucretia...@yahoo.co.uk, Nov 14, 2009
[...] stop trying [to] make C safe
and use Ada instead,
[...] Just a thought, eh Google!?
[7.31:
. what?! Go is hardly a safer C;
it's basically a quicker Python .]
[7.30:
. like most dev'houses, google has to
reach for what the market will offer;
most hot-shot coders will not
touch that verbose language;
c is low-level, but has a huge library,
and your glue code is compact,
not verbose;
your tools are solid and familiar .]

Comment 891 by wrolufsen, Nov 15, 2009
My vote is for Go@, pronounced like 'goat'.

Comment 932 by graham.p...@gmail.com, Nov 16, 2009
Google probably already knew about
"Go!" (or "gobang" as it may be pronounced,
[...] So "Go" without the "!"
is like "C#" without the "#".
Also, I doubt you can trademark "Go"
so legally there's not leg to .

many suggested calling it plan9:

"( The name Plan 9 from Bell Labs
is a reference to the 1959
cult science fiction B-movie
Plan 9 from Outer Space.)

)

. issue 9 is still open .

. McCabe's old blog's about-page is taking comments .

review of other clashes with "(go):

. another name clashing with "(go)
is a different sort of "(language):
"(the cultural roadmap for the city girl)
... then for a lack of language there is
"(go) family news portal by Disney .

. finally, for some foreign language,
"(Go!) is the english version of weg!:
. an Afrikaans language outdoor and travel magazine;
focuses on affordable destinations in South Africa
and the rest of Africa.

supercomputer power promised in the CHaPeL (Cascade Hi'Productivity Lang)

7.26: news.adda/lang"Chapel(Cascade Hi'Productivity Lang'):

2005.9 PGAS Programming Models Conference/Chapel (pdf):

. the final session of the
Parallel Global Address Space(PGAS)
Programming Models Conference was
devoted to DARPA's HPCS program
(High Productivity Computing Systems):
Cascade[chapel]
, X10[verbose java]
, Fortress[greek]
, StarP[slow matlab].

Locality Control Through Domains:
[7.29:
. domains are array subscript objects,
specifying the size and shape of arrays,
the represent a set of subscripts;
and so, applying a domain to an array
selects a set of array elements .
(recall that term"domain is part of
function: domain-->codomain terminology)
. domains make it easy to work with
sparse arrays, hash tables, graphs,
and interior slices of arrays .]

. domains can be distributed across locales,
which generally correspond to CPUs in Chapel.
This gives Chapel its fundamentally
global character, as contrasted to
the process-centric nature of MPI or CAF,
for example.
When operations are performed on
an array whose domain is distributed,
any needed IPC is implicitly carried out,
-- (inter-processor communication) --
without the need for function calls.
Chapel provides much more generality
in the distribution of domains
than high performance Fortran, or UPC,
but, will not take the place of
the complex domain decomposition tools
required to distribute data for
optimum load balance and communication
in most practical parallel programs.

[7.29:
. HPC systems today are overwhelmingly
distributed memory systems,
and the applications tend to require
highly irregular communication
between the CPUs.
This means that good performance
depends on effective locality management,
which minimizes the IPC costs .
whereas,
productive levels of abstraction
will degrades performance.
Chapel simply allows mixing
a variety of abstraction levels .]

Cray's Chapel intro (pdf):

Chapel (Cascade High Productivity Language)
is Cray's programming language for
supercomputers, like the Cascade system;
part of the Cray Cascade project,
a participant in DARPA's HPCS program
(High Productivity Computing Systems ) .
influences:
. iterators:
CLU, Ruby, Python
. latent types:
ML, Scala, Matlab, Perl, Python, C#
. OOP, type safety:
Java, C#:
. generic programming/templates:
C++
. data parallelism, index sets, distributed arrays:
ZPL (Z-level Programming Language)
HPF (High-Performance Fortran),
. task parallelism, synchronization:
Cray's MTA's extensions to Fortran and C.
(Multi-Threaded Architecture)

Global-view vs Fragmented models:
. in Fragmented programming models,
the Programmer's point-of-view
is a single processor/thread;
. the PGAS lang" UPC (unified parallel C),
has a fragmented compute model
and a Global-View data model .
. the shared mem' sytems" OpenMP & pThreads
have a trivially Global View of everything .
. the HPCS lang's, including Chapel,
hava a Global View of everything .

too-low- & too-high-level abstractions
vs multiple levels of design:
. openMP, pthreads, MPI are low-level & difficult;
HPF, ZPL are high-level & inefficient .
Chapel has a mix of abstractions for:
# task scheduling levels:
. work stealing; suspendable tasks;
task pool; thread per task .
# lang' concept levels:
. data parallelism; distributions;
task parallelism;
base lang; locality control .
# mem'mgt levels:
. gc; region-based;
manual(malloc,free) .

. chapel`downloads for linux and mac .
readme for Chapel 1.1

The highlights of this release include:
parallel execution of all
data parallel operations
on arithmetic domains and arrays;
improved control over the
degree and granularity of parallelism
for data parallel language constructs;
feature-complete
Block and Cyclic distributions;
simplified constructor calls for
Block and Cyclic distributions;
support for assignments between,
and removal of indices from,
sparse domains;
more robust performance optimizations on
aligned arithmetic domains and arrays;
many programmability
and correctness improvements;
new example programs demonstrating
task parallel concepts and distributions;
wide-ranging improvements
to the content and organization of the
language specification.
This release of Chapel contains
stable support for the base language,
and for task and regular data parallelism
using one or multiple nodes.
Data parallel features
on irregular domains and arrays
are supported via a single-threaded,
single-node reference implementation.
impl' status:
No support for inheritance from
multiple or generic classes
Incomplete support for user-defined constructors
Incomplete support for sparse arrays and domains
Unchecked support for index types and sub-domains
No support for skyline arrays
No constant checking for domains, arrays, fields
Several internal memory leaks
Task Parallelism
No support for atomic statements
Memory consistency model is not guaranteed
Locality and Affinity
String assignment across locales is by reference
Data Parallelism
Promoted functions/operators do not preserve shape
User-defined reductions are undocumented and in flux
No partial scans or reductions
Some data parallel statements are serialized
Distributions and Layouts
Distributions are limited to Block and Cyclic
User-defined domain maps are undocumented and in flux

7.27 ... 28:
IJHPCA` High Productivity Languages and Models
(Internat' J. HPC App's, 2007, 21(3))

Diaconescua, Zima 2007`An Approach to Data Distributions in Chapel (pdf):
--. same paper is ref'd here:
Chapel Publications from Collaborators:
. they note:
"( This paper presents early exploratory work
in developing a philosophy and foundation for
Chapel's user-defined distributions ).

User-defined distributions
are first-class objects:
placed in a library,
passed to functions,
and reused in array declarations.
In the simplest case,
the specification of a
new distribution
can consist of just a few lines
of code to define mappings between
the global indices of a data structure
and memory;
in contrast, a sophisticated user
(or distribution writer)
can control the internal
representation and layout of data
to an almost arbitrary degree,
allowing even the expression of
auxiliary structures
typically used for distributed
sparse matrix data.
Specifically,
our distribution framework is designed to
support:
• The mapping of arbitrary data collections
to units of locality,
• the specification of user-defined mappings
exploiting knowledge of data structures
and their access patterns,
• the capability to control the
layout (allocation) of data
within units of locality,
• orthogonality between
distributions and algorithms,
• the uniform expression of computation
for both dense and sparse data structures,
• reusability and extensibility
of the data mapping machinery itself,
as well as of the common data mapping patterns
occurring in various application domains.

Our approach is the first
that addresses these issues
completely and consistently
at a high level of abstraction;
in contrast to the current programming paradigm
that explicitly manages data locality
and the related aspects of synchronization,
communication, and thread management
at a level close to what assembly programming
was for sequential languages.
The challenge is to allow the programmer
high-level control of data locality
based on the knowledge of the problem
without unnecessarily burdening the
expression of the algorithm
with low-level detail,
and achieving target code performance
similar to that of
manually parallelized programs.

Data locality is expressed via
first-class objects called distributions.
Distributions apply to collections
of indices represented by domains,
which determine how arrays
associated with a domain
are to be mapped and allocated across
abstract units of uniform memory access
called locales.
Chapel offers an open concept of distributions,
defined by a set of classes
which establish the interface between
the programmer and the compiler.
Components of distributions
are overridable by the user,
at different levels of abstraction,
with varying degrees of difficulty.
Well-known regular standard distributions
can be specified along with
arbitrary irregular distributions
using the same uniform framework.
There are no built-in distributions
in our approach.
Instead, the vision is that
Chapel will be an open source language,
with an open distribution interface,
which allows experts and non-experts
to design new distribution classes
and support the construction of
distribution libraries that can be
further reused, extended, and optimized.
Data parallel computations
are expressed via forall loops,
which concurrently iterate over domains.

. the class of PGAS languages
including Unified Parallel C (UPC)
provide a reasonable improvement
over lower-level communications with MPI.
. UPC`threads support block-cyclic
distributions of one-dimensional arrays
over a one-dimensional set of processors .
and a stylized upc forall loop
that supports an affinity expression
to map iterations to threads.

. the other DARPA`HPCS lang's
provide built-in distributions
as well as the possibility to create
new distributions from existing ones.
However,
they do not contain features
for specifying user-defined
distributions and layouts.
Furthermore,
X10’s locality rule requires
an explicit distinction between
local and remote accesses
to be made by the programmer
at the source language level.
The key differences between
existing work and our approach
can be summarized as follows.
First,
we provide a general oop framework
for the specification of
user-defined distributions,
integrated into an advanced
high-productivity parallel language.
Secondly,
our framework allows the
flexible formulation
of data distributions,
locale-internal data arrangements,
and associated control mechanisms
at a high level of abstraction,
tuned to the properties of
architectures and applications.
This ensures
target code performance
that is otherwise achievable only via
low-level control.

. Chapel Publications and Papers .
. Chapel Specification [current version (pdf)] .


Parallel Programmability and the Chapel Language (pdf)
bradc@cray.com, d.callahan@microsoft.com, zima@jpl.nasa.gov`
Int.J. High Performance Computing App's, 21(3) 2007

This paper serves as a good introduction
to Chapel's themes and main language concepts.

7.28: adda/concurrency/chapel/Common Component Architecture:
Common Component Architecture (CCA) Forum

2005 CCA Whitepaper (pdf):

. reusable scientific components
and the tools with which to use them.
In addition to developing
simple component examples
and hands-on exercises as part of
CCA tutorial materials,
we are growing a CCA toolkit of components
that is based on widely used software packages,
including:
ARMCI (one-sided messaging),
CUMULVS (visualization and parallel data redistribution),
CVODE (integrators), DRA (parallel I/O),
Epetra (sparse linear solvers),
Global Arrays (parallel programming),
GrACE (structured adaptive meshes),
netCDF and parallel netCDF (input/output),
TAO (optimization), TAU (performance measurement),
and TOPS (linear and nonlinear solvers).

Babel (inter-lang communication):

Babel is a compiler
that generates glue code from
SIDL interface descriptions.
(Scientific Interface Description Language)
SIDL features support for
complex numbers, structs,
and dynamic multidimensional arrays.
SIDL provides a modern oop model,
with automatic ref'counting
and resource (de)allocation.
-- even on top of traditional
procedural languages.
Code written in one language
can be called from any of the
supported languages.
Full support for
Remote Method Invocation (RMI)
allows for parallel distributed applications.

Babel focuses on high-performance
language interoperability
within a single address space;
It won a prestigious R&D 100 award in 2006
for "The world's most
rapid communication among
many languages in a single application."

. Babel currently fully supports
C, C++ Fortran, Python, and Java.

-- Chapel is coming soon:
CCA working with chapel 2009:
Babel migration path for chapel:

Collaboration Status: Active
TASCS Contact: Brad Chamberlain, Cray mailto:bradc@cray.com
Collaboration Summary:
Cray is developing a Chapel language binding
to the Babel interoperability tool.
The work is purely exploratory
(source is not publicly available yet)
and Babel is providing
whatever consulting and training services
needed to facilitate.

doc's:
Common Component Architecture Core Specification
Babel Manuals:
User's Guide (tar.gz)
Implement a Protocol for BabelRMI (pdf)
Understanding the CCA Specification Using Decaf (pdf)
CCA toolkit
CCA tut's directory
CCA Hands-On Guide 0.7.0 (tar.gz)

Our language interoperability tool, Babel,
makes CCA components interoperable
across languages and CCA frameworks.
Numerous studies have demonstrated
that the overheads of the CCA environment
are small and easily amortized
in typical scientific applications.

specification of components.
. using SIDL,
The current syntactic specification
can be extended to capture
more of the semantics
of component behavior.
For example,
increasing the expressiveness
of component specifications
(the “metadata” available about them)
makes it possible to catch
certain types of errors automatically
. we must leverage the
unique capabilities of
component technology
to inspire new CS research directions.
For example,
the CCA provides a dynamic model
for components,
allowing them to be
reconnected during execution.
This model allows an application to
monitor and adapt itself
by swapping components for others.
This approach, called
computational quality of service,
can benefit numerical, performance,
and other aspects of software.
Enhanced component specifications
can provide copious information
that parallel runtime environments
could exploit to provide
the utmost performance.
. the development and use of
new runtime environments
could be simplified by integrating them
with component frameworks.

PGAS replacing MPI in supercomputer lang's

7.26: news.adda/concurrency/PGAS (partitioned global address space):

. PGAS is a parallel programming model
(partitioned global address space) [7.29:
where each processor
has its own local memory
and the sharable portion of local space
can be reached from other processors
by pointer rather than by
the slower MPI (message passing interface) .
. since a shared space has an
"(affinity for a particular processor),
things can be arranged so that
local shares can be
accessed quicker than remote shares,
thereby "(exploiting
locality of reference).]

The PGAS model is the basis of
Unified Parallel C,
and all the lang's funded by DARPA`HPCS
(High Productivitiy Computing Systems)
{ Sun`Fortress, Cray`Chapel, IBM`X10 }.
[@] adda/concurrency/NUCC/lang's for supercomputer concurrency
The pgas model -- also known as
the distributed shared address space model,[7.29:
provides more of both performance
and ease-of-programming
than MPI (Message Passing Interface)
which uses function calls
to communicate across clustered processors .]

As in the shared-memory model,
one thread may directly read and write
memory allocated to another.
At the same time, [7.29:
the concept of local yet sharable]
is essential for performance .

7.26: news.adda/lang"upc (Unified Parallel C):
The UPC language evolved from experiences with
three other earlier languages
that proposed parallel extensions to ISO C 99:
AC, Split-C, and Parallel C Preprocessor (PCP). [7.29:

AC (Distributed Data Access):

. AC modifies C to support
a shared address space
with physically distributed memory.
. the nodes of a massively parallel processor
can access remote memory
without message passing.
AC provides support for distributed arrays
as well as pointers to distributed data.
Simple array references
and pointer dereferencing
are sufficient to generate
low-overhead remote reads and writes .

Split-C

. supports efficient access to a
global address space
on distributed memory multiprocessors.
It retains the "small language" character of C
and supports careful engineering
and optimization of programs
by providing a simple, predictable
cost model. -- in stark contrast to
languages that rely on extensive
compile-time transformations
to obtain performance on parallel machines.
Split-C programs do what the
programmer specifies;
the compiler takes care of
addressing and communication,
as well as code generation.
Thus, the ability to exploit
parallelism or locality
is not limited by the compiler's
recognition capability,
nor is there need to second guess
the compiler transformations
while optimizing the program.
The language provides a small set of
global access primitives
and parallel storage layout declarations.
These seem to capture
most of the useful elements
of shared memory, message passing,
and data parallel programming
in a common, familiar context.

Parallel C Preprocessor (pcp):

. a parallel extension of C for multiprocessors
(eg, scalable massively parallel machine, the BBN TC2000)
for sharing memory between processors.
The programming model is split-join
rather than fork-join.
Concurrency is exploited to use a
fixed number of processors more efficiently
rather than to exploit more processors
as in the fork-join model.
Team splitting, a mechanism to
split the processors into subteams
to handle parallel subtasks,
provides an efficient mechanism
for exploiting nested concurrency.
We have found the split-join model
to have an inherent
implementation advantage,
compared to the fork-join model,
when the number of processors becomes large .]

GCC Unified Parallel C (GCC UPC):

UPC 1.2 specification compliant, Based on GNU GCC 4.3.2
Fast bit packed shared pointer support
Configurable shared pointer representation
Pthreads support
GASP support, a performance tool interface
for Global Address Space Languages
Run-time support for UPC collectives
Support for uniprocessor
and symmetric multiprocessor systems
Support for UPC thread affinity
via linux scheduling affinity and NUMA package
Compatible with Berkeley UPC run-time version 2.8 and up
Support for many large scale machines and clusters
in conjunction with Berkeley UPC run-time
Binary packages for x86_64, ia64, x86, mips
Binary packages for Linux Fedora, SuSe, CentOS, Mac OS X, IRIX
. for uniprocessor and symmetric multiprocessor systems:
Intel x86_64 Linux (Fedora Core 11)
Intel ia64 (Itanium) Linux (SuSe SEL 11)
Intel x86 Linux (CentOS 5.3)
Intel x86 Apple Mac OS X (Leopard 10.5.7+ and Snow Leopard 10.6)

. GCC-UPC@HERMES.GWU.EDU archives .
. Programming in the pgas Model at SC2003 (pdf) .

Programming With the Distributed Shared-Memory Model at SC2001 (pdf):

. Recent developments have resulted in
viable distributed shared-memory languages
for a balance between ease-of-programming
and performance.
As in the shared-memory model,
programmers need not explicitly specify
data accesses.
Meanwhile, programmers can exploit data locality
using a model that enables the placement of data
close to the threads that process them,
to reduce remote memory accesses.
. fundamental concepts associated with
this programming model include
execution models, synchronization,
workload distribution,
and memory consistency.
We then introduce the syntax and semantics
of three parallel programming language
instances with growing interest:
Cray's CAF(Co-Array FORTRAN),
Berkeley's Titanium JAVA
and (IDA, LLNL, UCB) 1999` UPC (Unified Parallel C)

-- upc`history (pdf):

. IDA Center for Computing Sciences
University of California at Berkeley,
Lawrence Livermore National Lab,
... and the consortium refined the design:
Government: ARSC, IDA CSC, LBNL, LLNL, NSA, US DOD
Academia: GWU, MTU, UCB
Vendors: Compaq, CSC, Cray, Etnus, HP, IBM, Intrepid, SGI, Sun,

It will be shown through
experimental case studies
that optimized distributed shared memory
can be competitive with
message passing codes,
without significant departure from the
ease of programming
provided by the shared memory model .

. the openMP model is
all the threads on shared mem';
it doesn't allow locality exploitation;
modifying shared data
may require synchronization
(locks, semaphores) .
. in contrast,
the upc model is
distributed shared mem';
it differs from threads sharing mem' by
each thread has its own partition slice;
and within that slice, there's a
{shared, private} division:
slice#i has affinity to thread#i
-- that means, a thread tries to
keep most of its own obj's
within its own slice;
but, it's share.pointers can target
any other thread's sharable slice .
"(exploit locality of references) .
. message-passing as a
sharing mechanism
isn't a good fit for many app's in
math, science, and data mining .
. a dimension of {value, pointer} types is
{ shared -- can point in shared mem' .
, private -- points only to thread's private mem .
} -- both can access either {dynamic, static } mem .
. all scalar (non-array) shared objects
have affinity with thread#0 .
. pointers have a self and a target,
both of which can
optionally be in shared mem';
but it's unwise
to have a shared pointer
accessing a private value;
upc disallows casting of
private pointer to shared .
. casting of shared to private
is defined only if the shared pointer
has affinity with
the thread performing the cast,
since the cast doesn't preserve
the pointer`thread.identifier .

attributes of shared pointers:

. int upc_threadof(shared void *ptr);
-- thread that has affinity to pointer
int upc_phaseof(shared void *ptr);
-- pointer's index (pos' within the block)
void* upc_addrfield(shared void *ptr);
-- address of the targeted block

other upc-specific attributes:

upc_localsizeof(type-name or expression);
-- size of the local portion of a shared object.
upc_blocksizeof(type-name or expression);
-- the blocking factor associated with the argument.
upc_elemsizeof(type-name or expression);
-- size (in bytes) of the left-most type
that is not an array.

Berkeley UPC intro:

array` affinity granularities:
# cyclic (per element)
- successive elements of the array
have affinity with successive threads.
# blocked-cyclic (user-defined)
- the array is divided into
user-defined size blocks
and the blocks are cyclically distributed
among threads.
# blocked (run-time)
- each thread has affinity to a
tile of the array.
The size of the contiguous part
is determined in such a way
that the array is
"evenly" distributed among threads.

To define the interaction between
memory accesses to shared data,
UPC provides two user-controlled
consistency models { strict, relaxed }:
# "strict" model:
the program executes in a
Lamport`sequential consistency model .
This means that
it appears to all threads that the
strict references within the same thread
appear in the program order,
relative to all other accesses.
# "relaxed" model:
it appears to the issuing thread
that all shared references within the thread
appear in the program order.

The UPC execution model
is similar to the SIMD used by the
(Single Instruction Multiple Data)
message passing style (MPI or PVM).
-- an explicitly parallel model.
In UPC terms, the execution vehicle
for a program is called a thread.
The language defines a private variable
- MYTHREAD - to distinguish between
the threads of an UPC program.
The language does not define any
correspondence between
a UPC thread and its OS-level
counterparts,
nor does it define any
mapping to physical CPU's.
Because of this,
UPC threads can be implemented
either as full-fledged OS processes
or as threads (user or kernel level).
On a parallel system,
the UPC program running with shared data
will contain at least
one UPC thread per physical processor .

what are delegates? not oop!

7.25: web.adda/delegates:

replacing delegates with inner classes:

. Sun believes bound method references [delegates]
are unnecessary because
another design alternative, inner classes,
provides equal or superior functionality.
In particular,
inner classes fully support the requirements
of user-interface event handling,
Programs that use delegation patterns
are easily expressed with inner classes.
In the AWT event model,
user-interface components send
event notifications by invoking methods of
one or more delegate objects
that have registered themselves with the component.
A button is a very simple GUI component
that signals a single event when pressed.
Here is an example using the JButton class:

[. this is my conversion of Java to adda:]

SimpleExample.class extends JPanel:
  button.JButton = ("Hello, world")
  ;
  ButtonClick.class implements ActionListener:
  ( actionPerformed(e.ActionEvent).proc:
       println("Hello, world!")
  )  ...
  init(x):
    ( x`button`addActionListener(anon.x`ButtonClick)
    ; add(x`button)
    )
** vs delegates: **
SimpleExample.class extends JPanel:
  button.JButton = ("Hello, world")
  ;
  button_clicked(e.ActionEvent).void:
    println("Hello, world!")
  ...
  init(x):
    (x`button`addActionListener(
       anon.ActionDelegate(x`button_clicked`body))
    ; add(button)
    )

. in this case, the delegate is an enclosing object.
Keeping the identifiers the same
to make the remaining differences stand out clearly:
When the user clicks on the button,
it invokes the actionPerformed method
of its delegate,
which can be of any class that implements
the ActionListener interface.
In a language that uses
bound method references,[delegates]
the corresponding code
would be quite similar. Again,
a button delegates a click to an action method.
The key conclusion is quite simple:
Any use of [a delegate] to a private method
can be transformed,
in a simple and purely local way,
into an equivalent program
using a one-member private class.
Unlike the original,
the transformed program will compile under
any JDK 1.1-compliant Java compiler.

[. delegates] require the programmer to
flatten the code of the application
into a single class
and choose a new identifier for each action method,
whereas
the use of adapter objects
requires the programmer to nest the action method
in a subsidiary class
and choose a new identifier for the class name.

exceptions of Golang and Vala

7.17: news.adda/google'golang/design/exceptions:

. coupling exceptions to a
control structure,
as in the try-catch-finally idiom,
results in convoluted code.
. a routine that signals an exception
will then run a recovery routine
as it is closing for the return
-- sufficient to handle catastrophe
but requires no extra control structures
and, when used well,
can result in clean error-handling code.

7.25: news.adda/exceptions/Vala:

. vala has checked exceptions, not class-based, no wrapping
[. I changed the lexicon to match Ada's,
and much of the syntax to match adda's:]
. error domain with multiple error codes:
MyError.exception: { FOO, BAR }

. must be declared in method signature,
part of the contract:
method().proc raises MyError:
( raise FOO (message: "not enough foo") );

. must be handled or propagated:
[this is a declare block
(having an ex.label (exception)
defines a try-catch block)]
.(
method ();
ex:
e.MyError:
stderr(e`message);
)

. Although the compiler emits
warnings for ignored errors
it does not abort the compilation process.
This allows prototyping without proper error handling
and will hopefully prevent
forgotten empty catch blocks.

naming syntax struggles

7.18: sci.adda/type naming confused with file names:
. if type names and file paths
can be used in the same places,
then the type name"(/.T) (ptr to type"T)
could be confused with
a root folder containing
a file having a null name and ext"type .

7.21: adda/url syntax:
. my revised url syntax differed by
using dos drive letters as domain names,
C:\ was //C.drive/
and anything in the .drive domain
was a subdomain of .local;
eg, //mypc.local/E.drive
is a typical dos volume .
. while dos does auto-assign
labels to drives,
the software on dos can also
assign labels to a
subfolder on a drive .
. since std url's use the "file " protocol
to mean a local space,
could file be the name for {drives, subdir's}?
eg, //c.file/ ?
no:
file is an obvious protocol only within
the syntax of file:// .
. if you change the syntax or omit it,
the meaning naturally reverts to std english:
eg, c.file is a file!
. even if a label is pointing to a subdir,
app's still think it's a drive,
so stick with the use of .drive .
. whether unix or dos,
the machine can be reached by .local;
if dos and a drive is not specified
then c.drive is assumed to be root .

web.adda/golang/case-for-visibility rule:
7.17:
. an exported identifier must
begin with an upper-case letter .
7.25: web: case-for-visibility advantages
> In my brain:

> - lower case = Common, normal things. Normal words.
> - Upper Case = Unique, special things. Name of unique thing.
> - ALL UPPERCASE = Things need special care, a kind of warning.

"Rob 'Commander' Pike" Date: Sat, 9 Jan 2010:

. in English, capital words are "proper nouns".
They're important, public things
like names and places.
The analogy with
public and private names inside a package
is a bit of a stretch
but does make sense.
Public things are more important,
and so they're capital.
The ability to
glance at a name in a package
and know, without finding its declaration
(or some keyword somewhere near the declaration)
that it is a publicly visible name,
is a great thing
that more than makes up for
the inability to use some of the styles you mention.

adda's subfile support

7.17: adda/subfile support:
. adda can use adde ml to define subfiles
and can also treat any of html's
hierarchical structures as subfiles .
[7.27: intro:
. the idea of subfiles as chunks of text,
is to have an documents arranged like
folder trees of files
without depending on the platform's
native folder and file system .
. files usually contain several subfiles,
and in this way act as folders .
. conversely, a folder of files
may be identified as one document .
. how subfiles are packed into files
depends on how the subfiles are used:
documents that are frequently modified
will be packed into more files .
. very large doc's will be packed into
more folders .
. adde ml (markup language) is a
simplified form of html that uses
an exclamation point in brackets
as an escape for text about text,
such as identifying the title of a subfile .]

Golang's targeted platforms include mac

7.17: web.adda/google'golang/how many platforms targeted?:

golang.org:

"( we considered using LLVM for 6g
but we felt it was too large and slow
to meet our performance goals.
The Go tool chain is written in C.
. you need to have GCC, the standard C libraries,
the parser generator Bison, make, awk,
and the text editor ed installed.
On OS X, they can be installed as part of Xcode.
On Linux, use
$ sudo apt-get install bison gcc libc6-dev ed gawk make
Why doesn't Go run on Windows yet?
We understand that
a significant fraction of computers in the world
run Windows
and it would be great if those computers
could run Go programs.
A group of volunteers has made significant progress
toward porting Go to MinGW.
You can follow their progress
at the Go Wiki's WindowsPort page.) .

7.27:
. a tutorial shows xp safely sandboxed
in a mac {vmware, parallels} virtual machine .

D language

7.16: news.adda/lang"D:

. reasons to rewrite C to D?

. C is missing module support?
-- that feature helps figure out
where a function is defined;
but, cscope does that too .

. D was designed to remove
most of the flaws from C and C++,
while keeping the same syntax,
and even binary compatibility with C...
and almost binary compatibility with C++.
[. actually,
C syntax is a flaw too,
having decl's only a compiler could love .]

. D is also a mix of low-level
(x86 assembly is part of the language)
and higher-level characteristics
(optional gc, modules,
a fix of C++ templates, etc.).
. implementations include
front ends for LLVM and gcc .
. tango is the community replacement for
digitalmars`Phobos system library .
[7.27:
. unless there's something your lang needs
that simply can't be done in C,
the most simple and portable way
to enjoy working in your own lang,
is to just translate your lang to C .
. that route is esp'ly handy for
platforms like Apple's,
where you really need to be
using their tools .
. having translated to C,
I'm already half-way toward
translating for obj'c and cocoa .]

dependency versioning

7.10: adda/lib'mgt/dependency versioning:

[7.27: summary:
. these are rough ideas for
how to keep a library of software components
modifiable and evolvable,
without having problems with version conflicts .]

. intro to versions conflicting with portability:

"(
Forth was interesting in that
not only it was written in itself,
but all the internals were available
to any programs written in it,
and, furthermore, could be changed at will.
The thing is, Forth lacked abstraction.
Yes, you could change the compiler.
But, of course, you’d have to know
how the specific compiler worked:
eg, for threaded code, variants included:
generated {indirectly, directly, subrouting},
and generated {bytecode, machine coded}.
So it had no portability at all.
)

--. that inspired these ideas:

. in addition to there being
versions for systems;
any component that allows being changed,
must have it's own version number
for both the spec' or interface changes,
and the impl' or build changes .
. if the author (ie, the name of a fork)
is not specified with the version number;
then it's assumed to be from the main source .

. to be portable with forks,
the original site must provide
version numbers for
all components, and all forks .

. in order for a fork to use
the original project's name,
it must subscribe to the original's
versioning protocol:
a component lists its dependencies
which includes the release version
of a particular fork
modified with the release versions
for particular components .

. that means when a library or app
is introduced to a platform's lib mgt,
it gives the expected release versions:
this is described economically by saying
most components' versions are that of
some author's system release;
while a few components vary from that release,
and are described individually .
. the lib'mgt then checks these dependencies,
and tries to procure them if not present .

. the user should also be notified
when an installation has
many unique dependencies;
at least if the practical size
is significantly larger
than the app's stated size .

. each version requires its own
space in the library,
along with pointers to
whatever other units are using it;
so then, if you uninstall a unit,
all its bloat goes with it,
unless a dependency is still used
by other units .

adda's virtual preemptive multitasking

 7.4: adda/translate/tasklets:

. tasklet here means
a chunk of app that is considered by
the task scheduler to be
a unit of time slicing,
allowed to be run uninterrupted .
. it comes from Ada's "(task),
a thread of execution,
and is a way of translating c's
single-threaded model
into a multi-threaded one .
--
. definitions of tasklet by others
include stackless`threads
and linux:
"(Tasklets are a deferred-execution method
as a way for interrupt handlers to schedule work
to be done in the very near future.
a tasklet is a function(of data pointer)
to be called in a software interrupt
as soon as the kernel finishes running
an interrupt handler .
)[7.7:
. app's are calling to the system
only for system resources:
for things they need permission to share
like ipc, and file sharing .]
. if app's have to call back
to system for so much,
why chop them into tasklets?
. why can't apps simply
call task mgt periodically
and in every loop ?
#:
. also need to watch for
mutual recursion loops
where stack over-use
is more dangerous than time hogging .
#:
. periodic calls to system might work for
multi-tasking system work
like when an app gives the system's
controller-view obj's a chance to
interact with user;
but, for giving many concurrent app's
a fair share of time slices,
that requires tasklets .

. c has no concept of threads,
each with their own stack,
so c's one system stack must be reserved for
only the tasklet-running system;
the calls to apps are not c calls at all .

tasklets are modeled from asm:
. each thread's program
is an array of tasklets;
pc: index into array of tasklets;
sp: ptr to thread's stack;
task mgt calls are
(thread#pc)(sp) .

7.6: adda/tasklets/goto's:
. the tasklet plan is complicated by
the looping made by goto's?
just treat each loop's {entry, exit} points
as instructions,
and the blocks between those instr's
will be tasklets .
. this will put at least 2 yield points
in every loop .

adda/tasklets/declare.blocks:
. recall nested declare.blocks
can create new locals,
so at level#n,
the thread has a local's array
with n ptrs to [decl'block` act'rec]'s
(one for each nest level) .

7.7:
. threads do need their own artificial stack:
each call to another sub
is considered a tasklet
unless it's calling a sub that is so brief
that it hasn't been broken into tasklets .
[7.27:
. there's basically 2 kinds of calls:
app calls that run an algorithm;
and type'mgt calls that do
brief operations on their members .
app calls are kept on the thread's
artificial stack,
whereas, calls to brief operations
are allowed to run on the c`stack .]

. one way is already written:
"(. all c routines
take an arg of where to start .
. the routine exits by returning
what step it should continue at .
)
-- if that is written somewhere
it's a defunct writing:
.the better way is to replace all calls
with the system's style of calls .

. I'm worried that any arificial stack idea
will be no more efficient than
just doing vm code?

. every artificial call
becomes a stack & exit
but not every call
needs to be a separate tasklet .

. also, the tasklet idea is meant only for
the app's that want multi-threading;
so, tasklets can be as unresponsive
as app's can tolerate;
adda's insured gui response,
on the other hand,
comes from adda embedding code into app's,
so that all code is frequently
calling the system to handle real-time .

addx guarding

6.26: addx/IPC monitoring:
. how to assure that IPC ability is fair?
need to keep a rough count of each process's
use of message-sends,
if things are getting bogged down,
and someone is out-mailing a lot,
then need to alert user to the app
for ideas about why that app might need to out-mail so much .

6.29: addx/malloc/privacy assurance:
. when doing mem alloc,
privacy could be done with minimal erasing
if each sensitive domain
did its own recycling .
. also if the next domain is trusted (the os)
then erasing would not be needed,
suggesting that erasure not be done until
the block is reallocated .
. on most practical systems though,
addx will be a system on an untrusted system,
so then you'd need to erase as soon as possible,
and still not be sure of privacy!

much robotics can be done via simulation

6.24: addx/robotics/much can be done via simulation:

. when video is recognized,
ie, tells us what's happening in a world,
it's because we are
parsing a matrix of pixels
into a theatre of actors and props,
as we understand them to exist from experience .
. the list of actors and their positions
is what the robotic intelligence algorithms
can work with;
so, having a lot of practice with
programming this symbol-binding transform:
pixel matrix -> actor set configuration
is how a desk engineer can
contribute to robotics .

. the robot has a working model or world
like the database of a game
(a 3-D simulated world) .
. the main test needed
is to have a game that makes
vid's of your 3-D world,
and then see how fast and completely
your transform can recreate that 3-D world
from that vid .
. before thinking that is ok
we need to deal with how
real vid's are fuzzy and smeared .
. this poor quality of real-time vid
is why the algorithms need to find
the edges of fuzzy masses,
and identify joint points
among the shape's changes .
. this can't be done with
scanning one frame at a time,
there must be a quick way to show how
2 frames differ from eachother:
only movement can reveal movable masses
among the camoflage of noise .
. if the quality of the vid is very high,
then instead of having to
analyze an entire stream differential,
it could study just one snapshot .

. once the id is bound to a database obj,
then most analysis time can be spent on
reaffirming the actor's position and posture:
eg, which way is it facing,
what vector is the movement following, etc .

. just as a pixel mass is bound to an actor,
joint points are bound via outline of mass,
changes in joint points are easily tracked that way
and are key to identifying
potential and current actions .
. another analysis
-- that can be done concurrently --
is finding new objects .
. if the robot's scan of area
can't be persistent
or an obj's trail is lost
the actors have to be sortable by
spheres of possible location .

mvc and observer notifications

6.21: addx/mvc and observer notifications:

. if there is a std graphics api like opengl,
shouldn't scripted app's have access to it ?
[6.27: how to keep mvc separation?
the model is using graphics to
make data components;
then the view can still have control in display
(graphics can be thumb-nailed,
displayed in a box of the intended size
but with descriptive text inside, etc ) .]

. the core system needs to build from
mac's idea of encouraging
all app's to provide an interface
that other app's and scripts can use
for using the services of that app .
[6.27:
. if the model does have some
new way of letting the user generate
data that is based on graphics,
then in a strict mvc model,
the model programmer will need to
contribute to the controller's code
through a plug-in architecture .
eg, the basic adde will have only
the sort of gui dynamics needed for
generating text and pictures .
. these behaviors will be bound to
the relevant datatypes;
a model can extend adde to do games
by binding a plug-in to game types
that explains how to display interaction .
. the proof of mvc separation will be
that adde, or the user's chosen controller
can show robots or networked users
playing the same game .
(game architecture should be based on
layers of activity
so as to minimize redrawing
and instead rely on compositing;
when something affects a model
all it's observers (view controlers and robots)
need to be notified .) ]

mvc with client-server architecture

6.20: addx/mvc with client-server architecture:

. strict mvc means model lets
controller do the image writing to screen .
. if part of the model includes
generating images,
then these can be composited
by the controller
to wherever the chosen view is .

. the x server allows remote servers
to be treated similar to local servers .
. the addx server model is not
concerned with such networking;
the main point is mvc modularity:
. the servers providing an mvc`model
need to be working for mvc`controllers
not for the human users directly .
. the model can then be used either by
human views or robotic views;
most typically, the point is to provide
more choice for the human
by letting a robot filter what the
model is intending for the human,
similar to the way bodyguards work .
[6.26:
. this isn't meant to escape from
contracts like google's
where the ad's must be human-visible
and the inquiries human-initiated;
it simply means that all models
are capable of robot accessability .
. one way this can help while
fulfilling the google contract
is to have the robot help with
making wise keyword choices:
it presents a list of suggestions
and the human can merely press enter .
. robots can also be reading along
to help humans spot the best leads .
]
. in addition to that mvc architecture,
the model may want to divide itself
into server and client parts
in order to efficiently modify the model
from afar .
[my intuitive assumption:]
. the x window system works on the idea that
it would be most efficient if
the os could provide a generic client part
that many servers could use,
since servers often need to make similar moves .

api:
. don't I need to have a
carefully selected set of api's
in order to know how to do the
x thing (where the model has
separate server and client parts) ?
. one could take it
one example at a time:
cloud computing depends heavily on x:
. some things like controlling the cursor
can't be done quickly eno' by remote .
. if the model does some
major modifications
the data needs to be with the server;
that's where it originates from,
so no problem .
. letting the user quickly scan
large portions of data
means that the data should be
mirrored there too
unless the platform is tiny .

review of higher view:
. mvc works naturally with x,
since the Server = Model
and the Controller along with its
choice of View = Client .
. the View consists of
the Controllers way of arranging
the data in a form that is
optimal for viewing .
. remote does complicate things a bit:
the non-remote controller has
instant access to the model,
but now with the model so far away,
the controller is having to think about
how much to ask for in advance
depending on the size of the local system .


6.26: addx/x protocol/server client terminology:

The term server and client have different meanings.
XServer runs on your graphically rich desktop
(it serves up the view)
X11-client (like app is client of OS)
may run anywhere on the connected network

lwn.net:

The program which provides access to
system resources (display, keyboard, mouse)
is the server.
The shorter-lived applications
which connect to the server
to make use of these shared resources
are the clients.
This is perfectly consistent with
every other use of "(client/server) .
What throws people is that
we're used to thinking of servers as
a particular type of *hardware*,
steadily ticking away the CPU-hours
in some climate-controlled back room,
out of sight, whereas "clients"
are the visible workstation PCs
at everyone's desk.
However, the X client/server terminology
is perfectly reasonable
from a *software* point-of-view,
and X is software, not hardware.

NetVM and cap'based security

6.19: addx/security/vmnet and cap'based:
Joanna Rutkowska Jun 17, 2010 at 4:53 AM
qubes-devel@googlegroups.com

"(you might not have VT-d working,
in which case you would not benefit from
additional security provided by sandboxed NetVM, ...)

. how could this NetVM idea
merge with cap-based security?
. with very fast switching
every app could have virtual access to
every other component .
. of these connections, you could
watch or log them, stop them,
or undo them by allowing only
virtual access (not modifying the
original object, but some mirror)
. the reason we need L4 [hypervisor]
is to reuse untrusted wares
and to have cooperation among
the mutually distrusting .
. ok L4 is a great platform because
it can act as a microkernel too .
. need to see how that means the
face changes for app dev's
-- is it just good news for sys' eng's?
. even without having L4 as an os,
the process-isolating architectures
like seL4 and Microsoft's Singularity
can still be useful as part of an
app framework that can give app dev's
more confidence in their own wares .
[6.24: ie, the basic stack has been:
(basic OS providing common lib's,
between hardware, and apps)
and then in addition to the basic OS,
there could be a security layer
provided by an app framework OS .]
[6.30: L4 vs seL4:
seL4 adds cap'based security to L4 .]

6.20: addx/seeing the light:
[6.30: todo: prove this!]
. it can seem rather pointless
trying to develope a secure app
on an insecure OS;
but for personal use, at least,
there can be security if
all the apps use that framework .
. even if you can't find hardware
that is designed for okL4 (open seL4)
it could help to have an app framework
that is based on an seL4 .

addx/security/what's needed on secure platform:
6.21:
. after seeing this googlecode blog
I was thinking about how to include
cap'based security .
6.27:
. chrome os already has good
separation between app's,
but what is that allows a browser
to rifle through your files
after hitting the go-away button?
has chrome browser really fixed that?
[6.30:
I've not had this problem on Chrome yet;
but what is "(sandboxed) mean
if it's so easy to steal my permission? ]
. if the guard is finally here,
then the only needed
cap'based permissions would be
the design of an app like addx
that is a scripting environment .

viewing changing amounts

6.11: adde/mem display:

. the system should be showing

how much mem it's using;

but, if the mem is changing rapidly,

then then display will be unreadable:

in that case,

the minimal display has both the amount,

and the rate of change;

so, then if you see the rate of change

is non-zero

then you know the amount will change soon,

it is only frozen temporarily for readability .

library mgt integrated with torrent client

6.1: adde/library mgt integrated with torrent client:
. I had this idea that all components
should be torrented by default
with the torrent client built into the apt-get service
but a problem with torrents generally,
is that when getting service from an isp,
the isp is a bottleneck;
because,
you're a subnode of the isp;
the isp can efficiently flow only when
not everyone is streaming at once;
whereas, torrenting expects the opposite:
if any streaming occurs,
all nodes should be streaming until the job is done .
. nevertheless, if it's good eno' for large downloads,
it should also be helpful with
masses of small downloads, [right?]
. when you open the torrent client,
it's listing all the modular packages
you got from the debian library,
having them in original form .
todo:
. do the packages get modified during installation?
if so, this would double the amount space
needed for this to work .

. another problem is that with smaller loads,
it may not be easy to convince people to give
when they're not getting as much .
. I got the idea from
hearing that skype is a torrent;
when you sign in to take calls,
you are also volunteering to torrent calls .
. the apt-get client could at least torrent
for the time it was downloading .
. the client could explain
this is one way you could help with
paying for the cost of
streaming packages and updates .
. there could be an adjustment for
stopping when the fan got noisy, etc .

journaling . elder care helps dev's too

6.1: adde/journaling/elder care helps dev's too:

. so many computer problems and clues;

trouble trusting my mem now ...

. computers for elders definitely aren't here yet!

we need a modular architecture

where no app does anything

-- inputting our outputting --

without a log recording it .

. you would think this is

the first thing designers would do;

they are always fielding calls like:

"( I don't know what I did .

how can reproduce the problem?)

and always giving advice like:

"( please be aware

of every little keystroke you make;

so that you can recreate the problem,

noting (eg, logging)

what conditions are triggering the behavior

-- the platform, its config', and your input).

gooey gui portability

6.28: pos.adda/dev.gui/stay with the mainstream tools:

. sdl (simple directmedia layer)

reminds me that being low level
could get you into trouble;
the original adda plan
to simply write a native code compiler
for each system, might be
not only cleaner, but also safer,
since a platform's own code and tools
are likely to take better care of you
if not with a safer language,
then at least with documentation
that warns you where the landmines are .

. a browse through sdl-style portability
points out that c is not multi-core .
. the goal of portability
-- as in maximal c coding --
could interfere with some
very smart c concurrency extensions .
. if concurrency should be pervasive,
your code will be riddled with non-std c
(don't even bother trying for portable).

[6.30: but the mvc pattern could help with

turning concurrency into a module? ]

adda exceptions performance

6.27: web.adda/exceptions/performance:
. I'm wondering how c++ exceptions
were inherently slowing c programs:
the addx vision of exceptions
is that there is always a scheduler
pre-empting functions for multitasking;
and, this scheduler is
gathering all the exceptions in a library;
... but it's not just routines
that handle exceptions:
every sub.block with a routine
can provide a unique handler for an exception;
and, these "(catch blocks) can be nested .
impl' idea#1:
. catch.blocks need to push their address
onto an exception stack;
then after an exception,
the scheduler can look on this exception stack
to the addresses of handlers .
. a program that has no handlers
would have an exception stack with
only one item:
the system's set of handlers .

web: Exceptions performance penalty:

In C++ Programming Language 3rd edition section 14.8
Stroustrup writes that it's possible to
implement exception handling
in such a way that there is
no run time overhead
when no exception is thrown
-- but it's not easy .
There is generally some overhead
as we have to keep track of the local objects
whose constructors have run,
so when exception is thrown
their destructors are called.
see Scott Meyer's "More effective C++" .

. g++ creates exception tables.
From the value of program counter
you can know which function/scope you're in.
For each scope you then know
which destructors have to be called
and using the stack address
you delete the apropriate objects.

We suppose that you can get
{ __builtin_return_address (LEVEL)
, __builtin_frame_address (LEVEL)
} for the ESP and EPC
of the caller of level LEVEL.

plain old c ... minus

6.25: pos.adda/plain old c ... minus:
[6.30:
. google's native-code sandboxes
in Android and in Chromium OS
are an important platform,
and should be considered before
deciding what "std" C is .] [6.28:
. many platforms will restrict the std libraries
in order to further enhance security
or adapt to smaller, energy-wise situations .]

. keep in mind what's available in
the android ndk:
libc (the C library), -- as Bionic, not glibc.
JNI interface, -- controller-view of your mvc;
OpenGL ES {1.1, 2.0} (3D graphics)
libm (Math),
libz (Zlib compression)
liblog (Android logging)
libjnigraphics (JNI Pixel buffer access)

. the API is included with the sdk;
so, get that installed on ubuntu

(eclipse runs better there than mac?) .

details of android's custom libc:

. customizing libc (as Bionic)
was necessary for three reasons:
* License:
keep GPL out of user-space.
Bionic code uses the BSD license.
* Size:
Bionic is half the size of glibc
* Speed:
custom pthread implementation.
Bionic has built-in support for
important Android-specific services
such as system properties and logging.

It doesn't support certain POSIX features,
like wide chars (wchar_t)
-- The world has moved on to Unicode
with its various fixed and variable
width encodings,
which the wide character type
is not particularly useful for.

. use ICU for internationalization support,
(International Components for Unicode)
instead of LOCALE .
ICU is a mature, widely used set of
C/C++ and Java libraries providing
Unicode and Globalization support
for software applications.
ICU is widely portable
and gives applications the same results on all platforms
and between C/C++ and Java software.
threads
The bionic C library has it's own thread API,
not the same as either original LinuxThreads or NPTL.
It implements only support for Dalvik JVM,

Mutexes, rwlocks, condvars, etc
are all implemented using kernel futexes,
-- see Ulrich Drepper's futex whitepaper.
There is no pthread_cancel().
Threads can exit,
but can not be killed by another thread.
There is no pthread_atfork().
Thread local storage is implemented,
with up to 64 keys handled.
Android reserves several of these for its own use:
the per-thread id and errno,
as well as two variables related to OpenGL
whose function I do not understand.

stdio, stdlib, string, unistd:
. removed the LOCALE support from strtod()
(i.e., is the decimal point a period or a comma?
In the Bionic library it is always a period).

{openlog(), syslog()} are replaced with
__libc_android_log_print()

Bionic uses Doug Lea's malloc, dlmalloc.
Bionic also provides a hash table
to track allocations
looking for leaks, in malloc_leak.c.

no asynchronous AIO routines
aio_read() or aio_write().

no crypt()
-- uses OpenSSL for that .
. dispenses with most file-based Unix administration.
no getfsent,
because no /etc/fstab.
. there is a /var/run/utmp,
and so there is a getutent()
Android implements its own account management,
and does not use /etc/passwd.

. no getpwent(),
and getpwnam()/getpwuid() are implemented as
wrappers around an Android ID service.
At present, the Android ID service consists of
25 hard-coded accounts in android_filesystem_config.h

. no termios support (good riddance).

Bionic is a BSD-based libc with support for
Linux system calls and interfaces.
If limitations prove untenable,
the syscall and pthread mutex implementation
could be repurposed into the heavier
FreeBSD/NetBSD/OpenBSD libc,
though handling thread cancellation
while using the Bionic mutexes
could require additional work.

oop theories, designs, implementation notes

6.1: adda/oop/based on vari-sized records:

. the 3 sorts of inheritance:
# subtyping:
. inheritors are assignment compatible .
# symbolic description:
. describes one type in terms of
another (incompatible) type .
# generics:
. an object's function is selected by
the function's name (or hash of that)
-- rather than referring to methods by
index, pointer, or offset .
-- this is the obj'c meaning of inheritance; [6.7:
obj'c also supports subtyping . 6.30:
... actually,
any system that supports generics,
implicitely supports subtyping .]
. after review oop ideas,
I wondered if the the trailer idea
would be useful for oop subclassing .

. parameters that vary in size
are accommodated by replacing value with
a mini-pointer (an index into
an associated trailer array)
that then leads to the value .
. a vari-sized record is organized as
a head (the mini-pointers)
and body (the trailer).
. in the vari-length record,
the number of fields is fixed,
whereas in multi-inheriting oop,
not only does the body grow
but the head too is vari-sized .
. provide a mini-filesystem (fs)...
[6.30:
. each class expects to find
it's own instance variables
arranged in the usual way
even when it's one of many ancestors
of the obj's class .
. in a mini-fs, the directory
doesn't have to be on the obj;
just containing a type.tag,
will be eno', since that is a
key to using the lib mgr's map:
(instance's class, ancestor.class) ->
offset into instance .
-- this map shows what offset to use
into the instance
to get to an inherited class's
embedded instance record .]

6.30: todo.adda/oop/obj'c dynamic linking:
. when there is multi-inheritance,
you have an instance variable where
there are separate sections
for each subclass's privates .
. it would be efficient if type-mgr is
passed ptr's to the operated obj's;
and generally,
how is each type-mgr supervised
so as to insure that they
don't affect any surrounding bits?
. you trust your compiler,
but then in obj'c,
you can relink modules dynamically;
and, the new module might be
code from another compiler .
. need to review obj'c
for how to best use,
and not be used by, that subsystem .

6.5: adda/oop/how does c do obj'c dynamic bindings:
. the generic-oop functionality of obj'c
comes from being able to launch a subrogram
when given a function`name and a type.tag;
this could be done in unix by calling to the os
to execute a shell command;
. without that ability,
each addition to the class library
would require editing the obj'c run-time engine
and re-linking it with the expanded class library .

6.7: adda/oop/subclasses as subtypes:
. what is a new type, vs a subtype, in adda?
subtypes can't remove or ignore
inherited functions,
but new types can do a reuse but remove;
how does Ada do this ?
inheritors are new types (meaning it's
not assignment-compatible
with the inherited type);
if you want a variable to have subtype polymorphism
(ie, be assignment-compatible with
all inheritors of Baseclass),
then you need to declare the var's type to be
Baseclass'class .

6.23: pos.adda/obj'c and cocoa bindings:
. the reason to not interface cocoa
is that it's very inefficient with its
ref'counting and its (typical) way
of implementing oop .
. to the extent there does need to be
a binding to it (for the sake of
getting the usual services)
the binding will not be direct:
the mac gives all the power to the developer,
whereas addx will give the user
most of the gui design .
. for example, in nu,
"( locally-scoped assignments are
instances of NSMutableDictionary; )
this might look like an
excellent chance to reuse,
but you might be using thousands of those,
which means juggling a lot of within
both your subsystem's heap
and the mac's system heap .
. it might be ok for prototyping,
but eventually, you want basic services
to be based on your own mem'mgt .