safer C coding

9: news.cyb/dev.c/avoiding remote code execution:
leafsr:

. some C library functions are often used incorrectly,
and that consequently result in
remote code execution vulnerabilities:
strcpy, sprintf, memcpy, CopyMemory,
RtlCopyMemory, strcat, alloca
and many more Win32 specific stuff.
. much of the legacy software that is
still critical to many enterprises
contains code that calls these vulnerable library functions.
Despite modern memory protections
like ASLR and DEP
the vulnerabilities these functions introduce
are still exploitable under the right conditions .

 . here's the safe C library
– that implements strcpy_s and friends
as an open source library (MIT license).

19: news.cyb/dev.c/SAFECode updates secure dev guide:
.  pubs from safecode.org .
scmagazine:

The Software Assurance Forum for
Excellence in Code (SAFECode),
a nonprofit seeking to advance software assurance,
released an updated guidance document .
The free report includes verification methods and tools
that can be used to confirm whether
development teams have followed prescribed practices.

the standardized subtype.tag

12.8: adda/oop/the standardized subtype.tag:
27: summary:
. the most important feature of object orientation
is the support of polymorphism,
such that a "supertype" can declare variant formats
known as "subtypes" and know the current variant
by the value of a subtype.tag .
. if a type mgt supports subtyping,
then it also supports an object format
that puts the subtype.tag in a standard location,
so that it can be co-managed
by both the type mgt and the system run-time .
. subtypes codify not only the format variant
but also whether the object is constant,
and whether it is a value vs pointer vs function .

in-mode param use cases

12.8: adda/oop/in-mode param use cases:
8.731: 9.812:
. in-mode parameters that are offering values
(vs pointers) don't need an is-constant bit,
since they are copies that the callee can operate on;
in-mode pointers could use an is-constant bit
for describing whether the target can be modified?
the compiler can enforce constancy
by ensuring that the in-mode pointer's components
are passed only to other in-mode param's .
8.748:
. the primary reason for
in-mode param's being pointers
is when the callee expects the value to be
a large data structure,
and is agreeing to treat the pointer's target
as read-only rather than a writable copy .
. but another reason for an in-mode pointer
is that the pointer's target
is under the control of a different process
that will be changing the value in real time .
. so the formal in-mode pointer
needs to declare whether it honors read-only;
if it doesn't and an actual parameter is read-only
there could be a copy-on-write mechanism
where if the callee's code branched to
a place where it wanted to modify the target,
then the pointer was replaced to point to
a local copy of the target .
9.817: conclusion:
. an actual param has several cases:
# is a value (not a pointer):
. the value is copied,
and the callee's local version is modifiable .
# is a ptr real-time modified:
. the callee is getting a newsfeed
so there is no local version to modify
and it is considered read-only by callee .
# is a ptr not real-time modified:
. if the callee decides to modify the local version
one is created on demand (copy ptr target on modify).

subtypes of values vs addresses #terminology

12.10: mis.adda/oop/terminology/subtypes of values vs addresses:
. I'm misusing the term "subtype"?
let's review,
2010/04/polymorphic-vs-evolutionary-subtyping.html
2012/02/def-supertype-supratype-surtype.html
2012/10/infratypes-of-surtypes-phyla-of-taxa.html

"you mean that N is a subset of Z,
but then that implies Z is a subtype of N?
that depends on the direction:
you can represent any N value with a Z,
but if I need the address to an N,
a Z is no substitute because it
doesn't enforce my range constraint ."

so, "subtype" is the right word sometimes .

# for passing a value in:
. if the actual is a subrange of the formal,
then it will fit, and the actual is
serving both types, and that's what a subtype is .
. if the actual is a
constant version of the formal's type,
no changes are needed so input is accepted,
and constant T is a subtype of T here .

# for passing a value out:
. if the actual is a subrange of the formal,
the formal is saying
"I might send out a large number"
and the actual is saying
"I can't handle large numbers";
therefore access to subrange of T
is not a subtype of access to T .
. if the actual is
a constant version of the formal
the constant fails to accept any of the output;
therefore access to constant T
is not a subtype of access to T .

. after reading notes on terminology
I decided on yet another revision to that:
"surtype" is the system's typeclass
(pointer, valuetype, subprogram, aggregate, ...);
because, a surtype can classify
every object found in a programming environment .
. a polymorphic typeclass like number.type
is an example of a "polymorphtype" .
. the types {N,Z,Q,C,R}
are all the "morphtypes" of number;
"morphsubtypes" are constrained subtypes of morphtypes;
"infratypes" are implementations of a morphtype,
eg, the machine type Z32 (32-bit integer)
may implement a subrange of morphtype Z .

design by autocrat

11.20:  addx/design by autocrat:
. what if you are deluded, and need others' help
to show you that your design is fundamentally flawed;
but you are drugged by the freedom of working alone?
well, I've wanted to reuse some other work
but I've often found the documentation unreadable,
so this is about building an understanding
through finding that my design was flawed .
. at least I will be on the step toward
both designing a program I like,
and writing some documentation I can read
(likely others can read it too,
because I'm very forgetful,
so I will need to be thorough).

Parython is a #ParaSail version based on #python

11.22: news.adda/co/Parython is a ParaSail version based on python:
. a recent ParaSail article mentions Parython, (sources)
more on that earlier? yes but just a mention:

Achieving parallelism and safety at the same time
by eliminating rather than adding features
has worked out better than we originally expected.
One of the lessons of the process has been that
just a small number of key ideas are sufficient to
achieve safe and easy parallelism.
Probably the biggest is the elimination of pointers,
with the substitution of expandable objects.
The other big one is the elimination of
direct access to global variables,
instead requiring that a variable be
passed as a (var or in out) parameter
if a function is going to update it.
Other important ones include
the elimination of parameter aliasing,
and the replacement of exception handling
with a combination of more
pre-condition checking at compile time
and a more task-friendly
event handling mechanism at run time.
So the question now is whether
some of these same key ideas
can be applied to existing languages,
to produce something with much the same look and feel
of the original, while moving toward a much more
parallel-, multicore-, human-friendly semantics.

news.adda/co/ParaSail at oopsla:
. parasail is featured at recent oopsla:
# bringing value semantics to parallel oop,
# parallel tutorial with 
decomposition and work-stealing  .

#stackless #python dis- #continuations

11.20: news.adda/stackless python/dis-continuations:
Christian Tismer 2000:

. this paper presents an implementation
of "Stackless Python" (a Python which
does not keep state on the C stack) .
By decoupling the frame stack from the C stack,
we now have the ability to keep references to frames
and to do non-local jumps.
This turns the frame stack into a tree,
and every leaf of the tree can now be a jump target.
While exploring this idea,
we will recognize ordinary function calls and returns
to be just special cases of so-called continuations, ... .

why should stackless be popular?

. the maximum recursion level would be a user option
and not a limit set by C's stack .
There would be pickleable execution states;
that means you can save your running program
and send it in a file to another computer
where it can continue running .

multi-language to univeral data format (UDF)

addx/multi-lang like .net:
10.2: 12.11:
. the addx system needs to be like
Microsoft's ".NET" such that it lets us
command addx with a variety of languages
-- adda would just be the default language .
. a new language being ported to addx
would provide a compiler that emits
addm's high-level code (HLC)
that is similar to .NET's assemblies .

efficient Levenshtein distance

10.15: news.adds/algorithm/efficient Levenshtein distance:
. the Levenshtein distance between two words
is the minimum number of single-character edits
(insertion, deletion, substitution)
required to change one word into the other.
. finding the Lev' distance takes a long time,
on the order of (length of longer string
times length of shorter string),
so we are looking for ways to avoid it .

constant.subtype vs mutable.subclass

adda/oop/mutable types as subclasses of a constant type:
10.20:
. I'm reviewing Apple's Cocoa Objective-C,
and noticing again a mutable type is
seen as a subclass of a constant?
it would seem more intuitive if
constancy was an orthogonal attribute:
any type can optionally be described as
constrained to a constant state .

module permissions

adda/cstr/module/permits:
12.8: intro:
. just as corporations are composed of
people with various levels of trustability
and access to classified resources,
a program is composed of modules
that are trusted with various sensitive resources
by being being granted a permit to gain access .
. obvious things in need of permissions
include access to the file system, internet,
or the user's contact list .
. recently under consideration
is the idea that modules need to get permits
before they can access any other modules .

implicit parameters

10.4: adda/cstr/module/specification/implicit params:
. another dimension of the module's specification
is access to certain implicit parameters:
ie, it would expect every caller to have
certain locals having specific names,
as well as specific types .
. that's no different than a var param
except it's constant; ie the declaration is saying
I don't want just any local you care to assign,
I want a local with a certain name .
12.8:
. this is like what a nested subprogram can do;
but is trying to apply the same idea to
non-nested subprograms .
. the generic way to give a subprogram
an implicit parameter,
is have the subprogram packaged in a tuple,
and have the other fields of the tuple
be pointers that can linked to locals .
. the tuple is declared locally,
the links are set to locals,
and then, whenever the subprogram is called,
it uses the links as implicit parameters .

type.tags for aggregates

10.17: adda/oop/type.tags for aggregates:
[12.8: intro:
. if we were to have a generic aggregate,
what would its type.tag look like?
(the purpose of a generic aggregate
is having something that packs objects together,
and can be easily created and traversed .

MS`Portable Class Libraries

10.2: news.adda/MS`Portable Class Libraries:

Windows 8 programming model is like Xamarin,
[ where you write your apps entirely in C#,
and can run the resulting code on many platforms:
iOS, Android, Windows and Mac.]-12.8
. MS does this with Portable Class Libraries (PCL's)
which are subprograms that can be used anywhere
by attaching to whatever gui is on the current device
(supporting device-specific user interfaces).
PCLs let you wrap common code into a library,
ready for use by various gui's .

12.8: adda gui access is based on similar idea:

"( subprograms should not be controlling a gui interface,
rather they should be designed primarily for
use by other subprograms,
just as unix tools are meant to be .
(well, unix takes a shortcuts by
making its tool language a character string
that could also be readable by humans
but that was a security blunder because of parsing errors
that confuse critical datatypes like
{filenames, command names}).
. so, anyway, back to how humans fit in:
in order for a human to interact with
a subprogram that speaks type-tagged binary,
the human needs an agent who will
convert this robot speak into a gui .
. this agent is the Controller of the Model subprogram,
and it creates a View the human can interact with .)

10.2: 12.8:
. rather than having a subprogram export gui interactions
they are generating output like html
meant for describing what to display
but instead of being text a human can understand
it is encoded for quick access by other subprograms
including the human user's interaction agent
which builds a gui like html does .
. all native data types,
from which all other data types are composed,
have a gui presentation .
for instance, a tuple data type
is naturally represented as a dialog window,
and dialogs are then nestable like tuples can be .

target-owning vs aliasing pointers

12.7: summary:
. this explores types of pointers
needed for adda's space-efficient style of oop,
and support of safe pointers .

.NET Gadgeteer

10.22: news.adds/openware/.NET Gadgeteer:
. the .NET Gadgeteer

 was created by researchers at Microsoft
as an internal prototyping tool,
but because of external interest,
particularly from educators and hobbyists,
MS turned it into open source software
which now has a vibrant hardware ecosystem
from multiple manufacturers.
The platform is built on the .NET Micro Framework,
which allows small devices to be
programmed in the C# language
and make use of Visual Studio's developer tools .

. components can be bought here .
zdnet:

. it is praised for being simple to build and code,
and for offering processors clocked at
several times the speed of those on
the vast majority of Arduino boards.
However Gadgeteer modules also cost more
compared to modules for the similar
Grove prototype system.

many fewer soldiers needed soon #robotics

10.1: news.adds/robotics/many fewer soldiers needed soon:
. within the next 2 years
.mil is going to transition 100,000-plus
out of the military soon .
--[ for some balance there are medical jobs: ]
The American Association of Medical College
Center for Workforce Studies estimates

 the country will need 45,000 more PCPs [primary care physician]
and 46,000 more surgeons and medical specialists
 within the next decade.
. medical schools are turning to a
3-year diploma solution
 to hedge against the looming
 primary care physician (PCP) shortage,
offering an accelerated program
and major savings to future primary care doctors.

12.4:
. surgery will be easier to learn
and easily packed into a 3-year degree
because of the use of robotic assistance .

mem'mgt for a top-down stack layout

9.4: adda/mem'mgt/top-down stack layout:
[11.23: intro:
. the adda`oop system (unlike other oop impl's)
seeks to avoid generating garbage,
and that requires that when a function returns an object,
the return-obj's needed memory was allocated by the caller,
and then the function was given a pointer to that memory .
. that means assignments and function calls
are fused into one operation;
y`= f(x) is implicitly:
y`= f(x, ptr to y),
and this is true even when y is a parameter:
g( f(x) ) is implicitly:
stack g(y); stack f(x, ptr to y); call f; call g .

stackless mem'mgt

9.3: adda/mem'mgt/stackless:
[11.20: what stackless python does:
. a tasklet is like a separate program:
it has its own stack,
but for multi-threading it also
shares some state with some other program  .
. pickling is the saving of a running program
into a file until needed,
when the file is unpickled to become
the same running program it was before .
pickling stackless python's tasklets:
This allows you to take a tasklet mid-execution,
serialise it to a chunk of data
and then unserialise that data at a later point,
creating a new tasklet from it
that resumes where the last left off.]

. stackless python allows simulation of the stack,
so it can freeze your program's long operation,
and restart it when you have the computer back .
. even though we cannot freeze the C`stack,
we can impl' our own stack .
. see also notes on a system for ensuring
user interaction was not interrupted by
a long operation from an uncooperative subprogram
(in our system for ensuring this,
the compiler is rewriting each subprogram
by sprinkling it with calls to the gui;
so that no subprogram causes the gui to be
starved for cpu time).

extension modules

19: 21: adda/extension modules:
. Python' C interface has 2 uses:
one face simply models python
(for calling python from c),
whereas the other face is for writing extensions:
calling c from python .
. the primary purpose of adda,
on the other hand, is to avoid C:
all extension modules are written in adda,
and then adda translates that to C,
like what Cython is doing: Python to C .

semi-type-specific subheaps

14: adda/oop/type-specific heaps:
[29: intro:
. in a type-specific heap,
each object reference has 2 parts:
# an ID of the type it belongs to,
# a serial number unique to that type . 31:
. this promotes encapsulation,
because the reference isn't a pointer,
the only one who knows the object's address
is the type mgt that owns the object .
]
. given the idea of type-specific heaps,
how does that mix with adda's oop?
--that's the efficient form of oop;
the inefficient way of doing oop
uses garbage collection
and that is a choice independent of
which heap an object is owned by,
whether local or global .
. using pointers is not what causes
pointer semantics and garbage:
that's caused when assignments are overwriting
the pointer instead of the pointer's target:
to avoid garbage collection,
assignments need to overwrite the objects themselves
not the pointers to the objects .
. instead of a global type-specific heap
local vars would be placed into
the subprogram's local heap
that can have within it type-specific subheaps .

reviewing the costs of SOA

31: addm/reviewing the costs of soa:
. when looking at SOA architecture,
how does that affect the
cross-module communications costs ?
there could be a devil in the details;
but, the bird's eye view is that
it could actually be quite minimal .

list of online directories

4.18: news.adds/lib/directories:
. web directories for subject-specific searches:
http://en.wikipedia.org/wiki/List_of_web_directories
classic dmoz (but no longer mozilla
http://www.dmoz.org/
-- owned by AOL Time Warner)
(become an editor)
http://www.dmoz.org/docs/en/help/become.html
classic yahoo
http://dir.yahoo.com/
joeant (subject then rank sorted)
http://www.joeant.com/
(apply to be accepted as an editor)
the best of blogs
http://blogs.botw.org/
the best of web (dinky)
http://botw.org/

skipping versioning for small edits

4.26: adde/versioning/skipping versioning for small edits:
. while inputting a string,
when the normal rules of versioning are in play,
stopping to do a correction should
cause the string input to be terminated,
then the new command is a string replace .

. one way around that is [word modify mode]
that can suspend the versioning system
until you are finished editing a particular word;
and, it reduces the damage of the modification
to just a particular word,
so that the versioning system is less needed . 5.13:
. it doesn't consider the word saved by versioning
until you are actually getting into the next word .

versiwiki

4.12: adds/versiwiki:
. in wikipedia we often see "citation needed";
this reminded me of a need for versioning;
the default version is completely referenced:
it hides everything that needs a citation,
or that has a citation that doesn't apply .
. if you want to see flagged content
(original work, contested material)
then you click the view-all button .
. we can also extend that idea
to have buttons for many versions:
show original content, or show controversial .
. when the controversial view is shown
it also shows the tags people have added:
pornographic, hate speech, ethnocentric, etc .

free online books

4.20: adds/lib/free books list

5.29: summary:

. I found a list of free academic books online,

and this is my reading list todo . :)

security through instruction restrictions:

3.19: adda/exceptions/security through instruction restrictions:
. my style of exceptions has been
having the raiser call the caller's handler;
but, if the caller's code can be corrupted,
then the exception raiser could be exploited .
. it's fun to think about a perfect language,
one that's not corruptable;
but it's safer to have layers of security,
and not depend on a perfect language system .
. say the exception raiser is in ROM
(as it is a library routine)
but the exception handler has a jump instruction
that bypasses a safety guard;
what could save us is a VM that knows when it is
inside an exception handler,
and never executes a jump in such situations .

extending the const-var architecture

3.31: addm/extending the const-var architecture:
. what I'm calling the constant-var architecture
makes use of hardware isolation mechanisms
by safely dividing the system into
constant code, and variable data .
. there is another segment to consider:
# write-once-read-many:
. what's variable across process instances
can be constant during execution .
# special permissions:
. the implied permission is that the
data in the process belongs to that process .
. the activation record's resource display
is an example of special permissions:
the process has permission only to
read the resource display, not write to it;
but the supervisor can modify it .

privacy assurance while reporting errors

3.19: addx/privacy rights/reporting errors:
. if an error should be reported to the coder
the report should be generated in such a way that
the user can see it contains no personal data;
or, it should say it does contain data
and ask if it is too sensitive .

parser algorithm for postfix operators

2.1: adda/syntax/parser algorithm for postfix operators:
. when checking for existence,
the word (is) could be used in 2 ways:
(is x) checks for existence,
(x is t) checks for x's type = t .
. another idea is having support for
english-style postfix operators:
then you could write (x exists)
and you'd parse this like an infix
except you're not looking for a 2nd arg .
. its syntax for being declared
could be similar to that of other functions:
myPrefix(arg.argType).ReturnType,
(arg.t)myPostfix.ReturnType,
myInfix(arg1,arg2:t).ReturnType .
-- anything with 2 args is an infix operator .

combined hardware-virtual isolation

addm/security/combined hardware-virtual isolation:
2.9: 3.31: intro:

. certain attributes of data are essential to security;
eg, by retagging arbitrary data so that it is
usable as a pointer to code,
we can treat malware data as instructions to follow .
. generally all data can be tagged
just as it is done by xml .

. there are 2 possible ways to enforce
process isolation and ROM attributes:
# HW (hardware) mem'mgt,
# VM (virtual machine) mem'mgt .
. hardware mem'mgt can enforce VM mem'mgt:
the VM's run-time exec never needs to change;
so, HW mem'mgt can see that code as const;
also, any file that the VM is trying to interpret
can be treated by the HW mem'mgt as
something that only the VM process can modify .
. finally, the VM has its own process space
and this should keep other processes
from corrupting its work space .

vmm (virtual machine monitoring)

2.5: addx/vmm/virtual dom0:
. just as addm is a VM (virtual machine),
addx is a VM monitor (vmm);
so it should do things like Vmware does,
only with some enhancements and extensions; eg,

pythonic software construction

2.2: news.addx/pythonic software construction:

Fred Brooks` Mythical Man Month:
. Brooks suggested we plan to throw one away
because we will throw the first attempt anyway;
10years later he noted that incremental development
can reduce much of that loss;
nevertheless,
what he said still applies on a smaller scale:

platforms and requirements to target

1.23: addx/targets are {python, c, obj'c, parasail}:
. if parasail is not on all the needed platforms
the base lang should be obj'c not c;
because, obj'c has better programming-in-the-large features;
well, does it support concurrency?

concurrency4all Parallella

1.4: news.adds/Parallella/
concurrency4all got funding:
Parallella project will make parallel computing
accessible to everyone.
first seen here:
www.geekosystem.com/99-dollar-computer-
3.9: sign up here: parallella.org .

ratical.org and other libraries

1.6: news.adds/lib/ratical.org:
ratical.org/startHere.html
Today the internet is primarily described as
electronic commerce.
Before 1995 it was lauded as
a so-called information superhighway.
rat haus reality is guided by
the paradigm of web-as-library.
It seeks to implement a collection of
library reference materials
to assist students
in the classroom of life and our world.

10: web.adds/lib/e-books for free:

virtual machine for obj'c services

1.23: addm/
simulates obj'c when obj'c is not available:

adda`lib folder system

1.15: mis.adda/cstr/pkg/how do packages work?/
adda`lib folder system:
. I seemed to have missed the point of
de-anonymizing a type:

converting messy data formats

adde/universal conversion via screen scraping
adde/convert clonespy links
adde/browser/filename cleaner

user interface ideas

1.19: adde/gui/window resize needs a quick mode
adde/gui/backed by vcs (version control system)
23: adde/gui/message window/priority treed

managing self-modifying code

1.31: adda/managing self-modifying code:
(inspired by python unpickle vulnerability)
. the safe pickle is built by the system .
. it can be compared to the decompile,
how is it extensible? that is to ask
how are objects built in the first place?

exceptions ok unless requirements preclude

1.30: adda/cstr/exceptions/ok unless requirements preclude them:
. I thought Parasail's author explained somewhere
how exceptions really messed up multi-threads;
review my blog of that ...
(parasail-is-big-win-for-reliable).
. I decided a thread hang was no big deal;
in critical applications
exceptions are absolutely useless;
but if our point is to encourage programming
we should cater to all styles of thinking .
. we just need to protect the coder's user too:

avoiding the dangling pointer

1.16: adde/fs/bookmarks/avoiding the dangling pointer:
. mac spends a megabyte per bookmark,
but it can't find the moved file
even when it's inside the same volume as the link ?
. all it needs is a checksum of the file,
and then if it's not at the path indicated,
it reports that this could take a while,
unless you know where it is;
and, it offers to cancel this backgrounded search job .
. it looks for a file of the same name,
then checks it has the same checksum .
. here's how to find the file in the case where
it was edited and no longer has the same checksum:
a bookmark can include a list of keywords,
and like google we can produce a list of files
that gives a decending-order best fit
of all files that contain more of those keywords .
. a more thorough way is more expensive,
and requires all file moves happen via the file mgt
who then has to provide a system for
knowing when a file has been linked in the past,
and has subsequently been both moved and edited .
todo:
. I think that design has been discussed earlier;
need to merge that with this idea .

relative url syntax

1.20: adda/syntax/relative url/

globalizing #SOA with web services

 1.30: web: cs/soa/globalizing SOA with web services
Filtering to Inspect XML: an Operational Framework for
Service Oriented Architecture Network Security
www.tacoma.uw.edu ... rbunge.pdf
Robert Bunge1, Sam Chung1, Barbara Endicott-Popovsky2, Don McLane1
1 Computing & Software Systems; Institute of Technology
University of Washington, Tacoma
{rbunge, chungsa, dmclane}@u.washington.edu
2 Center for Information Assurance and Cybersecurity
University of Washington, Seattle
endicott@u.washington.edu
1.30: summary of this paper:

#SOA -- where is #security today?

1.29: web.cs/soa/where is security today?:
. I was looking for updates on
how SOA is preventing malware;
instead, I find this claim about SOA's vulnerabilities:

"( Modern buffer overflows are more difficult to exploit
than Aleph One's
smashing the stack for fun and profit.
You should look into modern bypasses to ASLR
such as heap spraying or heap feng shui.
Attacks like jmp2reg (jmp2esp jump2ebx ect...)
are also interesting for bypasses for ASLR.
Attacking ActiveX components is fun.
I used H.D. Moore's AxMan with great success.
Here is the remote code execution exploit I found
using AxMan .
. Here are more exploits that I have written .
. The best fuzzer is by far PeachFuzz,
and writing some pit files for it
can be very fruitful research.
Buffer overflows and sql injection
are the most talked about,
but there are a couple hundred categories for vulnerabilities
and they are identified by CWE numbers.
Its worth exploring, I think it will surprise you
what NIST thinks a vulnerability is.

. that had nothing to do with SOA;
it should have been subtitled
"exploits you can avoid by using SOA" .

virtual C

1.11: adda/dstr/safe pointers/virtual C:
. adda's pointers can feature arithmetic exactly like c;
yet it can still remain safe because
the addresses are not absolute;
the pointers are actually actually just offsets .