avoiding the dangling pointer

1.16: adde/fs/bookmarks/avoiding the dangling pointer:
. mac spends a megabyte per bookmark,
but it can't find the moved file
even when it's inside the same volume as the link ?
. all it needs is a checksum of the file,
and then if it's not at the path indicated,
it reports that this could take a while,
unless you know where it is;
and, it offers to cancel this backgrounded search job .
. it looks for a file of the same name,
then checks it has the same checksum .
. here's how to find the file in the case where
it was edited and no longer has the same checksum:
a bookmark can include a list of keywords,
and like google we can produce a list of files
that gives a decending-order best fit
of all files that contain more of those keywords .
. a more thorough way is more expensive,
and requires all file moves happen via the file mgt
who then has to provide a system for
knowing when a file has been linked in the past,
and has subsequently been both moved and edited .
todo:
. I think that design has been discussed earlier;
need to merge that with this idea .

portable cross-volume hardlinks

2012.2.26: adde/fs/volume id for cross-volume linking,
[2011.10.2: preface:

. addx is meant to be a portable environment
available for mac, linux, pc, ...;
and one problem to be faced
is a filesystem that has portable links:
they are all incompatible with each other,
except where mac and linux use unix links;
but those are not generally useful;
eg, I can't put a unix link on a fat32 volume,
not even a symbolic (vs hard) link?! [12.2.26:
( a symbolic link is just a text file
that includes the pathname of the resource);
one problem with unix allowing a link on fat32
is that it can be seen by multiple unix versions
each with its own style of pathnames:
# mac unix: /Volumes/myDrive/
# linux unix: /media/myDrive/
but that problem would be true for any removable drive,
not just one formatted with fat32 .
. what's peculiar about fat32 is a competing link format .
. it appears the volume's file system
is responsible for supporting links,
when what is needed is to have a file type
associated with link data ).]

. Apple has a friendly link that works in {mac,fat32},
and it keeps pointing to its target
even if you move the target;
however, each such link weighs half a megabyte!

[2011.10.2: intro:

. for link targets to be recognized across all platforms,
the portable filesystem manager (adde)
must have its own way to identify
every possible link target,
and all the volumes they could be on .]

2012.2.26: converting other OS's links:
. when dealing with other links
(those created by {windows, linux, mac})
adde needs to convert these .
. other apps see the addx link as a text file,
but when asking adde to traverse the file system,
it will follow the link's directions .

12.2.27: tracking unwritable volumes:
. any time a new volume is mounted
adde tries to register it by adding the volume ID;
however, some volumes are not writable,
so they can't be registered in the usual way;
in that case,
adde needs a list of those volumes that were
found to be unregisterable .
. does each volume have some readable serial#?
. you can't depend on checksumming;
because, being unwritable doesn't mean it can't be
modified by another platform
(eg, a writable mac volume will be
unwritable on linux);
so, the only reliable way is to
keep a list of unregisterable volumes,
and ask user which one this is .
. the menu should be sorted by device;
for instance, if this is a mac volume on usb,
there's no use showing cd's on the list).

12.2.27: the 3 types of pointers:

. filenames are pointers to content units;
links are usually considered to be aliases,
where pointers are sharing the same target .
. there are subtle differences of intention
among the various uses of pointers:

# the absolute address:
. goto whatever's at this address
(the address includes the filename
but not the file being named);
if nothing's at this address,
then make a new file with this filename .

# the relative address:
. use the address relative to some ancestor folder
or relative to the current volume,
rather than at a specific full path .

# the object reference:
. stay connected to what's currently at this address
(at the time the link was created);
if the file was moved, then find that file again .
. adde is putting the file ID inside the file;
that makes it easy to stay linked to an object reference .

2012.2.26: types of link exceptions:
# broken link:
. the target file was moved or renamed;
# unmounted volume:
. the target volume is not currently available .
. adde has agreements with the user
for each type of link exception,
whether to report exceptions via mailbox,
or interrupt the user with an alert dialog .

2011.9.2: details:

. adde is assigning a unique ID to every
volume, folder, and file; [12.2.28:
within the addx filesystem only:
for external files the object linking is less robust .]
. since volumes can be shared among acct's,
the volume itself must have a file ID database;
and then each acct must use that to update
the ID database kept on the acct's volume .

. mac's .dmg (disk image) is a virtual volume:
the enclosing volume considers it to be one file,
but after mounting it, it's like an external drive;
so, when looking for volumes to register,
it needs to be on the look-out for mounted .dmg's .

2012.2.26:
. the volume ID has 2 parts:
# the name of the acct that created it;
# the date of this volume's registration .
. the user's ID could be something like their email address,
or a social networking acct address .

file ID uniqueness:
. if there is an endless stream of creations and deletions,
then a volume could run out of file-ID's;
whereas, if they were reused,
then we must make sure there are no clashes;
so, the file ID should include the creation date .
[12.2.28:
. the file ID in a volume database
is specific to that volume
so it need be nothing more than a timestamp,
but that's only true for an infinite precision timestamp;
because, if the timestamp changes only every second,
then the ID needs a serial number for the situation where
the user has asked for thousands of links,
and they were all generated within one second .
. alt'ly, we could keep track of the last timestamp used,
and then we'd have use of
every timestamp between then and now .
... so then it seems,
the simplest idea is to have a serial number
but one of infinite type, like dates are:
we never run out of file-ID's
because we can just keep adding digits .]

[12.2.28: the addx file ID:
. this article has used the same term, file ID,
for both a volume database index,
and for the addx file ID tag inside every file .
. the addx internal file ID is formatted as
creation.date & subject name
but if these files are moved to a shared drive
then to make the ID unique,
the creators name will need to be added .
. and so, in anticipation of this,
an object reference link should always include
the author's name like so:
. check the addx file for an author's name,
if it has none,
then the link's object identifier should include
the name of the person making the link
since it is their acct,
and the file's in it are by default theirs .]

2012.2.26: volume db separate vs integrated:
. in building a volume's file ID database,
the user has the option of efficiency or less clutter:
. some are not happy about hidden system files
because when you bring your removable to another OS,
all the hidden files become visible .
. if hidden folders are considered clutter,
then adde could avoid such clutter by
building a 2nd partial folder system
instead of reusing the current one .

12.2.27: the acct db:
. each user's acct needs an acct database
that contains the list of familiar volume ID's .
. when a link is being resolved
it first has to translate the volume ID
into the string understood by the current OS .
. it has a list of mounted volumes
and expects each to have an adde system file,
containing the volume's ID
and the file ID database .
. it then collects these from each volume
into the acct database
in a table of (volumes currently mounted)
that provides the maps:
volume ID -> volume pathname,
and its inverse .
. any time adde makes a link across volumes
it uses this table to convert the volume's name
into the corresponding volume ID .

12.2.27: sorting unmountables:
. a volume shared with other users
could have links to
volumes that are unknown to the user's acct
so the acct database has 3 lists:
# volumes known to user:
(mountable even if not currently mounted)
# unmountable volumes:
(user indicated these can't be found)
# volumes not asked about yet:
(we won't ask a user whether a volume is mountable
until they click on a link that targets that volume).

12.2.27: volume database has a list of target volumes:
. for each volume, self,
if any of the links on self are
pointing into a volume, v, other than self,
then v's volume ID should be copied into
self's list of target volumes .
. for each volume ID on this list,
there is volume meta info,
(such as a user hint about how to
locate and identify the volume;
or a picture of the removable media);
and during a mount, this volume meta info is
copied into the acct database to build the
table of (volumes currently mounted).

2012.2.26: version# simple links:

. the simplest capability doesn't include
dealing with broken links;
it is creating links and having an engine for
translating them into system-specific links .

. in version# simple links, the link has 3 parts:
# a text string indicating the pathname
(either absolute or relative);
# a volume ID .
[12.2.28:
. those 2 fields specify an file address
(ie, a pointer to a file with a given name)
but for an object reference
(where we are tracking a particular file)
there could also be a 3rd part:
# file ID:
. an object reference means that
when you get to this address,
make sure the file there is the one I had linked to .
. the file's data should include the indicated file ID .
. a file ID can also be used for
jumping into the middle of a file,
to point at a subfolder .
. if the volume and address are unspecified
then it means search for this file ID,
and return me a folder of links to any files containing it .]

2012.2.26: version# broken link finder:
 
[12.2.28: intro:
. in this advanced version,
we not only make a universal link,
but also repair links that are missing targets .
. of course, this concerns only links to
object references (particular content blocks)
rather than address references
which require only a particular pathname .]

. it might be the case that, after a file rename,
the only way to tell a file's identity
is by searching for the same attributes:
(size, modify date, checksum).
. if the file has been modified and moved or renamed
we might still be able to find the identity
by checking for partial checksums .
. this is possible for files like text,
where blank lines naturally define paragraphs .
. of course in the case of addx files,
we just look inside the file for the file ID
rather than relying on the file's name .

. we cannot assume that every file system will
divide the job into file pointers and content nodes;
therefore we can't depend on
hardlinks to provide a sort of file ID .

12.2.27: web: fat32 hardlink limitations:
. in fat32, the directory entries contain
file pointers rather than handles:
they tell you where on the disk the content is,
rather than where an immovable info node is
that will contain the current pointer;
so then compacting the drive moves the files
thus breaking any hardlinks you had .
. and if you delete a link's target file,
then your hardlinks are pointing at garbage .
. here's a unix command for making hardlinks in FAT32:
mount -o bind /origdir /newdir
. chkdsk will "fix" hardlinks:
. it reports them as cross-links and repairs by
making copies for each alias to own .
. fat32 symbolic links (aka shortcuts)
are before the era of POSIX symbolic links
which are supported by Windows only in Vista+ .

. to support version# broken link finder,
the link's 3rd part, file ID, becomes mandatory .

. back at the volume where the target is,
the file ID locates meta info about that file:
(size, checksum, name
, current url (a cache built from folder list)
, pointer to parent folder
)
. and from there it has the full chain of folders
because each folder has a pointer to its parent folder
(2011: doing it that way saves a lot of
file pathname rewrites
when one of the top level folders gets moved ).

. if the folder system has not been modified recently
then this goes very quickly because
the file meta info includes the full pathname
but if things have changed, then that cache will be stale,
and it has to rebuild the cache by
following the chain of parent folders
and copying their name strings into a unix pathname .

. if the link is dangling,
and the volume database is in a separate folder
rather than integrated with the volume's file system
then it needs to sync the database
with the current file system . [12.2.27:
. the database is only a partial copy of the file system
since the only files included are the aliased ones,
and the only folders included
are the ones that contain the aliased files .]

. to see if the missing file was just moved,
search everywhere starting on the expected volume;
if still not found,
then ask the user whether to search more
or just use a backup .

. if the missing file was both moved and renamed
then this could be a case where
the file has been repurposed,
and so the link should just be deleted .
. the user may know what to do,
but some file changes are made by programs .
[12.2.28:
. a (move, rename, modify) of a non-addx file
is usually indistinguishable from a {delete, create};
and, if the name is quite common,
then a (move, modify) is not always fixable either .
. if it was a text file, then adde would have done
some partial checksums on it,
and in that case, the file is likely findable
because a file modification doesn't usually affect
every single paragraph in the file .

. adde needs to run in the background
and watch the file system for changes .
. modifications can be quite frequent,
so, adde has to be watching first for
file renames and moves,
and check with the volume's file ID database
for whether that's an aliased file or not .
. it should then check for file changes
in order to promptly update partial checksums .]

documents as folder trees

5.11: adde/documents as folder trees:
[4.21:
. just as trees of folders can be merged,
the same should apply to documents as well,
with each subtitle being like a subfolder .]
. that introduces a 3rd dimension of merging
(volume, folder path, file);
and,
many merge styles:
# mac style:
. don't merge anything .
# pc style:
. merge folders, replace files .
# versioned merge:
. merge folders and avoid replacing files
by renaming them instead
with appended version numbers .
. the versions numbers rotate through 0...n;
so eventually data is lost .
# subfile merge:
. merge folders and files;
paragraphs are combined rather than replaced
(default is new subsections are appended
to older subsections).
# journaled merge:
. this is like the subfile merge (above)
except that there are instructions such as
delete the nth paragraph of a subsection;
or run a patch over it .
[4.21:
. the journal's files would pay attention to
subtitle pathname
and only then consider character position .
. this is in contrast to the typical
{revision, version}* control system
which measure all changes in terms of
character position .]
--
. intra-document structuring is
necessarily language-specific,
except at the level of characters,
whereas the unix way is tool reuse,
and so the patch tool must be
language-agnostic .
*:
revisions vs versions (terminology proposal):
# revisions are corrections or
efficiency enhancements;
# versions are parameterized configurations
for adapting to various situations
or new design ideas .

journaling with efficient versioning

2.11: adde/journaling/efficient versioning:

. notice reverse diff can't be done by journaling,
needs exact replace commands:
eg, suppose block has x's
and I use command to replace all x's with y's,
but there are already y's .
then a reverse journal changing y's back to x's
is going to change too many y's
and therefore won't represent the previous version .
. reversing must keep track of either whole block versions
or use address of changes
-- in some cases the instruction for reverse
would be heavier than just keeping both versions .

vnc integration

11.23: adde/vnc integration:

. the mvc and journaling arch' can make it easy to

do a vnc system;

so, try to be more inclusive of that feature .

that should be considered a design goal .

adde's file.editor

8.12: adde/fs/pathname`ordering:

. when sorting pathnames for listing,

the comparison is per directory level

as if each level in the path were a letter of the name;

the opposite of this way

would be converting the whole pathname to string

and then comparing strings .

8.12: adde/menu`customization:

. why does the mac user no longer get to add to the menu?

keeping the menu fixed may help other parts of the system

know what's in the menu?

. the only reason they got to change things anyway

was to add mini-app's to the apple menu,

and now that's been moved to the dashboard .

. mac users have never had much control over their menu's;

ms`office97 had customizable menu's, but the editor was buggy .

. the adde`user needs a timemachine-like functionality

for identifying time-oriented menu` changes

as well as to help with purpose-oriented changes .

8.30: adde/fs/virtually complete volumes:

. in the virtual fs (filesystem),

all files are just pointers to shared db entries,

but then also each volume

(a separate storage device or removable medium),

has its own version of the db;

which is sync'able (like with git) .

adde's journaling

todo.unreadable: 8.7: adde/journaling/assuring privacy:

. how to assure logging without key logger vulnerabilities?

3 types of log:

( sw config

, user cmd input

, user data input (content and pass )

) . the system must insure (verify)

that outcome doesn't depend on user content .

. the exception is when content includes scripts .

. if script affects only content

then the only thing to report is the allowed cpu and mem access .

. general scripts are part of software,

and become part of system config'

-- or need log of subset of system accessed .

log of pass and content is replaced with

cmd of get these from user .

--. inputs assumed to filter checking type and string`length .

. log also includes a ptr of use only locally

describing where the content is stored .

. the pass needs an id too (which pass did I use ?),

you have an id made like file names (creation date, owner) .

pluggable system-wide param's

7.28: adde/pluggable system-wide param's:

. part of journaling is keeping track of

every change and option made on the system

in a pluggable form,

so that others wanting to help

can simulate your system by plugging in your journaled param's .