| DNess -11868 052533 |
/<$Simple DB in K - Ver DN-1B(5) - [GSR-XDDP]$>
|
db Data Structure
The data is initially loaded into a 3-dimensional array, in a structure
that broadly parallels that of a directory. db[n;i;j] where
n counts the objects, i runs through the attributes and
j of 0 is the attribute symbol, 1 is the value and 2 is null (reserved
for format etc.) At this point db[;;0] is a list of each
of the attributes of each Object and db[;;1] is a list
of the corresponding values.
|
| This is where `log' data is collected. |
lg:""
|
| The g function is used to chop up lines into symbol, value, null triplets |
| g:{l:(3#1,0 1+*&"="=x) _ x; |
|
Cut up the lines at the first equal sign.
|
|
|
(`$l[0]),(,l[2]),,_n} |
Return a triple with the symbol at as the first element, the value
as the second element and a null.
|
|
|
The Original Data File
The original data file dealt with by this program consists of descriptions
of a number of `objects' (in this case restaurants, people, residences, ...)
These objects are stored in an ASCII file. A typical entry might look like
|
{{ObjID=Bernardin}{Name=Le Bernardin}{ObjClass=EatSite}
{City=New York}{Phone=(212) 489-1515}{Address=155 West 51st Street
(bet. 6th and 7th Aves.)}{State=NY}{Code=10019}{Text=A serious candidate
for the best restaurant in NYC. Also costs like it. Snotty, too, but
when you are the best, perhaps you have a right to be}{Drupelet=2143}}
|
|
| This is the first `workhorse'. It loads the whole data base and stores it in the db array. The code here relies on one
(lousy) `pun'. After cutting, lines which are three or less
characters long can be dropped as they don't contain data (i.e.
they are just mid-item blanks, etc.)
|
|
|
,/((0:(F:_i[0]),".db"),\:" ") |
The database is loaded, blanks are appended to every line and then
the data is strung out into a single line.
|
|
|
(&_sm[e;"{"])_ e:(&e _lin "{}") _ e:_ssr[(above);" ";" "] |
|
This gets rid of multiple blanks and then cuts the data at every left and right curly brace.
It also forms items that begin with left curly brace alone.
|
|
| A:!#db:g''e@'{&3<x}'#:''e: (above) |
|
|
The chopped data is stored into the e array, and this is then cut
into items. The g function is then used to cut each of
the items into symbol, value, nil triples and they are stored in
db. A is made available as a vector of the
items in db.
|
|
|
| f returns the index of the various `symbols' that constitue each occurrence of the items in the argument list. |
f:{&:'db[;;0] _lin\: ,/x}
|
| d[`Sym1 ...;ItemLst] returns the values of the given elements in `Sym1 ... for each of the items in ItemLst. |
>d:{db[y;f[x][y];1]}
|
| whr[`Symbol;"Value"] returns the indexes of items that have an attribute `Symbol with value "Value". |
whr:{&,/_sm[*:'(d[x]'A);y]}
|
| v[`Sym] lists all of the `values' of `Sym across all of the data items. |
v:{t:d[x]'A;?t@'<:'t}
|
| vp pretty much does the same thing as v, but it returns null strings for non-existent values, not nils.
|
vp:{t:v[x];if[~0=#r:&0=#:'t;t[r]:(#r)#,,""];t}
|
| oo is used to help with output. It calls oi with each argument, It puts a left curly at the left of the first
item and a right curly after the last item.
|
oo:{l:,/oi'x;l[0]:"{",l[0];l[-1+#l],:"}";l,,""}
|
| oi outputs each item by formatting a left curly, the text value of the symbol, an equal sign and then the value,
followed by a right curly. |
oi:{chop["{",($x[0]),"=",x[1],"}"]}
|
| osz sets the maximum width of output lines. |
osz:60
|
| The chop function simply chops its argument into lines that are not longer than osz
|
| chop:{r:();while[osz<#x; |
|
|
A result array is established, and then a while
loop is entered. It runs until the argument is less than
the desired length.
|
|
|
t:osz#x; v:osz - (|t)?" "; |
|
Take the first osz characters. Look for the last blank
(first blank in reverse).
|
|
|
|
r:r,,v#x; x:v _ x];r,,x} |
Append the cut line to the result array. Chop
characters from the argument and close the
loop. When done append characters that are left
and return the whole result.
|
|
|
| ad[x] appends a `Drupelet' attribute to each Object. |
ad:{db[x]::db[x],,(`Drupelet;$x;)}
|
| wr[] writes out a fresh copy of the data base, and then quits. |
wr:{(F,".new")0:,/oo'db;."\\\\"}
|
| sv[] saves a copy of the data in K internal format |
sv:{F 1:db;."\\\\"}
|
| nil[`Sym] returns a binary vector with 1s corresponding to items that have null values of `Sym. |
nil:{1-(#x)=#:'d[x]'A}
|
| mtch[n;List] returns the items in the list List which occur exactly n times. |
mtch:{&@[(#db)#0;y;+;1]_lin ,/x}
|
| This fragment appends a `Drupelet' attribute to each item that doesn't already have one. |
ad'&nil[`Drupelet]
|
| shw[x] simply appends the message `x' to the current log. By changing this one function it could be dispayed as well. |
shw:{lg::lg,x}
|
| This takes a text list and formats it for output. |
shwl:{shw[(1_ ,/"\n\t",/:*:'x),"\n"]}
|
| This lists the Object IDs for the objects listed |
list:{shwl[d[`ObjID]'x]}
|
| rep[n;S;T] reports the members of S that occur n times with the descriptive text T.
n can be a list.
|
| rep:{l:mtch[x;y];if[~0=#l; |
|
Generate list of items that meet criteria and prepare to continue
only if there are some to be listed.
|
|
|
shw[z,": "];list[l];shw["\n"]]} |
Show the text message and then list the items.
|
|
|
| fd[`Sym] returns a vector that contains the value of the `Sym attribute of every data element.
|
| fd:{adb:db,\:,(x,_n,_n); |
|
Create a temporary data base that attaches a `null' value of
a `Sym attribute to each item. This means lookup of `Sym can't
fail for any item. It will just return the real value if there
is one, or a null if there is not.
|
|
|
adb[;;1]@'(adb[;;0])?\:x} |
Retrive the values corresponding to the indicate attributes.
|
|
|
| The audit[] function performs a number of data base consistency checks. At the moment it is run everytime the data
base is loaded.
|
| audit:{ NNm::&nil[`ObjID] |
Make a list of Object Names
|
|
| if[0<#NNm;shw["No ObjID: ",(,/" ",'$NNm),"\n"]] |
Objects that don't have ID's have to be listed by number
|
|
| objl:fd[`ObjID] |
Make a list of Object IDs
|
|
| if[~(@objl)~@=objl;shw["object ID conflict\n"]] |
Consistency check: Do objects (properly) contain their own ID as their `ObjID attribute?
|
|
| Res:whr[`Cat;"Restaurants"] |
Make a list of objects that are restaurants
|
|
| rep[1;Res,whr[`ObjClass;"EatSite"]; "Restaurants/EatSite mismatch"] |
Restaurants should also have a `Class' designatiing them as "EatSite"s
|
|
| rep[2;Res,&nil[`Phone];"Restaurants with no Phone"] |
Report restaurants that don't have phone numbers listed in db
|
|
| rep[2;Res,&nil[`Address];"Restaurants with no Address"] |
Report restaurants that don't have addresses indicated.
|
|
| rep[1;&nil[`ObjClass];"Unclassed"] |
Report objects that don't have an `ObjClass attribute
|
|
| rep[1;&nil[`Cat];"No Cat"] |
Objects that don't have a category
|
|
| rep[1;&nil[`Text];"No Text"] |
Objects that don't have descriptive text
|
|
|
|
|
| Run the audit function |
audit[]
|
| Display the Log |
`0:lg
|
| Build a list of all distinct Keys |
Keys: ?,/db[;;0]
|
| s is used to help dump the Keys and uses the Global Keys to do this
|
| s:{shw[($x:Keys[0]),":\n"]; |
|
Show the name of the key
|
|
|
Keys::1_ Keys;shwl[vp[x]]} |
Trim current key from the list, and dump the distinct values
|
|
|
| This actually dumps the keys and their distinct values |
keydmp:{while[#Keys;s[]]}
|
| Dump the current log |
logdmp:{"log.tmp"6:lg;."\\\\"}
|
| Build an Object List |
Obj:,/`$fd[`ObjID]
|
The O data base
is an alternative form of the data in db[]. O is `subscripted' by
symbols made up of Object IDs. Each symbol has a separate directory
where the attributes are variables and their values can be accessed
and maniuplated directly. As an example, the entry used above to
describe the original data base would result in
O.Bernardin.City having the value "New York".
This could also be referred to as O[`Bernardin;`City] if
that was more comfortable.
|
| This deceptively simple line really does a huge amount of work, It basically builds a copy of the data base where each object is an
element of the O array |
O:. Obj,'.:'db
|
| Essentialy does a nub sieve |
fs:{x[*:'=x]}
|
| A (probably silly) checksum function. I'd be happy to replace it with something more intelligent, but it also probably doesn't matter much
|
| CK:{a:0;while[#x; a:((a*179424691) |
|
The idea of this checksum is to run through a vector calculating
a reproducable number. Since its main use is to see if an object
has changed,
|
|
|
+*x)!2038074751; x:1_ x];a} |
Finish up the calculation and return the result
|
|
|
| Checksum for strings |
CKQ:{CK[_ic[x]]}
|
| Builds a text string from an attribute entry |
tv:{[x;y;z]($x),"=",y}
|
| Again this does a lot of work. It builds a checksum table for each attribute in the data base |
fcs:CKQ''tv .'' db
|
| Builds the checksum for each Object. Notice that it canonicalizes attribute order before checksumming |
CSm:CK'fcs@'<:'db[;;0]
|
| Calculates the checksum of any object in the O array |
| Cal:{rv:. O[x];av:tv .' rv; |
|
Cal[`Sym] calculates the checksum by retrieving the object.
Then it converts each attribute to a string.
|
|
|
av:av[<rv[;0]];CK[CKQ'av]} |
Then the attributes are ordered, and the checksum of the ordered
attributes is formed.
|
|
|
| This function is used to `reconcile' two copies of the data base. Its basic purpose is to check through the elements of the O array, and
reconcile changes in that data with the form of data in the
db array
|
| BB:{Old:Obj::,/`$fd[`ObjID] |
Build a list of all Objects in db
|
|
| Ct:=Old,New,New:(. O)[;0] |
Build a list of Objects in O
|
|
| Both:fs[&3=c:#:'Ct] |
Items that appear in the list 3 times are in both lists
|
|
| NewO:NewO[>NewO:fs[&2=c]] |
Items that appear twice are only in the New list
|
|
| OldO:fs[&1=c] |
Items that appear once are only in the Old list
|
|
| while[#Both;t:*Both; |
|
|
Run throught the `Both' list
|
|
|
if[~CSm[t]=Cal[Obj[t]]; Update[t]]; |
|
Checking to see if any items have changed. Update them if so
|
|
|
|
Both:1_ Both] |
Continue looping.
|
|
| while[#NewO;Create[*NewO];NewO:1_ NewO] |
Create items that are `New Only'
|
|
| while[#OldO;Erase[*OldO];OldO:1_ OldO] |
Erase items that are `Old Only'
|
|
|
|
|
| Kill an item |
Kill:{O::O _di x}
|
| Erase from db |
Erase:{db::db _di x}
|
| Create an item |
Create:{db::db,,. (. O)[x;1]}
|
| Update the data base |
Update:{db[x]::. (. O)[x;1]}
|