Mon Nov 18 10:21:07 1996 OK here is the first pass. It is pretty weak in the area of the handler interface (i.e. the handler section is almost empty;) the actual names of functions and details of the data structures are a little flexible; but the document is beginning to take its final form. There is a heavy OMI based bias in this spec. What I have been doing is reading the OMI spec in great detail and asking at every point, "Do we need this? It is specific to OMI or does every handler need it? How should we generalize to equivalent functionality?" I am using OMI as the most formally worked out MUMPS database handler. I have copied the OMI standard document format up to a point. I do this because OMI is the one place in the MUMPS standards where a boundary between MUMPS applications/interpreters and a global handler is clearly delineated. Also I am taking advantage of the analysis of the MUMPS experts that wrote and reviewed the OMI spec. Basically this specification is designed to support multiple OMI interfaces (as multiple handlers) in a way that is general enough so that any global handler can be substituted. I elected to use some data types that OMI specifies where they seemed general enough. Please read and comment. I am already writing code today so the sooner the better. I would appreciate both functional, and copy edit comments (gramar, syntax, spelling, unclear and/or ambiguous text etc. etc.) NOTE: For the sake of existing code I am writing two wrappers, one that allows existing application code to use the new manager without change. The other is a special handler that calls the current dbopen handler. I will test these with the existing QA tests. For emacs fans I have defined the right variables at the end of the file that allows you to use emacs outline mode on this file. I spent several days of 4th watch time making this work. I couldn't have written this document otherwise. Regards, Steve Morris Table of Contents Abstract Forward 1 - Scope, purpose and application 1.1 - Scope 1.2 - Purpose 1.3 - Application 2 - Normative references 3 - Definitions 3.1 - X11.2-1994 defintions 3.2 - handler 3.3 - environment 3.4 - channel 3.5 - channel name 3.6 - full global reference 4 - General description 5 - Relationship to OMI 6 - Interface header files 6.1 - gum_agent.h 6.2 - gum_handler.h 7 - Manager interface to agents 7.2 - Data structures. 7.2.1 - Global reference 7.2.2 - values 7.2.3 - User identification. 7.2.4 - transaction status. 7.3 - Transaction flow 7.4 - Function specifications. 7.4.1 - GUM management operations 7.4.1.1 - Initialization 7.4.1.2 - Channel list 7.4.1.3 - Cleanup 7.4.2 - Channel control operations 7.4.2.1 - Connect channel 7.4.2.2 - Channel status 7.4.2.3 - Channel disconnect 7.4.3 - agent operations 7.4.3.1 - query response 7.4.3.2 - Global update operations 7.4.4.2.1 - SET 7.4.4.2.2 - SET $PIECE 7.4.4.2.3 - SET $EXTRACT 7.4.4.2.4 - KILL 7.4.3.3 - Global fetch operations 7.4.3.3.1 - $GET 7.4.3.3.2 - $DEFINE 7.4.3.3.3 - $ORDER 7.4.3.3.4 - Reverse $ORDER 7.4.3.3.5 - $QUERY 7.4.3.4 - Lock operations 7.4.3.5.1 - LOCK 7.4.3.5.2 - UNLOCK 8 - Manager interface to handlers 8.1 - Channels and their relationship to handlers 8.2 - Data structures 8.2.1 - Global reference 8.2.2 - Channel context 8.2.3 - Channel switch 8.3 - Installing a handler 8.3.1 - 8.3.2 - 8.4 - Function specifications 8.4.1 - Configuration utilities 8.4.2 - control operations 8.4.3 - agent operations appendix B - Configuration files appendix B - gum_agent.h appendix C - gum_handler.h ----------------------------------------------------------------------------- 0.1 - Abstract GUM defines a global data manager which allows a MUMPS interpreter or other program to interface to one or more global data handlers in a common way. GUM defines two C callable interfaces, one for user programs and one for global handlers. 0.2 - Forward GUM provides standard interfaces between interpreters and other programs (referred to in this document as agents) and one or more global handlers. This specification defines two (C callable) software interfaces; the interface between agents and GUM, and interfaces between GUM and global handlers. It also defines the functionality that must be provided by the global manager through these interfaces. This document is based on and frequently refers to X11.3-1994 (OMI) and related documents which define a method for network access to MUMPS database by agents. OMI is used in this document as a model for the services that a global handler must provide to an agent. OMI is not a part of this specification. It is expected that there will be OMI handlers but that is out of the scope of this document. For the sake of this document a global handler is software that provides the same kind of functionality to agents that OMI does. The critical additional support provided by GUM is the transparent support of multiple global handlers. The primary purpose of this documents dependance on X11.3 is that we want to depend on a standards document which describes a minimum useful set of MUMPS database functions so that we don't have to duplicate the effort of defining such a set of functions. Contents ??? 1 - Scope, purpose and application 1.1 - Scope The Global Manager is software that implements a an interface that allows applications to transparently use one or more global database handlers, possibly from many vendors. Application code (possibly a MUMPS interpreter) uses the Global manager Interface which hide implementation level details. Backend interfaces called handlers translate from the standard interface to the specifics of a specific global database interface. This document specifies the application or agent software interface and the handler interface and describes the functional relationship between them. These interfaces provide all basic operations on the sparse tree-structured MUMPS database. These operations are not fully documented here even though they are implemented by the Global Manager. To completely understand Global manager functionality it is necessary to understand the underlying MUMPS functionality as described in ANSI/MDC X11.1. In some cases full X11.1 functionality is not supplied. In these cases the agent must supply the additional functionality. In such cases the implemented functionality is modeled after the equivalent functionality in X11.2-1994 (OMI.) Two examples of such functionality are naked references and multiple LOCKS. Neither are implemented by the Global manager. 1.2 - Purpose Because there are many vendor specify ways to access a MUMPS database a common software interface is desirable to allow access to a variety of such implementations without changing the agent software. 1.3 - Application Designed primarily to provide a common interface between the GUM interpreter and various MUMPS database implementations, the Global Manager may also be used by other software to gain access to a mumps database or to provide a MUMPS database service. 2 - Normative references ANSI/MDC X11.2-1994 Communications Protocol - Open Mumps Interconnect (OMI) ANSI/MDC X11.1 3 - Definitions 3.1 - X11.2-1994 defintions This document uses several terms defined in ANSI/MDC X11.2-1994 Section 3, including but possibly not limited to client, agent, session, user, user ID, group ID. 3.2 - handler A handler is a software service that provides basic database operations in accordance with the handler interface specification in this documents. One handler might consist of a library of functions that access a database directly. Another handler might provide an interface to OMI client software. Handler software may support multiple instances, i.e it may be possible to open a handler for multiple database files or an OMI handler may support connections to multiple servers. 3.3 - environment An environment is a string which specifies a namespace of global variable names. 3.4 - channel A channel is a specific instance of a global manager connection. Each channel has its own environment. 3.5 - channel name 3.6 - full global reference A full global reference consists of a channel name, and environment, a global name and a list of subscripts. Channels are specified by a name string. 4 - General description The Global Manager supplier MUMPS database services to an agent. The global manager acts as an intermediary between agent software and one or more global database handlers. Agent software accesses the Global manager by opening one or more channels. Each channel has its environment and related global namespace. A complete specification of a global node thus consists of a channel name string, an enviroment string, a global name and a list of subscripts. Each handler will support one or more channels. For example an OMI handler may support access to one or more server. The mapping of channels names to handler is configured at startup by reading a configuration file. The channel name is provided to the handler at connect time and is used to access handler specific configuration information. It may be an error to assign more than one channel to a handler. It is not an error to open the same channel multiple times. A handler can be opened once per client (i,e. to support multiple users in an interpreter) or multiple times for the same client. The initial purpose for allowing multiple connects for the same user is to support a special purpose mapping handler. A mapping handler translates an agent specified full global reference to a different full global reference and makes a recursive call to the global manager to satisfy the new request. This special handler allows administrators to map databases from on server to another, or to map one or more globals onto a node of another global. The details of this mapping handler will be discussed in appendix ??? when I write it. To support multitasking access the Global manager supports non blocking asynchronous transactions. To do this the global manager uses a request/response dialog. An agent first makes a request for service which returns a unique transaction identifier. The agent then asks for the result of that transaction in a separate call. The agent can optionally block on a request. Handlers which do not support nonblocking will silently block regardless of whether blocking is requested. It curently an error to have multiple transactions outstanding. This cancels much of the benefit of blocking calls, however is expected that this restriction will be lifted in a future release. 5 - Relationship to OMI OMI is not part of this specification although it is possible to implement an OMI client interface as a Global Handler. The OMI specification is used here as a reference for the definition of various terms, data types and a specification of required handler functionality. It is recognized that OMI supports the same functionality that any Global Handler needs to support. The model used to define the global manager interfaces is one agent talking to multiple OMI client connections. 6 - Interface header files There are two header files which define the C callable interfaces. 6.1 - gum_agent.h 6.2 - gum_handler.h 7 - Manager interface to agents 7.2 - Data structures. 7.2.1 - Global reference A global reference is a string of characters that specifies an environment, a global name and 0 or more subscripts. The format of a global reference is identical to the OMI global reference documented in X11.2-1994 Section 5.3.3. 7.2.2 - values A values structure contains parameters that constrain a connection. These values are passed in by an agent. The handler responds with the agent value if possible, If the handler can't respond with all values in the requestes ranges it returns an error. typedef struct { int minValues; int maxValues; int minReference; int maxReference; int minSubscript; int maxSubscript; } values; 7.2.3 - User identification. A userSpec is a structure that contains all of the information necessary to identify a user for database access and locking. typedef struct { int user; int group; char job[]; /* $JOB string. */ } userSpec; 7.2.4 - transaction status. typedef enum { } status; 7.3 - Transaction flow There are three levels of GUM interface functions. The outer level concern all of GUM (initialization and cleanup.) The middle level concern themselves with the management of a specific channel (connection, status and disconnect.) The lowest level 7.4 - Function specifications. 7.4.1 - GUM management operations 7.4.1.1 - Initialization status GUM_gumInit(void) Initializes The Global Manager. Reads the configuration file. Loads and initializes handlers. 7.4.1.2 - Channel list char *GUM_channelOrder(char *) channelOrder allows applications to walk the list of available handlers. 7.4.1.3 - Cleanup status GUM_gumTerminate(void) Unloads all handlers, frees up memory and cleans up gracefully. 7.4.2 - Channel control operations 7.4.2.1 - Connect channel int GUM_channelConnect(char *channelName, userSpec u, values *v) The channel connect operation establishes a connection between a specific user job and an instance of a handler. Valid channel values are positive integers. A negative channel is a negated error value. The connect operation negotiates specific values of parameters indicated in the values structure. The requested values are passed into this function. channelConnect will edit these values as required by server limitations. The agent must examine the return value. The channel connect operation blocks until it succeeds or fails. 7.4.2.2 - Channel status This exists in OMI and returns server status but I can't figure out what server status means. There is no list of legal values. It seems to have been left out of the spec. If I can't figire out what it is for I will delete it from this spec. 7.4.2.3 - Channel disconnect status GUM_channelDisconnect(channel chan) The disconnect operation releases a connection. It blocks until it succeeds or fails. 7.4.3 - agent operations In a mumps interpreter which runs as a single task and supports multitasking internally it is not desirable for operations run on behalf of a task to block waiting for a resource. This wastes time that could be used by other tasks. The Global Manager resolves this problem by doing asynchronous I/O. Transactions are requested in one function call and then tested for completion later with another. Each function implementing an agent request returns a transaction identifier which is used by the query routine to test for the correct response. This strategy closely models OMI transactions which send a request packet and wait for a response packet. This is only useful if handlers can support asychronous transactions. Otherwise the request will be serviced in the request function call and the query is merely a formality. However the query cannot be skipped in these cases since only on transaction is allowed to be pending on a channel. The query clears the pending status. 7.4.3.1 - query response status GUM_queryResponse(channel chan, transaction tran, bool block) The query operation tests for and optionally waits for the completion of a transaction that was started previously. If block is true queryResponse will not return until the transaction completes. 7.4.3.2 - Global update operations Global update operations change the contents of the database by adding a value to a node, modifying an existing value, or deleting a node. Deleting a node deletes all of the nodes below it. 7.4.4.2.1 - SET transaction GUM_set(channel chan, gref, value) The set operation assigns a value to a global variable creating the variable if it doesn't exist. 7.4.4.2.2 - SET $PIECE transaction GUM_setPiece(channel chan, gref, value, start, end, delimiter) 7.4.4.2.3 - SET $EXTRACT transaction GUM_setExtact(channel chan, gref, value, start, end) 7.4.4.2.4 - KILL transaction GUM_kill(channel, gref) The kill operation deletes a global variable and all of its descendants 7.4.3.3 - Global fetch operations 7.4.3.3.1 - $GET transaction GUM_get(channel chan, gref, &value) The get operation returns the value of a global variable. Query will return an error if the global variable does not have a value. 7.4.3.3.2 - $DEFINE transaction GUM_define(channel chan, gref, &nodeStat) The define operation indicates whether a global variable has a value and whether it has descendants. 7.4.3.3.3 - $ORDER transaction GUM_order(channel chan, gref, &subscript) The order operation returns the following global name if thee are no subscripts in gref and the following subscript otherwise. 7.4.3.3.4 - Reverse $ORDER transaction GUM_revOrder(channel chan, gref, &subscript) Identical to order except that the collating sequence is reversed. 7.4.3.3.5 - $QUERY transaction GUM_query(channel chan, gref, rref) 7.4.3.4 - Lock operations 7.4.3.5.1 - LOCK transaction GUM_lock(channel chan, gref) The lock operation claims exclusice use of an nref which has the form of a global reference. Unlike the MUMPS LOCK command this transaction will succeed or fail. It will not wait until a pending lock is release. 7.4.3.5.2 - UNLOCK transaction GUM_unlock(channel chan, gref) The unlock operation releases a claimed nref. 8 - Manager interface to handlers 8.1 - Channels and their relationship to handlers 8.2 - Data structures 8.2.1 - Global reference 8.2.2 - Channel context 8.2.3 - Channel switch A handler switch is a structure which contains pointers to all of the information required to access a handler. It includes pointers to handler functions and data including the handler name. The handler name is used to identify a particular handler. A handler name also defines a configuration namespace. 8.3 - Installing a handler 8.3.1 - 8.3.2 - 8.4 - Function specifications 8.4.1 - Configuration utilities 8.4.2 - control operations 8.4.3 - agent operations appendix B - Configuration files appendix B - gum_agent.h appendix C - gum_handler.h typedef struct { char *name; /* This name is used to map configuration data onto specific handlers. */ void *internal; /* Pointer to internal data. This could be useful when installing reentrant handler code multiple times. */ connect status disconnect } handlerSwitch; appendix Z - emacs magic ----------------------------------------------------------------------------- /* Emacs magic. Must be at end of file. Local Variables: mode: outline outline-regexp: "\\(appendix\\|[0-9]+\\(\\.[0-9]+\\)*\\)" eval: (defun outline-level () "Support n.n.n style headers." (setq num 0) (save-excursion (cond ((looking-at "[0-9]+") (progn (goto-char (match-end 0)) (setq num (1+ num)) (while (looking-at "\\.[0-9]+") (goto-char (match-end 0)) (setq num (1+ num))))) ((looking-at "appendix") (setq num 1)))) num) End: */