go-vise

dev.texi (12854B)

---

 @node dev
 @chapter Developing with vise
 
 
 @section Code repository structure
 
 @table @code
 @item asm
 Assembly parser and compiler.
 @item cache
 Holds and manages all loaded content.
 @item db
 Provides interface and implementations for data storage and retrieval backends.
 @item engine
 Outermost interface. Orchestrates execution of bytecode against input. 
 @item lang
 Validation and specification of language context.
 @item logging
 Logging interface and build tags for loglevels.
 @item persist
 Provides `state` and `cache` persistence across asynchronous vm executions.
 @item render
 Renders menu and templates, and enforces output size constraints.
 @item resource
 Resolves bytecode, translations, templates and menu symbols from external symbols.
 @item state
 Holds the bytecode buffer, error states and navigation states.
 @item vm
 Defines instructions, and applies transformations according to the instructions.
 @end table
 
 
 @section Interacting with @code{vise}
 
 Implementers of @code{vise} should interface with the system using the @code{engine} module.
 
 The engine comes in two implementations, one volatile base implemetnation and a subclass that includes persistent state.
 
 
 @subsection Modes of operation
 
 The @code{engine} module provides three different modes of operation for the engine implementations.
 
 
 @subsubsection Manual operation
 
 Directly interaction with an @code{engine.Engine} instance.
 
 The engine is manually initialized, and execution must be explicitly triggered with input every time the VM yields control.
 
 Output flushing must also be operated manually.
 
 The interface is the same for both persistent and volatile operation.
 
 
 @subsubsection Synchronous loop
 
 Receives input from a reader and writes into to a writer, and executes the underlying @code{engine.Engine} with given inputs until execution is terminated.
 
 The loop may be either persistent or volatile.
 
 This mode drives the interactive driver execution tool. 
 
 
 @subsubsection Asynchronous one-shot
 
 Compatible with e.g. a network socket or HTTP frontend. The @code{engine.RunPersisted} method restores a persisted state and runs one single input until VM yield after which the new state is persisted.
 
 This mode of operation can only be used with persistent state.
 
 
 @subsection Configuration
 
 The engine configuration defines the top-level parameters for the execution environment, including maximum output size, default language, execution entry point and more.
 
 Please refer to @code{engine.Config} for details.
 
 
 @anchor{sessions}
 @subsection Sessions
 
 The @code{engine.Config.SessionId} is used to disambiguate the end-user that is interacting with the engine.
 
 For example, in a @abbr{USSD} context, the @code{SessionId} may be the @emph{phone number} of the end-user.
 
 
 @anchor{execution_context}
 @subsection Execution context
 
 The engine stores the @code{SessionId} aswell as the current chosen @code{lang.Language} in the execution context. This is passed through to the VM operation, and is available for client code, specifically:
 
 @itemize
 @item When resolving symbols with @code{LOAD}. (@code{resource.EntryFunc}).
 @item When resolving menu symbols (@code{resource.Resource.GetMenu}).
 @item When retrieving node templates (@code{resource.Resource.GetTemplate}).
 @end itemize
 
 
 @subsection Blocking execution 
 
 Using the @code{engine.SetFirst()} method, a function may be defined that executes before the pending bytecode in the VM state.
 
 The function uses the same signature as the external functions executed by @code{resource} for @code{LOAD} instructions.
 
 This can be for example be used to prevent access to execution for a blocked user account, or as an override while doing maintenance.
 
 To prevent VM execution from the pre-VM check, the flag @code{TERMINATE} should be set in the @code{resource.Result.FlagSet} array.
 
 
 @section Resolving resources
 
 The core of implementation code is defined by implementing the @code{resource.Resource} interface. This is also described in the @ref{load_handler, LOAD handler} section.
 
 In addition to resolving external code symbols, @code{resource.Resource} implementations also translate @emph{menu labels} and @emph{templates} based on the current language context, and retrieves bytecode for execution nodes.
 
 @subsection Memory resource implementation
 
 One of two reference implementations of @code{resource.Resource} is the @code{resource.MemResource} class. It enables the client to register all node and symbol resolutions at runtime, using its functions prefixed with @code{Add...}. 
 
 The @code{resource.MemResource} implementation is primarily useful for use in tests.
 
 
 @subsection Filesystem resource implementation
 
 The Filesystem based resource implemementation is used by the @code{dev/interactive} tool, aswell as the executable examples in @file{examples/} directory.
 
 It is instantiated with a base directory location relative to which all resources are read.
 
 
 @subsubsection Bytecode (@code{resource.Resource.GetCode})
 
 Read from @file{basedir/<node>.bin}.
 
 
 @subsubsection Templates (@code{resource.Resource.GetTemplate})
 
 If language has been set, the template will be read from @file{basedir/<node>_<lang>}. For example, the @emph{norwegian} template for the node @code{root} will be read from @file{basedir/root_nor}.
 
 If reading the language specific template fails (or if no language has been set), template will be read from @file{basedir/<node>}.
 
 A missing template file will result in load failure and program termination.
 
 
 @subsubsection Menus (@code{resource.Resource.GetMenu})
 
 If language has been set, the template will be read from @file{basedir/<label>_<lang>_menu}. For example, the @emph{norwegian} template for the menu label @code{foo} will be read from @file{basedir/foo_nor_menu}.
 
 If reading the language specific menu label fails (or if no language has been set), label will be read from @file{basedir/<label>_menu}.
 
 If this also fails, the implementation returns the original label used for lookup.
 
 
 @subsubsection External symbols (@code{resource.Resource.FuncFor})
 
 The implementation allows setting resolver functions for symbols at runtime, using the @code{resource.FsResource.AddLocalFunc} method. This registers an @code{resource.FsResource.EntryFunc} with the lookup symbol as key. Note that the @code{EntryFunc} receives the language setting through the execution context.
 
 If no function has been registered for the requested symbol, it will be looked up in the filesystem on @file{basedir/<symbol>_<lang>.txt}. For example, the @emph{norwegian} entry for the symbol @code{foo} will be read from @file{basedir/foo_nor.txt}.
 
 If reading the language specific entry fails (or if no language has been set), entry will be read from @file{basedir/<symbol>.txt}.
 
 A missing entry will result in load failure and program termination.
 
 The implementation contains no built-in handling of the @code{SessionId} supplied by the context.
 
 
 @section Data provider
 
 The @code{db.Db} interface provides methods to get and set data to key-value stores.
 
 The storage keys are partitioned according to the @ref{sessions, session} context, aswell as what type of data is being stored or retrieved.
 
 The interface and the data types are defined in @code{db/db.go}.
 
 The included implementations are:
 
 @table @code
 @item MemDb
 An volatile, in-process store. Used in most tests.
 @item FsDb
 A filesystem-backed store using subdirectories to separate sessions.
 @item GdbmDb
 A @url{https://www.gnu.org/software/gdbm/gdbm,gdbm} backed store.
 @item PgDb
 A @url{https://www.postgresql.org/,Postgres} backed store, using a single table with two @code{BYTEA} columns and a connection pool.
 @end table
 
 
 @subsection Uses
 
 @code{db.Db} may fulfill all local data requirements in @code{vise}, including:
 
 @itemize
 @item Resource retrieval
 @item State and cache persistence
 @item Application data
 @end itemize
 
 
 @subsection Using data provider with resources
 
 The @code{resource.dbGetter} assists in using a @code{db.Db} implementation.
 
 Its functions may be assigned individually to a @code{resource.MenuResource}, allowing for co-existence of @code{db.Db} backed resources, aswell as from other sources.
 
 
 @subsection State persistence
 
 Any asynchronous or consecutive synchronous operation of the @code{engine.Engine} requires persistence of the associated @code{state.State} and @code{cache.Memory}. This is achieved using @code{persist.Persister}, instantiated with a @code{db.Db} implementation.
 
 The @code{db.Db} used for persistence does not need to be the same as e.g. used for retrieval of resources, or even for application data.
 
 
 @section Logging
 
 Loglevels are set at compile-time using the following build tags:
 
 @itemize
 @item @code{lognone}
 @item @code{logerror}
 @item @code{logwarn}
 @item @code{loginfo}
 @item @code{logdebug}
 @item @code{logtrace}
 @end itemize
 
 Only use @strong{ONE} of these tags.
 
 The default tag is @code{lognone} which disables logging completely.
 
 @code{logging.Logger} defines the logging interface. It is faintly inspired by the experimental @url{https://pkg.go.dev/golang.org/x/exp/slog) package, in that it differentiates explicit context logging, slog}.
 
 
 @section Tools
 
 Located in the @file{dev/} directory of the source code repository. 
 
 
 @subsection Test data generation
 
 @example
 go run ./dev/gendata/ <directory>
 @end example
 
 Outputs bytecodes and templates for test data scenarios used in `engine` unit tests.
 
 
 @subsection Interactive runner
 
 @example
 go run ./dev/interactive [-d <data_directory>] [--root <root_symbol>] [--session-id <session_id>] [--persist]
 @end example
 
 Creates a new interactive session using @code{engine.DefaultEngine}, starting execution at symbol @code{root_symbol}
 
 @code{data_directory} points to a directory where templates and bytecode is to be found (in the same format as generated by @file{dev/gendata}).
 
 If @code{data_directory} is not set, current directory will be used.
 
 if @code{root_symbol} is not set, the symbol @code{root} will be used.
 
 if @code{session_id} is set, mutable data will be stored and retrieved keyed by the given identifer (if implemented).
 
 If @code{persist} is set, the execution state will be persisted across sessions.
 
 
 @subsection Assembler
 
 @example
 go run ./dev/asm <assembly_file>
 @end example
 
 Will output bytecode on STDOUT generated from a valid assembly file.
 
 
 @subsection Disassembler
 
 @example
 go run ./dev/disasm/ <binary_file>
 @end example
 
 Will list all the instructions on STDOUT from a valid binary file.
 
 
 @subsection Interactive case examples
 
 Found in @file{examples/}.
 
 Be sure to @code{make examples} before running them.
 
 Can be run with:
 
 @example
 go run ./examples/<case> [...]
 @end example
 
 except helloworld which is run as
 
 @example
 go run ./dev/interactive -d ./examples/helloworld [...]
 @end example
 
 The available options are the same as for the @file{dev/interactive} tool.
 
 Contents of the case directory:
 
 @table @file
 @item *.vis
 assembly code.
 @item *.bin
 bytecode for each node symbol (only available after make).
 @item *.txt.orig
 default contents of a single data entry.
 @item *.txt
 current contents of a single data entry (only available after make).
 @end table
 
 
 @section Assembly examples
 
 See @file{testdata/*.vis}
 
 
 @section Bytecode example
 
 Currently the following rules apply for encoding in version @code{0}:
 
 @itemize
 @item A code instruction is a @emph{big-endian} 2-byte value. See @file{vm/opcodes.go} for valid opcode values.
 @item @code{symbol} value is encoded as @emph{one byte} of string length, after which the  byte-value of the string follows.
 @item @code{size} value is encoded as @emph{one byte} of numeric length, after which the @emph{big-endian} byte-value of the integer follows.
 @item @code{signal} value is encoded as @emph{one byte} of byte length, after which a byte-array representing the defined signal follows.
 @end itemize
 
 
 @subsection Example
 
 (Minimal, WIP)
 
 @verbatim
 000a 03666f6f 05746f666f6f    # MOUT tofoo foo  - display a menu entry for choice "foo", described by "to foo"
 0008 03666f6f 03626172        # INCMP bar foo   - move to node "bar" if input is "FOO"
 0001 0461696565 01 01         # CATCH aiee 1 1  - move to node "aiee" (and immediately halt) if input match flag (1) is set (1)
 0003 04616263 020104          # LOAD abc 260    - execute code symbol "abc" with a result size limit of 260 (2 byte BE integer, 0x0104)
 0003 04646566 00              # LOAD def 0      - execute code symbol "abc" with no size limit (sink)
 0005 04616263                 # MAP abc         - make "abc" available for renderer
 0007                          # HALT            - stop execution (require new input to continue)
 0006 0461313233               # MOVE a123       - move to node "a123" (regardless of input)
 0007                          # HALT            - stop execution
 @end verbatim