dev.html (22648B)
1 <!DOCTYPE html> 2 <html> 3 <!-- Created by GNU Texinfo 7.1, https://www.gnu.org/software/texinfo/ --> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> 6 <!-- Released 2023 under AGPL3 --> 7 <title>dev (vise)</title> 8 9 <meta name="description" content="dev (vise)"> 10 <meta name="keywords" content="dev (vise)"> 11 <meta name="resource-type" content="document"> 12 <meta name="distribution" content="global"> 13 <meta name="Generator" content="makeinfo"> 14 <meta name="viewport" content="width=device-width,initial-scale=1"> 15 16 <link href="index.html" rel="start" title="Top"> 17 <link href="index.html#SEC_Contents" rel="contents" title="Table of Contents"> 18 <link href="index.html" rel="up" title="Top"> 19 <link href="cookbook.html" rel="prev" title="cookbook"> 20 <style type="text/css"> 21 <!-- 22 a.copiable-link {visibility: hidden; text-decoration: none; line-height: 0em} 23 div.example {margin-left: 3.2em} 24 span:hover a.copiable-link {visibility: visible} 25 ul.mark-bullet {list-style-type: disc} 26 --> 27 </style> 28 29 30 </head> 31 32 <body lang="en"> 33 <div class="chapter-level-extent" id="dev"> 34 <div class="nav-panel"> 35 <p> 36 Previous: <a href="cookbook.html" accesskey="p" rel="prev">Common patterns</a>, Up: <a href="index.html" accesskey="u" rel="up">Introduction</a> [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>]</p> 37 </div> 38 <hr> 39 <h2 class="chapter" id="Developing-with-vise"><span>11 Developing with vise<a class="copiable-link" href="#Developing-with-vise"> ¶</a></span></h2> 40 41 42 <ul class="mini-toc"> 43 <li><a href="#Code-repository-structure" accesskey="1">Code repository structure</a></li> 44 <li><a href="#Interacting-with-vise" accesskey="2">Interacting with <code class="code">vise</code></a></li> 45 <li><a href="#Resolving-resources" accesskey="3">Resolving resources</a></li> 46 <li><a href="#Logging" accesskey="4">Logging</a></li> 47 <li><a href="#Tools" accesskey="5">Tools</a></li> 48 <li><a href="#Assembly-examples" accesskey="6">Assembly examples</a></li> 49 <li><a href="#Bytecode-example" accesskey="7">Bytecode example</a></li> 50 </ul> 51 <div class="section-level-extent" id="Code-repository-structure"> 52 <h3 class="section"><span>11.1 Code repository structure<a class="copiable-link" href="#Code-repository-structure"> ¶</a></span></h3> 53 54 <dl class="table"> 55 <dt><code class="code">asm</code></dt> 56 <dd><p>Assembly parser and compiler. 57 </p></dd> 58 <dt><code class="code">cache</code></dt> 59 <dd><p>Holds and manages all loaded content. 60 </p></dd> 61 <dt><code class="code">engine</code></dt> 62 <dd><p>Outermost interface. Orchestrates execution of bytecode against input. 63 </p></dd> 64 <dt><code class="code">lang</code></dt> 65 <dd><p>Validation and specification of language context. 66 </p></dd> 67 <dt><code class="code">logging</code></dt> 68 <dd><p>Logging interface and build tags for loglevels. 69 </p></dd> 70 <dt><code class="code">persist</code></dt> 71 <dd><p>Interface and reference implementation of ‘state‘ and ‘cache‘ persistence across asynchronous vm executions. 72 </p></dd> 73 <dt><code class="code">render</code></dt> 74 <dd><p>Renders menu and templates, and enforces output size constraints. 75 </p></dd> 76 <dt><code class="code">resource</code></dt> 77 <dd><p>Retrieves data and bytecode from external symbols, and retrieves templates. 78 </p></dd> 79 <dt><code class="code">state</code></dt> 80 <dd><p>Holds the bytecode buffer, error states and navigation states. 81 </p></dd> 82 <dt><code class="code">vm</code></dt> 83 <dd><p>Defines instructions, and applies transformations according to the instructions. 84 </p></dd> 85 </dl> 86 87 88 </div> 89 <div class="section-level-extent" id="Interacting-with-vise"> 90 <h3 class="section"><span>11.2 Interacting with <code class="code">vise</code><a class="copiable-link" href="#Interacting-with-vise"> ¶</a></span></h3> 91 92 <p>Implementers of <code class="code">vise</code> should interface with the system using the <code class="code">engine</code> module. 93 </p> 94 <p>The engine comes in two implementations, one volatile base implemetnation and a subclass that includes persistent state. 95 </p> 96 97 <ul class="mini-toc"> 98 <li><a href="#Modes-of-operation" accesskey="1">Modes of operation</a></li> 99 <li><a href="#Configuration" accesskey="2">Configuration</a></li> 100 <li><a href="#Sessions" accesskey="3">Sessions</a></li> 101 <li><a href="#Execution-context" accesskey="4">Execution context</a></li> 102 </ul> 103 <div class="subsection-level-extent" id="Modes-of-operation"> 104 <h4 class="subsection"><span>11.2.1 Modes of operation<a class="copiable-link" href="#Modes-of-operation"> ¶</a></span></h4> 105 106 <p>The <code class="code">engine</code> module provides three different modes of operation for the engine implementations. 107 </p> 108 109 <ul class="mini-toc"> 110 <li><a href="#Manual-operation" accesskey="1">Manual operation</a></li> 111 <li><a href="#Synchronous-loop" accesskey="2">Synchronous loop</a></li> 112 <li><a href="#Asynchronous-one_002dshot" accesskey="3">Asynchronous one-shot</a></li> 113 </ul> 114 <div class="subsubsection-level-extent" id="Manual-operation"> 115 <h4 class="subsubsection"><span>11.2.1.1 Manual operation<a class="copiable-link" href="#Manual-operation"> ¶</a></span></h4> 116 117 <p>Directly interaction with an <code class="code">engine.Engine</code> instance. 118 </p> 119 <p>The engine is manually initialized, and execution must be explicitly triggered with input every time the VM yields control. 120 </p> 121 <p>Output flushing must also be operated manually. 122 </p> 123 <p>The interface is the same for both persistent and volatile operation. 124 </p> 125 126 </div> 127 <div class="subsubsection-level-extent" id="Synchronous-loop"> 128 <h4 class="subsubsection"><span>11.2.1.2 Synchronous loop<a class="copiable-link" href="#Synchronous-loop"> ¶</a></span></h4> 129 130 <p>Receives input from a reader and writes into to a writer, and executes the underlying <code class="code">engine.Engine</code> with given inputs until execution is terminated. 131 </p> 132 <p>The loop may be either persistent or volatile. 133 </p> 134 <p>This mode drives the interactive driver execution tool. 135 </p> 136 137 </div> 138 <div class="subsubsection-level-extent" id="Asynchronous-one_002dshot"> 139 <h4 class="subsubsection"><span>11.2.1.3 Asynchronous one-shot<a class="copiable-link" href="#Asynchronous-one_002dshot"> ¶</a></span></h4> 140 141 <p>Compatible with e.g. a network socket or HTTP frontend. The <code class="code">engine.RunPersisted</code> method restores a persisted state and runs one single input until VM yield after which the new state is persisted. 142 </p> 143 <p>This mode of operation can only be used with persistent state. 144 </p> 145 146 </div> 147 </div> 148 <div class="subsection-level-extent" id="Configuration"> 149 <h4 class="subsection"><span>11.2.2 Configuration<a class="copiable-link" href="#Configuration"> ¶</a></span></h4> 150 151 <p>The engine configuration defines the top-level parameters for the execution environment, including maximum output size, default language, execution entry point and more. 152 </p> 153 <p>Please refer to <code class="code">engine.Config</code> for details. 154 </p> 155 156 </div> 157 <div class="subsection-level-extent" id="Sessions"> 158 <h4 class="subsection"><span>11.2.3 Sessions<a class="copiable-link" href="#Sessions"> ¶</a></span></h4> 159 160 <p>The <code class="code">engine.Config.SessionId</code> is used to disambiguate the end-user that is interacting with the engine. 161 </p> 162 <p>For example, in a <abbr class="abbr">USSD</abbr> context, the <code class="code">SessionId</code> may be the <em class="emph">phone number</em> of the end-user. 163 </p> 164 165 <a class="anchor" id="execution_005fcontext"></a></div> 166 <div class="subsection-level-extent" id="Execution-context"> 167 <h4 class="subsection"><span>11.2.4 Execution context<a class="copiable-link" href="#Execution-context"> ¶</a></span></h4> 168 169 <p>The engine stores the <code class="code">SessionId</code> aswell as the current chosen <code class="code">lang.Language</code> in the execution context. This is passed through to the VM operation, and is available for client code, specifically: 170 </p> 171 <ul class="itemize mark-bullet"> 172 <li>When resolving symbols with <code class="code">LOAD</code>. (<code class="code">resource.EntryFunc</code>). 173 </li><li>When resolving menu symbols (<code class="code">resource.Resource.GetMenu</code>). 174 </li><li>When retrieving node templates (<code class="code">resource.Resource.GetTemplate</code>). 175 </li></ul> 176 177 178 </div> 179 </div> 180 <div class="section-level-extent" id="Resolving-resources"> 181 <h3 class="section"><span>11.3 Resolving resources<a class="copiable-link" href="#Resolving-resources"> ¶</a></span></h3> 182 183 <p>The core of implementation code is defined by implementing the <code class="code">resource.Resource</code> interface. This is also described in the <a class="ref" href="cache.html#load_005fhandler">LOAD handler</a> section. 184 </p> 185 <p>In addition to resolving external code symbols, <code class="code">resource.Resource</code> implementations also translate <em class="emph">menu labels</em> and <em class="emph">templates</em> based on the current language context, and retrieves bytecode for execution nodes. 186 </p> 187 <ul class="mini-toc"> 188 <li><a href="#Memory-resource-implementation" accesskey="1">Memory resource implementation</a></li> 189 <li><a href="#Filesystem-resource-implementation" accesskey="2">Filesystem resource implementation</a></li> 190 </ul> 191 <div class="subsection-level-extent" id="Memory-resource-implementation"> 192 <h4 class="subsection"><span>11.3.1 Memory resource implementation<a class="copiable-link" href="#Memory-resource-implementation"> ¶</a></span></h4> 193 194 <p>One of two reference implementations of <code class="code">resource.Resource</code> is the <code class="code">resource.MemResource</code> class. It enables the client to register all node and symbol resolutions at runtime, using its functions prefixed with <code class="code">Add...</code>. 195 </p> 196 <p>The <code class="code">resource.MemResource</code> implementation is primarily useful for use in tests. 197 </p> 198 199 </div> 200 <div class="subsection-level-extent" id="Filesystem-resource-implementation"> 201 <h4 class="subsection"><span>11.3.2 Filesystem resource implementation<a class="copiable-link" href="#Filesystem-resource-implementation"> ¶</a></span></h4> 202 203 <p>The Filesystem based resource implemementation is used by the <code class="code">dev/interactive</code> tool, aswell as the executable examples in <samp class="file">examples/</samp> directory. 204 </p> 205 <p>It is instantiated with a base directory location relative to which all resources are read. 206 </p> 207 208 <ul class="mini-toc"> 209 <li><a href="#Bytecode-_0028resource_002eResource_002eGetCode_0029" accesskey="1">Bytecode (<code class="code">resource.Resource.GetCode</code>)</a></li> 210 <li><a href="#Templates-_0028resource_002eResource_002eGetTemplate_0029" accesskey="2">Templates (<code class="code">resource.Resource.GetTemplate</code>)</a></li> 211 <li><a href="#Menus-_0028resource_002eResource_002eGetMenu_0029" accesskey="3">Menus (<code class="code">resource.Resource.GetMenu</code>)</a></li> 212 <li><a href="#External-symbols-_0028resource_002eResource_002eFuncFor_0029" accesskey="4">External symbols (<code class="code">resource.Resource.FuncFor</code>)</a></li> 213 </ul> 214 <div class="subsubsection-level-extent" id="Bytecode-_0028resource_002eResource_002eGetCode_0029"> 215 <h4 class="subsubsection"><span>11.3.2.1 Bytecode (<code class="code">resource.Resource.GetCode</code>)<a class="copiable-link" href="#Bytecode-_0028resource_002eResource_002eGetCode_0029"> ¶</a></span></h4> 216 217 <p>Read from <samp class="file">basedir/<node>.bin</samp>. 218 </p> 219 220 </div> 221 <div class="subsubsection-level-extent" id="Templates-_0028resource_002eResource_002eGetTemplate_0029"> 222 <h4 class="subsubsection"><span>11.3.2.2 Templates (<code class="code">resource.Resource.GetTemplate</code>)<a class="copiable-link" href="#Templates-_0028resource_002eResource_002eGetTemplate_0029"> ¶</a></span></h4> 223 224 <p>If language has been set, the template will be read from <samp class="file">basedir/<node>_<lang></samp>. For example, the <em class="emph">norwegian</em> template for the node <code class="code">root</code> will be read from <samp class="file">basedir/root_nor</samp>. 225 </p> 226 <p>If reading the language specific template fails (or if no language has been set), template will be read from <samp class="file">basedir/<node></samp>. 227 </p> 228 <p>A missing template file will result in load failure and program termination. 229 </p> 230 231 </div> 232 <div class="subsubsection-level-extent" id="Menus-_0028resource_002eResource_002eGetMenu_0029"> 233 <h4 class="subsubsection"><span>11.3.2.3 Menus (<code class="code">resource.Resource.GetMenu</code>)<a class="copiable-link" href="#Menus-_0028resource_002eResource_002eGetMenu_0029"> ¶</a></span></h4> 234 235 <p>If language has been set, the template will be read from <samp class="file">basedir/<label>_<lang>_menu</samp>. For example, the <em class="emph">norwegian</em> template for the menu label <code class="code">foo</code> will be read from <samp class="file">basedir/foo_nor_menu</samp>. 236 </p> 237 <p>If reading the language specific menu label fails (or if no language has been set), label will be read from <samp class="file">basedir/<label>_menu</samp>. 238 </p> 239 <p>If this also fails, the implementation returns the original label used for lookup. 240 </p> 241 242 </div> 243 <div class="subsubsection-level-extent" id="External-symbols-_0028resource_002eResource_002eFuncFor_0029"> 244 <h4 class="subsubsection"><span>11.3.2.4 External symbols (<code class="code">resource.Resource.FuncFor</code>)<a class="copiable-link" href="#External-symbols-_0028resource_002eResource_002eFuncFor_0029"> ¶</a></span></h4> 245 246 <p>The implementation allows setting resolver functions for symbols at runtime, using the <code class="code">resource.FsResource.AddLocalFunc</code> method. This registers an <code class="code">resource.FsResource.EntryFunc</code> with the lookup symbol as key. Note that the <code class="code">EntryFunc</code> receives the language setting through the execution context. 247 </p> 248 <p>If no function has been registered for the requested symbol, it will be looked up in the filesystem on <samp class="file">basedir/<symbol>_<lang>.txt</samp>. For example, the <em class="emph">norwegian</em> entry for the symbol <code class="code">foo</code> will be read from <samp class="file">basedir/foo_nor.txt</samp>. 249 </p> 250 <p>If reading the language specific entry fails (or if no language has been set), entry will be read from <samp class="file">basedir/<symbol>.txt</samp>. 251 </p> 252 <p>A missing entry will result in load failure and program termination. 253 </p> 254 <p>The implementation contains no built-in handling of the <code class="code">SessionId</code> supplied by the context. 255 </p> 256 257 </div> 258 </div> 259 </div> 260 <div class="section-level-extent" id="Logging"> 261 <h3 class="section"><span>11.4 Logging<a class="copiable-link" href="#Logging"> ¶</a></span></h3> 262 263 <p>Loglevels are set at compile-time using the following build tags: 264 </p> 265 <ul class="itemize mark-bullet"> 266 <li><code class="code">lognone</code> 267 </li><li><code class="code">logerror</code> 268 </li><li><code class="code">logwarn</code> 269 </li><li><code class="code">loginfo</code> 270 </li><li><code class="code">logdebug</code> 271 </li><li><code class="code">logtrace</code> 272 </li></ul> 273 274 <p>Only use <strong class="strong">ONE</strong> of these tags. 275 </p> 276 <p>The default tag is <code class="code">lognone</code> which disables logging completely. 277 </p> 278 <p><code class="code">logging.Logger</code> defines the logging interface. It is faintly inspired by the experimental <a class="url" href="https://pkg.go.dev/golang.org/x/exp/slog)%20package">slog</a>. 279 </p> 280 281 </div> 282 <div class="section-level-extent" id="Tools"> 283 <h3 class="section"><span>11.5 Tools<a class="copiable-link" href="#Tools"> ¶</a></span></h3> 284 285 <p>Located in the <samp class="file">dev/</samp> directory of the source code repository. 286 </p> 287 288 <ul class="mini-toc"> 289 <li><a href="#Test-data-generation" accesskey="1">Test data generation</a></li> 290 <li><a href="#Interactive-runner" accesskey="2">Interactive runner</a></li> 291 <li><a href="#Assembler" accesskey="3">Assembler</a></li> 292 <li><a href="#Disassembler" accesskey="4">Disassembler</a></li> 293 <li><a href="#Interactive-case-examples" accesskey="5">Interactive case examples</a></li> 294 </ul> 295 <div class="subsection-level-extent" id="Test-data-generation"> 296 <h4 class="subsection"><span>11.5.1 Test data generation<a class="copiable-link" href="#Test-data-generation"> ¶</a></span></h4> 297 298 <div class="example"> 299 <pre class="example-preformatted">go run ./dev/gendata/ <directory> 300 </pre></div> 301 302 <p>Outputs bytecodes and templates for test data scenarios used in ‘engine‘ unit tests. 303 </p> 304 305 </div> 306 <div class="subsection-level-extent" id="Interactive-runner"> 307 <h4 class="subsection"><span>11.5.2 Interactive runner<a class="copiable-link" href="#Interactive-runner"> ¶</a></span></h4> 308 309 <div class="example"> 310 <pre class="example-preformatted">go run ./dev/interactive [-d <data_directory>] [--root <root_symbol>] [--session-id <session_id>] [--persist] 311 </pre></div> 312 313 <p>Creates a new interactive session using <code class="code">engine.DefaultEngine</code>, starting execution at symbol <code class="code">root_symbol</code> 314 </p> 315 <p><code class="code">data_directory</code> points to a directory where templates and bytecode is to be found (in the same format as generated by <samp class="file">dev/gendata</samp>). 316 </p> 317 <p>If <code class="code">data_directory</code> is not set, current directory will be used. 318 </p> 319 <p>if <code class="code">root_symbol</code> is not set, the symbol <code class="code">root</code> will be used. 320 </p> 321 <p>if <code class="code">session_id</code> is set, mutable data will be stored and retrieved keyed by the given identifer (if implemented). 322 </p> 323 <p>If <code class="code">persist</code> is set, the execution state will be persisted across sessions. 324 </p> 325 326 </div> 327 <div class="subsection-level-extent" id="Assembler"> 328 <h4 class="subsection"><span>11.5.3 Assembler<a class="copiable-link" href="#Assembler"> ¶</a></span></h4> 329 330 <div class="example"> 331 <pre class="example-preformatted">go run ./dev/asm <assembly_file> 332 </pre></div> 333 334 <p>Will output bytecode on STDOUT generated from a valid assembly file. 335 </p> 336 337 </div> 338 <div class="subsection-level-extent" id="Disassembler"> 339 <h4 class="subsection"><span>11.5.4 Disassembler<a class="copiable-link" href="#Disassembler"> ¶</a></span></h4> 340 341 <div class="example"> 342 <pre class="example-preformatted">go run ./dev/disasm/ <binary_file> 343 </pre></div> 344 345 <p>Will list all the instructions on STDOUT from a valid binary file. 346 </p> 347 348 </div> 349 <div class="subsection-level-extent" id="Interactive-case-examples"> 350 <h4 class="subsection"><span>11.5.5 Interactive case examples<a class="copiable-link" href="#Interactive-case-examples"> ¶</a></span></h4> 351 352 <p>Found in <samp class="file">examples/</samp>. 353 </p> 354 <p>Be sure to <code class="code">make examples</code> before running them. 355 </p> 356 <p>Can be run with: 357 </p> 358 <div class="example"> 359 <pre class="example-preformatted">go run ./examples/<case> [...] 360 </pre></div> 361 362 <p>except helloworld which is run as 363 </p> 364 <div class="example"> 365 <pre class="example-preformatted">go run ./dev/interactive -d ./examples/helloworld [...] 366 </pre></div> 367 368 <p>The available options are the same as for the <samp class="file">dev/interactive</samp> tool. 369 </p> 370 <p>Contents of the case directory: 371 </p> 372 <dl class="table"> 373 <dt><samp class="file">*.vis</samp></dt> 374 <dd><p>assembly code. 375 </p></dd> 376 <dt><samp class="file">*.bin</samp></dt> 377 <dd><p>bytecode for each node symbol (only available after make). 378 </p></dd> 379 <dt><samp class="file">*.txt.orig</samp></dt> 380 <dd><p>default contents of a single data entry. 381 </p></dd> 382 <dt><samp class="file">*.txt</samp></dt> 383 <dd><p>current contents of a single data entry (only available after make). 384 </p></dd> 385 </dl> 386 387 388 </div> 389 </div> 390 <div class="section-level-extent" id="Assembly-examples"> 391 <h3 class="section"><span>11.6 Assembly examples<a class="copiable-link" href="#Assembly-examples"> ¶</a></span></h3> 392 393 <p>See <samp class="file">testdata/*.vis</samp> 394 </p> 395 396 </div> 397 <div class="section-level-extent" id="Bytecode-example"> 398 <h3 class="section"><span>11.7 Bytecode example<a class="copiable-link" href="#Bytecode-example"> ¶</a></span></h3> 399 400 <p>Currently the following rules apply for encoding in version <code class="code">0</code>: 401 </p> 402 <ul class="itemize mark-bullet"> 403 <li>A code instruction is a <em class="emph">big-endian</em> 2-byte value. See <samp class="file">vm/opcodes.go</samp> for valid opcode values. 404 </li><li><code class="code">symbol</code> value is encoded as <em class="emph">one byte</em> of string length, after which the byte-value of the string follows. 405 </li><li><code class="code">size</code> value is encoded as <em class="emph">one byte</em> of numeric length, after which the <em class="emph">big-endian</em> byte-value of the integer follows. 406 </li><li><code class="code">signal</code> value is encoded as <em class="emph">one byte</em> of byte length, after which a byte-array representing the defined signal follows. 407 </li></ul> 408 409 410 <ul class="mini-toc"> 411 <li><a href="#Example" accesskey="1">Example</a></li> 412 </ul> 413 <div class="subsection-level-extent" id="Example"> 414 <h4 class="subsection"><span>11.7.1 Example<a class="copiable-link" href="#Example"> ¶</a></span></h4> 415 416 <p>(Minimal, WIP) 417 </p> 418 <pre class="verbatim">000a 03666f6f 05746f666f6f # MOUT tofoo foo - display a menu entry for choice "foo", described by "to foo" 419 0008 03666f6f 03626172 # INCMP bar foo - move to node "bar" if input is "FOO" 420 0001 0461696565 01 01 # CATCH aiee 1 1 - move to node "aiee" (and immediately halt) if input match flag (1) is not set (1) 421 0003 04616263 020104 # LOAD abc 260 - execute code symbol "abc" with a result size limit of 260 (2 byte BE integer, 0x0104) 422 0003 04646566 00 # LOAD def 0 - execute code symbol "abc" with no size limit (sink) 423 0005 04616263 # MAP abc - make "abc" available for renderer 424 0007 # HALT - stop execution (require new input to continue) 425 0006 0461313233 # MOVE a123 - move to node "a123" (regardless of input) 426 0007 # HALT - stop execution 427 </pre></div> 428 </div> 429 </div> 430 <hr> 431 <div class="nav-panel"> 432 <p> 433 Previous: <a href="cookbook.html">Common patterns</a>, Up: <a href="index.html">Introduction</a> [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>]</p> 434 </div> 435 436 437 438 </body> 439 </html>