Goals

• Whitespace should not be signifiant to the document structure.
• Unstructured text should be ignored, and should not cause parsing errors.
• Data structures should be familiar to non-programmers, such as arrays denoted by bullet points / asterisks.

Definitions

• A token is defined as either a key, or a namespace for a key, either of which can use any of the following characeters: alphanumeric (upper- and lower-case), hyphens (-) and underscores (_). In addition, periods (.) may be used to specify nested keys or namespaces, but they may not appear at either end of a token.
• The output is an object-like hash of data that will be returned by the parser.
• Whitespace, in reference to invisible characters that may be symantically ignored by the parser, can be either spaces or tabs.

1.1 Special tokens

Parsers should implement a system by which whole lines are interpreted either as a command line, or a plain text line to be read into a buffer.

"Command" lines are defined as those that begin with any of the following general patterns, with token defined as above:

• token:
• :token
• {token}
• [token]
• {}
• []
• *

Any non-newline whitespace may appear at either end of a token or any of the punctuation used above without affecing the parsing. Special lines should be recognized based on how they start; all text after the patterns mentioned above are valid and should not affect the line's status. For example, each of the following is a valid :token line:

          :end
          :end this line
          :endthisline
        

When a command line is found, logic should be executed that immediately affects the output. For example, if a key/value pair is encountered, the value specified on that line should immediately be added to the output. A document may end at any time, and the output object should always be in a state ready to be returned by the parser.

Based on context, some lines that fit one of the above patterns may not be defined as special; for example, bullet point lines ("* anything") are only special when the parser is inside an array. In these cases (defined below), the line should be interpreted as a plain text line.

All lines that are not commands are interpreted as plain text. When they are encountered, they should not immediately affect the output in any way. Instead, every plain text line should be incrementally added to a buffer that is used during multi-line value parsing, and emptied whenever a special line is encountered.

1.2 Conflict resolution

ArchieML does not throw syntax errors. This is to make parsing of documents as simple as possible, but also to avoid requiring writers be directly aware of the parsing mechanism. Documents are parsed line by line, and any line that does not conform to one of the special line syntaxes above is interpreted as plain text. Syntax errors that cause a command line to be treated as plain text thus do not "break" the parsing of a document, but instead generally omit that line from the output.

Many of the command lines are set up to "reset" to document to some state. This generally means that any mistakes that occur in a document remain local, and do not harm the rest of the document.

If a key is encountered twice, the latter occurrence should take precedence. Arrays that are defined on top of an existing value or array should also be replaced. Thus, no validation should be made as to whether a key is already taken. In the following example, the output should contain second value as the value for key.

          key: first value
          key: second value
        

A partial exception to this is in defining object blocks: if an object already exists as the definition key, the value should not be deleted. In the following example, both first and second should remain defined.

          {scope}
          first: 1
          {}

          {scope}
          second: 2
          {}
        

When redefining a value, it's possible to change the data structure of the output. For example, a key can change from holding a string to being an object namespace with keys underneath it. In these cases too, the latter definition should take precedence, and parsers must override the existing data structure of the output as necessary to accomodate the new data type (such as replacing a string value with an object). Here, the output should define key as {"subkey": "subvalue"}

          key: value

          {key}
          subkey: subvalue
          {}
        

1.3 Parser options

Parsers should accept several options for modifying how documents are processed.

• Case-sensitivity. By default, keys must be case-sensitive, with the result that one should be able to use the exact key from the AML definition when accessing the key's value from the output. When a key is used as an array's element delimiter, only that precise casing of the key should be used to identify the delimiter. But parsers may allow a case-insensitive mode, in which keys with any casing should overwrite each other if re-encountered, and should be interchangable as element delimiter keys.

3.1 Multi-line values

All plain-text lines should be stored into a buffer that is emptied whenever any command line is encountered. This is because values that span multiple lines are defined by including an "ending anchor tag", :end, to the line following the value. Thus the parser can not know ahead of time whether lines 2+ of a value need to be included, until the value has been defined in full.

          key: This value should immediately be saved.
          This will be read into a buffer.
          So will this.
          :end Now the buffer should be emptied into "key", adding on two additional lines to the value.
        

When a command line beginning with :end is encountered, the buffer should be emptied. If the previous command that was parsed was either a token: or a * line (each of which signals the start of a value), then the contents of the buffer should be appended to that value, which should already contain the first line of the value.

Because the first line of a value is always inserted into the output immediately (before the rest of the value is parsed), and surrounding whitespace around the initial values is discarded, care should be taken to preserve newlines between the first and second line of a value. A simple way to accomplish this is to send any trailing newline whitespace from the first line of a value into the buffer, so that it is included when you append the buffer to the value's first line.

As characters are added to the buffer, all whitespace should be preserved, including newlines. The end effect should be that leading whitespace on the first line, and trailing whitespace on the last line is discarded, and nothing else.

6.1 Object arrays

Arrays of objects can be defined to create multiple instances of an object. They begin by declaring an array line, similar to an object block above; however, arrays use square brackets instead of curly brackets.

          [array]
          item: 1
          item: 2
          item: 3
          []
        

When an array definition is encountered, the parser should immediately create an empty array at the given key.

When an array begins, the parser should take note of the first key that is defined. This is the element delimiter key. Every time that key is encountered again, the parser should create a new element inside the array, and that key's value, and subsequent values (until a later occurrence of the delimiter key), should be added to that new object.

Keys within the array block should be assigned to an object within that array. All rules defined above relating to keys still apply, but within the scope of a particular object inside the array. Keys containing dot-notation are relative to individual items within the array.

          [array]
          scope.one: 1
          scope.two: 1

          scope.one: 1
          scope.two: 1
          []
        

This example would result in two items, each containing a top-level scope key. This is distinct from the following array, where the resulting items would contain only the top-level keys one and two:

          [array.scope]
          one: 1
          two: 1

          one: 1
          two: 1
          []
        

6.2 String arrays

Arrays can also be created that contain simple strings instead of objects. They begin the same way as object arrays do, and the same rules apply around naming a string array. String arrays contain values defined by lines beginning with *. All characters following an *, minus surrounding whitespace, should be stored as a new value inside the array.

          [array]
          * item 1
          * item 2
          * item 3
          []
        

When an array is opened, the parser should take note of which command line is defined first: either a key/value line, or an asterisk (*) line. If the former is defined first, then the array is defined as an object array. If a * line appears first, it is defined as a string array.

Object arrays should treat token: lines as commands, and * lines as plain-text. String arrays should treat token: lines as plain-text and * lines as commands.

Multi-line values are allowed within both object and string arrays:

        [object]
        key: value...
        ...with multiple lines.
        :end

        [object]
        * value...
        ...with multiple lines
        :end
        

6.3 Nested arrays

Array elements can also contain nested-arrays (i.e., sub-arrays). To open a nested array, add a . to the front of the array's key: [.subarray]. This signfies that subarray should be an array called subarray within the current array element, instead of closing the current array and opening a new one.

        [array]
        [.subarray]
        key: value
        

This should produce the following structure: {"array": [{"subarray": [{"key": value"}]}]}.

Nested array keys can be mixed freely with normal key/value pairs. Note that subarrays must be closed with empty brackets ([]) in order to "return" to the parent element.

        [array]
        key: Depth 1
        [.subarray]
        subkey: Depth 2
        [] This returns us to the first element in "array."
        anotherkey: Depth 1
        

This should produce a top-level array with three keys: key, subarray and anotherkey.

The nested array's key functions the same as regular keys for determining an array's element delimitor. If a nested array subarray is the first key within a parent array, then subarray will become the array's delimitor.

        [array]
        [.subarray] First element
        key: value
        []
        [.subarray] Second element
        key: value
        []
        

Here, array will contain two elements, each with its own subarray.

Nested arrays can be either Object or String arrays, using the same rules as top-level arrays for determining the type. Only object arrays can contain other arrays, but string arrays can exist at any depth level.

6.4 Freeform arrays

Where Object arrays build up data structures with multiple key/value pairs with no inherent ordering of keys, Freeform arrays preserve the order of every line. Every line within a Freeform array results in an individual object in the array. Sequential lines result in as many elements in the array; the values are not folded into the same object.

Each array element is an object with two attributes: type and value. What is normally treated as the key on each line becomes the type.

This type of array is useful when you want to tie ArchieML input more directly to the presentation order of content. The order of content can then be directly preserved between the text input and the presentation on a page.

          [+array]
          title: Middlemarch
          author: George Eliot
          []
        

          {array: [
            {type: "title", value: "Middlemarch"},
            {type: "author", value: "George Eliot"}
          ]}
        

In this example, moving title below author changes the output, which is not the case in standard Object arrays.

Lines of text which do not contain a key are preserved in Freeform arrays, and not ignored as they are in Object arrays. These lines are given a presumed type of text. In this sense, Freeform arrays are collections of lines, each of which may or may not be tagged with a type. Lines without a type are given one by default.

          [+array]
          h1: This is a header
          h2: This is a sub-header
          This is normal text.
          []
        

          {array: [
            {type: "h1", value: "This is a header"},
            {type: "h2", value: "This is a sub-header"},
            {type: "text", value: "This is normal text"}
          ]}
        

To open a Freeform array, prepend the array's key with a + character. This serves as an array modifier similar to the leading . used in nested arrays. That array should then be treated as a freeform array.

Freeform arrays can also be nested under other arrays by using both modifiers. The order of the modifiers must not matter.

          [array]
            [.+nestedFreeform]
              Nested Freeform content.
            []
          []
        

8.1 End command

The specifics of :end command keys are described above in "Multi-line values."

It should be noted that :end lines which do not occur after a key/value line or array element line (*) should have no effect on the output.

8.2 Skip blocks

When a line beginning with :skip is encountered, the parser should begin to ignore all lines of text. Plain text lines should not be added to the buffer, and all command lines should be ignored. This is with the exception of two command keys which serve to end the skip block: :endskip (which closes the block and signals to resume normal parsing) or :ignore (which stops parsing altogether).

This allows for creating blocks of text where even lines that fit the formal of command lines are ignored and do not affect parsing.

:endskip should resume normal parsing. No special actions need be taken upon resuming parsing, and the buffer should be empty at this point.

8.3 Ignore

As soon as a line beginning with :ignore is encountered, parsing should stop immediately, and the output should be returned. This is a safety mechanism to allow for a safe comment / scratchpad area that has no chance of ending up in, or affecting, the output.