Menu Tools

Entity Converters

This tool handles documents, which contain symbols, not supported by the local hardware architecture. It substitutes the symbols with entities according to the standard ISO 8879 and vice versa. Currently, this tool supports 19 sub-sets of entity-char conversions. Each of them can be activated or deactivated. The more sub-sets are activated, the more time is needed for processing (conversion). One reason for excluding some of the sub-sets is the following : sometimes not all the symbols have to be converted, for example: commas, dots, colons, semicolons ....

Example: ("дума" in Bulgarian is the equivalent of "word")

"дума" <-- entity conversion --> "&dcy;&ucy;&mcy;&acy;"

The tool operates on the document which is currently opened in the system. It can be started from the main menu:

The dialog window:

The window shows the current active converters (filters). Each of them can be deactivated by removing it from this list (button 'Remove filter').

In order to see the symbols (entities respectively), the user can press the View filter button. Having pressed it, a table appears on the screen containing a detailed information about the filter. Each row represents one pair entity-symbol. The table has 3 columns: the first for the entities, the third for the symbols and the second is for the unicodes of the symbols represented as entities.

In order to activate one or more filters, which are not in the list yet, the user can press Add filter button. A new dialog window appears which contains a list of all the available non-active filters. By selecting the checkbox opposite to each of them the user can activate filters. Here he/she can see the currently selected (non-active) filter content before adding it (button Preview). Optionally all filters can be added with button Add All.

When the entity filter management has been completed, the new settings can be updated by Done button and then the window is closed.

There are two more buttons (Apply "Entity --> Char" and Apply "Char --> Entity") which apply the corresponding conversion upon the current document opened in the system. It performs the same action as the conversion tool from the main menu of the CLaRK System. In addition, the changes in the active filters(if any) are updated.

Grammars

A Grammar in the CLaRK System is defined as a set of rules. Each rule consists of three regular expressions and a category (represented as an XML fragment, called Return Markup). The three regular expressions are called: Regular Expression, Left Regular Expression, Right Regular Expression. The Regular Expression determines the content which the rule can be applied to. The Left and the Right Regular Expression determine the left and the right context of the content the rule recognises (if there is no constraints over one of the contexts, then the corresponding expressions are empty). When the rule is applied and recognises some part of an XML document, the part is substituted by the return markup of the rule. If it is necessary to keep the recognised part, it can be cited by using the variable \w. If the user needs to use the string \w in the return markup, he/she can avoid the \w variable in the following way: ^\w

The regular grammars in the CLaRK System work over token and/or element values generated from the content of an XML document and they incorporate their results back in the document as XML mark-up.

The tokens are determined by the corresponding tokenizer.

Before having been used in the grammar, each XML element is converted into a list of textual items. This list is called element value for the XML element. The element values are defined with the help of XPath keys, which determine the important information for each element.

In the grammars, the token and element values are described by token and element descriptions. These descriptions could contain wildcard symbols and variables. The variables are shared among the token descriptions within a regular expression and can be used for the treatment of phenomena like agreement.

Here is the list of the token and element descriptions:

  1. "token" -> describes the token itself. This description can be matched to the token itself and nothing else.

  2. $TokenCategory -> describes all tokens of the category TokenCategory. In the grammar input this description is matched against exactly one token of this category.

  3. Wildcard Symbols: #, @, % -> describe substrings of a given token. # - describes a substring of arbitrary length from 0 to infinity, @ - describes a substring of arbitrary length from 0 or 1, % - describes a substring of arbitrary length one. Here are some examples:

    • "lov#": matches exactly one token which starts with "lov": "lov", "love", "loves", and many others.

    • "lo#ve": matches exactly one token which starts with "lo" and ends with "ve": "love", "locative", "locomotive" etc.

    • "%og":matches exactly one token which ends in "og": "bog", "dog", "fog", "jog" etc.

    • "do":matches exactly one token which starts with "do": "do", "doe", "dog", "don" and many others.

  4. Variables: &V -> describes some substring in a token, when initialised for the first time, then matches to the same substring in the same token, or some following tokens. The scope of a variable is one grammar rule. The variable can be used in the return mark-up and in this case the value of the variable is copied into the return mark-up. Each variable consists of the symbol & followed by a single Latin letter. Each variable has positive and/or negative constraints over the possible values. Both the positive and the negative constraints over variables are given by lists of token descriptions. The value assigned to a variable during the application of the grammar has to be described by one of the positive constraints and must not be described by any of the negative constraints. These token descriptions can contain wildcard symbols, but no other variables. Here are some examples: "A&N&G", "Nc&N&G". These token descriptions can be used in a rule to ensure the agreement in number and gender between an adjective and a noun.

  5. Complex token descriptions. The user can combine the above descriptions in one token description. Some examples: "lov%#", "Vp&N&G#&P"

  6. Element description is a regular expression in angle brackets: < Regular Expression >. Here the Regular Expression is over token descriptions which is matched against the element value. Examples:

The application of a rule works in the following way. The element value is scanned from left to right. Its Regular Expression is evaluated from the current point. If it recognises a part of the element value (this part we will call a match of the rule), then the regular expressions for the left and for the right contexts are evaluated (if they are not empty). If they are satisfied by the context of the match, then the match is substituted in the return markup for each presence of the variable \w (The user must be careful if he/she has, for example, text like \word in the return markup, the beginning \w will be substituted by the match. In this case the variable must be escaped). After these substitutions, the new markup is substituted in the XML document instead of the match place.

When a regular expression is evaluated from a given point within the element value, there is a possibility for several matches to the expression. For instance, the expression (A,B)+ over the element value L,A,B,A,B,A can recognise two matches from the second possition: A,B and A,B,A,B. This allows for a non-deterministic choice in this place. One can choose either the shortest match, or the longest one, or some in between. Generally, there are no universal principles for making such a choice. This is why in the CLaRK system we allow for user definition of a strategy to choose a match among more choices. We envisage four strategies: shortest match - in this case the system always selects the shortest possible match; longest match - in this case the system always selects the longest possible match; any up - in this case the possible matches are enumerated from the shortest to the longest possible match up to the moment when the left and/or the right context of the match satisfy the Left and/or the Right Regular Expression. If there is no Left and Right Regular Expressions then any up strategy is the same as shortest match; any down - it is similar to any up except for the fact that the possible matches are enumerated from the longest to the shortest one. These strategies are specified within the grammar queries. This allows the same grammar to be applied with different strategies over different documents.

The definition and application of a grammar are separated within the CLaRK system. The grammar itself is defined at one place, the parameters for its application are defined at another place in the form of grammar queries. This separation allows the use of the same grammar with different parameters like different tokenizers, different element values, different filters etc. Each of them has an XML representation. These XML representations allow grammars and their queries to be exchanged among different users. Also this allows the grammars to be constructed out of the system and then imported within it.

The grammar definition consists of a set of rules, variable definitions and context evaluation parameter (Check Context Order). The rules have already been discussed. The variable definitions are given by the positive and negative constraints over the variable. The context evaluation parameter determines the regular expression for which the context will be checked first - the left and then the right, or vice versa.

The grammar application determines: the elements which the grammar will be applied to; the element values for those elements, including the tokenizer and the filter; whether the textual elements will be normalized; the application strategy (longest, shortest match, any up and any down match).

Grammar Manager

The grammar manager is the user interface in the CLaRK System for management of grammar definitions. It supports the user in the creation, modification, deletion of grammars. The main dialog contains the list of the available grammars, some of their features and buttons for management of the grammars.

Each grammar has to be compiled in order to be used in the system. The compilation converts the regular expressions into a finite-state automaton. Because the compilation is a heavy process, sometimes it is better to postpone the grammar compilation (For example, when a large grammar is imported into the system). The column Compiled in the table represents information whether the grammar is compiled or not. If the grammar is compiled the corresponding box is checked.

The system also allows the user to export and import already compiled grammars. This option is very useful in cases of large grammars, when the compilation takes longer time or the user wants to exchange just the compiled grammar, but not its source. If such a grammar is used in the system, it cannot be edited. In such cases, the check box in the column Editable is not checked.

Here is the main window of the Grammar Manager:

There are several grammars. The grammar Slovnik One is editable and also compiled. Thus if necessary, the user can open it in the Grammar Editor and can modify it. The grammar Slovnik One[1] is editable, but not compiled. Thus it cannot be used immediately, first it has to be compiled. The grammar Slovnik One[3] is available only in compiled form. It cannot be modified in this form, but it can be used to process the relevant documents.

Under the table with the available grammars there are 8 buttons (New, Edit, Remove, Rename, Copy, Compile, Apply, Exit) and 3 menus (Compiled Grammar, File I/O, XML Editor ). They can be used to manage the grammars.

Buttons:

Menus:

The three menus allow the import and export of grammars from and to the system. As it was said above, the grammars in CLaRK have an XML representation. Thus the user can load such grammar from an external file, or from a document within the system. Also, the user can save a grammar created within the system in an external file or as a document in the system. In this way the user can exchange grammars with other users, or make backup copies of the them, or can process them with other tools in system, such as sorting, searching etc. Additionally, there is a format for saving and loading compiled grammars.

Grammar Editor

This is the editor for grammars in the CLaRK system in table format. The editor contains the following elements: Rules, Option, Variables, and three buttons Save, Compile, Exit . Here is the main window of the Grammar Editor:

Rules

The table Rules contains the rules of the grammar. Each row of the table represents one rule. The columns follow the structure of the grammar rules in CLaRK. The column Regular Expression has to contain the regular expression which will be matched with respect to the element content. The column Return Markup is the second obligatory element in a rule. It contains the XML fragment which will be substituted with the matched part of the element content. There are two columns for the regular expressions which determine the left and the right context of the match - Left Regular Expression and Right Regular Expression. The last column is for comments on the rule.

The content of the table cells is not checked for consistency before the compilation.

Context Check Order

This option gives the user a possibility to choose which context will be checked first - the right or the left. In this way a preference over rules can be defined. The two orders are: first the left context, then the right one (Left->Right); the right context, then the left one (Right->Left).

Variables

This table contains the definitions of the variables for the grammar. Each row of the table represents the definition of one variable. Each definition consists of the following elements: name of the variable (Name) which is a capital Latin letter; The positive constraints for the variable are given as a list of token descriptions in the cell Positive Values. If the cell is empty, then the variable is not constrained and can have any non-empty value; The negative constraints for the variable are given as a list of token descriptions in the cell Negative Values. All the values that can be described by some of the token descriptions in the cell are forbidden as values of the variable. If the cell is empty, then the variable is not negatively constrained and can have any value described be the positive constraints; Like every token description in the CLaRK system, a variable also can match several strings starting from a position. The user has a possibility to define which match to be chosen. There are two options for the Match cell: Longest - in this case the variable is assigned the longest possible value, and Shortest - in this case it receives the shortest possible value.

The management of the variable table is done by a pop-up menu which appears when the user clicks with the right button of the mouse on the cells in the table. The possible choices are: Insert row which allows the user to define a new variable; Delete row which allows the user to delete the definition of a variable; Edit Cell which allows the user to modify the list of token descriptions for the positive or the negative constraints for the variable; Up and Down allow the user to rearrange the list of the variable descriptions for her/his own convenience.

When the user edits the constraints for a variable (Edit Cell) the following dialog appears for the positive constraints:

or for the negative constraints:

In both cases the constraints are represented as a list of token descriptions (one per line). The user has the possibility to enter a new description by the button Add Value - in this case a text edit field appears and the new token description has to be entered in it. The user can delete some token descriptions by selecting them in the list and clicking on the button Remove Value(s). The buttons OK and Cancel are used for the acceptance or rejection of the changes.

Buttons

The buttons at the bottom of the dialog give the user the possibility to save the grammar - Save; to compile the grammar - Compile. In case of errors in the grammar a corresponding error message appears; to exit the dialog - Exit. The system prompts for unsaved changes.

Apply Grammar

As it was described in the Grammars menu, the definition and application of a grammar are separated within the CLaRK system. In the Grammars menu the user can find a description of the grammars and how to construct their definitions in the CLaRK system. In this section the user can read how to apply a grammar over one or several XML documents.

Here is the main dialog of the Apply Grammar tool:

The application of a grammar requires the following types of information: the name of the grammar (text field Grammar), the target of application (text field Apply to), how the input to be prepared (the second row of options: Tokenizer, Filter, Normalize, and Element Values), how the rules of the grammar to be applied (the third row of options: the match options for the left context regular expression (Left), for the main regular expression (Body), for the right context regular expression (Right)), and whether the context can be backtracked (Context Backtracking). A combination of all these options is called a grammar query.

Additionally, the dialog allows the user to consult the definitions which are connected with some DTD in the system. Generally, if in a particular grammar query some of the necessary information is not presented, then the system checks the corresponding information connected with the DTD of the document. This can be done by the menu Features.

The user can save the current settings of a query as an XML document by choosing Queries check box. Then the user has the possibility to save the query with some comments in the Info text field. Also, the user can select a previous query from the list of queries. The grammar query XML documents are saved in the group SYSTEM:Queries:Grammar.

Also, the user can specify whether the grammar to be applied over the currently opened document or on some documents stored internally in the system. This can be done by choosing the Multiple Apply. In this case the user can select several documents which to apply the grammar to.

At the bottom of the dialog there are three buttons: Apply for application of the currently stated query (also one loaded from the XML representation); Close for closing the dialog; and Select for navigation over the current document and manual annotation (see below).

Element Values

When a grammar is applied over a content which contains XML elements, the system converts each such element into an element value. How exactly this conversion is performed is stated as Element Value definitions. These definitions can be connected with a particular DTD, but the grammar query allows the user to change the DTD settings and to define them locally into the query. Each element value is connected with a element tag and a sequence of XPath expressions (called keys) which define the sequence of tokens or elements for the element value. See below for examples. The dialog of the Element Values editor is as follows:

Each row of the table represents an element value for an element. The first column represents the names of the elements for which element value is defined. In this case they are w and pt. The Keys column contains the keys for each definition. In this case the value of the element w is the text in its ph element and the text in its ta element. For the pt elements the definition says that the element value is their text content.

The user can edit the value of the right column by selecting the Edit Cell item from the pop-up menu which appears when the user presses the right mouse button over the cell (The menu is visible on the screen shot). There are two modes for the element value Tool and User. In the following screen shots the w element is shown in both modes:

Each row of the table represents one key. All keys in the table define the value of one element. One key consists of a key name (option), a key value (XPath expression), a normalize option and a tokenizer name. If the element value is in Tool mode, then the normalize option and the tokenizer are taken from the grammar. The key in the table is called Grammar Key. It can be saved and loaded into the system memory. This is done by selecting Load Key and Save Key menu items in the context menu shown when the user right clicks over a cell in the table. The user can also add remove keys with items from this menu. The normalize option and the tokenizer name determine the input word, which is created for the elements when the grammar is applied. An interesting option here is "No Tokenizer" tokenizer. If it is selected, then the text nodes are treated as one token. When the OK button is clicked all XPath expressions in the table are compiled.

The element values are calculated in the following way:

  1. If the XPath expression selects textual content or the value of an attribute, the corresponding text is tokenized by the relevant tokenizer. Then the value is a sequence of tokens.

  2. If the XPath expression selects one or more elements then each element is represented as <tagname>, where tagname is the tag for the element.

  3. If there is no element value definition for the element then it is represented as <tagname>, where tagname is the tag for the element. The difference from the previous case is that if the element value is defined by self::*, then the element value will be <<tagname>>.

Select

This button allows the user to apply a grammar in an interactive mode. In this mode the grammar is executed on the current document and for each match it stops and allows the user to see the selected content and to perform some actions like: to go to the next selection (Next button), to go to the previous selection (Previous button), to add the return mark-up to the content (Mark button), and to exit the mode (Exit).

Grammar Groups

This tool is a means for applying a set of Grammar Queries in a row. The application itself is done in a cascaded grammar style, i.e. the output from each grammar is an input for the next one. The result from the last grammar is a result from the whole tool. The advantage of this tool is the better efficiency which is a result from the fact that the input for the grammars is prepared only once. Otherwise, the input should be prepared (preprocessed) each time a single grammar query is to be applied. This can be crucial for huge amounts of data.

Here is what the Grammar Groups dialog window looks like:

The tool dialog basically represents a list of Grammar queries. The user can Add Grammar Query to the end of the list and/or Remove Grammar Query from the list by using the buttons on the right side of the panel. The order of the different grammar queries can be changed by selecting a query and dragging it to the desired position.

This tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or load the current tool settings, i.e. XML Tool Queries are supported here.

Constraints

Regular Expression Constraints

The Regular Expression Constraints is a means for setting restrictions on the content of certain nodes in a document. The restrictions are set as regular expression patterns. The syntax of the patterns is the same as the one in the Grammar tool. The nodes whose content the constraint will be applied to are selected by an XPath expression. The selected nodes must only be of type Element (as no other types of nodes can have content). A node satisfies a certain constraint if its content matches the pattern given in the constraint. The application of a constraint gives the user the possibility to navigate either through the nodes which satisfy the constraint or through the nodes which do not satisfy it.

The constraint application is similar to a DTD validation of certain nodes of a document, but here we offer a more powerful instrument. On the one hand, the nodes which the constraints are applied to are selected not only by name (as it is in the DTD), but depending on the context in which they appear (XPath determined context). The context can be an absolute or relative document position, based on properties of the selected nodes or nodes relative to them, properties located in other documents, etc. The node selection qualification uses the full expressive power of the XPath engine, implemented in the system. On the other hand, the regular expressions of the Grammar allow writing patterns not only at the level of text nodes, but at the level of tokens within the text nodes as well. Even more, the user can specify a tokenizer which will be used for segmenting the text and a filter to discard the unmeaningful tokens during application. In the patterns the user can write wildcard symbols in token descriptions in the same way they are written in the Grammar tool.

With the help of these constraints the user can cover some of the features of XML Schema usage, by defining patterns for text nodes.

Regular Expression Constraints Structure

Each Regular Expression Constraint consists of 2 main parts and 3 additional (optional) parts:

Edit Regular Expression Constraints (REC)

This section is responsible for the regular expression constraint management. Here the REC can be created, modified, removed, saved as a file and loaded from a file. Here is a picture of the dialog window:

The left side of the window is a table with all RECs in the system. The first column contains the names of the constraints. The second one contains the regular expressions for each of the constraints. Having selected a row in this table, the user can apply a manipulation over a constraint by using the buttons on the right.

Description of the buttons on the right:

Regular Expression Constraint Editor

Here is the interface view of the editor window for the REC:

The last three fields are optional. The tokenizer and filter lists contain all tokenizers and filters defined in the system. The regular expression may consist of: tags, token categories, token values and token value templates (wildcard descriptions).

Apply Regular Expression Constraints

The actual applying of the Regular Expression Constraints can be performed in two ways:

Here we describe the latter case. The user chooses 'Apply Regular Expression Constraints' from the menu Tools/Constraints/Regular Expression Constraints/Apply Regular Expression Constraints. Then the following dialog window appears:

The first input field Select nodes contains the XPath which is evaluated in order to select nodes for the constraints operation. If the default XPath expression is specified for the constraint, then it appears in this field as a default text.

The second field selects by name a constraint to be applied.

The last two fields are activated when the current constraint tokenizer and filter are ignored and new ones have to be defined explicitly.

Having pressed the Apply button, the XPath is evaluated and a set of nodes is selected. Then for each of them the constraint is applied. If the node's content satisfies the constraint, then the node is marked as Valid. Otherwise it is marked as Non Valid. In this way two groups of nodes are formed and each of them can be observed separately. Here is a picture of the navigation panel window:

In this window the user can change the group under observation by using the two radio buttons. Pushing Next and Previous buttons the user changes the current selection in the editor. On the top of the window there is some information about the constraint and the nodes which satisfy or do not satisfy it. For the example above, the pattern '$NUMBER+,$SPACE*' concerns the content of text nodes. The items which satisfy the constraint with this pattern are all element nodes whose text content is a sequence of one or more tokens of category NUMBER, followed by zero or more tokens of category SPACE. Thus, strings which match the pattern are: "1234", "256  ", "666        ", etc.

Value Constraints

The constraint engine is a means for setting restrictions on the content or other related information of nodes in XML documents, which cannot be expressed by the DTD. The nature of the restrictions is based on the existence of certain values (tokens and/or tags) at certain places. The constraints of this type specify the pieces of information which are restricted and define the set of admissible values for each of them (usually by pointing to a location they are stored in, or by encoding the values themselves explicitly).

Value Constraint Structure

In general a value constraint consists of two parts: a target section and a source section.

Target Section

In this part one can find a description of the nodes which the constraint will be applied to. The target nodes for a constraint are selected by an XPath expression evaluated on the document which the given constraint is to be applied to. The result from the evaluation is expected (required) to be a node set with nodes compatible with the specific constraint application. If the result set contains nodes of types other than the required ones, they are automatically excluded (example: the selection contains text and attribute nodes, but the constraint checks the child nodes of its targets). This way of target selection uses the full expressive power of the XPath language in order a context dependency to be expressed.

Source Section

Here the possible values for the target nodes (selected by the previous section) are defined. The possible values are tag names and tokens depending on the type of the constraint. The source list can be selected by an XPath expression or by typing the choices explicitly as an XML markup. If the selection is made by a relative XPath expression, then the current target node is taken as a context node for the constraint. If a text node is selected as a source, then its text value is tokenized and the tokens are added to the source list, excluding the node itself. It is possible that the source for the constraint is an external document. The only requirements in such cases are the following: the external document has to be in the internal database of the system and the XPath expression cannot be relative.

There are four types of value constraints, currently supported by the system. They are distinguished by their target and the way of their usage. Here is a description of each value constraint separately:

  1. Parent Constraint

    This type of value constraint sets limits on the possible parents of a node. There are two ways of applying this constraint type: by changing the parent of a node (local) or explicitly running the constraint engine (global).

    The first possibility is changing the parent of a node (or a set of nodes at one level). The list of all the relevant parent nodes can be restricted further by applying other constraints. The final list contains the intersection between the source of the constraints and its former content. If the operation - changing the parent of a set of nodes - is performed, then all compatible (parent)constraints are applied.

    The second possibility is running the Constraint Engine. It works in the following way. First, the targets are selected (by their tag names and an XPath restriction). Then the source is compiled. If there is more than one choice, the user is asked to select one option from a list. If the choice happens to be exactly one element, it can be automatically inserted as a parent of the target. The action of a constraint depends on the Application Mode set for the constraint.

    The source list of each constraint must contain only tag names. All tokens in the list are ignored.

  2. All Children Constraint

    This type of value constraints sets limits on the names of a node's children and the content of its text children. All children, that are tags, must have names coinciding with the name of some node from the source list. Then all the data in text children is tokenized and a list A of tokens is formed. After that all the data in text nodes in the source list is tokenized and a list B of tokens is formed. For every token in A there must exist a token in B such that the values (not categories) of A and B are equal. This type of value constraints can be applied (checked) from menu item Apply All Children or from the toolbar button. The list of all invalid nodes according to the constraints is given in the Error message area together with the rest of the errors (if any). The user is given a possibility to navigate through all invalid for the constraints nodes.

  3. Some Children Constraint

    This is a special type of a value constraint, because its main task is not only to set limits on the node's content. Instead, it can be used for a value restriction when the operation inserting a child in a node is performed. This constraint type is not applied each time a new node is inserted. These constraints are used separately. Here the target node is the node where the insertion takes place. The constraint is blocked when:

    • there is a child of the target node that is a tag and there is a node in the source list, such that both nodes have identical names.
    • there is a text node in the target node that has a token, whose value equals the value of a token in the source list.

    To sum up, when there is a non-empty intersection between the source list and the target node's content, the constraint is satisfied and there is nothing more to be done. In cases when the source list is empty and the target content is also empty, then the constraint is satisfied.

    When the source list is not empty and there is no intersection with the target's content, the user is offered a list with the possible values from the source list for the target node. The user can choose one item to insert. The action of a constraint depends on the Application Mode set for the constraint.

  4. Some Attributes Constraint

    This constraint is very similar to the previous one. The only difference is that the target here is an attribute of an Element node. Also the target selection includes a selection of an attribute defined in the DTD for the selected tag name.The action of a constraint depends on the Application Mode set for the constraint.

Application Mode

The Value Constraints have two modes of application, concerning the treatment of the target nodes:

Edit Value Constraints

The screen shot on Fig. 1 is the dialog window of the value constraints editor.

VConst.gif (19423 bytes)

Fig. 1

The editor is separated into 5 sections which are responsible for different parts of the constraint definition. On the top of the window there is a Summary information panel which shows the current constraint settings (Type, Mode, Target, etc.). The sections are:

VConst1.gif (5396 bytes)

Fig. 2

VConst2.gif (3835 bytes)

Fig. 3

VConst3.gif (4583 bytes)

Fig. 4

VConst4.gif (5414 bytes)

Fig. 5

Value Constraints Manager

In the preceding section a description of the Constraint Editor was presented. It is evoked whenever a change on a Value Constraint is needed or a new constraint is defined. The Value Constraint management is handled by the following manager dialog window:

VConst_m.gif (22410 bytes)

Within the CLaRK System this module can be evoked from the menu: Tools/Constraints/Edit Value Constraints.

The Constraint Manager represents a table of all constraints defined in the system. Each constraint is represented as one row in the table. The ordering in the table is important only if the constraints applications depend on each other. The constraint in the first row is applied first, then the second constraint is applied and so on. But the ordering can be changed by the two buttons on the right side: Move Up and Move Down. They swap the position of the selected row with one of its neighbours above or below.

Sometimes it is useful to deactivate some of the constraints temporarily. It can be done just by unselecting the check box on the constraint row. For example, in the picture above the second constraint remains deactivated.

The remaining buttons:

Value Constraints Group

The constraints described so far work in the following way: the first constraint is applied to all targets, then the second one is applied, and so on. The constraint groups, however, work in a slightly different way. First, a context node is selected and then all the constraints from the group are applied within this context. Each group contains three parts:

The Value Constraint Group Manager:

VGroup.gif (8672 bytes)

The Group management includes: a creation of a new group (New Group), a modification (Edit Group) and removal (Remove Group) of an existing group, an application of a group of constraints (Apply Group). Each operation (except for New Group) requires selecting a group from the list.

Apply Value Constraints

This is a tool specialized in applying Value Constraints on the current document or on a set of documents. The user is expected to create a list of single Value Constraints or whole Value Constraint Groups. The constraints and groups are applied in the order they appear in the list. They can be reordered by simply using drag-and-drop technique, i.e. pressing a list entry and moving upwards or downwords until the desired position is reached. Each entry in the list contains: either a constraint name followed by constraint description in brackets (for a constraint) or a group name followed by the '(group)' suffix (for a constraint group).

The user can modify the content of the Constraints List by pressing the following buttons:

VConst5.gif (5953 bytes)

This tool supports two modes of application: to the current document and Multiple Apply mode. For details see Tool Application Modes. If the tool is run in Multiple Apply mode there is one significant difference in the application: if a constraint uses Insertion Mode, a real insertion is performed only if the possible source value is one. In case there are more choices - some human intervention is needed. But the Multiple Apply does not allow it.

The user can also save or load the current tool settings, i.e. XML Tool Queries are supported here.

Number Constraints

This constraint type in general restricts numeric values related with nodes and their properties within XML documents. Such values can be: a number of occurrences of some specific elements within the content, values returned by XPath functions or operators. The target constrained values are produced from the evaluation of an XPath expression. This XPath is evaluated according to the result from the evaluation of Context XPath expression which determines the nodes which the constraint will be applied to. Independently, for each initially selected context node, one XPath result is produced. Depending on the result, a numeric value is formed as follows:

Note: if the newly produced value is the Not-A-Number value then the corresponding context node does not satisfy the constraint.

A context node satisfies a constraint if the result numeric value of the XPath is within the range of Minimum Size and Maximum Size values. The latter two values can be either numbers or XPath expressions which are expected to return numeric values. If an XPath returns a non-numeric value, the system tries to transform it automatically to a number. In case the Minimum Size value for a constraint is not defined or the defined value produces Not-A-Number value, the system assumes that at this place there is no limitation and all target values which are under the corresponding maximum satisfy the constraint. By analogy, if a Maximum Size value is omitted or it is Not-A-Number, then no upper limitation is assumed. If an XPath expression is used for setting minimal or maximal limit, its context for evaluation is each initially selected context node. In this way the boundaries of one constraint can vary for different contexts.

The Number Constraint Manager dialog:

NumConst.gif (7470 bytes)

In the example above, the fourth constraint has no upper limit. The fourth column (Use It?)  is responsible for the activation/deactivation of the constraints. It becomes a necessity when the user would like to apply only a certain subset from all the constraints. Applying the (active) constraints can be done by pressing the Apply button. This button is disabled when there in no document in the editor. After applying the constraints, the user receives information about each applied constraint and the number of the satisfied nodes (contexts) as well as the non-satisfied ones . In the picture below there is an example result dialog:

NumConst1.gif (4912 bytes)

Here the user has the ability to navigate through all satisfying or not satisfying a certain constraint nodes. This can be done by selecting a row in the result info table and using button Details. The user is shown a small navigation dialog which allows successive traversal of the nodes in forward or reverse direction. Here follows a picture of the navigation dialog from the preceding example picture:

NumConst2.gif (5137 bytes)

The dialog contains several sections:

XSLT

Apply XSLT

This function applies an XSL Transformation either on the current document in the system editor or on a set of selected internal documents. In the case a transformation is applied on the current document, the result XML document is loaded automatically in the system. If the transformation which has been applied does not produce any result, a warning message 'No result produced!' appears. Otherwise the user is asked to supply a DTD for the result document. In case a transformation is applied on a set of internal documents the user has to specify a DTD and result document names in advance.

Another not traditional application of the XSL Transformations is the so called Local Transformations. Their application is performed in the following way: a set of nodes are extracted from a document (current or internal). For each of them, independently an XSLT is performed and the result (if any) is incorporated back in the original location where the extract was taken. Thus, no new result document is created but the original document is modified. The extracted nodes to be transformed are selected either by an XPath expression or by direct selection in the tree. The result from each transformation application is a Document Fragment (DOM) which substitutes the context node for which it is produced. The context node is removed from the tree and all sub elements of the fragment are inserted at the position of the context. The application of the transformation is followed by a result information message. It contains four pieces of information:

  1. The number of nodes selected by the XPath expression as contexts;
  2. The number of context nodes which have been replaced by a result fragment;
  3. The number of context nodes to which the transformation did not produce any result;
  4. The number of context nodes which have been lost during applying a preceding transformation. This can happen when a node and its descendant node are selected as contexts and the transformation has succeeded on the parent node.

If no transformations are available in the system, a warning message appears. User can apply XSL Transformation by means of the Multiple Apply module, or save the current settings for further use from Queries module.

For details about the management of the transformations see module XSLT Manager.

This tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or load the current tool settings, i.e. XML Tool Queries are supported here.

XSLT Manager

This component is responsible for the management of all XSL Transformations in the CLaRK System. Although the documents containing XSLT are regular well-formed XML documents, they are treated as a separate class of read-only documents. They are stored in a special separate list and only tools using XSLT can access it. The acceptable operations on the list are adding, removing and overwriting a list entry (XSLTransformation document).

Here is the dialog window of the manager:

XSLTManager.gif (6074 bytes)

Buttons:

Validate XSLT

This option validates the current document in the editor if it can be used as an XSL Transformation. This option can be used when a new transformation document is created (or imported) in the editor. The XSLT Validator checks the content and if there are no errors, an information message is shown. Otherwise, an error message appears and the location of the error in the document is pointed to.


Concordance

Concordance - a system tool for information extraction. It allows an extraction of certain units (words, phrases, etc.) within bigger units (sentences, paragraphs, etc.). The result extraction is shown in a table where on each row a result is shown. The searched items, the left and the right contexts are distinguished in separate columns. The tool is implemented on the basis of the XPath engine, regular grammar engine and a sorting module.

The field at the top of the dialog (Define Context) is used for defining the context nodes within which the extraction will be done. The user is expected to supply an XPath expression which after evaluation returns a node set. The context for evaluation is the root node of the document. The user can perform two types of concordance extraction: grammar based and XPath based. They will be described in details in the following paragraphs.

The result from the concordance is stored in an XML document and for convenience it is shown in a table. The structure of the XML document is a sequence of <L> elements standing for the found items (lines). Each concordance line has the following XML structure:

<L>
   <LC> the left context </LC>
   <I> the data we are searching for </I>
   <RC> the right context </RC>
   <!-- user commentary -->
</L>

and the corresponding table representation:

When the user sees a result from the concordance in a table s/he must always have in mind that there is an XML structure behind the dialog window, especially when a table rows sorting has to be performed.

Grammar Concordance Search

This type of concordance uses regular expressions patterns for searching items in other items. The patterns are defined as grammars in the Grammar tool. The items which match the patterns (tokens, XML tags or mixed) are shown in the result table (document), accompanied by the context in which they appear. Initially the context is determined by the XPath expression mentioned above. Further restrictions on the context can be set by another grammar pattern, i.e. the target searched items will be extracted only from items matched by another grammar pattern as contexts discarding everything else.
If the Text only (The mark-up will be ignored)? option is selected the concordance engine will ignore the mark-up inside the initially selected contexts while checking. Here follows an example how the mark-up can be ignored during the data extraction. Let us have the following simple XML document as a source of extraction:

If the target item of extraction is the word loves within the context of the TEXT element the Grammar pattern must only describe the word itself without specifying that it appears in any mark-up (in this case in the content of tag verb. Here is the result from the query 'loves' within the context of 'TEXT':

If the target of interest is the sequence John loves Mary within the context of the TEXT element, the query pattern will be: "John","loves","Mary". Although these three words appear in different XML structures, after filtering the mark-up the result will be:

Here is how the Concordance dialog window appears in a configuration set for a Grammar concordance search:

The Concordance dialog offers three sub-dialogs for setting a grammar search query corresponding to three levels of complexity, each supplying different sets of options. Each of the three dialogs is accessible by choosing the corresponding item from the Usage Mode panel. The possible items are:

XPath Concordance Search

Searching items in this mode of concordance extraction is based on XPath queries within initially selected context nodes. The content which will be shown in the Item column will be a result from the evaluation of the XPath expression from the field Search Elements. If for a context the returned result contains more than one node, each node will appear in a separate row in the table. For each single result node the XPath expressions from Left Context and Right Context are evaluated to form the content of the corresponding table cells for the contexts. If any of these two fields does not contain an XPath expression, in the corresponding table cell the whole content before/after the found item will be shown.

Here is the Concordance dialog window in a configuration set for XPath concordance search :

Concordance Options

The options which appear in the both modes of extraction are:

Table View

The Table View tool is created to represent the information extracted from the concordance tool in a more readable table form. Each line of the table represents one line of the concordance result.

If the user wants to use this feature, s/he has to open an XML document which is produced as a result from the Concordance tool. Heaving the Table View menu item selected, the system tries to detect the required structure in the currently opened document in the system editor. In case of failure, the system produces an appropriate error message. Otherwise, the document content is shown in a table (the picture below). The required document structure of the input for the Table View is described in the Concordance tool.

The data in the "Context" columns does not represent the whole context but only the amount of data that can fit in the column length. At the beginning it is only 30 symbols. To increase the context the user should press the settings button and from there to determine the context in symbols. The user can also set the width of the comment column. If the user wants to see the context without expanding the column data s/he can do it with right click on the "Left Context" and "Right Context" column. If the user wants to add some commentary to a concordance line s/he can do so by filling a value in the "Comment" column or by right clicking a row in the "Item" column. To navigate faster through the table the user can rely on the combo box at the top for accessing a row.

The user can also sort the lines of the table. To do so, s/he must select which column to apply each sort keys to (which element of the concordance line will be the context LC, I, RC or comments). If no column is selected then the key will be executed with the line element for context.

A useful option of the Table View is the "Edit Layout". The user can filter the tags that are shown in the table. For example, if the POS information is separated in a tag, the user can hide it in order to view only the text.


Extract

The extract tool task is to extract nodes from a document or from multiple documents and to save them as a new document. The document data extraction is based on XPath expressions. The text field at the top of the dialog is used for defining an XPath expression which selects the elements in the document(s). The context node for this evaluation is the root node of the document(s). The result from the extraction is an XML document in which all extracted nodes are children of the root element (This element is named "Extract" by the system).

The Include subtree option allows the extraction not only of the selected nodes but the entire subtrees below them as well.

The Create result tag option allows each extracted node to have a parent element which is used to separate the different results. For example, if we extract only text node, then in the new document all the text nodes will be concatenated. If the Create result tag is selected, then for each result node there will be added a parent element. The name of the parent element is taken from the corresponding text field.

If Create source attribute option is selected, then the extract tool adds an attribute with the source document name to either the auxiliary element or the root of each result root element. The name of the attribute is taken from the corresponding text field. In case the result is not an element-rooted structure and no auxiliary element is added, this option does not change anything.

If Create path attribute option is selected, then the extract tool adds to each result structure an attribute with a value which is an XPath expression expressing the location of the result in the original source document. In other words if this XPath expression is evaluated on the source document, the result will be exactly the extracted result node. The name of the attribute is taken from the corresponding text field. In case the result is not an element-rooted structure and no auxiliary element is added, this option does not change anything.

If Create number attribute option is selected then the extract tool adds an attribute with the extract result number to the auxiliary node or to the root element. The name of the attribute is taken from the corresponding text field. In case the result is not an element-rooted structure and no auxiliary element is added, this option does not change anything.

This tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or the load current tool settings, i.e. XML Tool Queries are supported here.

Sort

The sort tool is used for reordering nodes in the tree representation of a document. The sort operation changes the order only for nodes which are siblings, i.e. nodes with the same parent. If sort is applied on a set of nodes with different parent nodes, the nodes will be sorted only in the scope of their parents. The nodes selection for sorting and the sort criteria (Sort Keys) are written in XPath expressions.

For sorting the user has to specify the following two things:

  1. The target nodes for sorting.
  2. The keys for each node.

The first is done by defining an XPath expression in the Select Elements field. If the field is empty, the sort tool will show an error message. For context node the XPath engine assumes the node selected in the tree panel of the system or the root elements of the internal documents. The sort tool compares only element nodes which have a common parent. The sort tool splits the result returned from the XPath evaluation into groups according to the parent node. Each group is sorted separately.

Keys are calculated for every node the user wants to sort. Each row in the table represents one key. The sort tool compares two nodes key by key. The key is the list of nodes returned from the XPath engine after evaluating the expression defined in the column Key of the table. The context node in this evaluation coincides with the node for which the user wants to create the key. The other columns of the table represent settings used in the list comparison. The lists are compared node by node as follows.

For each key the user can define different order ( Ascending | Descending ). The order of the keys in the table is very important because this is the order in which they will be used. If two keys have equal nodes but one of them has additional elements, then the one with the smaller number of nodes is considered smaller.

The difference between the DTD sort and the Advanced one is that the sort tool takes the tokenizer and the number option from the DTD (Element Features, Attribute Features). For attribute nodes the sort tool also takes from the DTD the order of enumeration values.

Examples:

Example 1

Example 2

This tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or load current tool settings, i.e. XML Tool Queries are supported here.


XPath Insert ...

XPath Insert Attribute

This tool (Fig. 1) gives the possibility to set a certain attribute to nodes selected by an XPath expression. The user specifies an Attribute Name and Value. Additionally, s/he can tune the tool to set attributes only to nodes which do not have such yet. If the checkbox Skip Existing Attributes is unselected, the tool will set the given attribute with the given value to each element node returned by the XPath. Otherwise, it will skip all element nodes which already have this attribute and in this way it will preserve their original values. If the result from the evaluation of the XPath expression includes nodes other than Element nodes, they are ignored during the processing time.

This tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or load current tool settings, i.e. XML Tool Queries are supported here.

insattr1.gif (5435 bytes)

Fig. 1

XPath Insert Child

This tool (Fig. 2) allows the user to insert certain child nodes in the content of other Element nodes. The target nodes which the insertion will be applied to are selected by an XPath expression. The result from the evaluation of the XPath expression must be a list of Element nodes. All the other types of nodes are discarded. This tool can insert two types of child nodes: Element and Text nodes. If the new children are of type Element, the tool expects from the user to supply a valid tag name. Otherwise, i.e. when the new children are Text nodes, the tool accepts any non-empty textual data. The user can also set on which position the new children will appear in their parents' content. Here the counting starts from 0, i.e. the first child is denoted by 0, the second - by 1, etc. If the position field remains empty, then the new nodes will be appended to the target nodes' content. Any non-numerical data in the position field will produce an error.

This tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or load current tool settings, i.e. XML Tool Queries are supported here.

inschild.gif (5618 bytes)

Fig. 2

XPath Insert Parent

This tool (Fig. 3) enables insertion of parent Element nodes of selected by an XPath expression nodes. The selected target nodes by the XPath expression can be either Element or Text nodes. Any other types of nodes are discarded from the selection. This tool expects from the user to specify a valid tag name for the new parent nodes.

The tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or load current tool settings, i.e. XML Tool Queries are supported here.

inspar.gif (5260 bytes)

Fig. 3

XPath Insert Sibling

This tool (Fig. 4) allows inserting sibling nodes of nodes selected by an XPath expression. The target selected nodes can be of any kind,but Attribute nodes. If the root node is selected, it is discarded during the processing time. The new nodes for insertion can be of type Element or Text. If the new nodes are of type Element, the tool expects from the user to supply a valid tag name. Otherwise, i.e. when the new siblings are Text nodes, the tool accepts any non-empty textual data. The user can also set the position where the new sibling nodes will appear. The options are: previous (preceding the target node sibling) and next (following the target node sibling).

The tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or load current tool settings, i.e. XML Tool Queries are supported here.

inssib.gif (5502 bytes)

Fig. 4


XPath Remove

This tool (Fig. 5) gives the possibility of removing parts from XML documents selected by an XPath expression. The target selection can list all types of nodes (including attributes). The only node which cannot be removed is the root of the document and if it is included in the selection it is discarded during the processing time. If a root node is detected in a selection, a warning message is shown. The removal can be done in two modes: either removing the selected nodes and their content (when Delete subtree is selected) or removing only the nodes without their content. In the latter case, the content of the deleted nodes is inserted in the content of their parent(s), in the places where the deletion was performed. The attribute nodes are not considered as content of the Element nodes they belong to, so they are removed in both cases.

The tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or load current tool settings, i.e. XML Tool Queries are supported here.

remove.gif (4859 bytes)

Fig. 5

XPath Rename

This tool (Fig. 6) allows the user to rename Element nodes in an expression selected by an XPath. The user is expected to supply a valid New name. The selected nodes are renamed without changing their attributes and content. If the selection contains nodes of type different than Element, a warning message is shown and these nodes are discarded from further processing.

The tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or load the current tool settings, i.e. XML Tool Queries are supported here.

rename.gif (5043 bytes)

Fig. 6

XPath Transformations

This dialog is used for various "mass" commands for document restructuring. Generally, the scenario is the following: (1) a list of nodes (subtrees, text elements) is chosen by the Source field. In this way the user defines what will be copied or moved in the document; (2) a list of nodes is chosen by the Target field. In this way the place(s) where the source elements will be copied or moved is defined; (3) the elements from the source list are attached to the elements of the target list. There are several options defining the way of performing the above action. It refers to such things as: whether the elements of the source are copied to or cut from the document before being attached to the target, the mapping between the source and the target elements - there are possibilities elements of the source to be attached to each element of the target, or each element of the source to be attached to the corresponding element of the target.

A more detailed description of each field and action follows.

Source

This is a description of the data which is to be copied or moved. The description can be an XPath expression, an XML markup data or some text. If the description is an XPath expression then it is evaluated to a list of nodes depending on the document selected in the combo box under the source field. The XPath expression is evaluated before the actual change of the document to take place. If the source is XML markup data, then it does not need a root element. The XML markup data is parsed to a list of XML nodes. If only text is given, this very text is considered as one element list.

Copy versus Cut

When the Source is defined by an XPath expression, the elements in the resulting list can be either kept in the document or deleted from it before the rest of the processing. If one chooses to copy elements, first a copy of the source list is created and then the operation proceeds further with that copy. If one uses cut, then the elements are first removed from the tree and then the operation continues. When one cuts the elements, the destination XPath cannot be relative.
Note: If you cut the elements, then they will not be present in the tree when the destination XPath is evaluated.

Include subtree check box

If this check box is on, then the source list contains for each chosen node the entire subtree under the chosen node. If it is off, then only the local information for each node is put in the source list. The local information includes the tag name and the set of attributes as well as their values. When only a node with the local information is chosen and it has to be cut, then its children are inserted as immediate children of its parent. The insertion is made in the position of the deleted node.

Treat source as a XML markup check box

By this check box the treatment of the Source field is controlled. If it is on, then the source is treated as XML markup data . If the XML markup data does not contain tags, then it is treated as text.
If the check box is off, then the contents of the Source field is treated as an XPath expression.

Target

An XPath defining the target list of nodes, e.g. the nodes where the source will be included.

Absolute or Relative?

When Absolute, the XPath is calculated from the beginning of the document, which is selected in the document combo box under the destination field, e.g. the XPath "self::*" will return the document element.

When Relative, the XPath is calculated for every node from the source with this node as a context. As a result the user gets a list of nodes for each node in the source.

Insert node(s)

Defines the position where the source to be included, relative to a node in the target list.

As a parent

The nodes from the source become parents (ancestors) of destination.

As a child

The nodes from source become children of the destination in an appropriate position. If the number in the box is less than 0 or it is not a number, then they become last children of the destination nodes.

As a sibling

The nodes from the source become siblings of destination in the appropriate position. 1 means next sibling, 2 means the sibling after the next sibling and so on. -1 means previous sibling, -2 means the sibling before the previous sibling and so on.

Options

Copy all nodes in the source list to every node in the target list

This is allowed only when the target XPath is Absolute. It works according to the selected position.

Although the program always makes a copy, if cut is selected the nodes, in the source list will be deleted.

Copy each node from the source list into the corresponding node in the target list

This option allows an inclusion of the source list into the target list. When the target is Absolute, the first node from the source list is attached to the first node in the target list, the second node from the source list - to the second in the target and so on until there are no nodes left in the source or in the target list. When the target is Relative, each node in the source list is attached to the first node in its relative target list. There is an option to check whether the two lists are of equal sizes (when Absolute) or the relative lists contain a single node (when Relative ).

Copy all nodes from the source list as parents of all nodes in the target list

The elements in the source list are treated as a path in an XML document where the first element is a parent of the second, the second of the third and so on. If the source list contains whole subtrees, then each element in the source list is considered the last child of the previous element. This path is inserted in the document as a common path of parents for all nodes in the target list. This is done by searching for the nearest common parent for all nodes in the target list (note that this parent exists!). Then all nodes which are children of that parent and are placed between the first and the last node in the target list are removed and in their place the first node in the source is put, instead. Then the removed nodes are added as the last children of the last node in the source list.

When the target XPath is Relative, the above procedure is repeated for each node in the target list.

Buttons:


Statistics

The Statistics tool is used for counting the number of nodes or/and token occurrences in XML document(s). The items to be counted initially are selected by an XPath expression (field Select (XPath)). The selection returned by the XPath evaluation is a node set. At this point the Value Keys defined by the user are taken into consideration. Each key contains an XPath expression which is meant to point the essential properties of the selected nodes. The value keys are similar to the ones in the Sort tool. For each node of the initial selection the values from the Value Keys are calculated independently. If for two nodes the corresponding calculated values are the same, they are assumed to belong to the same class. In this way each of the selected nodes is classified in one class. If the statistics has to be applied not only on XML nodes, but on tokens the user must select a tokenizer from Choose Tokenizer: field. In this way the text nodes will be segmented in meaningful tokens. In addition the user can filter the tokens by category in order to receive information only for certain types of tokens (using the button Customize). Only tokens whose categories are in the list will be counted. All the rest will be discarded. If no tokenizer is selected, the text nodes will be processed as a whole node.

The result from the statistics application is a list of all classes formed by the selected nodes. The information which is kept for each class is:

This tool supports two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.The user can also save or load the current tool settings, i.e. XML Tool Queries are supported here.

Statistics on Current Document: The result of this type of statistics is shown as a table below.

The "Category" column contains categories from the filter which exist in the chosen text nodes or "<Element>" if the row represents element node, and "#text" if the node is text.

The "Element" column contains tokenized text (The value of the filtered tokens), or node names.

The "#" column contains number of occurrences of the corresponding item.

The "%" column contains information about the percentage of the corresponding item.

The "Key Value" column contains the value of sort keys created for the corresponding node or nothing if the line contains token.

Closing the table, the user can choose from the following options:

Statistics on Multiple Apply:
<!ELEMENT statistics (documents, item+ , all )>
<!ELEMENT documents (document+)>
<!ELEMENT document (#PCDATA)>
<!ELEMENT item (category, element, number, percent, keyvalue)>
<!ELEMENT category (#PCDATA)>
<!ELEMENT element (#PCDATA)>
<!ELEMENT number (#PCDATA)>
<!ELEMENT percent (#PCDATA)>
<!ELEMENT keyvalue (#PCDATA)>
<!ELEMENT all (number, percent)>
]>

documents tag is a list of selected for statistic documents, where each document name appears in a document tag; item tag corresponds to a line from a result table as follows:

all tag corresponds to the last row of result table. It contains the number of all the occurrences of the selected elements and information for percentage.

Pressing "Cancel" button closes the window and restores the main window of the system.

Node Info

The following item gives information about the number of occurrences of specified tags or tokens in a set of internal documents. When the user starts this tool, s/he is asked to provide several things:

Here is a screen-shot of the initial dialog window:

The main components of the dialog window are:

If the Show Info button is pressed, the system starts to process the selected documents one by one. While processing the documents, the status bar of the system shows the current process state. Having processed all the selected documents, the system shows the result in a new window. Here are two example results, one for Word Info and one for Tag Info:


Text Replace

A default shortcut Ctrl+T
An icon on the text area frame toolbar

This tool is used for searching for patterns in the text and replacing (marking) them in an appropriate way.
The dialog has three fields :

There are two modes of application:

Buttons:


MultiQuery Tool

This tool is designed to call other tools. It does not work directly with XML data. The tool uses a list of XML Tool Queries which are executed one by one in the order of their appearance. The result from each single tool application is an input for the next single tool application. The result from the last single tool application is the result from the MultiQuery Tool. The tools query list is represented in the manager window by a table in which each row keeps information for one query. Each row contains the Name of the query in the second column, which is its document name. The third column shows the Type of the query, i.e. the tool it represents. In the fourth column the Info data from each query is shown. This is optional text which can be saved for each query when it is created or updated. The Info input field is part of the Queries panel in the different tools.

The first column (Label) of the table contains important information about the Conditional Control Operators or short Controls, specific for this tool. These Controls allow changing the order of application of the different queries or/and conditional application of certain operations. For more information see Controls description below.

Here is what the MultiQuery Tool dialog looks like:

multi.gif (7531 bytes)

The operations which the user can perform during the creation of a list of queries are:

The tool also supports the two modes of application: on the current document and Multiple Apply mode. For details see Tool Application Modes.

The user can save or load the current tool settings, i.e. XML Tool Queries are supported here. This feature allows the creation of tool queries. There is no limitation for the level of inclusions in the queries. If a cyclic inclusion is detected (in the multi-query or in any of its sub-queries) the system produces an error.

Controls

The Control operator allows changing the order of application of the queries in the MultiQuery tool. The usual order of applications is starting from the first one and proceeding one by one up to the last one. Using Controls operators some queries can be applied only if certain conditions are true. Such conditions are: the true or false value of a result from an XPath evaluation; whether the preceding single tool application has or has not modified the working document; or unconditional (always succeeding). When a condition for a Control is true, the next query (or another Control), which will be applied, is defined in the control itself. Otherwise, the application proceeds with the next entry in the order (query or another control). The Control operators address their targets (queries or controls to be applied in case of success) by pointing their labels. Each entry(row) in the table of the MultiQuery Tool can have a label (unique identifier) which can be referred by to control operators. It is an error if a Control operator uses a target label which does not exist.

Each Control operator may consist of three parts:

By using the Controls-labels technique, the user can model the famous IF-THEN-ELSE and WHILE-CONDITION-DO structures in order to make the processing more flexible. The composition of different Controls allows the user to create varied 'programs' or 'scripts' capable of doing certain jobs. It is up to the user to create efficient and reliable processing procedures.

Controls Editor

Here follows a description of the Control Operator Editor:

The editor design follows the Controls structure in three sections: