Home
Description
Publications

Available Resources
Text Acknowledgements
Related links


Events


CLaRK System

CLaRK System Online Manual


Bulgarian dialects'
electronic archive


Pragramme in Computational Linguistics



 

 

 

 

 

 

 


Menu File

This menu item includes subitems for management of documents in the CLaRK system. The documents that the system recognizes are XML well-formed documents. These documents can be divided into two kinds. The documents of the first kind are outside of the system. The system can work with these documents if they are imported inside the system. The external documents must have one of the following character encodings: ASCII, UNICODE (UTF-8), UNICODE (UTF-16BE) or UNICODE (UTF-16LE). The documents of the second kind are imported into the system and stored in an internal representation. When a document is processed inside the system, it has to be connected with a document type definition (DTD). The DTD has to be compiled in an internal representation before being assigned to the document. The document loaded in the active view of the system is called a current document.

Sub-Menus:

New

A default shortcut Ctrl+N
An icon on the toolbar

Choosing this item, the user creates a new empty document. First, a menu for choosing a DTD appears. The user can create a document with respect to one of the DTDs compiled into the system. If the appropriate DTD is not compiled, then the user has to close the dialog and compile the required DTD. After choosing the DTD, a new document, containing only the root element with empty content is created. In most cases this document will not be valid.

Open

A default shortcut Ctrl+O
An icon on the toolbar

By choosing this item, the user opens document(s) saved in the internal document database of the system. A Document Manager window is opened on the screen.

The user has to point out to a certain document and click the Open button. Then the corresponding document is loaded into the system. The internal documents are stored together with information for their DTD. If their DTD is compiled in the system, then they are opened with respect to it. If the DTD is deleted (for some reason) from the DTD database of the system, the user is asked to choose a DTD from the available ones.

The user can choose document(s) to open in two modes:

  • selecting one or more documents from a list of all internal documents. It is an option provided by the Document Manager;
  • the internal documents are arranged in hierarchically organized groups. The user can select document(s) from the current group.

If the Cancel button is chosen the, manager window closes and no document is loaded into the system.

The user can also open a document in the manager by performing a double click on a certain document name either in the list of all documents or in the list of the current group.

Note that Open menu does not open text files. If the you want to load an XML document from a text file, then choose Import XML item.

Save

A default shortcut Ctrl+S
An icon on the toolbar

By choosing this item, the user saves the current document in the internal document database of the system. The user has to give a name to the document in a dialog box. Optionally the user can specify a document group in which the document should be included. Then the system tries to save the document. If there exists a document with the specified name, the user is asked to confirm the overwriting of the document or to choose a new name.

Document Manager

This component is responsible for the management of the internal documents in the CLaRK System. It keeps a record of all documents saved in the system. In addition, there is a system of document groups which are arranged in a tree, similar to the directory structure. Each document group contains zero or more internal documents and zero or more sub-groups. The main group is called "Root" and it cannot be renamed or deleted by the user. It contains the following system groups:

  • SYSTEM - It is expected to contain documents for special purposes (help documents for the constraints, documents containing XSL Transformations and others).
  • Results - This group contains a folder for each tool in the system. In this way the results from the application of the tools are stored by default in the particular folder.
  • Queries - This group contains a folder for each tool in the system. Thus each tools' query is stored in the particular tool folder (a query tool includes all the settings of the tool dialogue).

None of the above groups can be deleted or renamed by the user.
The possible operations for a group are: creating a new group; removing a group with its content; adding document(s) which are present in the group but are saved in the system. It is possible for one document to be included in more than one group. In this way different views of the document database can be created. Removing a document from a group does not mean that it is removed from the database. It is just excluded from the group. Here is one example picture of the Document Manager window:

The manager window consists of two main parts:

  1. The panel on the left. It contains the tree representations of the group hierarchy: User, SYSTEM, Results. Note that the User group is created by the user. This group includes 3 documents and another group TEI2. All other groups are system ones. When the user selects a node in this tree, the content of the corresponding group is loaded in the component on the right side.
  2. Current group monitor. This is the panel situated on the right side of the window. It is a list with the content of the currently selected group in the tree. The list components, which are in blue color, are sub-groups. The other entries are the documents included in this group. They are colored in black or red, depending on whether they are valid or not, according to their DTDs.

    There are five additional buttons which can be used for modification the content of the current group:

    • New Group - creates a new sub-group of the current one. The user is asked to give a name for it;
    • Remove - removes the selected entries from the list. The removal is preceded by a confirmation message. If the selection includes sub-groups, they are also removed with their entire contents.
    • Add Document - gives a list of all internal documents which are not present in the current group. The user is expected to choose one or more documents to be included in the current group.
    • Hard Remove! - the button is visible only when the Document Manager is started from menu File/Document Manager. This function can be used for removing documents from the internal document database. It can be applied only to single documents, not to whole groups. Groups are excluded from any selections. The removal of documents is preceded by a confirmation message. The documents to be removed, are excluded from all the groups they may belong to.
    • Apply - This function is used when applying the saved queries. The user can choose one or more queries to be applied. This is performed by selecting documents from the right panel. The "Apply" button is enabled only if the documents are queries. Pressing the button, the MultyQuery Tool manager opens and all the selected queries are inserted automatically.

Navigation in the group structure can be made also in the panel on the right. When the user wants to see the content of a certain sub-group of the current group, s/he just has to perform a double click on the desired sub-group. This will change the current group to the new one. This represents the movement from a group to a sub-group. The movement in the other direction is also possible. For each document group (excluding the Root), a special sub-group is included, named: ". .". By performing a double click on it, similarly to most file systems, the current group is changed to its parent one.

The Document Manager also provides a list of all the internal documents, no matter which group they are included in. The following information appears for each document: date - when it has been last modified; if it is a query, which tool it refers to; and which is its DTD. When selecting documents, the right button of the mouse is used to visualize the Pop-up menu with the following operations on selected documents:

  • Info - This item shows the following information about the selected documents:
    • document name
    • document size
    • document's dtd name
    • whether the document is valid
    • group of the document


  • Add In Group... - This item shows a dialog with the hierarchical structure for the groups in the system and the user can choose a group in which to place all the selected files .

  • Hard Remove! - It is described above.

Here is one example picture of all internal documents in CLaRK:

When the document management is done, the user can update or cancel the changes made over the document database structure by using the two buttons on the bottom of the window.

Delete

By choosing this item, the user deletes document(s), saved in the internal document database of the system. A Document Manager window is opened on the screen.

The user has to point out to a document (or a set of documents) and click the Remove button. Then a confirmation dialog appears. If the user clicks on the Yes button, the chosen document(s)  is(are) permanently deleted from the database. If the user clicks on the No button the document(s) is(are) not deleted.

Similarly to the "Open" function, the user can choose documents in two modes: from all documents or from a certain group. When documents are removed from the list of all documents, the groups they belong to are also updated.

The user can also delete a document in the manager by performing a double click on a certain document name either in the list of all documents or in the list of the current group.

Close

A default shortcut Ctrl+F4

By choosing this item, the user closes the current view of the current document and removes it from the memory.

If this is the only view of the current document and the document has been edited, then the user is asked whether he/she wants to save the changes.

When the view is closed, if there is at least one more view, loaded in the system, the previous active view becomes a current view. If there is no more documents, the system blank view appears.


Import XML

An icon on the toolbar

By choosing this item, the user loads a text file containing an XML document within the system. A standard dialog for choosing a file appears. The user has to specify the name of the file to load and its character encoding in Files of type: choice box. The possible encodings for files are: ASCII, UNICODE (UTF-8), UNICODE (UTF-16BE) or UNICODE (UTF-16LE). Then the user is asked to choose a DTD from the DTD database of the system. If the document contains a DTD, this DTD is ignored during the parsing.

Remark: If a document is opened in ASCII format (and it is sure it is an ASCII file) and in the system it contains different symbols from the expected ones, the possible problems are two:

  1. Inappropriate system font (for details see Options/Fonts);
  2. Inappropriate character encoding converter from ASCII to Unicode (for details see Options/Encoding Correction).

The system tries to parse the document. If the document is well-formed, then it is parsed into the internal representation. The system creates a view of the document, validates the document and if the document is not valid with respect to the chosen DTD, a list of errors is printed in the error window. A description of all validation errors can be found in Validation error messages section.

If the document is not well-formed, then the system reports an error. A detailed description of all parsing errors can be found in Parse error messages section. The errors of this kind have to be repaired within an external to the system editor. The error message reported by CLaRK system can help the user to find the error in the document.

ReImport

A default shortcut Ctrl+R

When the user tries to load a document, which is not well-formed, an error message is reported by the CLaRK system and the user has to edit the document outside the CLaRK system. Then he/she would like to load the same document. In order not to go throughout the whole process of choosing the file and so on, the user can use ReImport item to load the last chosen document.

The system tries to parse the document. If the document is well-formed, it is parsed into the internal representation. The system creates a view of the document, validates the document and if the document is not valid with respect to the chosen DTD, a list of errors is printed in the error window. A description of all validation errors can be found in Validation error messages section.

If the document is not well-formed, then the system reports an error. A detailed description of all parsing errors can be found in Parse error messages section. The errors of this kind have to be repaired with an editor outside the system. The error message reported by CLaRK system can help the user to find the error in the document.

Export XML

An icon on the toolbar

By choosing this item, the user can save the current document into a text file outside the system. A standard dialog for choosing a file appears. The user has to specify the name of the file which will contain the document and the character encoding of the exported text in Files of type: choice box. The possible encodings for files are: ASCII, UNICODE (UTF-8), UNICODE (UTF-16BE) or UNICODE (UTF-16LE). If the files already exist the user is asked whether he/she wants to overwrite it.

Remark: If a document is exported in ASCII format but the content of the output file is not comprehensible for other programs or even for the CLaRK System, then the problem can be in an inappropriate character encoding converter from Unicode to ASCII (for details see Options/Encoding Correction).

The text in the file is formatted according to the layout specified for the DTD of the document.


Export Options

By choosing this item, the user can specify the information, to be added in the external file.

Here is a screen shot of the dialog :

The dialog contains the following components:

  • None - By selecting this radio button, no information about the file layout is added.
  • DTD Layout - By selecting this radio button, the DTD layout of the internal document is preserved in the external file.
  • Current Document Layout - By selecting this radio button, the text layout of the current document is preserved in the external file.
  • Export Line Offsets - By selecting this checkbox, the offset of the external file can be chosen according to the chosen layout.
  • Spaces count per Generation - inserts the offset spaces
  • Print DTD in Document - the DTD of the exported file is added.

Import RTF

By choosing this item, the user loads a RTF file within the system. A standard dialog for choosing a file appears. The user is asked to point to the RTF file to load. The expected file encoding is ASCII as it is specified in the RTF specification. The result from importing an RTF file is an XML document whose structure follows the TEI.2 DTD. During importing the user is asked to point to a DTD name where the TEI.2 DTD is stored. If there is no such DTD available, any other DTD can be selected and the result document will not be valid (which is not a problem).

RTF to XML Conversion

Not all the data from an RTF document is detected and transferred into XML. From the structures which are not recognized in the RTF file only the text content is taken. The types of data for which the system is aware are: heading information (title, author, keywords, creation/last modification date and time), structural information (headers, paragraphs, sections) and text layout information (bold, italics, underlined). Here follows the correspondence table from RTF to XML (TEI.2).

RTF Heading Data

XML(TEI.2) Corresponding Data

title TEI.2 > teiHeader > fileDesc > titleStmt > title
author TEI.2 > teiHeader > fileDesc > titleStmt > author
operator TEI.2 > teiHeader > fileDesc > titleStmt > respStmt > name
company TEI.2 > teiHeader > fileDesc > publicationStmt > publisher
category TEI.2 > teiHeader > encodingDesc > classDecl > taxonomy > category
keywords TEI.2 > teiHeader > profileDesc > textClass > keywords > term
creation date/time TEI.2 > teiHeader > profileDesc > creation > date / time
words count TEI.2 > teiHeader > fileDesc > sourceDesc > bibl > extent > XXX words
characters count TEI.2 > teiHeader > fileDesc > sourceDesc > bibl > extent > XXX characters
characters count incl. whitespaces TEI.2 > teiHeader > fileDesc > sourceDesc > bibl > extent > XXX characters with ws.
Structure & Layout Data XML Tags
paragraph <p> ... </p>
section <div> ... </div>
header <head> ... </head>
footer <trailer> ... </trailer>
bold <hi rend="Bold"> ... </hi>
italics <hi rend="Italics"> ... </hi>
underlined

<hi rend="Underlined"> ... </hi>

Remark: The notation in the table above: "xxx > yyy > zzz" stands for the following XML markup: "<xxx><yyy><zzz></zzz></yyy></xxx>"

The system supports Unicode encoded RTF documents as well as 23 different ASCII text encodings. If no ASCII encoding is detected, the system defaults to US Codepage (OEM-437).


Import from URL

By choosing this item, the user can load an XML document from an URL address. The user should write: URL address, file encoding: ASCII, UNICODE (UTF-8), UNICODE (UTF-16BE) or UNICODE (UTF-16LE). Then the user is asked to choose a DTD from the DTD database of the system. If the document contains a DTD, this DTD is ignored during the parsing.

Remark: If a document is opened in ASCII format (and it is sure it is an ASCII file) and in the system it contains different symbols from the expected ones, the possible problems are two:

  1. Inappropriate system font (for details see Options/Fonts);
  2. Inappropriate character encoding converter from ASCII to Unicode (for details see Options/Encoding Correction).

The system tries to parse the document. If the document is well-formed, then it is parsed into the internal representation. The system creates a view of the document, validates the document and if the document is not valid with respect to the chosen DTD, a list of errors is printed in the error window. A description of all validation errors can be found in Validation error messages section.

If the document is not well-formed, then the system reports an error. A detailed description of all parsing errors can be found in Parse error messages section. The errors of this kind have to be repaired with an editor outside the system. The error message reported by CLaRK system can help the user to find the error in the document.


Here is a screen shot of the dialog :


Import Text

This item allows the user to import pure text documents (without XML markup) in the system. The input documents can be encoded in ASCII or Unicode. The text from the input file is represented as one big text node. For compatibility, the CLaRK System puts an auxiliary tag (textdata) as root of the document and the text node is its only child. This auxiliary tag has an attribute source which keeps information about the location of the source file for this text. Having chosen this item, the user is asked to point to a DTD for the new document.

Export Text

This item allows the user to export the text content of XML document as pure text. The text is serialized in a file without any preprocessing like the substitution of special control symbols ('<', '>', '&', etc.) with entities and others. Only the text nodes are printed in the order they appear in the document. All other data is ignored. The encoding of the text data can be either ASCII or Unicode.


Multi-Import

By choosing this item, the user can load several text files containing XML documents within the system. A standard dialog for choosing a file appears. The user has to choose one or more files from a directory. The files have to contain XML documents in the same character encoding (ASCII, UNICODE (UTF-8), UNICODE (UTF-16BE) or UNICODE (UTF-16LE)). The character encoding can be stated in the Files of type: choice box.

Remark: If a document is opened in ASCII format (and it is sure it is an ASCII file) and in the system it contains different symbols from the expected ones, the possible problems are two:

  1. Inappropriate system font (for details see Options/Fonts);
  2. Inappropriate character encoding converter from ASCII to Unicode (for details see Options/Encoding Correction).

After choosing the files to be imported, a Multi-Import Manager dialog appears. Here is a screen shot of one example dialog window configuration:

The manager window contains the following components:

  • Default DTD for documents - this component suggests a DTD for the documents selected for importing in the system. This chooser is default, because the user can specify a DTD for each document separately and if the DTD of the document is not specified, the system takes the DTD from here. The initial selected value in this chooser is the last DTD used in the system.
  • Send Documents to group - This is an optional chooser. It lets the user send the imported documents to an existing document group in the internal document database. By default, the selection in it is set to the last group which is used in the system. If nothing is selected in this chooser, the documents are saved but not included in any groups. This item is disabled if item Save Documents is not selected.
  • Use directory "valid" Checkbox - This item, if selected, moves all files which are valid according to the corresponding DTD to a sub-directory "valid" of the current directory (the directory which the imported documents are taken from). If such a directory does not exist, the user is prompted to confirm its creation. If the user cancels, this option is disabled.
  • Use directory "well-formed" Checkbox - This item, if selected, moves all files which are well-formed but not valid according to the corresponding DTD to a sub-directory "well-formed" of the current directory (the directory which the imported documents are taken from). If such a directory does not exist, the user is prompted to confirm its creation. If the user cancels, this option is disabled.
  • Documents Preview Table - this component shows all documents which will be imported in the system. It is a table with three columns:
    1. File Name

      This is a list of all files to be imported with their full directory path. This column is not editable.

    2. DTD Name

      This column assigns a DTD to each document to be imported. By clicking on each cell, a list of all DTDs in the system appears. If for a document there is no DTD selected, the system takes the default selected DTD.

    3. Document Name

      This column determines the internal name, under which the document will be saved in the system. This column is editable and the user can select an arbitrary name for each document. If a name for a document ends with a number in brackets, this means that the selected name is already used for some document(s).

  • Start Button - Starts to import the selected documents one by one. While importing, the status bar indicates which document is being currently processed. When the operation is completed a result message window appears with detailed information about each imported document: whether it is valid, well-formed and so on.
  • Cancel Button - Cancels the operation of multi-importing documents.
  • Save Documents Checkbox - This checkbox determines whether or not the imported documents will be saved in the system. Not saving the documents is useful for checking the validity and the well-formedness of a set of documents. By default this item is selected.

If a document is not well-formed, then the system reports an error. A detailed description of all parsing errors can be found in Parse error messages section. The errors of this kind have to be repaired with an editor outside the system. The error message reported by CLaRK system can help the user to find the error in the document.

Multi-Export

This item allows the user to export a set of internal documents: whole group or/and single documents. The user selects an output directory and encoding for all selected documents. Here is a screen-shot of the Multi-Export Manager window:

The manager window consists of:

  • Output Directory section

    This section contains two parts: Directory chooser and Encoding chooser. The Directory chooser shows the output directory, where the document will be stored in XML files. The file names are generated automatically on the basis of the internal document names. If an internal name contains symbols which are not allowed to be in a file name, they are substituted by an underscore symbol ('_'). The output directory can be changed by using button "Change Directory". Having pressed this button, the user is shown a standard file chooser to point to a new output directory. The second component is used for setting an output character encoding. The options are four:

    • ASCII Text File
    • UNICODE (UTF-8) Text File
    • UNICODE (UTF-16BE) Text File
    • UNICODE (UTF-16LE) Text File.

  • Internal Documents chooser

    This component is used for selecting documents to be exported. The component itself represents a standard for CLaRK system Document Selector, which will be described in details below. This selector is used not only in this part of the system, but in all places where a multiple selection of documents is needed. Having selected documents for exporting, the user must use one of the following buttons:

  • Export button

    Starts exporting the selected documents one by one in to the selected directory with the selected character encoding. The status bar of the system shows which file is being currently processed. After completion of the operation an information message about the result of the exported documents is shown to the user.

    Remark: If a document is exported in ASCII format but the content of the output file is not comprehensible for other programs or even for the CLaRK System, then the problem can be in an inappropriate character encoding converter from Unicode to ASCII (for details see Options/Encoding Correction).

  • Cancel button

    Cancels the whole operation.

  • No Overwrite Warnings

    If this item is selected, no confirmation messages for overwriting the existing files will be shown.

Document Selector

The Document Selector is an universal component which is used in almost all system tools, it can work with more than one document. The selector provides the possibility for the user to point to the internal documents a particular tool will be applied to. It consists of two sub-components: a selection list and a selecting dialog window. The selection list shows the currently selected documents. Initially this list is empty. Here follows a picture of the selection list. Usually it is embedded in another dialog window.

Three operations can be applied to this list: adding new list entries, removing selected list entries, clearing the whole list. These operations can be applied by using the buttons on the right:

  • Add Documents - By using this button a dialog window for selection of documents is shown. The functionality will be described below. When new documents are selected, they are added to the list on the left. If a document is selected, but it already is presented in the list, it is not added again.
  • Remove Documents - Removes the selected items in the list on the left. If no selection is made, nothing is done.
  • Clear All - Removes all the entries from the list on the left. If the list is empty, nothing is done.

The second sub-component of the Document Selector is a dialog window for adding entries to the Selected Documents list. This dialog window is visualized when the button Add Documents is pressed. Here is a preview of the dialog:

It supports three modes of document selection:

  1. Selecting a whole document group

    In order to select a whole group, the user must do two things. First, to select a group from the choice list "Document Group". It contains all groups in the system. The group opened in the Document Selector is the last selected group. Changing the selection in this list updates the content of the list on the left with the new selected group content. The second thing is to select the radio button Add Whole Group, situated in the bottom right corner (visible on the picture above). This makes all entries of the list on the left to get selected and the selection cannot be changed.

  2. Selecting single documents from a document group

    The user again is expected to select a group from the "Document Group" list and then to switch the radio button Single Documents on. Changing the selected group updates the content of the list on the left with the new group content. The next thing to do is to select manually a document from the list to be added.

  3. Selecting single documents from a list of all documents

    In order to select items from the list of all documents the user must select the option All Documents situated on the top of the window. The default selection here is always In Groups. When "All Documents" is selected, a list of all documents is shown. The user can select single documents irrespectively to the groups they belong to.

When the selection of documents is completed, the user must confirm it by using button OK or reject it with button Cancel. In case of confirmation, the new selected documents are added to the Selected Documents list.


Exit

By choosing this item, the user closes all opened files and exits the system. If there are unsaved documents, the user is asked whether to save the changes or not.