[ Previous | Next | Table of Contents | Index | Library Home | Legal | Search ]

General Programming Concepts: Writing and Debugging Programs

Creating Indexes of your Documentation

The search engine does not search your actual documentation files. Instead it searches indexes that are created from your documentation. Very simplistically, indexes are compressed copies of your files. This greatly speeds up the searches. Therefore, if you want to use the search service to search your documents, you must create at least one index that will be installed with your documents.

Requirements

Before beginning to create your indexes, make sure you meet the following requirements:

Building the Indexes

  1. Choose a Unique Index Name
  2. Create a New Directory
  3. Create an ASCII File
  4. Choose a Title for your Index
  5. Create an Empty Index
  6. Add your Documents to the Update List
  7. Start the Index Updating Process to Build your Index
  8. Update the Registration Table
  9. Copy your HTML Documents from the Build Directory into the Documentation Directory
  10. Test your Index
  11. Final Step

Each index you create will have its own selection checkbox in the search form. Typically, you create one index that contains text from multiple documents. Each time that index is selected for search, all the documents in that index will be searched. So when you combine documents into an index, you should think about what documents your user will want to search together.

Also, if you are creating an installp package, all documents that are within one index should be placed inside the same installable unit (fileset) of documentation. Otherwise users might only install some of the documents within the index and they would get missing document errors when they try to open the documents from the search results page.

For each index you want to create, repeat the following steps:

  1. Choose a Unique Index Name

    When you create a search index for a document you must specify an eight (8) character name for the index. However, the search service will not let you register your new index if there is already a registered index that has the same name as your index. To reduce the probability of naming conflicts, it is recommended that certain naming conventions be followed:

  2. Create a New Directory

    Create a new directory to hold the documents that will go into the index. We will call this directory the build directory. The build directory can be any place you want it. In our examples we are building indexes for a calculator application, so our build directory will be named /usr/work/calculator. Inside this build directory, arrange the documents into a directory tree structure exactly as you want them to be installed/placed relative to each other on a documentation search server computer.

    The result is that each document will have a full pathname that is composed of a "temporary" part, and a "permanent" part. The temporary part is the pathname of the build directory. The permanent part of the path specifies the location of the document inside your document tree. Once an index is built, the permanent part of a document's pathname cannot be changed. The one rule about the pathnames is that the first directory in the permanent part of the pathname must be the index name.

    For example, your application is called calculator. The online documents for the application are written in US English. There are two user guide documents (doc1, doc2) and one administrator document (doc3). You could place the documents like this in the filesystem on the computer on which you are building the indexes: 

    /usr/work/calculator/user/doc1.html
/usr/work/calculator/user/doc2.html
/usr/work/calculator/admin/doc3.html

    You can place your build directory anywhere, but all documents that go into a single index must under a single directory which acts as the common top directory so that they form a single tree. In the example, calculator is the common top directory.

  3. Create an ASCII File

    For each index, create a document list file. Place inside this file a list of all the documents you want to be in the index. For each document, list it by using the full pathname that specifies where the document can be currently found on your development computer. Note that the working locations of these documents do not need to be the same location where the documents will be eventually installed on a documentation server. This document list file can be named anything and placed in any directory. Put each pathname on its own line in the file.

    If you arranged your documents like the example above, your ASCII file would have the following contents: 

    /usr/work/calculator/user/doc1.html
/usr/work/calculator/user/doc2.html
/usr/work/calculator/admin/doc3.html

    Next you must indicate where the temporary part of each pathname ends and where the permanent "installed" part of the pathnames start. You do this by replacing the last slash (/) in the temporary part of the document pathnames (the build directory pathname) with a commercial at symbol (@). When the index is created, only the part of each pathname that is to the right of the @ will be saved in the index.

    For example, the above example file would now be modified to look like this: 

    /usr/work@calculator/user/doc1.html
/usr/work@calculator/user/doc2.html
/usr/work@calculator/admin/doc3.html

    The slash before the application name (calculator) was replaced with an @ since it is the last slash in the temporary part of the path.

  4. Choose a Title for your Index

    The title of your index is the text that will appear next to the index's checkbox in the search form. The title should uniquely describe the document or documents that are in the index and contain a maximum of 150 characters.

  5. Create an Empty Index

    You must then create an empty index. After the index is created you will fill it. To prepare for index creation, you must check the following:

    Your user id must be a member of the imnadm group to use the steps that follow. Before you can create your first index you will need to change ownership of the /usr/docsearch/indexes directory so that it is owned by the user imnadm. You will only need to do this step before you create your first index.

    chown imnadm:imnadm /usr/docsearch/indexes

    Creation of an index requires three steps.

    1. Create the empty index.

      The index creation command has the syntax (all 5 lines on one command line): 

       /usr/IMNSearch/bin/itecrix -s server -x index_name
 -p /usr/docsearch/indexes/index_name/data
 -pw /usr/docsearch/indexes/index_name/work
 -lsse itelsswt
 -t NORM | -t NGRAM
 -ccsid <codeset_id>

      Where index_name is the 8 character name of the index. The values for -t and -ccsid depend on the language of the documents in the index. Note that all single-byte languages have a -t value of NORM. All multi-byte languages have a -t value of NGRAM and you also must add the -ccsid value when creating a multi-byte index. The following table specifies the values to use for each language:

      Language -t -ccsid -lang
      English (United States) ISO8859-1 NORM 819 EN_US
      English (United States) ISO8859-15 NORM 923 EN_US
      English Great Britain ISO8859-1 NORM 819 EN_GB
      Catalan ISO8859-1 NORM 819 CA_ES
      Catalan ISO8859-15 NORM 923 CA_ES
      French ISO8859-1 NORM 819 FR_FR
      French ISO8859-15 NORM 923 FR_FR
      French Canadian ISO8859-1 NORM 819 FR_FR
      German ISO8859-1 NORM 819 DE_DE
      German ISO8859-15 NORM 923 DE_DE
      German Switzerland ISO8859-1 NORM 819 DE_CH
      Icelandic ISO8859-1 NORM 819 IS_IS
      Italian ISO8859-1 NORM 819 IT_IT
      Italian ISO8859-15 NORM 923 IT_IT
      Norwegian ISO8859-1 NORM 819 NO_NO
      Portuguese, Brazil ISO8859-1 NORM 819 PT_BR
      Portuguese, Portugal ISO8859-1 NORM 819 PT_PT
      Portuguese, Portugal ISO8859-15 NORM 923 PT_PT
      Russian ISO8859-9 NORM 878 RU_RU
      Spanish ISO8859-1 NORM 819 ES_ES
      Spanish ISO8859-15 NORM 923 ES_ES
      Swedish ISO8859-1 NORM 819 SV_SE
      Japanese IBM-932 NGRAM 932 JA_JP
      Korean IBM-eucKR NGRAM 949 KO_KR
      Traditional. Chinese big5 NGRAM 950 ZH_TW
      Simplified Chinese IBM-eucCN NGRAM 1381 ZH_CN

      Following our example, to create a single byte English index, you could type (all 5 lines on one command line): 

       /usr/IMNSearch/bin/itecrix -s server -x cal413en
 -p /usr/docsearch/indexes/cal413en/data
 -pw /usr/docsearch/indexes/cal413en/work
 -lsse itelsswt
 -t NORM
    2. Next you must specify the language and codeset of the documents that will be inserted into the index. The language specfication command has the following format (all on one line): 

       /usr/IMNSearch/bin/iterulix -s server -x index_name -dfmt HTML
 -ccsid <codeset_id
 -lang <language

      where index_name is the same name that was used in the previous command and codeset_id and language are the values from the table above.

      Following our example, you would now type (all on one line): 

       /usr/IMNSearch/bin/iterulix -s server -x cal413en -dfmt HTML -ccsid 819
 -lang EN_US
    3. After you create your index, you should check to make sure that your index is listed with the Documentation Library Service by typing:

      /usr/IMNSearch/bin/itelstix -s server

  6. Add your Documents to the Update List

    Next you must tell the Documentation Library Service the name of the file that contains the list of the documents that will go into the the empty index you just created. Then later you will run an update command and those documents will be indexed and the results will be inserted in the index.

    Use the following command to add your documents to the list of documents that will get inserted into the index 

    /usr/IMNSearch/bin/itequeue -s server -x index_name -add -l document_list_file

    (where document_list_file is the name of the ASCII file you created that contains your list of documents):

    Note: Test to make sure that your documents were queued successfully. Type: 

    /usr/IMNSearch/bin/itestaix -s server -x indexname

    The number after Number of indexing requests scheduled should equal the number of documents in your index.

  7. Start the Index Updating Process to Build your Index

    Start the index updating process. This will take the documents that are in your document update list, index them, and put the results into the empty index to build your final complete index.

    Note: Indexing may take a significant amount of time to complete. You CANNOT move onto the Update the Registration Table step until indexing is complete. Use the status command below to tell when indexing is done.

    Use the following update command: 

    /usr/IMNSearch/bin/iteupdix -s server -x index_name

    Note: Test to make sure that your documents were indexed successfully. Type: 

    /usr/IMNSearch/bin/itestaix -s server -x indexname

    The number after Number of documents in the primary index should equal the number of documents in your index.

  8. Update the Registration Table

    Next you need to register the new index in the registration table of your development computer so that the search service knows the index exists and you can do a test searches of the index.

    To update the registration table on the development computer, do the following (all on one command line):

    Note: There must be a final slash (/) after the application_name. 

    /usr/IMNSearch/bin/itedomap
    -p /var/docsearch/indexes -c -x index_name
    -sp /doc_link/locale/application_name/
    -ti index_title

    The locale variable is the name of the language directory under /usr/share/man/info where the index's documents are stored. The variable index_name is the name of your index, and index_title is the title of your index. The title is the text you want the user to see in the bottom of the search form when they are selecting which indexes to search. Remember that the title should be written using the same language and codeset as the documents inside the index.

    Additionally, for index titles, it is recommended that you specify the title as an HTML link. The title will then appear as a link in the search form. This allows a user to click on the title in the search form to open the first document in the index for reading.

    Note:

    Every web server has an internal document home directory where it starts its search for documents. When the Documentation Library Service is installed and configured, a filesystem link is placed in this directory. This link points to the standard location of documents in the AIX filesystem: /usr/share/man/info. Since your web server will automatically go to this location to find your documents, the search engine only needs the portion of the document path from this location forward.

    The link that the Documentation Library Service puts into your web server's starting directory is:

    doc_link -> /usr/share/man/info

    Your web server will be able to serve the documentation with URLs like:

    http://your.machine.name/doc_link/en_US/calculator/user/doc1.html

    For example, you might want the title of your index to be Calculator Application Manuals. You have three documents(manuals) inside this one index which is named cal413en. You decide that when the title link is clicked it would be best for the Beginners Guide document to be opened. So, you would insert in the title the URL that opens the Beginners Guide document. If you would normally type the URL /doc_link/en_US/calculator/user/doc1.htm (doc_link is the link to the /usr/share/man/info directory) to open the Beginners Guide document, you would use the following update command (all on one command line): 

    /usr/IMNSearch/bin/itedomap
    -p /var/docsearch/indexes -c -x cal413en
    -sp /doc_link/en_US/calculator/
    -ti <A HREF='/doc_link/en_US/calculator/user/doc1.htm'>Calculator Application Manuals</A>
  9. Copy your HTML Documents from the Build Directory into the Documentation Directory

    You must now copy your HTML documents into the location where they can be read by your users. Your documents should be placed under the directory /usr/share/man/info/locale/application_name/ index_name. Using our example, the Calculator Application's English documents would be placed in /usr/share/man/info/en_US/calculator/.

    1. Find out if the language directory /usr/share/man/info/ locale already exists. If it does not exist, create it. When you create this directory, make sure that it is executable and readable by all users.

      Using our example, the English directory is named: /usr/share/man/info/en_US.

    2. Create your application directory under the language directory. The directory structure should now look like: /usr/share/man/info/locale/application_name.

      Using our example, the application's directory is named: /usr/share/man/info/en_US/calculator.

    3. Copy your documents and place them under the application directory you just created. The directory structure should now look like:

      /usr/share/man/info/locale/application_name/documents.

      Using our example, you would use the following command to copy the calculator's documents from the build directory into the directory where they will be read by users. 

      cp -R /usr/work/calculator/* /usr/share/man/info/en_US/calculator

      The calculator's documents would then end up in these locations: 

      /usr/share/man/info/en_US/calculator/user/doc1.html
/usr/share/man/info/en_US/calculator/user/doc2.html
/usr/share/man/info/en_US/calculator/admin/doc3.html
  10. Test your Index

    You have now completed the creation and registration of an index on this development computer. You should now test the index by opening the search form, selecting the new index for search, and searching for words that you know are in the index. If the index does not work properly and you need to remove it so you can build it again, go to the section called Removing Indexes in your Documentation (Removing Indexes of your Documentation). When you are satisfied that the index is working correctly, go on to the next step

  11. Final Step

[ Previous | Next | Table of Contents | Index | Library Home | Legal | Search ]