Class HTMLtools

java.lang.Object
  Globals
      HTMLtools

public class HTMLtools
extends Globals

Program HTMLtools main class.

This application converts a set of tab-delimited data files to various HTML < TABLE > formats with many mapping options available. We use the term Table with an upper case 'T' to indicate the FileTable data structure used throughout the program. The command line arguments are parsed by the Switches class. It may be used iteratively if the argument is the '-batchProcess:{some batch file}' switch.

The program is documented in the MANUAL.html file that includes an introduction, list of switch commands, a number of examples, instructions, and additional documentation. See TODO.java class for [TODO] List for software Development.

Note that the application could be tightened up and optimized more. However, since most of the files it processes are relatively small, it may not be worth the effort.

List of Class modules

HTMLtools.java - main class and contains the top-level process loop. It extends class Globals.

Globals.java - contains the global variables.

Convert.java - contains the Table-to-HTML converters and various mapping methods.

CvtGUI.java - GUI for the converter.

JTVconvert.java - JTV array-name mapping and HTML generation methods

FileIO.java - methods for file I/O and zip file I/O.

FileTable.java - Table file readers and manipulation methods.

GenBatchScripts.java - generate batch scripts from methods.

MakeFlipTable.java - methods for making a flipped Table.

MakeTestsIntersectionTbl.java - create a Tests-Intersection Table.

ProcessData.java - Process data thread used with CvtGUI.

ProcessDataSearch.java - Process data thread used with SearchGUI.

SearchGUI.java - Search database GUI.

Sort.java - sort methods used in sorting Table rows.

Switches.java - the command line switch parser.

UtilCM.java - various short utility methods.

TODO.java - Comments TODO/DONE lists.

O2PlibSS/gui - various GUI classes used by CvtGUI/SearchGUI.

 List of Methods
 HTMLtools() - Constructor for converter main class
 createGUI() - create GUI instance. Do this only one time.
 createSearchGUI() - create Search GUI instance. Do this only one time.
 isAbortProcessingFlag() - check abort processing flag abortProcessingFlag.
 setAbortProcessingFlag() - set abort processing flag abortProcessingFlag.
 main() - program main when used as stand-alone application.
 cmdLineProcess() - main processing to parse  command lines, process all files.
 initCvt() - get OS variables, set up class instances & some data structures.
 processTableFile() - process an input file into an output HTML file.
 setupPrologEpilogue() - get the prolog and epilogue if they exist.
 mapFileSpecificPrologKeywords() - map file specific keywords in prolog.
 mapFileSpecificEpilogueKeywords() - map file specific keywords in epilogue.
 preprocessTable() - pre-process the Table prior to conversion to HTML.
 makeConcatTable() - create a new concatenated tab-delimited .txt 
 makePrefaceHTML() - if -makePrefaceHTML, then setup preface file 
 doEpilogueExtract() - if row extraction from resource file, mod. epilogue
 saveIndexMapFile() - save the index map table for the edited Table
 closeProgram() - close out the log file if logging messages and exit.
 fastEditFile() - fast process an input file into a tab-delimited file.
 buildColumnEditMap() - build column editing map for fast table editing.
 postCopyFilesAndDirectories() - copy files post-conversion
 joinTableToCurrentTable() - join another table to ft before preprocessing
 createFlipTable() - generate transposed flipped Table files: .txt, .html
 makeIndexMapFile() - read file header, make Index-Map in Table structure
 addNameOfHTMLfile() - append name of HTML file path last generated  
 clearGenHTMLpathList() - clear Generated HTML path List
 getGenHTMLpathList() - get Generated HTML path List
 incrUpdateStatusLine() - if GUI, then do incremental statusLine update.
 

This code is available at the HTMLtools project on SourceForge at http://htmltools.sourceforge.org/ under the "Common Public License Version 1.0" http://www.opensource.org/licenses/cpl1.0.php.

It was derived and refactored from the open source MAExplorer (http://maexplorer.sourceforge.org/), and Open2Dprot (http://Open2Dprot.sourceforge.net/) Table modules.

$Date: 2009/12/12 11:45:56 $ $Revision: 1.41 $
Copyright 2008, 2009 by Peter Lemkin E-Mail: lemkin@users.sourceforge.net http://lemkingroup.com/

Field Summary

static boolean	abortProcessingFlag Abort processing at major points: at the end of each file being converted.
static CvtGUI	gui CvtGui instance if it exists
java.lang.String	initialReport Initial report to user on startup
static SearchGUI	sGui SearchGui instance if it exists
static boolean	usingGUIflag Set if started up as a GUI of any type
static boolean	verboseLoggingFlag
static java.lang.String	VERSION Title line version numbers

Fields inherited from class Globals

addFCrangesForTestsIntersectionTableFlag, addOutfilePostfixName, addRangeOfMeansToTItableFlag, addRowNumbersFlag, addTableName, addTableNameFlag, altRowBackgroundColor, ALWAYS, argsV, batchCwd, batchParamsSpecs, batchProcessFile, batchProcessFlag, batchProcessStr, bigCellFontSize, bigExportCellThr, bigShrinkCellThr, colHdrIdxHmapData, colHdrNameHmapData, colNameIndexMap, colorMapScaleHTML, concatInputFilesWithSameFieldsFlag, concatOutputHtmlFile, concatOutputTxtFile, convert, copyDstFile, copyFileFlag, copySrcFile, copyTreeDataFlag, copyTreeDstDir, copyTreeSrcDir, curBatchJobNbr, curBatchParamsSpecs, curInputFile, cvt, dataPrecisionHTMLtable, date, DBUG_FLAG, dirIndexHtml, dirIndexHtmlFlag, dirIndexHtmlOverrideFlag, dropColNames, epilogue, epilogueFile, EXPORT_BIG_CELL_SIZE, exportBigCellsToHTMLfileFlag, extractColName, extractHtmlStyle, extractResTblFile, extractRowFromResTblAsHTMLflag, extractRowNbr, fastEditFileFlag, fileSep, filterDataField, filterDataList, filterTestField, filterTestsList, fio, flipAclass, flipBclass, flipColumnFile, flipColumnName, flipColumnValues, flipDataFile, flipDirectory, flipExcludeColumnName, flipFCthreshold, flipFCthrFlag, flipIndexMapFile, flipOrderHdrColList, flipRowFilterNames, flipRowFilterNamesFile, flipRowGSPIDfilters, flipSaveOutputFile, flipTableByIndexMapFlag, flipTableUseExactMatchFlag, fromLongHdrNamesMap, ftConcat, ftL2S, genAnalysesDir, genBatchDir, genBatchScriptFlag, genHTMLpathList, genInputTreeDir, genJTVDir, genMapEGdetails, genMapIntroduction, genParamsScriptsDir, genParamTemplateFiles, genParamTemplateFlag, genParamTemplateNames, genSummaryDir, genSummaryTemplateFlag, genSummaryTemplateNames, genSummaryTemplateOrderNbrs, genSupportFilesToCopy, genTestsIntersectionFlag, genTestToDoListFile, genTreeCopyDataFlag, genTreeDstTreeDir, genTreeSrcTreeDir, glbMaxRowVal, glbMeanRowVal, glbMinRowVal, glbStdDevRowVal, hasEmptyLineBeforeTableFlag, hasStatIndexMapFileFlag, hasTableHeaderRowFlag, heatMapColors, hrefHeaderRowFlag, ignoreDuplicateFieldsFlag, inputDataDir, inputExt, inputFiles, inputNames, isMacFlag, isSunFlag, isWinPCflag, joinTableFile, jtv_mAdbArraySummaryFile, jtv_mapHdrNamesFile, jtvButtonName, jtvCopyJTVjars, jtvDescription, jtvHTMLgenerateFlag, jtvInputDir, jtvMapFlag, jtvOutputDir, jtvReZipConvertedFiles, jtvSpecificInputFileList, jtvUseMapFromColNbr, jtvUseMapToColNbr, keepColNames, limitMaxTableRows, limitMaxTableRowsFlag, limitRowsSortAscendingFlag, limitRowsSortByColName, logFileName, long2ShortFromColNbr, long2ShortToColNbr, MAKE_LOGGING_FILE, makeHTMLflag, makeIndexMapFile, makeIndexMapFileFlag, makeMapColNames, makeMapTblFileName, makeOptionsListFromSemicolonListFlag, makePrefaceHTMLflag, makeStatisticsIndexMapFlag, makeTestsIntersectionTableFlag, makeTXTflag, mapDkeywords, mapDollarsignsFlag, mapDtoString, mapHdrNamesFile, mapHdrNamesFlag, mapQkeywords, mapQtoString, mapQuestionmarksFlag, mapTokenInUrlHmapData, MAX_BATCH_SPECS, MAX_COL_EDITS, MAX_COL2URL_MAPS, MAX_COPY_FILES_AND_TREES, MAX_IN_TOKENS_PER_ROW, MAX_INPUT_FILES, MAX_MAP_DOLLARSIGNS, MAX_MAP_QUESTIONMARKS, MAX_SORT_COLUMNS, MAX_TEMPLATES, maxBatchParamsSpecs, maxFlipSeekRowsToExtract, nbrTableHdrLines, nColNameIndexMap, nColUrlHmapData, nCopyFile, nCopyTreeDir, nDirIndexHtml, nDropColNames, NEVER, newHdrMapColNames, newline, nFlipColumns, nFlipExcludeColumns, nFlipOrderHdrColList, nFlipRowFilterNames, nGenParamTemplates, nGenSummaryTemplates, nGenSupportFilesToCopy, nGenTreeCopy, nHdrMapColNames, nInputFiles, nKeepColNames, nLong2ShortHdrNamesMap, nMakeMapColNames, nMapDollarsigns, nMapQuestionmarks, noBorderFlag, nReorderColName, nSortTable, oldHdrMapColNames, oldUserDir, openSourceNotice, osName, outputDataDir, outputExt, outputFiles, paramMapDir, paramMapFile, programName, programSummary, prolog, prologFile, referenceManual, reorderColName, reorderColNbr, reorderRemainingColumnsFlag, reportFoldChangeFlag, rmvTrailingBlankRowsAndColumnsFlag, saveEditedTable2File, saveEditHTMLflag, saveEditTableFlag, showDataHeatmapFlipTableFlag, SHRINK_BIG_CELL_SIZE, SHRINK_CELL_SIZE, shrinkBigCellsFlag, sortFlipTableByColumnName, sortRowsByColumnFlag, sortTableAscendingFlag, sortTableColName, sParamsStr, specificInputFileList, startTableAtKeywordLine, sw, tableDataDir, testsToDoTblFile, title, toShortHdrNamesMap, urlBaseLinkHmapData, useHTMLfontSize, useOnlyLastHeaderLineFlag, userDir, util

Constructor Summary

HTMLtools()
HTMLtools() - Constructor for converter main class

Method Summary

void	addNameOfHTMLfile(java.lang.String nameOfHTMLpath) addNameOfHTMLfile() - append name of HTML file path last generated to the this.genHTMLpathList[] list.
private int[]	buildColumnEditMap(FileTable ft, boolean reorderRemainingColsFlag) buildColumnEditMap() - build column editing map for fast table editing.
void	clearGenHTMLpathList(boolean forceFlag) clearGenHTMLpathList() - clear Generated HTML path List if not using the GUI and not doing batch processing.
void	closeProgram(int exitNbr) closeProgram() - close out the log file if logging messages and exit with the exit number.
boolean	cmdLineProcess(java.lang.String[] args) cmdLineProcess() - main processing for CvtTabDelim2HTML.
private boolean	convertTableFiles() convertTableFiles() - process list of Table files into HTML files.
boolean	createCvtGUI() createCvtGUI() - create GUI instance.
private boolean	createFlipTable() createFlipTable() - generate transposed flipped Table files (.txt and .html).
boolean	createSearchGUI(java.lang.String searchParamMapDefault) createSearchGUI() - create Search GUI instance.
private boolean	createTestsIntersectionTable() createTestsIntersectionTable() - create Tests Intersection Table and do related processing and .txt and .html file generation.
private java.lang.String	doEpilogueExtract(FileTable ft, java.lang.String sEpilogue) doEpilogueExtract() - test if doing row extraction from a resource table to modify the epilogue.
private boolean	doGenBatchScripts() doGenBatchScripts() - do GenBatchScripts processing to create batchScripts/ directory.
private boolean	fastEditFile(int fileNbr, int maxFiles, java.lang.String iDir, java.lang.String oDir, java.lang.String iFile, java.lang.String oFile, java.lang.String sName) fastEditFile() - fast process an input file into a tab-delimited file.
java.lang.String[]	getGenHTMLpathList() getGenHTMLpathList() - get Generated HTML path List cvt.genHTMLpathList[].
void	incrUpdateStatusLine(java.lang.String incrStr) incrUpdateStatusLine() - if GUI, then do incremental statusLine update.
void	initBase() initBase() - initialize basic data structures
boolean	initCvt(java.lang.String[] args) initCvt() - initialize data structures
static boolean	isAbortProcessingFlag() isAbortProcessingFlag() - check abort processing flag abortProcessingFlag.
boolean	joinTableToCurrentTable(FileTable ft, java.lang.String joinTableFile) joinTableToCurrentTable() - join another table to ft before preprocessing.
static void	main(java.lang.String[] args) main() - run CvtTabDelim2HTML as a stand-alone application.
private boolean	makeConcatTable(FileTable ft, int fileNbr, int maxFiles) makeConcatTable() - create a new concatenated tab-delimited .txt output file from the set of input text files IFF they have exactly the same column header names.
private FileTable	makeIndexMapFile(java.lang.String iFileNamePath, java.lang.String oDir, java.lang.String oIdxFileName) makeIndexMapFile() - read file header and make Index-Map in Table structure.
private java.lang.String	makePrefaceHTML(FileTable ft, java.lang.String oFile, java.lang.String sName, java.lang.String sProlog, java.lang.String sEpilogue) makePrefaceHTML() - if -makePrefaceHTML, then setup preface file name to link to in the 2nd file HTML we generate
java.lang.String	mapFileSpecificEpilogueKeywords(java.lang.String subTitle, java.lang.String epilogue) mapFileSpecificEpilogueKeywords() - map file specific keywords in the epilogue.
java.lang.String	mapFileSpecificPrologKeywords(java.lang.String subTitle, java.lang.String prolog) mapFileSpecificPrologKeywords() - map file specific keywords in prolog.
boolean	postCopyFilesAndDirectories() postCopyFilesAndDirectories() - copy files post-conversion using the set of -copyFile:{srcFile,destFile} and -copyTree:{srcTreeFiles,destPath} switches specified.
private boolean	preprocessTable(FileTable ft) preprocessTable() - pre-process the Table prior to conversion to HTML.
private boolean	processTableFile(int fileNbr, int maxFiles, java.lang.String iDir, java.lang.String oDir, java.lang.String iFile, java.lang.String oFile, java.lang.String sName, java.lang.String sProlog, java.lang.String sEpilogue, boolean use_ftConcatFlag) processTableFile() - Process an input file into an output HTML file.
private boolean	saveIndexMapFile(java.lang.String inputFilePath, java.lang.String oDir, java.lang.String oFile, FileTable ft) saveIndexMapFile() - save the index map table for the edited Table If doing '-makeIndexMapFile:{colName1,colName2,...,colNameN}' first generate the Index-Map Table using the makeIdxMapTable().
static void	setAbortProcessingFlag(boolean flag) setAbortProcessingFlag() - set abort processing flag abortProcessingFlag.
boolean	setupPrologEpilogue() setupPrologEpilogue() - setup the prolog and epilogue strings if the files exist.

Methods inherited from class Globals

reinitGBSglobalSwitches, reinitGlobalSwitches, reinitJTVglobalSwitches, reinitTIglobalSwitches

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VERSION

public static final java.lang.String VERSION

Title line version numbers

See Also:

Constant Field Values

gui

public static CvtGUI gui

CvtGui instance if it exists

sGui

public static SearchGUI sGui

SearchGui instance if it exists

usingGUIflag

public static boolean usingGUIflag

Set if started up as a GUI of any type

abortProcessingFlag

public static boolean abortProcessingFlag

Abort processing at major points: at the end of each file being converted. This flag is only set when used with the GUI and should never be set when operating as a command-line application.

verboseLoggingFlag

public static boolean verboseLoggingFlag

initialReport

public java.lang.String initialReport

Initial report to user on startup

Constructor Detail

HTMLtools

public HTMLtools()

HTMLtools() - Constructor for converter main class

Method Detail

createCvtGUI

public boolean createCvtGUI()

createCvtGUI() - create GUI instance. Do this only one time.

Returns:

true if it is successful.

See Also:

CvtGUI

createSearchGUI

public boolean createSearchGUI(java.lang.String searchParamMapDefault)

createSearchGUI() - create Search GUI instance. Do this only one time.

Parameters:

searchParamMapDefault - is default param .map file if not null.

Returns:

true if it is successful.

See Also:

SearchGUI

isAbortProcessingFlag

public static boolean isAbortProcessingFlag()

isAbortProcessingFlag() - check abort processing flag abortProcessingFlag. This is only called at major points: at the end of each file being converted. This flag is only set when used with the GUI.

setAbortProcessingFlag

public static void setAbortProcessingFlag(boolean flag)

setAbortProcessingFlag() - set abort processing flag abortProcessingFlag. This is only checked using isAbortProcessingFlag() at major points: at the end of each file being converted. This flag should only set when used with the GUI.

main

public static void main(java.lang.String[] args)

main() - run CvtTabDelim2HTML as a stand-alone application. Note global variables in the base class Global.java.

See Also:

cmdLineProcess(java.lang.String[]), closeProgram(int), UtilCM.setLogMsgs(boolean)

cmdLineProcess

public boolean cmdLineProcess(java.lang.String[] args)

cmdLineProcess() - main processing for CvtTabDelim2HTML. Note global variables in the base class Global.java. Note: it will return false if an abortProcessingFlag is set and the processing aborts.

Returns:

true if successful, false if error or detected abortProcessingFlag.

See Also:

Globals.reinitGlobalSwitches(), #parseSwitches, convertTableFiles(), doGenBatchScripts(), createTestsIntersectionTable(), postCopyFilesAndDirectories(), createFlipTable(), closeProgram(int), Convert.checkAndMakeIndexHtmlDirList(int, java.lang.String, java.lang.String[], boolean[]), FileIO.getDirOfFile(java.lang.String), FileIO.getFilesInDir(java.lang.String, java.lang.String), FileIO.readFileAsString(java.lang.String), JTVconvert, JTVconvert.convertAllJTVfiles(), Switches.parseSwitches(java.lang.String[]), UtilCM.logMsg(java.lang.String), UtilCM.mapCRLF2space(java.lang.String), UtilCM.dateStr(), UtilCM.changeWorkingDirectory(java.lang.String), UtilCM.cvs2Array(java.lang.String, java.lang.String)

initCvt

public boolean initCvt(java.lang.String[] args)

initCvt() - initialize data structures

Parameters:

args - is the command line arg list

Returns:

true if succeed

See Also:

UtilCM, UtilCM.dateStr(), UtilCM.logMsg(java.lang.String), Convert, FileTable, Switches

initBase

public void initBase()

initBase() - initialize basic data structures

convertTableFiles

private boolean convertTableFiles()

convertTableFiles() - process list of Table files into HTML files. The bulk of the conversion is done in the processTableFile() method.

Returns:

true if succeed to continue process, false if a problem or abortProcessingFlag was set.

See Also:

processTableFile(int, int, java.lang.String, java.lang.String, java.lang.String, java.lang.String, java.lang.String, java.lang.String, java.lang.String, boolean), setupPrologEpilogue(), UtilCM.logMsg(java.lang.String), UtilCM.replaceSubstrInString(java.lang.String, java.lang.String, java.lang.String)

processTableFile

private boolean processTableFile(int fileNbr,
                                 int maxFiles,
                                 java.lang.String iDir,
                                 java.lang.String oDir,
                                 java.lang.String iFile,
                                 java.lang.String oFile,
                                 java.lang.String sName,
                                 java.lang.String sProlog,
                                 java.lang.String sEpilogue,
                                 boolean use_ftConcatFlag)

processTableFile() - Process an input file into an output HTML file. This first reads the input tab-delimited data file into a FileTable ft structure. Processing is divided into 4 major phases:

  [1] Create a current FileTable instance.
  [2] Read and parse the tab-delimited input file to current FileTable.  
  [3] Optional (switch dependent) Table pre-processing including:
     -dropColumn list of columns
     -keepColumn list of columns
     -reorderColumn list of columns
     -mapLongToShortHdrNames
     -sortRowsByColumn data. 
  [3.1] optional -concatTables data from all input files with the 
      the same ft.tFields.
  [3.2] optional creation of preface HTML string (from 1st part of 
      current file before Table) for use later when convert and post
      process the current Table.
  [3.3] if -extractRow, then get the data from the resource Table
      file and insert it into the epilogue string for $$EXTRACT_ROW$$. 
  [4] Conversion of current File Table to HTML by the following:       
     Then it creates a Column-to-URL mapping lists.
   a) Add Web page header via the prolog and the TABLE title.
   b) Add TABLE header row
   c) Add all of the TABLE data rows, mapping some column data to
      the URL HREFs if match.
   d) Add epilogue HTML if it exists (before doing extractRow if 
      required).    
 [4.1] Write the HTML file from the current File Table. If doing 
       -makeIndexMapFile, then no HTML file is generated. If doing
       '-saveEditedTable2File:{outTblFile,opt. "HTML"}', then the
       HTML is conditionally written. If the outTblFile is null
       then use the input file name with the output file postfix
       string added to the input file.
 [5] optional post-conversion preface processing to create HTML file
     linked from the current Table HTML file.
 [6] If doing '-concatTables:{outputConcatenateFile.txt}', or
     '-makeIndexMapFile:{colName1,colName2,...,colNameN}' or
     '-saveEditedTable2File:{outTblFile,opt. "HTML"}', then it
     will write an output file. In the case of -makeIndexMapFile, it has
     the same name as the input file, but with an .idx file extensions.
 

Finally it writes the out into the output HTML file.

Parameters:

fileNbr - file number in the input files list. Special processing for 1st file

maxFiles - max # files in the input file list.

iDir - input file directory

oDir - output file directory

iFile - input file name with extension (no directory)

oFile - output file name with extension (no directory)

sName - name for the prolog HTML header if not null

sProlog - prolog string if not null

sEpilogue - epilogue string it not null

use_ftConcatFlag - it use the precomputed ftConcat Table else read the Table from the file.

Returns:

true if succeed

See Also:

UtilCM.logMsg(java.lang.String), FileTable#readFileAsTable, FileTable.setDuplicateFieldsFlag(boolean), FileTable.setNbrTableHdrLines(int), FileTable.setRmvTrailingBlankLinesFlag(boolean), FileTable.setRmvTrailingEmptyColumnsFlag(boolean), FileTable.setHasEmptyLineBeforeTableFlag(boolean), preprocessTable(FileTable), makeConcatTable(FileTable, int, int), makePrefaceHTML(FileTable, java.lang.String, java.lang.String, java.lang.String, java.lang.String), doEpilogueExtract(FileTable, java.lang.String)

setupPrologEpilogue

public boolean setupPrologEpilogue()

setupPrologEpilogue() - setup the prolog and epilogue strings if the files exist. The initial strings are used for multiple conversions. Look in the user directory first and then look in the input data directory. Leave the prolog and epilogue strings in the Globals.

Returns:

true if succeed

See Also:

FileTable#readFileAsTable, UtilCM.replaceSubstrInString(java.lang.String, java.lang.String, java.lang.String)

mapFileSpecificPrologKeywords

public java.lang.String mapFileSpecificPrologKeywords(java.lang.String subTitle,
                                                      java.lang.String prolog)

mapFileSpecificPrologKeywords() - map file specific keywords in prolog.

Parameters:

subTitle - - to use for "$$INFILENAME$$"

prolog - - prolog string to map.

Returns:

mapped string if succeed, if prolog is null then return null

See Also:

UtilCM.replaceSubstrInString(java.lang.String, java.lang.String, java.lang.String)

mapFileSpecificEpilogueKeywords

public java.lang.String mapFileSpecificEpilogueKeywords(java.lang.String subTitle,
                                                        java.lang.String epilogue)

mapFileSpecificEpilogueKeywords() - map file specific keywords in the epilogue.

Parameters:

subTitle - - to use for "$$INFILENAME$$"

epilogue - - epilogue string to map.

Returns:

mapped string if succeed, if epilogue is null then return null

See Also:

UtilCM.replaceSubstrInString(java.lang.String, java.lang.String, java.lang.String)

preprocessTable

private boolean preprocessTable(FileTable ft)

preprocessTable() - pre-process the Table prior to conversion to HTML. This may include 1) dropping or 2) keeping columns, 3) mapping long to short header names, 3) reordering columns and reorder remaining columns alphabeticly, 5) sorting rows by column data, 6) mapping header column names, and 7) concatenating a set of tables.

Parameters:

ft - - file Table to pre-process

Returns:

true if succeed

See Also:

FileTable.deleteTableColumnByName(java.lang.String), FileTable.reorderColumnsInTable(java.lang.String[], int[], int, boolean), Convert#mapLongToShortHdrNames, Convert#mapLongToShortHdrNames, Sort.bubbleSortIndex(java.lang.String[], int, boolean), FileTable.reorderRowsBySortIndex(int[])

makeConcatTable

private boolean makeConcatTable(FileTable ft,
                                int fileNbr,
                                int maxFiles)

makeConcatTable() - create a new concatenated tab-delimited .txt output file from the set of input text files IFF they have exactly the same column header names. The -outputDir switch specifies where the file is saved. Default is not to concatenate the input files. Data is appended into Table ftConcat. At the end of merging all of the files, it writes out the ftConcat Table into the output file. The '-concatTables:{outputConcatenateFile.txt}' switch enables this option by setting the concatInputFilesWithSameFieldsFlag switch. If -makeMapFile:{makeMapTblFileName,orderedCommaColList} is also set, then create and write out the .map file as well.

Parameters:

ft - is FileTable to process

fileNbr - is file number in list of Tables to concatenate

maxFiles - is size of list of Tables to concatenate

Returns:

true if succeed

See Also:

UtilCM.logMsg(java.lang.String), Convert.concatInputFilesWithSameFields(int, FileTable), FileIO.writeStringToFile(java.lang.String, java.lang.String), FileTable.setHasTableHeaderFlag(boolean), FileTable.setDuplicateFieldsFlag(boolean), FileTable.setNbrTableHdrLines(int), FileTable.setHasEmptyLineBeforeTableFlag(boolean), FileTable.lookupFieldIdx(java.lang.String), FileTable.getColumnData(java.lang.String), FileTable.addColumnToTable(java.lang.String, java.lang.String[]), FileTable.cvtTableToTabDelimStr(boolean)

makePrefaceHTML

private java.lang.String makePrefaceHTML(FileTable ft,
                                         java.lang.String oFile,
                                         java.lang.String sName,
                                         java.lang.String sProlog,
                                         java.lang.String sEpilogue)

makePrefaceHTML() - if -makePrefaceHTML, then setup preface file name to link to in the 2nd file HTML we generate

Parameters:

FileTable - ft

ofile - - output file

sName - - name of Table

sProlog - - prolog string

sEpilogue - - epilogue string

Returns:

preface HTML string for current ft

See Also:

FileTable, FileTable.parseTableFromString(java.lang.String), Convert.cvtPrefaceTable2HTML(FileTable, java.lang.String, java.lang.String, java.lang.String, java.lang.String), Convert.cvtText2HTML(java.lang.String, java.lang.String, java.lang.String, java.lang.String)

doEpilogueExtract

private java.lang.String doEpilogueExtract(FileTable ft,
                                           java.lang.String sEpilogue)

doEpilogueExtract() - test if doing row extraction from a resource table to modify the epilogue. First get and lookup a keyword in the table being processed at (colName,rowNbr) and then to search a resourceTblFile for that keyword. If it found, then it will extract the header row and the data row from the resource file and create HTML of htmlStyle. If $$EXTRACT_ROW$$ is in the epilogue, then replace it with the generated HTML else insert the HTML at the front of the epilogue. The htmlStyles may be DL, OL, UL and and TABLE. Default is no row extraction. The args are specified with the '-extractRow:{colName,rowNbr,resourceTblFile,htmlStyle}' switch.

Parameters:

ft - is FileTable

sEpilogue - to modify

Returns:

sEpilogue2 that may or may not have been modified.

See Also:

Convert.extractRowFromResourceTableAsHTML(FileTable, java.lang.String, int, java.lang.String, java.lang.String), UtilCM.replaceSubstrInString(java.lang.String, java.lang.String, java.lang.String)

saveIndexMapFile

private boolean saveIndexMapFile(java.lang.String inputFilePath,
                                 java.lang.String oDir,
                                 java.lang.String oFile,
                                 FileTable ft)

saveIndexMapFile() - save the index map table for the edited Table If doing '-makeIndexMapFile:{colName1,colName2,...,colNameN}' first generate the Index-Map Table using the makeIdxMapTable(). The ft.ftIdxMap FileTable is then written as an output file with the same name as the input file, but with an .idx file extension that is saved in cvt.makeIndexMapFile.

Parameters:

inputFilePath - - path of the Table input file

oDir - - directory to put the edited output Table and index file

oFile - - name of the Table output file

ft - - input file Table

Returns:

true if wrote out the index map file.

closeProgram

public void closeProgram(int exitNbr)

closeProgram() - close out the log file if logging messages and exit with the exit number. This method may be called when a fatal error is found to exit the program gracefully.

Parameters:

exitNbr -

See Also:

FileIO.writeStringToFile(java.lang.String, java.lang.String)

fastEditFile

private boolean fastEditFile(int fileNbr,
                             int maxFiles,
                             java.lang.String iDir,
                             java.lang.String oDir,
                             java.lang.String iFile,
                             java.lang.String oFile,
                             java.lang.String sName)

fastEditFile() - fast process an input file into a tab-delimited file. It does not generate a Table structure or a HTML output file. Instead, it processes input file data line by line and does not buffer the data in a Table structure, but remaps each line on the fly using '-mapLongToShortHdrNames:{L2SmapFile, opt. toCol#}', {-dropColumns or -keepColumns} followed by -reorderColumns. The buildColumnEditMap() builds the header column mapping mapInCols2OutCols[0:tCols-1] from the old to the new column indexes. Data is written immediately to a BufferedWrite stream so it can handle huge files. Because it is sequential, it can't do a -sortRowsByColumnData. This would generally be used to generate a tab-delim .txt files that can be random accessed or used in a RDBMS later. This -fastEditFile switch option is used instead of '-saveEditedTable2File:{outTblFile,opt. "noHTML"}' which overides the -saveEditedTable2File options. The filename is saved in saveEditedTable2File. This option is set by the '-fastEditFile:{opt. output file}' switch. Processing is divided into 4 major phases:

  [1] Read and save the file preface in tPreface. This could be
      saved in a separate file.
  [2] Read the file tHeader and tFields (no tData) into Table structure.
  [3] Optional (switch dependent) pre-processing including:
     -dropColumn list of columns
     -keepColumn list of columns
     -reorderColumn list of columns
     -mapLongToShortHdrNames mapping 
     -reorderRemainingColumns       
     -hdrMapColumnNames       
  [3.1] Build column mapper from preprocessing using buildColumnEditMap()
     that builds the header column mapping mapInCols2OutCols[0:tCols-1].
  [3.1] Map and save the modified header in the output stream.
  [4] Read and map each input data row using the column mapper
      and write the data to output stream.
 

Parameters:

fileNbr - file number in the input files list. Special processing for 1st file

maxFiles - max # files in the input file list.

iDir - input file directory

oDir - output file directory

iFile - input file name with extension (no directory)

oFile - output file name with extension (no directory)

sName - name for the prolog HTML header if not null

Returns:

true if succeed

See Also:

buildColumnEditMap(FileTable, boolean), UtilCM.logMsg(java.lang.String), FileTable

buildColumnEditMap

private int[] buildColumnEditMap(FileTable ft,
                                 boolean reorderRemainingColsFlag)

buildColumnEditMap() - build column editing map for fast table editing. This map is then used when doing Table editing of huge files using line by line processing using a BufferedReader/BufferedWriter combination. The map is used to map tField[0:tcols] data to an output field[] list. Columns that are dropped are set to -1. Columns with -1 indicate the column the tField data are ignored. This method builds a list of destination column indexes on the source column indexes for {-dropColumn or -keepColumn, and -reorderColumn) editing funtions.

Parameters:

ft - is the FileTable to analyze

reorderRemainingColsFlag - to sort the remaining columns not specified, but that are used, alphabetically.

Returns:

mapInCols2OutCols[] map if successful, null otherwise.

postCopyFilesAndDirectories

public boolean postCopyFilesAndDirectories()

postCopyFilesAndDirectories() - copy files post-conversion using the set of -copyFile:{srcFile,destFile} and -copyTree:{srcTreeFiles,destPath} switches specified.

Returns:

true if no problems with the copying.

joinTableToCurrentTable

public boolean joinTableToCurrentTable(FileTable ft,
                                       java.lang.String joinTableFile)

joinTableToCurrentTable() - join another table to ft before preprocessing. Add the contents of a joinTable file to the table being processed. This can allow us to add fields that can be used for sorting. Note: This switch can not be used with the -fastEditFile option.

Parameters:

ft - is current Table

joinTableFile - is the name of the Table file to join to ft

Returns:

true if succeed.

doGenBatchScripts

private boolean doGenBatchScripts()

doGenBatchScripts() - do GenBatchScripts processing to create batchScripts/ directory. If -genBatchScript:{test-ToDo-list.txt,batchDir,inputTreeDir, outputTreeDir}' is set, generate a set of parameter .map files in the batchDir directory.

Returns:

true if succeed

createTestsIntersectionTable

private boolean createTestsIntersectionTable()

createTestsIntersectionTable() - create Tests Intersection Table and do related processing and .txt and .html file generation.

Returns:

true if succeed

createFlipTable

private boolean createFlipTable()

createFlipTable() - generate transposed flipped Table files (.txt and .html). If '-flipTableByIndexMap:{flipDataFile,flipIndexMapFile}', then generate a transposed Table file using random access indexing to create a multi-line header (1 line for each column name in the list) using the list of columns previously specified when generating the index map file. Write out the file if succeed.

Returns:

true if succeed

makeIndexMapFile

private FileTable makeIndexMapFile(java.lang.String iFileNamePath,
                                   java.lang.String oDir,
                                   java.lang.String oIdxFileName)

makeIndexMapFile() - read file header and make Index-Map in Table structure. Note: The index-map is of the input file - not the edited input file since it builds the map as it reads the input Table file. It reads the Table header and then also reads the rest of the input Table file to get the indexMap. The map is saved in ft.ftIdxMap. This also sets ft.firstDataRowLineNbr that we need to set the BufferReader to the start of the data below.

Parameters:

ft - - Table instance to save result

iFileNamePath - - path of input Table file

oDir - - output Table directory

oIdxFileName - - name of index-map output file.

Returns:

the FileTable if successful, else null

addNameOfHTMLfile

public void addNameOfHTMLfile(java.lang.String nameOfHTMLpath)

addNameOfHTMLfile() - append name of HTML file path last generated to the this.genHTMLpathList[] list.

Parameters:

nameOfHTMLpath - is new HTML file path.

See Also:

clearGenHTMLpathList(boolean), getGenHTMLpathList()

clearGenHTMLpathList

public void clearGenHTMLpathList(boolean forceFlag)

clearGenHTMLpathList() - clear Generated HTML path List if not using the GUI and not doing batch processing. cvt.genHTMLpathList[].

Parameters:

forceFlag - to force it be be cleared

See Also:

addNameOfHTMLfile(java.lang.String), getGenHTMLpathList()

getGenHTMLpathList

public java.lang.String[] getGenHTMLpathList()

getGenHTMLpathList() - get Generated HTML path List cvt.genHTMLpathList[].

See Also:

addNameOfHTMLfile(java.lang.String), clearGenHTMLpathList(boolean)

incrUpdateStatusLine

public void incrUpdateStatusLine(java.lang.String incrStr)

incrUpdateStatusLine() - if GUI, then do incremental statusLine update. The base string for UtilCM.updateReportStatusLine() used to build incremental strings if running under a GUI.

Parameters:

incrStr -