Text file processing.

[HunterEC]

What class / command / function can I use to process (READ / WRITE ) Text files ? Are there any samples ? Thank you.

[Otto]

I use:
memowrit( "info.txt",cText)
Memoread(cAppPath + "\ini\update.txt")


Best regards,
Otto

[ukoenig]

The way to write a Textfile ( PRG ) with formatted Textlines , read with a Editor, compile and run the EXE-file :
A Example, to write the Source for a Window with included Buttons.
I use it in all of my Tools, to write the Source and create a EXE-File.

Code: Select all  Expand view

FUNCTION MAKE_EXE()

cNewFile := c_path + "\PROJECT.prg"
cNewFile1 := c_path + "\PROJECT.EXE"

DELETE FILE  "&cNewFile"
DELETE FILE   "&cNewFile1"

NEW_SOURCE() // Creates the Textfile ( PRG )

IF FILE ( c_path + "\PROJECT.PRG" )
    // File exists open with Editor
    WAITRUN( "Wordpad.exe " + c_path + "\PROJECT.prg" ) // copy from .PRG
    // Close Editor and create EXE-File
    IF MsgYesNo( "Do you want to Create / Run : PROJECT.exe ?","PROJECT")   
        WAITRUN( "GO.BAT"  ) // the Link-File, to create the EXE
        IF FILE ( c_path + "\PROJECT.EXE" )
            WINEXEC( "PROJECT.EXE" )
        ELSE
            MsgAlert( "The file : PROJECT.EXE " + CRLF + ;
                  "is not created !","Error" )
        ENDIF
    ENDIF
ELSE
    MsgAlert( "The File : " + CRLF + ;
            "PROJECT.PRG" + CRLF + ;
            "is not created !", "Error" )
ENDIF
SYSREFRESH()

RETURN NIL

// ------- Create the Text-File ( PRG ) --------------------------------

// A Example, how to include Var's :
// oText:Add('Gradient( hDC, { 0, 0, oDlg:nHeight, oDlg:nWidth }, ' + ALLTRIM( STR(W_COLOR1)) + ", " + ALLTRIM( STR(W_COLOR2)) + ', .F. )')
// Including a Textstring You must use ' and " it looks like : oText:Add('Adding Text-String : ' + "Test" + ' LineEnd' )

FUNCTION NEW_SOURCE()

cNewFile := c_path + "\PROJECT.PRG"
IF FILE ("&cNewFile")
    DELETE FILE  "&cNewFile"
ENDIF

oText := TTxtFile():New( "&cNewFile" )

IF oText:Open()
    oText:Add('#include "FiveWin.ch"')
    oText:Add('#include "TTitle.ch"')
    oText:Add('#include "Image.ch"')
    oText:Add("")
    oText:Add("static oWnd, oButtFont" )
    oText:Add("")
    oText:Add("FUNCTION Main()")
    oText:Add('LOCAL oBtn1, oBtn2, hDC, oBrush, oImage')
    oText:Add("LOCAL nWidth := GetSysMetrics(0)")
    oText:Add("LOCAL nHeight := GetSysMetrics(1)")
    oText:Add("")
    oText:Add("SET _3DLOOK ON" )
    oText:Add("SetBalloon( .T. )" )
    oText:Add('c_path := CURDRIVE() + ":\" + GETCURDIR()')
    oText:Add('oProgFont := TFont():New("Arial", ,-14,.F.,.F. , , , ,.F. ) ')
    oText:Add("")
    oText:Add('DEFINE WINDOW  oWnd  TITLE "VTitle Test" ')
    oText:Add("")
    oText:Add('oButtFont := TFont():New("Arial",0,-16,.F.,.T.,0,0,0,.T. ) ')
    oText:Add('DEFINE IMAGE oImage FILENAME c_path + "\Images\BACKGRD.JPG" ')
    oText:Add("")
    oText:Add("@ nHeight - 160, nWidth - 420 BTNBMP oBtn1 SIZE 200, 60 OF oWnd  2007 ;" )
    oText:Add('FILENAME c_path + "\Images\select.bmp" ;')
    oText:Add("LEFT ;" )
    oText:Add('PROMPT "  &Preview VTitle " ; ')
    oText:Add("FONT oButtFont ;" )
    oText:Add("ACTION Tools(oWnd)" )  
    oText:Add("oBtn1:lTransparent = .t. " )  
    oText:Add('oBtn1:cTooltip :=  { "Open" + CRLF + ;')
    oText:Add('       "the VTitlt-Painter","VTitle-Painter", 1, CLR_BLACK, 14089979 }' )
    oText:Add("")
    oText:Add("@ nHeight - 160, nWidth - 200 BTNBMP oBtn2 SIZE 150, 60 OF oWnd  2007 ;" )
    oText:Add('FILENAME c_path + "\Images\exit.bmp" ;' )
    oText:Add("LEFT ;" )
    oText:Add('PROMPT "  &Exit  " ;' )
    oText:Add("FONT oButtFont ;" )
    oText:Add("ACTION oWnd:End()" )  
    oText:Add("oBtn2:lTransparent = .t." )  
    oText:Add('oBtn2:cTooltip :=  { "Close" + CRLF + ;' )
    oText:Add('        "the VTitle-Painter","Close Window", 1, CLR_BLACK, 14089979 }' )
    oText:Add("")
    oText:Add('ACTIVATE WINDOW oWnd MAXIMIZED ;')
    oText:Add('ON PAINT ( DRAWBITMAP( hdc, oImage:hbitmap, 0, 0, nWidth, nHeight ) ) ;')
    oText:Add('VALID MsgYesNo( "Do you want to end?" )')
    oText:Add("")
    oText:Add('RETURN( NIL )')

    oText:Close()
ENDIF

RETURN(  NIL )
 

Best Regards
Uwe

[HunterEC]

Otto, Ukoenig:

Thank you for your responses. What I'm trying to do is process a huge (3.9GB) CSV file, read it line by line and split it in n lines pieces so I can import it to dbf files. I'm doing it because I have not found a single software that is able to import such a file. I think that the dBaseIII format has a 2GB limitation. So I'm importing it in various pieces. Basically I'm reading the line until I find CHR(13) + CHR(10) and then continue with the next. Thank you.

[Otto]

Best regards,
Otto

Please have a look at this threads:
Maybe you can use strtoken.

http://forums.fivetechsupport.com/viewtopic.php?f=3&t=12318&p=61651&hilit=strtoken#p61651

Antonio shows how to read large files:

http://forums.fivetechsupport.com/viewtopic.php?f=3&t=968&p=25897&hilit=search+otto#p25897

Some years ago we helped a company to test several third party tools for such purpouse and finally implemented our own solution to make the fastest search on all fields of a DBF.

We found with great surprise that the solution was to open the DBF as a standard file with FOpen(), read a bunch of bytes in memory and perform a simple At() to locate a string. Once found, you substract the DBF header length, then divide the offset by the record size and you get the record number. At() is an extremelly fast function as it is directly performed by the processor.

These days that we use 32 bits flat memory, I guess there is no need to use a bunch of bytes, so the entire DBF may be loaded in memory doing a MemoRead() of the DBF file, or several bunchs if it is too large, so the code may get simpler.

We compared this way with other available third party tools, and we found that ours was the fastest one

[Kleyber]

Hi Hunter,

Why don't you use the TExcel class or TOleauto class to import this CSV file into various dbf files? Just a suggestion.

Regards,

[Antonio Linares]

Gustavo,

If you need to read line by line, we have found that this is a very fast way to do it, much faster than using MemoLine():

Code: Select all  Expand view

function ExtractLine( cText, nFrom )

  local cLine, nAt

  nAt := At( CRLF, SubStr( cText, nFrom ) )

  if nAt > 0
    cLine := Substr( cText, nFrom, nAt - 1 )
    nFrom += nAt + 1
  else
    cLine := Substr( cText, nFrom )
    nFrom := Len( cText ) + 1
  endif

return cLine
 

nFrom must be supplied by reference and initialized with 1:

Code: Select all  Expand view

   local nFrom := 1
   local nLen  := Len( cText )

    While nFrom <= nLen
       cLine  := ExtractLine( cText, @nFrom )
       ...
    end