Help with HB_REALLOC ERROR 9009

[Adolfo]

Hello Fivewinners.
Hola Fivewinners

Need to open 160.000+ xml files and extract some info
Necesito abrir mas de 160.000 archivos xml y sacar informacion

I use tXmlDocument class
Uso la clase TxmlDocument

I have the file names in an Array
Tengo los nombres de los archivos en un arreglo

Code: Select all  Expand view

    For X=1 to 160.xxx

        If z=400
           HB_GcAll()
           SysRefresh()
           z:=0
        Endif

        LeeFileXml(hFile[x])
 
    Next
--------------------------------------
--------------------------------------
Function LeeFile Xml(hFile)
Local oXmlDoc := TXmlDocument():New( hFile )
Local oXmlIter := TXmlIterator():New( oXmlDoc:oRoot )
Local Data1:=0
Local Data2:=""

   while .T.
         oTagActual = oXmlIter:Next()
         If oTagActual != nil

            If oTagActual:cName ="END"
              Exit
            Endif

           If oTagActual:cName ="ValorXxx"
              Data1:=Val(oTagActual:cData)
           Endif

           If oTagActual:cName ="ValorYyy"
              Data2:=oTagActual:cData
           Endif

      End

FClose( hFile )
oXmlDoc:=Nil
oXmlIter:=Nil

 oSvr:Insert("detalle",{"NUMERO","DESCRIPCION"},{Data1,Data2})

Return Nil


 

It does what it is supposed to do, But I get this error each time Between 5.000 and 6.000 processed files
Hace lo que se supone debe hacer, pero tengo cada vez tengo este error entre los 5.000 y 6.000 archivos procesados

Application Internal Error - C:\SQL\xNewAqua\xNewII.Exe
Terminated at: 2024-03-20 16:18:46
Error irrecuperable 9009: hb_xrealloc no puede reubicar la memoria......

It fails in no specific file, just after 5000+ file
No falla en ningun archivo especifico, solo despues de los 5000

Any help will be appreciated
Cualquier ayuda se agradece

HARBOUR, FWH, MSVC, WINDOWS 11, 32GB ram, NVME disk

[Marc Venken]

Run out of memory ?

Maybe you can put a part of this code inside the loop en show the result while running. Than you see if there is a memory leak.

Code: Select all  Expand view

cErrorLog += "   Hardware memory: " + ;
                    cValToChar( Int( nExtMem() / ( 1024 * 1024 ) ) + 1 ) + " MB ( available: "+;
                    cValToChar (Int (nAvailMem() / (1024 * 1024 ) ) +1 ) + " MB )" + CRLF + CRLF
 

[Adolfo]

Hi Marc

To Avoid Memory leaks, within the loop I call an external function each time I read a xml file, close it and turn to nil each class instance.
For every 400 files I do a HB_GcAll() and a SysRefresh(), the biggest xml file is just 29 kb

Im going to put your code inside the loop, and show it every 1.000 files, hope it helps to know

Thanks

[Marc Venken]

nAvailMem() seems not to be standard.

These seems to be ok.

cErrorLog += " Dynamic memory consume:" + CRLF
cErrorLog += " Actual Value: " + Str( MemUsed() ) + " bytes" + CRLF
cErrorLog += " Highest Value: " + Str( MemMax() ) + " bytes" + CRLF

[Adolfo]

I found the error

Disable Avast Antivirus, and it works with no problem.

[karinha]

Code: Select all  Expand view

FUNCTION TU_Func()

   // inicio
   hb_gcAll( .F. )

   FOR X = 1 TO 160.xxx

      SYSREFRESH()

      IF z = 400

         // HB_GcAll()

         // SysRefresh()

         z := 0

      ENDIF

      LeeFileXml( hFile[ x ] )

   NEXT

   // Salida
   hb_gcAll( .T. )

RETURN NIL

FUNCTION LeeFile Xml( hFile )

   LOCAL oXmlDoc := TXmlDocument():New( hFile )
   LOCAL oXmlIter := TXmlIterator():New( oXmlDoc:oRoot )
   LOCAL Data1 := 0
   LOCAL Data2 := ""

   hb_gcAll( .F. )

   WHILE( .T. )

      SYSREFRESH() // aqui

      oTagActual = oXmlIter:Next()

      IF oTagActual != nil

         IF oTagActual:cName = "END"

            EXIT

         ENDIF

         IF oTagActual:cName = "ValorXxx"

            Data1 := Val( oTagActual:cData )

         ENDIF

         IF oTagActual:cName = "ValorYyy"

            Data2 := oTagActual:cData

         ENDIF

      ENDIF

   ENDDO // ??

   FClose( hFile )

   oXmlDoc  := Nil
   oXmlIter := Nil

   oSvr:Insert( "detalle", { "NUMERO", "DESCRIPCION" }, { Data1, Data2 } )

   hb_gcAll( .T. )

RETURN NIL
 

Regards, saludos.