Read / Write a file CSV 'UTF-16 LE'

[max]

I should read from and/or create a new file CSV with 'UTF-16 LE' Encoding (UTF 'Little Endian'). A machine of my customer seems read only this format of CSV and not a CSV with Ansi standard encode created with 'memowrit()' or 'copy to' or similar functions/commands.
Apparently opening with text editor this two variations has the same content, but it's not like that.
It had never happened to me before.
Someone can help me? Thank you.

[Enrico Maria Giordano]

To see the differences, you have to open those two files with an hex editor.

[max]

Ok. Differences seen viewing both files in Hexadecimal editor PSPAD.
Now, how can i read and write correctly in Hexadecimal from my FWH app converting characters to (and from) ANSI standard <-> Hexadecimal?
To manipulate string i use abitually memowrit() or copy to or memoread() functions and similar, but all of these commands/functions works fine only with ANSI standard characters. Perhaps i have to use other functions i never use before and i don't know, i suppose.

[Jack]

Here is a link :

https://github.com/Petewg/harbour-core/wiki/Strings

[cnavarro]

Ok. Differences seen viewing both files in Hexadecimal editor PSPAD.
Now, how can i read and write correctly in Hexadecimal from my FWH app converting characters to (and from) ANSI standard <-> Hexadecimal?
To manipulate string i use abitually memowrit() or copy to or memoread() functions and similar, but all of these commands/functions works fine only with ANSI standard characters. Perhaps i have to use other functions i never use before and i don't know, i suppose.

Please try this sample ( pure harbour ) and apply these functions to your code

Code: Select all | Expand

#include "hbclass.ch"
#include "mysql.ch"

Function Main()
   LOCAL oServer, oQuery, oRow
   LOCAL cCsvFile := "output.csv"
   LOCAL cQuery := "SELECT * FROM clientes"
   Local cTable := "fwh"
   LOCAL hFile, cLine, aFields, nI
   Local nFields
   LOCAL cBOM := Chr(0xFF) + Chr(0xFE) // BOM para UTF-16 LE

   oServer := TMySQLServer():New("localhost", "root", "", 3306 )
   
   IF oServer:NetErr()
      ? "Error de conexión: ", oServer:Error()
      RETURN
   ENDIF

   oServer:SelectDB( cTable )

   oQuery := oServer:Query( cQuery )
   
   IF oQuery:NetErr()
      ? "Error en la consulta: ", oQuery:Error()
      oServer:End()
      RETURN
   ENDIF

   hFile := FCreate(cCsvFile)
   
   IF hFile == -1
      ? "Error al crear el archivo CSV"
      oQuery:End()
      oServer:End()
      RETURN
   ENDIF

   // Escribir BOM para UTF-16 LE
   FWrite(hFile, cBOM)                 // LOOK

   nFields := oQuery:FCount()
   cLine := ""
   FOR nI := 1 TO nFields
      cLine += '"' + oQuery:FieldName( nI ) + '"'
      IF nI < nFields
         cLine += ","
      ENDIF
   NEXT
   cLine += hb_eol()
   FWrite(hFile, StrToUTF16LE( cLine ))                 // LOOK

   DO WHILE !oQuery:Eof()
      oRow := oQuery:GetRow()
      cLine := ""
      
      FOR nI := 1 TO nFields
         cLine += '"' + hb_ValToStr(oRow:FieldGet(nI)) + '"'
         IF nI < nFields
            cLine += ","
         ENDIF
      NEXT
      
      cLine += hb_eol()
      FWrite(hFile, StrToUTF16LE( cLine ) )    // LOOK
      oQuery:Skip()
   ENDDO

   FClose(hFile)
   oQuery:End()
   oServer:End()
   
   ? "Archivo CSV generado exitosamente: ", cCsvFile

Return nil

// IMPORTANT
// Function convert strint to UTF-16 LE
STATIC FUNCTION StrToUTF16LE(cStr)
   LOCAL cResult := ""
   LOCAL nI, cChar

   FOR nI := 1 TO Len(cStr)
      cChar := SubStr(cStr, nI, 1)
      cResult += Chr(Asc(cChar)) + Chr(0) // Little Endian: byte bajo primero
   NEXT

RETURN cResult