Steps to integrate AI

Post by **leandro** » Fri Jan 19, 2024 8:14 pm

Hello, good afternoon, how are you?

We want to start using this technology and would like to know what the best option is and what steps to follow. There is a lot of talk about ChatGPT in forums, and Antonio also posted something about Copilot. It's still a very new topic for us, and we want to know how we can start with the implementation without getting overwhelmed.

From what we understand, to use it in production, we need to contract with OpenAI. However, what we don't know is how to integrate this technology into our application, teach it certain things we need it to resolve, and ultimately implement a transparent process for the user where we make a request and receive the most accurate response, similar to what a regular function does with its return. We understand that the information we want to receive is more complex.

Does anyone have more experience with this topic who can guide us?

Thank you in advance.

Post by **Antonio Linares** » Sat Jan 20, 2024 9:12 am

Dear Leandro,

The first step is to create a dataset with questions and answers to train (fine tune) a base model such as Microsoft phi-2, TinyLlama, etc.

Once trained, a GGUF file is generated, which can be used with FWH using the llama64.dll.

This is the free and private way. The commercial way involves paying OpenAI and the inconvenience of providing our private information to an external company.

Post by **Jimmy** » Sat Jan 20, 2024 9:41 am

hi,

the "Problem" is to train (fine tune) a base model for own Data as it need much PC-Power

Question : is it possible to "rent" PC-Power to train own Model

Post by **Antonio Linares** » Sat Jan 20, 2024 4:14 pm

Jimmy wrote:hi,

the "Problem" is to train (fine tune) a base model for own Data as it need much PC-Power

Question : is it possible to "rent" PC-Power to train own Model

Dear Jimmy,

You can use Google Colab with T4. You have a certain amount of hours for free, and then you can pay very little (12 euros) for more.
Another option is kaggle (it belongs to Google too) where you get 30 hours for free.

Actually we are using a local GPU as we need to do many tests until we have a right workflow, and it is quite tiring to load the same python packages on those services again and again.
The idea is: do your research locally and when everything is ready do the final train on a cloud GPU if needed

Ruth · Post by **Ruth** » Sat Jan 20, 2024 4:44 pm

Dear team,

...same here ... I also wanted to use chatgpt api and already had a paid openai account. so i just needed the openai api key - which one should be careful to never reveal publicly.

i made a basic webpage where the user can input a prompt, send that prompt to the server that communicates with the openai api and display the response on the web page.

this is the html

Code: Select all | Expand

<!DOCTYPE html>
<html>
<head>
  <title>OpenAI API</title>
  <script>
    async function callAPI() {
      const promptText = document.getElementById("prompt").value;

      try {
        const response = await fetch("https://mybergland.com/chatgpt_api/serverside.prg", {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify({ prompt: promptText }),
        });
        
        // Check HTTP status code and log it
        console.log('HTTP Status Code:', response.status);
        
        // Read response as text first
        const textData = await response.text();
        
        // Log raw received data
        console.log('Received data:', textData);

        // Try to parse the text as JSON
        let jsonData;
        try {
          jsonData = JSON.parse(textData);
        } catch (error) {
          console.error('Invalid JSON:', textData);
          throw error;
        }
        
        // Do something with jsonData
        document.getElementById("result").innerText = jsonData.choices[0].message.content;
        
      } catch (error) {
        console.error('Fetch Error:', error);
      }
    }
  </script>
</head>
<body>

  <h1>OpenAI API </h1>
  
  <label for="prompt">Prompt:</label>
  <input type="text" id="prompt" />
  
  <button onclick="callAPI()">Call API</button>

  <pre id="result"></pre>

</body>
</html>

and this is serverside

Code: Select all | Expand

#include "c:\harbour\contrib\hbcurl\hbcurl.ch"
#define HB_CURLOPT_POST                      47
#define HB_CURLOPT_URL                        2
#define HB_CURLOPT_DL_BUFF_SETUP              1008
#define HB_CURLOPT_POSTFIELDS                 15
#define HB_CURLOPT_WRITEFUNCTION              11

FUNCTION Main()
  LOCAL hPairs := AP_Body()
  LOCAL hPost := {}
  LOCAL cAPIKey := "" // Getting API Key from environment variable
  LOCAL cUrl := "https://api.openai.com/v1/chat/completions"
  LOCAL cPrompt := ""
  LOCAL cResponse := ""
  
  LOCAL hCurl, cPayload
  LOCAL nResult := hb_jsonDecode( AP_Body(), @hPost )
  LOCAL aHeaders := ""
  LOCAL hPayload := HB_Hash()
  LOCAL aMessages := {}

  aHeaders := { "Content-Type: application/json", ;
                 "Authorization: Bearer " + " }
 

  IF nResult != 0  // Checks if the operation was successful
    IF HB_HHasKey( hPost, "prompt" )
      cPrompt := hPost[ "prompt" ]
    ENDIF
    
  ELSE
   
    cResponse := '{ "Error": "JSON decoding failed." }'
   
    AP_SetContentType( "application/json" )
    ?? cResponse  // Output the JSON error message and return
    RETURN NIL
  ENDIF

 aAdd( aMessages, { "role" => "system", "content" => "You are a helpful assistant." } )
 aAdd( aMessages, { "role" => "user", "content" => cPrompt } )

hb_HSet( hPayload, "model", "gpt-3.5-turbo" )
hb_HSet( hPayload, "messages", aMessages )
hb_HSet( hPayload, "max_tokens", 50 )

cPayload := hb_jsonEncode( hPayload )


  
  
  
  hCurl := CURL_easy_init()
  
  IF hCurl != NIL

  
    // Prepare headers
 
    // Set cURL options
    CURL_easy_setopt( hCurl, HB_CURLOPT_URL, cUrl )
    CURL_easy_setopt( hCurl, HB_CURLOPT_POST, .T. )
    CURL_easy_setopt( hCurl, HB_CURLOPT_POSTFIELDS, cPayload )

    CURL_easy_setopt( hCurl, HB_CURLOPT_HTTPHEADER, aHeaders )
    CURL_easy_setopt( hCurl, HB_CURLOPT_WRITEFUNCTION, @cResponse )
    curl_easy_setopt( hCurl, HB_CURLOPT_DL_BUFF_SETUP )
    curl_easy_setopt( hCurl, HB_CURLOPT_SSL_VERIFYPEER, .F. )


    // Perform cURL request
    CURL_easy_perform( hCurl )

    cResponse = curl_easy_dl_buff_get(hCurl)
    

    // Cleanup
    
    CURL_easy_cleanup( hCurl )

    // Output response
    AP_SetContentType( "application/json" )
    AP_RPuts(cResponse)  // Send the response back


  ELSE
    cResponse := '{ "error": "Could not initialize cURL" }'
    AP_SetContentType( "application/json" )
    ?? cResponse  // Output the JSON error message
  ENDIF
  
RETURN NIL

for me this project and information were very inspiring and helpful...
https://forums.fivetechsupport.com/view ... 1e#p259779

kind regards and a nice weekend
ruth

FiveTech Software tech support forums