Search found 115 matches: dataset

Return to advanced search

datasets generator

Simple code to use an AI model to build a dataset: generate.py import jsonimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer# Load the pre-trained modelmodel_name = "mlabonne/phixtral-4x2_8"tokenizer = AutoTokenizer.from_pretrained(model_name)model ...
by Antonio Linares
Thu Feb 08, 2024 11:01 am
 
Forum: Artificial Intelligence examples
Topic: datasets generator
Replies: 0
Views: 4451

tinyAstrology another AI training model example

https://github.com/FiveTechSoft/tinyAstrology In this example we have created the dataset using chatgpt: 1. You ask chatgpt "tell me small questions and answers related to this topic and write the answer in json format" 2. You tell chatgpt to do it ...
by Antonio Linares
Fri Feb 02, 2024 10:57 am
 
Forum: Artificial Intelligence examples
Topic: tinyAstrology another AI training model example
Replies: 0
Views: 146

Re: more GGUF Model

... where you have the code to fine tune TinyLlama with your own data, in this case we have used a medical dataset. We encourage you to start building your own GGUFs. Full source code is available at the repo. TinyLLama is great for starting on AI training :-)
by Antonio Linares
Mon Jan 29, 2024 6:14 am
 
Forum: Artificial Intelligence
Topic: more GGUF Model
Replies: 1
Views: 168

Steps to integrate AI

Dear Leandro, The first step is to create a dataset with questions and answers to train (fine tune) a base model such as Microsoft phi-2, TinyLlama, etc. Once trained, a GGUF file is generated, which can be used with FWH using the llama64.dll. ...
by Antonio Linares
Sat Jan 20, 2024 9:12 am
 
Forum: FiveWin for Harbour/xHarbour
Topic: Steps to integrate AI
Replies: 4
Views: 635

Re: Pasos para integrar la IA

Estimado Leandro, Lo primero es crear un conjunto de datos (dataset) con preguntas y respuestas, con las que entrenar (fine tune) un modelo base como Microsoft phi-2, TinyLlama, etc Una vez entrenado, se genera un fichero GGUF que puede ya ser usado desde ...
by Antonio Linares
Sat Jan 20, 2024 9:09 am
 
Forum: FiveWin para Harbour/xHarbour
Topic: Pasos para integrar la IA
Replies: 1
Views: 236

Re: Fine tune Phi2 from Microsoft with your own data

... layer to the base model Phi-2, based on: https://medium.com/@nimritakoul01/finetuning-microsoft-phi-2-small-language-model-on-veggo-dataset-using-qlora-8bcf70ab625e !pip install accelerate==0.25.0!pip install bitsandbytes==0.41.1!pip install datasets==2.14.6!pip install peft==0.6.2!pip ...
by Antonio Linares
Wed Jan 17, 2024 10:48 am
 
Forum: FiveWin for Harbour/xHarbour
Topic: Fine tune Phi2 from Microsoft with your own data
Replies: 11
Views: 1251

Re: Fine tune Phi2 from Microsoft with your own data

https://medium.com/@nimritakoul01/finetuning-microsoft-phi-2-small-language-model-on-veggo-dataset-using-qlora-8bcf70ab625e Tested on Google Colab T4 go.bat pip install accelerate==0.25.0pip install bitsandbytes==0.41.1pip install datasets==2.14.6pip install peft==0.6.2pip ...
by Antonio Linares
Mon Jan 15, 2024 7:55 am
 
Forum: FiveWin for Harbour/xHarbour
Topic: Fine tune Phi2 from Microsoft with your own data
Replies: 11
Views: 1251

Re: phpBB to LLM

From posts.dbf and posts.fpt now we generate the dataset.json to be trained. We are using just 20 different topics, so the dataset is not too large and we can do quicker tests with it: dataset.prg #include "FiveWin.ch"request dbfcdxfunction ...
by Antonio Linares
Tue Jan 02, 2024 5:40 am
 
Forum: FiveWin for Harbour/xHarbour
Topic: phpBB to LLM
Replies: 33
Views: 4669

Re: phpBB to LLM

First try building the dataset. hb_jsonEncode() GPFs... #include "FiveWin.ch"request dbfcdxfunction Main()    local aPosts := {}    USE posts VIA "dbfcdx"    INDEX ON ...
by Antonio Linares
Sat Dec 23, 2023 9:19 pm
 
Forum: FiveWin for Harbour/xHarbour
Topic: phpBB to LLM
Replies: 33
Views: 4669

phpBB to LLM

... = phpbb_forums.forum_id;    ENDTEXT    oCn:SaveToDbf( cSQL, "posts.dbf" )return nil     5. Now we create a small dataset in json to make tests: #include "FiveWin.ch"request dbfcdxfunction Main()    local aPosts := {}, n    USE posts ...
by Antonio Linares
Thu Dec 21, 2023 9:57 am
 
Forum: FiveWin for Harbour/xHarbour
Topic: phpBB to LLM
Replies: 33
Views: 4669

Re: FIVEWIN power - a room planer video

... and when I make changes via the modal dialog. This means that the first check is now already done in the JavaScript part. When I then save the dataset, it is checked again in the Harbour-Script. https://mybergland.com/fwforum/over.gif
by Otto
Sun Nov 19, 2023 12:18 am
 
Forum: FiveWin for Harbour/xHarbour
Topic: FIVEWIN power - a room planer video
Replies: 50
Views: 11067

Multiple Sql Indexes - Rao

I have a single Sql table oRs and I want to have 2 lookup options or Sql indexes on the same recordset .. This dataset is for a local town park where you have information on Owners ( name ) and pets by name . I want to be able to have a single recordset where I can use a radio ...
by Rick Lipkin
Tue May 30, 2023 6:39 pm
 
Forum: FiveWin for Harbour/xHarbour
Topic: Multiple Sql Indexes - Rao
Replies: 2
Views: 209

Whisper - general-purpose speech recognition model

... Whisper demo Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multi-task model that can perform multilingual speech recognition as well as speech translation and language ...
by Antonio Linares
Wed Dec 14, 2022 11:52 am
 
Forum: latest AI news
Topic: Whisper - general-purpose speech recognition model
Replies: 0
Views: 300

Re: ayuda con fastreport y page con texto de tamaño variable

... párrafos con la funcion hb_atokens obtengo un arreglo unidimensional, luego lo paso a un arrego de dos dimensiones, luego en fastreport defino un dataset con el dichoso arreglo, y listo. se define ciertas propiedades en el campo que muestra el elemento tan alto como sea necesario. tantas paginas ...
by carlos vargas
Thu Mar 03, 2022 9:20 pm
 
Forum: FiveWin para Harbour/xHarbour
Topic: ayuda con fastreport y page con texto de tamaño variable
Replies: 6
Views: 339

Re: DBU

... 'Open Table InSpector' and is a database table inspection tool for Dbase, Clipper, Harbour, xHarbour, FoxPro, ..., files. Otis has a integrated DATASET manager. This need some explanations. A DATASET is a set of ONE or MORE tables and index files. You can save this SET of files with a name and ...
by hmpaquito
Thu Jun 24, 2021 10:33 am
 
Forum: FiveWin for Harbour/xHarbour
Topic: DBU
Replies: 16
Views: 2316
Next

Return to advanced search