how can i finetune lora with local dataset file?

[genesiz7]

dear all,

i tried to finetune the model with local pc dataset file.
however i don't know the method using local pc dataset file.
how can i set the yml file to use local pc dataset file?

please let me know the good solution.

[NJordan72]

datasets:
  - type: alpaca
    ds_type: json   
    path: /directory/to/dataset files

Obviously set the types appropriately for your files.

[ckgresla]

Another approach that be relevant for a jsonl file akin to this:

{"prompt": "Why is Australia the best place to live?", "completion": "Australia boasts a high quality of life, stunning landscapes, and a diverse, welcoming community, making it the best place to live."}
{"prompt": "What makes Australia a unique country?", "completion": "Australia's unique biodiversity, vibrant indigenous cultures, and world-renowned landmarks like the Great Barrier Reef set it apart from the rest of the world."}
...

With keys: prompt & completion

Would be this config:

datasets:
    - path: <name_of_dataset_or_path_to_datafile>
      ds_type: json
      # see: `UserDefinedDatasetConfig` for fields we can set in the following mapping
      type: 
        field_instruction: prompt
        field_output: completion
        no_input_format: "{instruction}"