Configuration

In this section, we'll take a closer look at how you can quickly and easily set up a configuration file to start implementing any of your tasks. You can use absolutely any format of configuration file:

• JSON
• YAML
• TOML
• HCL (Terraform)

The main condition is that your config file must contain the sections necessary for csv2api to work.

A configured config.yml file usually looks like this:

save_filtered_pk_to_csv: true

columns_order:
  - id
  - order_id
  - is_paid

csv_separator: ','

api:

  base_url: my-api-server.com
  base_url_schema: https
  auth_method: Bearer
  token: '{{ CONFIG_API_TOKEN }}'
  request_timeout: 0

  update_endpoint:
    endpoint_name: /api/v1/order/%s
    content_type: application/json
    add_pk_to_endpoint_name: true
    http_method: PATCH
    endpoint_body:
      data:
        order: id
        attributes:
          tags: tags

filter_columns:

  - column_name: order_id
    condition: NEQ
    value: ''

update_fields:

  - field_name: tags
    values:
      - paid
    conditions:
      - column_name: is_paid
        condition: EQ
        value: '1'

Let's look at each section in more detail.

save_filtered_pk_to_csv

This section accepts a boolean (true or false) value. Allows you to specify whether to save the filtered primary keys (PK) to a CSV file.

This is useful when you need to keep track of which PK (IDs) have been filtered, to control how they change in the database or elsewhere.

columns_order

This section accepts a list of strings. Allows you to specify the order of the fields for filtering and updating.

❗️ Please note: the first element in this list is the primary key (PK), which will be used in other configuration sections.

csv_separator

This section accepts a string. Allows you to specify a separator between columns in a CSV data file.

❗️ Please note: the setting must match the parameters of the Go standard library: https://pkg.go.dev/encoding/csv#Writer

api

This section accepts many parameters and objects. Allows you to specify settings for your REST API instance.

base_url

This section accepts a string. Allows you to specify a base URL address (host) for your REST API.

base_url_schema

This section accepts the one of the values: http or https. Allows you to specify an HTTP schema for the base URL address (host) for your REST API.

auth_method

This section accepts a string. Allows you to specify an auth method for making requests to your REST API.

For example, Bearer.

token

This section accepts a string. Allows you to specify an auth token for making requests to your REST API.

WARNING: do not forget to add your config file to a .gitignore, if you place API token to this section!

It is recommended to keep such data under protection, for example by using environment variables. For example, you can specify a token like this when you start the parser:

MYVAR_API_TOKEN=secret-1234567 csv2api -c /path/to/config.yml -d /path/to/data.csv -e MYVAR

During initialization, the parser will insert your token from env into this configuration section by itself.

❗️ Please note: to be built into the configuration correctly, you must specify environment variables in object style, but use underscores (_) instead of dots (.).

request_timeout

This section accepts an int. Allows you to specify a request timeout for HTTP client.

update_endpoint

This section accepts object. Allows you to specify endpoint to update actions from your REST API.

endpoint_name

This section accepts a string. Allows you to specify an endpoint name (path).

For example, if you need to set a primary key (PK) to endpoint path, this section support Go markers (like you use in fmt.Sprintf function):

endpoint_name: /api/v1/order/%s # <-- '%s' is a stub where PK will be inserted

content_type

This section accepts a string. Allows you to specify an endpoint's Content-Type.

add_pk_to_endpoint_name

This section accepts a boolean (true or false) value. Allows you to set a primary key (PK) to the endpoint name (path).

❗️ Please note: if you set this setting to true, then you must necessarily specify a stub in the endpoint_name option.

http_method

This section accepts a string. Allows you to specify an endpoint's HTTP method.

endpoint_body

This section accepts an object. Allows you to describe in a simple format the structure of the body of a request.

For example, if your API endpoint has a body structure like this:

{
  "data": {
    "order": "1",
    "attributes": {
      "tags": [
        "new",
        "paid"
      ]
    }
  }
}

Then the configuration for this subsection will be like this:

api:

  # ...

  update_endpoint:

    # ...

    endpoint_body:
      data:
        order: id
        attributes:
          tags: tags

❗️ Please note: you must use exactly the field/column names, that you specified in the columns_order and update_fields sections.

filter_columns

This section accepts a list of objects. Allows you to describe in a simple format the structure of the filter for input data.

❗️ Please note: there is a logical AND between all filters.

column_name

This section accepts a string. Allows you to specify a column name for filtering.

condition

This section accepts the one of the values: EQ, NEQ, GT, LT, GTE, LTE. Allows you to specify a condition for filtering.

For string or int values:

• EQ – equals to the given value (==)
• NEQ – not equals to the given value (!=)

Only for int values:

• GT – greater than the given value (>)
• LT – less than the given value (<)
• GTE – greater than or equal to the given value (>=)
• LTE – less than or equal to the given value (<=)

value

This section accepts a string. Allows you to specify a column value for filtering.

❗️ Please note: if the value should be empty, specify it as ''.

update_fields

This section accepts a list of objects. Allows you to specify fields to be updated in your REST API.

field_name

This section accepts a string. Allows you to specify a field name for updating in your REST API.

values

This section accepts a list of strings. Allows you to specify field value(s) for updating in your REST API.

conditions

This section accepts a list of objects. Allows you to specify conditions for the field.

❗️ Please note: there is a logical AND between all conditions.

column_name

This section accepts a string. Allows you to specify a column name for filtering.

condition

This section accepts the one of the values: EQ, NEQ, GT, LT, GTE, LTE. Allows you to specify a condition for filtering.

For string or int values:

• EQ – equals to the given value (==)
• NEQ – not equals to the given value (!=)

Only for int values:

• GT – greater than the given value (>)
• LT – less than the given value (<)
• GTE – greater than or equal to the given value (>=)
• LTE – less than or equal to the given value (<=)

value

This section accepts a string. Allows you to specify a column value for filtering.

❗️ Please note: if the value should be empty, specify it as ''.