Skip to content

Consul Template plugin to write multiple files from a single template.

License

Notifications You must be signed in to change notification settings

heronimus/ct-plugin-string2files

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Consul-Template Plugin: string2files

Lightweight Consul-Template plugin that (basically) write string to file(s). Created because I need easy way to write multiple files from Consul KV tree with only single template ✌️.

(Update 2024-10-26) Build with V language compiler: V 0.4.8 14b1a14

Credits

This plugin heavily inspired by:

Both of above code already working nice, but I do some modification and mainly as an experiment to try new language (V-lang) and make it more lightweight binary (320~ kB) (it's seriously promising new prog. language and fun to try, they have an active community too on Discord).

Table of Contents

Features

  • Write/append string to file.
  • Split string and write them to multiple files.
  • Allow you to write multiple files from Consul KV Tree with only single Consul-Template file.
  • Combine Consul KV + Vault Secret and write it to files, and basicly write to file any string you can get from Consul Templates.
  • ...

Installation

  • Download the executable binary from releases page.
  • Give execute permission & copy to your path
    chmod +x string2files
    mv string2files-v0.1.0 /usr/local/bin/string2files
    

Usages

Usage: strings2files [commands] [flags] <arguments..>

Flags:
  -help               Prints help information.
  -f   --force        Create new directory/file from <path-file> if not exist.
  -nl  --new-line     Add new line in the end of file.

Commands:
  append              MODE: Append string to file.
  create              MODE: Write string to file.
  explode             MODE: Split text and write to multiple file.
  help                Prints help information.
  version             Prints version information.

Arguments:
  append              <file-path> <content>
  create              <file-path> <content>
  explode             <base-path> <separator> <content>

Examples (as Consul Template Plugin)

These are just a few examples that bring me to create this plugin. Basically, this plugin only writes value given into a file, so it may have many possibilities how you can make use of this plugin.


For Example: given below items on Consul KV & Vault Secret:

  • Consul KV:
    config/myservices
        │   app.conf         app.name="myservices"
        │
        └─── db/
            │  db.conf       db.host="127.0.0.1:5432"
    
  • Vault Secret:
    secret/myservices
        │   app.conf
        |      - app.token    "s3cr3t-t0k3n"
        │
        └─── db/db.conf
            │  - db.username  "mydb"
            │  - db.password  "pass1234"
    

Use cases example:

  • Example 1 (Consul KV Tree): example/example-consul-kv-tree.tpl

    {{/* "Loop on Consul KV path, then write K/V to files" */}}
    {{ range tree "config/myservices" }}
    {{- .Value | plugin "/usr/local/bin/string2files" "create" "-f" "-nl" (print "/opt/myservices/" .Key) -}}
    {{ end -}}
    

    execute command:

    consul-template -template "example-consul-kv-tree.tpl:debug.log" -once
    

    result:

    /opt/myservices/
        │   app.conf         app.name="myservices"
        │
        └─── db/
            │  db.conf       db.host="127.0.0.1:5432"
    
  • Example 2 (Consul KV Tree + Vault Secret appended): example/example-consul-vault-pair.tpl

    *Please notes that Consul KV and Vault Secret of this example have same directory structure.

    {{/* "Loop on Consul KV path" */}}
    {{ range tree "config/myservices" }} {{ $keytemp := .Key }}
    {{- .Value | plugin "/usr/local/bin/string2files" "create" "-f" "-nl" (print "/opt/myservices/" .Key) -}}
    
    {{/* "Get Secret from Vault" */}}
    {{ with secret (print "secret/myservice/" $keytemp) }}{{ range $k, $v := .Data }}
    {{ if $k }}
    {{- (print $k " = " $v) | plugin "/usr/local/bin/string2files" "append" "-f" "-nl" (print "/opt/myservices/" $keytemp) -}}
    {{ end }}{{ end -}}{{ end -}}
    
    {{ end -}}
    

    execute command:

    consul-template -vault-addr <vault-host> -vault-token=<vault-token> -vault-renew-token=false  -template "example-consul-vault-pair.tpl:debug.log" -once
    

    result:

    /opt/myservices/
        │   app.conf         app.name="myservices"
        |                    app.token = s3cr3t-t0k3n
        │
        └─── db/
            │  db.conf       db.host="127.0.0.1:5432"
            |                db.password = pass1234
            |                db.username = mydb
    
  • Example 3 (Consul KV Tree with explode mode): example/example-splits-from-string.tpl

    *The explode mode allows you to freely construct the data first, then pass the whole strings to the plugin, as the plugin will split the key/value by the separator and create file(s) based on that.

    {{/* "Initiate Variable" */}}
    {{ $separator := "=====" -}}
    {{ $data := "" -}}
    
    {{/* "Constructing string to .Data variable" */}}
    {{ range tree "config/myservices" }}
    {{- $data = (print $data .Key $separator .Value $separator) -}}
    {{ end -}}
    
    {{/* "Executing plugin exclude ..." */}}
    {{ with $data }}
    {{ . | plugin "/usr/local/bin/string2files" "explode" "-f" "-nl" "/opt/myservices/" $separator }}{{ end }}
    

    execute command:

    consul-template -template "example-splits-from-string.tpl:debug.log" -once
    

    result:

    /opt/myservices/
        │   app.conf         app.name="myservices"
        │
        └─── db/
            │  db.conf       db.host="127.0.0.1:5432"
    

Examples (as Standalone Binary)

  • Create file /opt/files/key-example.txt with file-content value-example:
    strings2files create /opt/files/key-example.txt "value-example"
    
    force directory creation if not exist (-f), and add new-line (-nl) on the end of the file.
    strings2files create -f -nl /opt/files/key-example.txt "value-example"
    

  • Append file-content value-example to/opt/files/key-example.txt:
    strings2files append /opt/files/key-example.txt "value-example-appended"
    

  • Explode combined key-value strings separated by delimiter:
    strings2files explode /opt/files/ ";" "key1.txt;value1;key2.txt;value2;child/key3;value3"
    
    will create following files tree:
    /opt/files/
    │   key1.txt        value1
    │   key2.txt        value2
    │
    └─── child/
        │  key3.txt     value3
    
    

References

More references about Consul Template and Consul Template plugin, please see official documentation:

Build from source

This plugin written in V, so you must have V compiler installed.

  • Install V (Linux, macOS, Windows, *BSD, Solaris, WSL, Android, Raspbian)

    git clone https://github.com/vlang/v
    cd v
    make
    
  • Symlink V

    • On Unix system
      sudo ./v symlink
      
    • On Windows
      .\v.exe symlink
      
  • Clone this repository

    git clone https://github.com/heronimus/ct-plugin-string2files.git
    
  • Build the code

    cd ct-plugin-string2files
    v string2files.v
    
  • Binary will created at working directory (string2files / string2files.exe for Windows)

*Notes: V allows you to cross compilation by passing flag -os <windows/linux>, m acOS binary only can be compiled on macOS platform. *Build with V compiler version: V 0.2.4 8315e82

More About V

Please visit official website and docs: