Add privateKey property to the AbstractGitTask to support SSH key-based authentication for all Git tasks

[smooth-baking-powder]

Describe the issue

This Flow not work with the WorkingDirectory inputFiles. Before i used the deprecated LocalFiles and it worked fine.

id: dev-test
namespace: dev

tasks:
  - id: start
    type: io.kestra.plugin.core.flow.WorkingDirectory
    inputFiles:
      id_rsa: "test"


    tasks:
      - id: clone_repository
        type: io.kestra.plugin.git.SyncNamespaceFiles
        namespace: prod
        gitDirectory: _files # optional; set to _files by default
        delete: false # optional; by default, it's set to false to avoid destructive behavior
        url: "{{ globals['repository-url'] }}"
        branch: "{{ globals['repository-branch'] }}"
        username: "{{ globals['repository-username'] }}"
        password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
        dryRun: true

Log:

2024-06-07T06:40:15.026Z INFO Provided 1 input(s).
2024-06-07T06:40:16.187Z INFO Start cloning from 'https://gitlab.xy.xy/xy/xy'
2024-06-07T06:40:16.204Z ERROR Destination path "spgOiPTaewpXqQS8vgGL0" already exists and is not an empty directory

If i use the old "LocalFiles" it works:

id: dev-test
namespace: dev

tasks:
  - id: start
    type: io.kestra.plugin.core.flow.WorkingDirectory

    tasks:
      - id: clone_repository
        type: io.kestra.plugin.git.SyncNamespaceFiles
        namespace: prod
        gitDirectory: _files # optional; set to _files by default
        delete: false # optional; by default, it's set to false to avoid destructive behavior
        url: "{{ globals['repository-url'] }}"
        branch: "{{ globals['repository-branch'] }}"
        username: "{{ globals['repository-username'] }}"
        password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
        dryRun: true
      
      - id: load_ssh_key
        type: io.kestra.core.tasks.storages.LocalFiles
        inputs:
          id_rsa: "{{ secret('SSH_KEY') }}"

Log:

2024-06-07T06:43:30.430Z INFO Start cloning from 'https://gitlab.c-lab.one/c-lab/automatisierung'
2024-06-07T06:43:31.851Z INFO Provided 1 input(s).
2024-06-07T06:43:31.857Z INFO Captured 0 output(s).

Environment

• Kestra Version: 0.17.1
• Operating System (OS/Docker/Kubernetes): Debian/Docker

[anna-geller]

it seems that you authenticate with your Git VCS using an SSH key. We may explore adding sshKey property to the AbstractGitTask so that you don't need to use WorkingDirectory anymore:

id: dev-test
namespace: dev

tasks:
      - id: clone_repository
        type: io.kestra.plugin.git.SyncNamespaceFiles
        namespace: prod
        gitDirectory: _files # optional; set to _files by default
        delete: false # optional; by default, it's set to false to avoid destructive behavior
        url: "{{ globals['repository-url'] }}"
        branch: "{{ globals['repository-branch'] }}"
        username: "{{ globals['repository-username'] }}"
        password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
        dryRun: true
        sshKey: "{{ secret('SSH_KEY') }}"

[anna-geller]

Related issue: #61

[anna-geller]

and kestra-io/kestra#3477

[smooth-baking-powder]

I use she "ssh_key" to add the ssh_key for ansible tasks for login on my target hosts. The Gitlab Token is another key (GITHUB_ACCESS_TOKEN in the secret file for kestra) and i dont have problems with gitlab. The Gitlab fails, cause the directory is already used.
I think the new ssh_key on workindirectory method make the problem. At the moment i use localfiles for the ssh_key to use the ssh_key in the container, that have to connect to my targets. often i need other files too, like inventory of my gitlab. so i need a directory local for my container.

With the deprecated localfiles the ssh key provide after the repository.

I follow here the tutorial of christian lempa: timestamp: 12:30
https://www.youtube.com/watch?v=PJG1-7hMHsE&t=778

I need this private key for all scripts (ansbile and more) outside of kestra. otherwise the flow failed, cause its not allowed to connect the target.

for example:

id: update-apt-packages
namespace: lab-0

inputs:
  - id: host
    type: STRING
    required: true
    description: "Der Host oder die Gruppe, die verwendet werden soll"
    defaults:
      - "update"
   
  - id: playbook_update
    type: STRING
    required: true
    description: "Der Host oder die Gruppe, die verwendet werden soll"
    defaults:
      - "ansible/update/update-apt-packages.yaml"

  - id: playbook_autoremove
    type: STRING
    required: true
    description: "Der Host oder die Gruppe, die verwendet werden soll"
    defaults:
      - "ansible/update/update-apt-autoremove.yaml"
  
  - id: inventory
    type: STRING
    required: true
    defaults: 
      - "ansible/inventory/inventory.yaml"

tasks:
  - id: start-task # this Task download the repository
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: clone-repository
        type: io.kestra.plugin.git.Clone
        url: "{{ globals['repository-url'] }}"
        branch: "{{ globals['repository-branch'] }}"
        username: "{{ globals['repository-username'] }}"
        password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"

      - id: load_ssh_key # this task add ssh_key for container
        type: io.kestra.core.tasks.storages.LocalFiles
        inputs:
          id_rsa: "{{ secret('SSH_KEY') }}"

      - id: update-servers # this task using repository inventory, playbook and ssh key to connect and update the targets
        type: io.kestra.plugin.ansible.cli.AnsibleCLI
        docker:
          image: cytopia/ansible:latest-tools
          pullPolicy: IF_NOT_PRESENT
          user: "1000"
        commands:
          - ansible-playbook -i "{{ inputs.inventory }}" --key-file id_rsa --extra-vars "hosts={{ inputs.host }}" "{{ inputs.playbook_update }}"
        env:
          ANSIBLE_HOST_KEY_CHECKING: "False"
          ANSIBLE_REMOTE_USER: "{{ globals['automation-ssh-user'] }}"

      - id: autoremove-old-packages
        type: io.kestra.plugin.ansible.cli.AnsibleCLI
        docker:
          image: cytopia/ansible:latest-tools
          pullPolicy: IF_NOT_PRESENT
          user: "1000"
        commands:
          - ansible-playbook -i "{{ inputs.inventory }}" --key-file id_rsa --extra-vars "hosts={{ inputs.host }}" "{{ inputs.playbook_autoremove }}"
        env:
          ANSIBLE_HOST_KEY_CHECKING: "False"
          ANSIBLE_REMOTE_USER: "{{ globals['automation-ssh-user'] }}"

triggers:
  - id: schedule_trigger
    type: io.kestra.plugin.core.trigger.Schedule
    cron: 0 22 * * *
    inputs:
      host: "update"