Template reclone / merge

[janfrl]

Describe the feature

I maintain a set of static files like dotfiles or other configs (.vscode/*, .editorconfig, eslint.config.ts, LICENSE, etc.) with mostly the same contents across dozens of repos. When I update any of those, I usually need to update most of them. Instead of doing this manually, I want to update one central template repository and pull changes to the others. I don't want to sync the entire repo but be able to select specific files to sync so I can easily deviate from the template repo.

Tools like GitPick already let you fetch arbitrary blobs from a Git provider’s raw endpoint. It would be great if giget could:

• Let me specify a list of files to pull.
• Integrate into npm scripts or a giget.config.(cjs|ts) (c12-style) so I can store my sync-list in source.
• Honor a conflict strategy (skip | overwrite | merge in the future) when a target file already exists.

Proposal

New CLI flag & API

I'm not sure yet if I should call the flag --files or --include. The latter has the advantage that an --exclude flag could be added as a filter, so I could --include .vscode --exclude .vscode/settings.json for example. But I think the best solution would be to use the .gitignore-Syntax to exclude files with an exclamation mark.

# sync only license and VSCode settings folder
giget gh:unjs/giget output-dir --strategy overwrite --files README.md --files .github

import { downloadTemplate } from 'giget'

await downloadTemplate('unjs/dotfiles', {
  dir: 'output-dir', 
  files: [
    'README.md',
    '.github'
  ],
  strategy: 'overwrite', // skip | overwrite | merge
})

• --files <paths…> (repeatable): list of file or folder paths to fetch (I guess this is how you use arrays with citty? Atleast it works).
• strategy (skip | overwrite | merge): what to do when a target already exists.
• raw-HTTP loop: for each file in files, attempt template.raw(path) → fetch → write.
• partial-tarball extract: if any path is a folder (or raw fetch throws), download the tarball and filter by those paths.

Config-file support (c12 integration)

// giget.config.ts
export default {
  dir: '.',
  files: [
    '.editorconfig',
    'eslint.config.ts',
    '.vscode/**/*',
  ],
  strategy: 'overwrite',
}

Then in your package.json:

import { resolve } from 'node:path'
import type { FileConfig, IncludeEntry, Source } from './src/config'

const config: FileConfig = {
  branch: 'main',
  strategy: 'overwrite',
  cache: false,

  sources: [
    {
      repo: 'github:unjs/giget',
      branch: 'develop', // can be overwritten for each source
      files: [
        // simple string → { src: 'path', dest: 'path' }
        '.editorconfig',
        'LICENSE',

        // object form to rename or relocate
        { src: '.vscode/settings.json', dest: resolve(process.cwd(), '.vscode/settings.json'), strategy: 'skip' },

        // sync an entire folder
        '.vscode/',
      ],
    },
    {
      // pick single file from a bitbucket repo:
      repo: 'bitbucket:acme/common-config',
      include: ['eslint.config.ts'],
    } as Source,
  ],
}

Implementation Notes

I have already implemented a working prototype. The following is summarized by Copilot based on my code. Maybe this helps with discussing early before I can submit a PR, but things might change:

1. --files flag
   • In cli.ts, define a multi-value files arg, normalize to string[].
2. Raw download pass
   • In downloadTemplate, if options.files is set and no folder-only paths remain, loop over each path:

     for (const p of options.files) {
       const url = template.raw(p)
       const res = await sendFetch(url)
       await writeFile(destDir/p, await res.arrayBuffer())
     }
     return early

3. Partial-tarball extraction
   • If any entry ends with / (folder), or if any raw fetch fails, download the .tar.gz, then:

     await tarExtract({
       file: tarPath,
       cwd: destDir,
       strip: 1,
       filter: entry => {
         const rel = entry.path.split('/').slice(1).join('/')
         return files.some(f =>
           rel === f || rel.startsWith(f + '/')
         )
       }
     })

4. Fallback & full extract
   • If --files is omitted, or after partial extract, proceed with the existing full-tarball logic + installDependencies.

---

Is this functionality wanted in giget? If so, I would be happy to submit a PR. Please let me know if you have better ideas to address my use case or if you would change some implementation details.

Additional information

• Would you be willing to help implement this feature?

[pi0]

I like the idea! Maintaining different merging strategies can be hard but what if we use local git command to treat clonnes tarball as git? (we can init a bare git project in clone dist and use git to handle sync and diffing)

[pi0]

Also lets split config loading and milti source topics. First thing i want to see how we can implement simple sync strategies either based on git or custom merge.

[janfrl]

Maintaining different merging strategies can be hard but what if we use local git command to treat clonnes tarball as git? (we can init a bare git project in clone dist and use git to handle sync and diffing)

Skip and overwrite are easy to handle, but I agree that merging would be hard. Git or an alternative tool sounds like a good solution. Would that mean that we need git as a dependency?

[janfrl]

Also lets split config loading and milti source topics. First thing i want to see how we can implement simple sync strategies either based on git or custom merge.

I drafted a PR and didn't include config support for now. Also I only included skip and overwrite for now and it still feels kinda big. Maybe it would be good to split the code or make the strategies modular, somewhat like unstorage drivers.