xtee

xtee is a command-line utility for reading from standard input and writing to both standard output and file, providing powerful features for file manipulation and data processing. It combines the functionality of tee, the soaking behavior of sponge, and more.

Resources

• Features
• Installation
  • Install release binaries (Without Go Installed)
  • Install source (With Go Installed)
    • go install ...
    • go build ... the development Version
• Usage
  • Examples
    • Basic
    • Deduplicating Files
    • In-Place Deduplicating Files
    • Appending Lines to File
    • Appending Unique Lines to File
  • Performance Considerations
• Contributing
• Licensing

Features

• Supports streams spliting, incoming standard input into standard output and file
• Supports soaking up input before writing, enabling safe in-place processing
• Supports appending and overwriting the destination entirely
• Supports deduplication, keeping destination unique
• Cross-Platform (Windows, Linux & macOS)

Installation

Install release binaries (without Go installed)

Visit the releases page and find the appropriate archive for your operating system and architecture. Download the archive from your browser or copy its URL and retrieve it with wget or curl:

• ...with wget:

   wget https://github.com/hueristiq/xtee/releases/download/v<version>/xtee-<version>-linux-amd64.tar.gz

• ...or, with curl:

   curl -OL https://github.com/hueristiq/xtee/releases/download/v<version>/xtee-<version>-linux-amd64.tar.gz

...then, extract the binary:

tar xf xtee-<version>-linux-amd64.tar.gz

Tip

The above steps, download and extract, can be combined into a single step with this onliner

curl -sL https://github.com/hueristiq/xtee/releases/download/v<version>/xtee-<version>-linux-amd64.tar.gz | tar -xzv

Note

On Windows systems, you should be able to double-click the zip archive to extract the xtee executable.

...move the xtee binary to somewhere in your PATH. For example, on GNU/Linux and OS X systems:

sudo mv xtee /usr/local/bin/

Note

Windows users can follow How to: Add Tool Locations to the PATH Environment Variable in order to add xtee to their PATH.

Install source (with Go installed)

Before you install from source, you need to make sure that Go is installed on your system. You can install Go by following the official instructions for your operating system. For this, we will assume that Go is already installed.

go install ...

go install -v github.com/hueristiq/xtee/cmd/xtee@latest

go build ... the development version

• Clone the repository

   git clone https://github.com/hueristiq/xtee.git 

• Build the utility

   cd xtee/cmd/xtee && \
   go build .

• Move the xtee binary to somewhere in your PATH. For example, on GNU/Linux and OS X systems:

   sudo mv xtee /usr/local/bin/

  Windows users can follow How to: Add Tool Locations to the PATH Environment Variable in order to add xtee to their PATH.

Caution

While the development version is a good way to take a peek at xtee's latest features before they get released, be aware that it may have bugs. Officially released versions will generally be more stable.

Usage

To display help message for xtee use the -h flag:

xtee -h

help message:

      _
__  _| |_ ___  ___
\ \/ / __/ _ \/ _ \
 >  <| ||  __/  __/
/_/\_\\__\___|\___|
             v1.0.0

USAGE:
 xtee [OPTION]... <FILE>

INPUT:
     --soak bool          buffer input before processing

OUTPUT:
     --append bool        append lines
     --unique bool        unique lines
     --preview bool       preview lines
 -q, --quiet bool         suppress stdout
 -m, --monochrome bool    stdout in monochrome
 -s, --silent bool        stdout in silent mode
 -v, --verbose bool       stdout in verbose mode

Examples

Basic

echo "zero" | xtee stuff.txt

cat stuff.txt

zero

Deduplicating Files

cat duplicates.txt

zero
one
two
three
zero
four
five
five

cat duplicates.txt | xtee --unique deduplicated.txt

zero
one
two
three
four
five

cat deduplicated.txt

zero
one
two
three
four
five

In-Place Deduplicating Files

cat stuff.txt

zero
one
two
three
zero
four
five
five

cat stuff.txt | xtee stuff.txt --soak --unique

zero
one
two
three
four
five

cat stuff.txt

zero
one
two
three
four
five

Note

Note the use of --soak, it makes the utility soak up all its input before writing to a file. This is useful for reading from and writing to the same file in a single pipeline.

Appending Lines to File

cat stuff.txt

one
two
three

cat new-stuff.txt

zero
one
two
three
four
five

cat new-stuff.txt | xtee stuff.txt --append 

zero
one
two
three
four
five

cat stuff.txt

one
two
three
zero
one
two
three
four
five

Appending Unique Lines to File

cat stuff.txt

one
two
three

cat new-stuff.txt

zero
one
two
three
four
five

cat new-stuff.txt | xtee stuff.txt --append --unique

zero
four
five

cat stuff.txt

one
two
three
zero
four
five

Note

Note that the new lines added to stuff.txt are also sent to stdout, this allows for them to be redirected to another file:

cat new-stuff.txt | xtee stuff.txt --append --unique > added-lines.txt

cat added-lines.txt

zero
four
five

Performance Considerations

• For large files (>1GB), --soak mode will use significant memory as it buffers everything
• --unique mode maintains all seen lines in memory - be cautious with huge datasets
• Stream processing (without --soak) is most memory-efficient for large inputs

Contributing

Contributions are welcome and encouraged! Feel free to submit Pull Requests or report Issues. For more details, check out the contribution guidelines.

A big thank you to all the contributors for your ongoing support!

Licensing

This package is licensed under the MIT license. You are free to use, modify, and distribute it, as long as you follow the terms of the license. You can find the full license text in the repository - Full MIT license text.