Add support for filtering

[sherriff]

Use case:

We have a bunch of Java applications installed using Solaris PKGs. To avoid duplication it would be nice to be able to use templates and just specify the differences using properties.

Example usage:

1. Have a start-up script template which can be reused by multiple projects. Java options, installation basedir and name of start-up jar file are examples of typical variables.
2. Reuse manifest files for Solaris SMF. Typical variables that can be substituted are: Path to start-up script and service name.
3. Reuse action scripts (post-install, pre-remove). Typical variables needed: installation basedir, application name

It seems likely that it is necessary to add configuration option to specify where the different files should be fetched from. E.g. target/unix/files/ and target/unix/scripts instead of src/main/unix/files and src/main/unix/scripts.

See also http://jira.codehaus.org/browse/MUNIX-8 and http://jira.codehaus.org/browse/MUNIX-9.

[trygvis]

Would something like this work?

<copyFile>
  <path>goals.txt</path>
  <toFile>/usr/share/hudson/README.txt</toFile>
  <!-- For a single replacement -->
  <replace>
    <pattern>SCRIPT_NAME</pattern>
    <value>app1</value>
  </replace>

  <!-- For multiple replacements -->
  <replacements>
    <replace>
      <pattern>SCRIPT_NAME</pattern>
      <value>app1</value>
    </replace>
    <replace>
      <pattern>SCRIPT_NAME</pattern>
      <value>app1</value>
    </replace>
  </replacements>
</copyFile>

Re "where to copy files from", I think this is covered already by just specifying whatever you want as <path> inside a <copyFile> or <from> inside a <copyDirectory>.

[sherriff]

Comments and questions:

1. I like the convention over configuration principle that have been used for example to add specific functionality when files are placed in src/main/unix/files and src/main/unix/scripts. A simple boolean to change whether variables in files and scripts folders are replaced or not would then be sufficient. E.g.:

   <configuration>
     <defaults> 
       <filtering>true</filtering>
     </defaults>
   </configuration>
   

2. Adding support for replacements in copyFile/copyDir seems like a good idea. I guess this can be the underlying implementation for the default filtering option in 1 as well?

3. I don't quite understand the syntax of the value tag.

   a. I would prefer to be able to turn replacements/filtering on/off with a flag and then expect the replacements that Maven normally would apply. I think this will result in less configuration than specifying all replacements explicitly. This solution would, however, probably solve the problem.

   b. So if I want to use standard maven properties as placeholders to replace it would look something like this?

   in unix-m-p config:

     <replacements>
         <replace>
           <pattern>${application.install.baseDirectory}</pattern>
           <value>/opt/application1</value>
         </replace>
   

   And I'd have to duplicate this information in the pom if the same value should be used other places? E.g.

     <properties>
       <application.install.baseDirectory>/opt/application1</application.install.baseDirectory>
      </properties>
   

4. I suggest to support replacements only, not a single replacement tag.

[sherriff]

I guess it is a separate issue, but with regards to "where to copy files from". Yes, this is supported by copyFile and copyDirectory. However, I want to use the default "copy rules" that are in effect for src/main/unix/files and scripts as much as possible. The design suggestion in bullet 1 in my previous comment this would be hidden for the user, but the implementation would use a folder containing filtered files instead of the unfiltered templates.

[trygvis]

(1) is a good idea that will control filtering on/off on the implicit copyDirectory. Don't know if I need one for file and one for scripts, but I guess the flexibility is good to have.

(2) yes, this will be how everything is implemented.

(3) The value is the exact value that will be used as a value for every regexp match. As outlined in the original example you would do something like this:

<replacements>
  <replace>
    <!--
        ${application.install.baseDirectory} would be replaced by the Maven XML
        reader. We don't want that.
    -->
    <pattern>@application.install.baseDirectory@</pattern>
    <value>${application.install.baseDirectory}</value>
  </replace>
</replacements>

...

<properties>
  <application.install.baseDirectory>/opt/application1</application.install.baseDirectory>
</properties>

But as you say, using the build-in filtering mechanism would be useful. This means that having this:

<properties>
  <application.install.baseDirectory>/opt/application1</application.install.baseDirectory>
</properties>

...

<copyDirectory>
  <path>target/foo</path>
  <to>/usr/share/foo</to>
  <filtering>true</filtering>
</copyDirectory>

would imply this:

<copyDirectory>
  <path>target/foo</path>
  <to>/usr/share/foo</to>
  <replacements>
    <replace>
      <pattern>\${application\.install\.baseDirectory\}</pattern>
      <value>/opt/application1</value>
    </replace>
  </replacements>
</copyDirectory>

[trygvis]

An alternative approach which is less work and possibly cleaner as it separates copying from filtering:

<copyDirectory>
  ...
</copyDirectory>
<copyFile>
  ...
</copyFile>

<filter>
  <!-- path refers to installed files -->
  <path>/opt/foo/etc</path>

  <!-- similar to resources plugin, as suggested above -->
  <filtering>true</filtering>

  <!-- as above -->
  <replacements>
    ...
  </replacements>
<filter>

Pros:

• Less code, I don't have to add support for filtering in 5 tasks (copy directory/file/artifact, extract file/artifact)
• Cleaner as each task type (copy vs filter) is more specific and hopefully easier to understand

Cons:

• The existing resources and assembly plugins does copy and filtering in one go which is what people might expect.
• The Zip package support will

[cholden14]

Any ETA on Filtering? I am about to have to rewrite our packaging but need filtering to make this particular plugin work.

[trygvis]

The code in the repository should work for the zip packaging. Which packaging are you using?

[cholden14]

Thanks for the reply I am using solaris and rpm's. I have a set of files for the pre and post functions that I want to use as a dependency for each project and have them filtered with application specific information (user, groups, perms).

Any plans to expand the filtering to work with all packages types?

[trygvis]

Planned, yes, timeframe, no. Given that people want it makes it more likely, but I don't have any workhours to spend on it now so don't hold your breath. It is doable to implement yourself if you have the time, use the zip code as a start.

[trygvis]

@cholden14 Is this still something you need? I'm working on this now.

[cholden14]

Yes sir it is!

Sent from my iPhone

On Feb 13, 2013, at 6:06 AM, Trygve Laugstøl notifications@github.com wrote:

@cholden14 Is this still something you need? I'm working on this now.

—
Reply to this email directly or view it on GitHub.

[trygvis]

Let me know when you've been able to try it out. I'm working on updating the handbook too.

[cholden14]

Trygve

I will look as soon as I can, might be Wednesday

Thank you!

Sent from my iPhone

On Feb 18, 2013, at 8:45 AM, Trygve Laugstøl notifications@github.com wrote:

Let me know when you've been able to try it out. I'm working on updating the handbook too.

—
Reply to this email directly or view it on GitHub.

[cholden14]

Trygvis,

Forgive me but which filtering option did you implement? I mean in terms of basic syntax?

Thanks
Chris