Various improvements related to repos/snapshots/publishes

[bbinet]

And also add support for more options when publishing a new snapshot.

(backward compatibility with existing pillar structure is preserved)

[bbinet]

Please wait before merging this PR.
I will extract the new snapshot publishing capability to a new aptly/server/publish.sls state so that it is not related to mirrors anymore (and can be used to publish repo snapshots too).

Any objection before I start the refactoring?

[fpytloun]

I am fine with it, go ahead :-)
Only two notes:

• it's recommended to use aptly-publisher for creating publishes as that tool is idempotent (won't do anything if there's nothing to do)
• I am planning to add feature to create up-to-date snapshots when publishing with aptly-publisher so it will not be needed to create them by other tools

[bbinet]

Yes, but as discussed in the other PR, I was not able to publish my filtered mirrors with aptly-publisher (probably because of an aptly api bug), so I'm now using salt directly, trying to make the salt formula publishing part idempotent too.

[fpytloun]

Yes, I am fine with using it this way.

[bbinet]

@fpytloun I've updated the PR with the refactoring and some other improvements:

• 5891a0c Add support for dropping mirror snapshots
• 48360ca Fix requirement issue when mirror.update is false
• 4e6621a Fix missing requirement when engine is not docker
• eb60e61 Minor renaming simplification
• f3f9369 Add support for snapshotting repos
• 1430653 Add support for publishing snapshots
• fde4d3c Only run aptly repo add if packages are available in pkg_dir
• ec80056 Add support for aptly repo add -remove-files option
• f4610cb Add pillar example in readme for aptly server publish

Every commit include a specific self-contained feature so that it is easier to review.

[bbinet]

I've updated the pull request with a new feature to add support for aptly repo include to be able to import .changes files (from dpkg-genchanges).
This also add support for importing gpg keys to verify .changes signature.

So here are now the various improvements of this PR:

• 5891a0c Add support for dropping mirror snapshots
• 48360ca Fix requirement issue when mirror.update is false
• 4e6621a Fix missing requirement when engine is not docker
• eb60e61 Minor renaming simplification
• 93fdfa8 Add support for snapshotting repos
• 09091ae Add support for publishing snapshots
• 5491532 Only run aptly repo add if packages are available in pkg_dir
• 84515a4 Add support for aptly repo add -remove-files option
• 1b62c7a Add pillar example in readme for aptly server publish
• 8de16c0 Add support for aptly repo include
• d71b350 Fix pillar example for gpgkeys syntax in readme
• 9fc0ef7 Cosmetics: cleanup .get('var', None) syntax

This PR is now ready to be reviewed.

[bbinet]

@fpytloun do you think this can be merged soon?

[bbinet]

@fpytloun any news on this one?