Imagefactory_API__Conductor_Work
This document is meant to facilitate collaboration and design with the factory team’s redux of their API, found [[image-factory:V2RESTAPI|here]]
The current factory model consists of 4 objects: (These are reasonably understood so I won’t go into detail)
- Image
- Build
- Target Image
- Provider Image
Essentially the Image and Build are meta data objects that are used to group target images and provider images. Build is used to group target images and provider images with guaranteed equivalence (i.e. Exact same packages/versions etc…) The image object grouped together builds that were all based on the same Template and help some extra data such as name. Equivalence is not guaranteed across builds within an image.
In it’s next major release factory intends to streamline its responsibilities and to concentrate on its end goal: “To Build and Push Images to Clouds” and to carry no burden of writing to another system based on the requirements of other upstream projects. This in turn requires changes to the factory object model, resulting in the removal of the Build object. Leaving only base image, target image and provider image. A further wrinkle here is that a snapshot style build will not have a target_ or provider_ image. though the conductor will need to preserve that relationship for its own business uses.
The new version of the factory will be storing state of objects, object metadata listed on wiki page at top of this doc. However, and this is a key point - it is NOT storing all the relationships that will matter to the conductor. It can be viewed as knowing about a similar object model from the conductor ‘build/version’ layer down to the actual provider image, modulo the fact that it does not recognize a snapshot build based on the same tdl as an upload build to have any connection to each other.
We will have to do a reasonable amount of work in conductor to update library calls (aeolus-image) and tests.
The factory model (as it currently stands) has been exposed directly by conductor. Users of conductors are exposed to what are essentially factory objects Image,Build etc… This is more obvious through the CLI than the UI though there is a considerable number of references in the UI. I feel that this has come about not through a requirement of conductor but as a result of using factory. With this in mind, we need to ask:
- What does Image, Build model buy us? Is this part of conductor
requirements?
-
Yes, these are needed by conductor
- Image- a common base object representing the abstract notion of an image with a given template (and has a reference to this template)
- Build- a reference to a given build/version of an image for a specific (set of) providers. This also has a reference to a template object, which may be different from that in the base image (for instance, if the template is later edited, after a build, the build still has a reference to a valid template)
-
Now, with the above in mind, that does NOT mean we cannot streamline the api of conductor. I think at least targetimages and providerimages still need to be exposed, but that may be all that are needed - everything else may be able to be done behind the scenes (we’ll see as we flesh out an updated conductor api/ui flow, hopefully keeping them consistent and having the same controllers this time).
-
-
Client wants to build an image, and specifies:
- Template
- Provider types (target)
- specific provider list
- credentials
- scope/environment
- other parameters
-
It receives back a uuid that it can then query for status that says when this image is ready to be deployed in a given cloud (plus whatever ‘environment’ scoping is needed)
Note: There is likely a similar case to the below ‘power user’ scenario thought the UI. In fact, the cli and web ui should largely be the same, though UI may end up being a superset, if it offers more complex flows than needed for cli user.
- Conductor calls factory to initiate a build on behalf of a user
- Based on , Conductor specifies whether this image should be a local or snapshot build, and tells factory which way to build via POSTing to the targetimages object for an upload build, or POSTing to providerimages and specifying a boolean to denote a snapshot build.
- Depending on factory api that was called, either wait for step to finish and initiate step 2 when notified of completion (api will support a webhook) or just wait for completion, then update conductor object hierarchy with details at that time.
Here we have a possibility of simplifying things a bit for the average user, should we want to. It could go something like this:
Joe User
# Select some action to ‘create new image’
# Upload tdl, select providers and any other requirements for using the
image when done.
# Submit the image to be ‘made available on chosen providers’
# Wait for image to show as available/complete, then launch instances
with it. The key here is that conductor makes the decision for the
user on whether to do a snapshot or upload style build for a given
provider. This covers cases where there are business rules that dictate
which way a certain OS on a specific provider should be built, as well
as simplifying the workflow for the user
UI
Polly Power User (intended to be something that is added/exposed
later, unless there is heavy demand immediately)
# Select some action to ‘create new image’
# Upload tdl, select types of providers to make this available for
# Choose, for each provider type, whether to use a snapshot or upload
style build
# Submit to build for upload/push for snapshot
# For upload style, once build is complete they can then be pushed to
chosen provider accounts
the conductor UI
Joe User
# Follow steps 1 & 2 above
# Select ‘share this image with other environments’
# Pick from a list of environments Joe has permission to build for
# Continue flow above
- There is some desire on the conductor side to have some way to say images in different environments (pool families) are equivalent, allowing them to then be used in a more portable way as defined in deployable xml. Apparently right now, that xml point to an image uuid, which is scoped to env. If we can consider solving this as part of the new data model, that would be helpful for conductor
- The concept of equivalence and versioning is not a high priority, and given our complete lack of ability to currently do this is a sane way, I propose we do not pretend to support it at this time. A conductor user could certainly ‘rebuild’ an image based on a given TDL, but the net effect is a new image is built, and it is then up to the user to give it a new name (perhaps they can include a version in the name for their own use). The point is, we do not guarantee anything here.
- Meaning and relevance to client of a local vs ‘snapshot’ build. IOW,
how important is the distinction to the api client? Conductor
client?
- My thought is, it makes sense for conductor to know this, but may be something that can be done by a power user, but does not HAVE to be (with conductor making the choice in the default case
The above model should largely be able to get reused within conductor,
with a few modification, including, but not limited to:
* Renaming some of the fields to something more sensible (like
providerimage.targetidentifier - maybe something like
‘externalimageid’ instead?)
* Addition of a field to represent the factory id for this object, if
it exists in factory
* We may be able drop the icicle object for now, as conductor has no
use for it (yet, anyway)
* Do not include ‘Base’
* (Possibly) rename Image > BaseImage, and Build> ImageVersion
* Add some fields from factory model, as appropriate, like snapshot
boolean, status, status_detail (unclear what, if any of ‘parameters’ we
need on conductor side.
* Consider additional layer of ‘ImageFamily’ to facilitate making
‘something’ based on a common tdl available across environments/pool
families.
In addition, there will need to be work done around a decision-making/policy engine type thing to allow conductor to determine which build type to request for a given provider_type, specific provider, and possibly even a certain account on that provider. Further details to come…
Factory V2 has significant changes:
- Change to Resource model, see github markdown
- Added support for different types of build (SNAPSHOT, BUILD)
- Dropped support for image imports.
- Web Hooks support
- No storing of Templates
These changes mean that conductor must store extra data and offer some of the services previously in factory as well as adding support for the Web Hooks.
Due to the changes in the resource model (Factory now only supports TargetImages, ProviderImages and BaseImage*) conductor must now add in the extra Image and Build (renamed to BaseImage, ImageVersion) objects that used to be managed by factory. In conductor we must add extra relational meta data to each of the TargetImage and ProviderImage objects such as ProviderType and Provider. This in addition to the import support (now handled in conductor) results in conductor needing to store TargetImage and ProviderImage objects. Conductor must also keep a reference to Templates since these will no longer be stored by factory. Potential resulting Model inc attributes is shown below:
*Not needed by conductor
Conductor must offer an endpoint in which factory can update the status of a particular Target/Provider Image. This will likely be added to the Conductor API.
Given the factory workflow changes we must add some extra model data to
support imports and to give the perception of a consistent workflow
through the different build types. One approach would be to model a
consistent BaseImage > ImageVersion> TargetImage -> ProviderImage
for all types of build (Import, Build, Snapshot) in conductor. This is
different from what is present in factory (Snapshots will not have a
targetimage).