-
Notifications
You must be signed in to change notification settings - Fork 3
Home
==About == This guideline is intended for anyone who wishes to use the CAB Configuration library. The library covers the following basic features: #Create a single configuration object that includes all the configuration of an appliacation. #A Factory API and a Spring Bean to get the configuration object into the application
The above supports 2 modes: #Central Configuration (DB) mode #File system mode
This will be detailed below.
== Scope == This wiki page is developer oriented.
The CAB Configuration library should be used in ALL CAB HE java components.
== Contact == [mailto:[email protected] Yair Ogen]
== Issue tracking ==
Rational Clear Quest
Area: CAB Area
Product: Cab Configuration
== Maven Info ==
== API Reference == {{prettynote|As we have new javadoc for eavh new release, please change the "{LATEST_VERSION}" with the real library version. E.g. http://maven-doc.nds.com/maven_sites/com.nds.cab.infra/cabConfiguration/3.0.1-2/apidocs/index.html.}} http://maven-doc.nds.com/maven_sites/com.nds.cab.infra/cabConfiguration/{LATEST_VERSION}/apidocs/index.html
== Documentation ==
==How it works==
===Central Configuration Mode=== Cab Configuration library acts as a client for a "Central Configuration Repository". You can verify you are in this mode if you see in the log file:
The enable and disable of CCP is done via CAB_Infrastructure/Build_Environment/RPM/Generic_Invocation_Script using environment variables. By default ccp is not enabled. If you want to enable/disable it you can add to your cfg file:
When CCP is enabled it will look for its data base connection details under the following environment variables:
These properties should be a part of the user profile by default, hence allowing a single source for every component on a given host. However, you can override any of the above in the cfg file as mentioned above.
In order to support CCP a component needs to define the parameter set A.K.A Namespace. See #CCP_Compliance for more details.
===File System Mode=== This mode should be used by developers and / or QC as it works with simple xml/property files. You can verify you are in this mode if you see in the log file:
This library delivers one single configuration object based on several sources of xml files. Legacy property file approach is still supported.
In order to support CCP a component needs to define the parameter set A.K.A Namespace. See #CCP_Compliance for more details.
The created object is an [http://commons.apache.org/configuration/apidocs/index.html apache common configuration Object] with all the needed API to work with the configuration parameters.
The CABConfiguration Library in this mode supports 3 levels of configuration:
The '''"Factory configuration"''' which is delivered as part of the components - is a simple xml file, called configSchema.xml that needs to sit in the root of the component jar. In this file will reside all the default configuration properties defined by the R&D for this specific component. The library will scan all the jars in the classpath in order to find all these files. The configuration object will include all the values defined in all these files.
The '''"Deployment configuration"''' - which is external to the running jars and enables a "site/host" flavor of configuration. The library will search for a property file called deploymentConfig.properties. This file can be created by the PD line when they integrate the components to the solution required by the customer. This is a single file that sits in the classpath or in the root of a jar in the classpath. All the property values in this file will overwrite the default values defined in stage 1.
The '''"Customer configuration"''' - which is external to the running jars and allows an instance-specific configuration set. The library will search for a single file called "config.properties" residing under the classpath. The aim of this file is to allow to configure properties that needs to be configured on site (such as URLs, users, passwords...). The library will take the configuration object and overwrite the stage 2 result with the properties values defined in this file.
{{prettynote|The config.properties file is mandatory and must exist in your classpath. One exception to this rule is when one is using the configuration for test library. see below more details.}}
After these 3 stages, the commons configuration object will be the final object that will be injected into the components all over the server.
{{prettywarning| When overriding a part of a structure array in config.properties the entire structure is re-written into the configuration map. This means that if you set default values in your configSchema.xml file (e.g. set 2 default array entries) overriding any array entry in config.properties mandates overriding the entire array. }} ====Multi-instance Support==== In some cases we a re required to support multiple instances of then same component running on the same host using the same install path. To allow loading different configuration files for each instance a new support was added. The new mechanism makes use of the system property "app.instance.name". When this property is set any non empty value, the configuration library will attempt to load a configuration file called: "config.<app.instance.name>.properties". If such a file does not exist the library will load the regular config.properties as it used to.
==How the R&D uses the CABConfiguration library==
Any component that wants to be CABConfiguration compliant needs to
==Maven Dependency== Please use the following maven dependency to get it:
==Expose the default configuration== In order to make the default configuration available to the CAB Configuration library, the developer needs to add an xml file including the default configuration values to the released jar.
- The format of this file needs to be xml file.
- This file needs to be called configSchema.xml (the correct place is under main/resources dir).
- The file needs to sit on the root of the delivered jar (handled automatically if, when using Maven, you put the file under src/main/resources).
{{prettynote|When you want to override a default value from your own set of configuration or perhaps from another library which you use and is in your dependency tree, you should use the configLocation notation as specified in the #CCP_Maven_Plugin.}}
==Get the configuration object using the Factory API==
==Get the configuration object using Spring Dependency Injection== Here is the change you need to do to your Spring XML in order to get the Cab Configuration object and to be able to inject it in your own bean.
==Property Place Holders== CAB Configuration library enables you to have place holders in your spring xml files. For example look at this xml snippet:
It doesn't make any sense hard coding such a value. Spring enables you to use this form:
And now in a configuration source you can set:
In the xml it would look like:
The CAB Configuration enables this Spring feature and applies it to any of the configuration layers described above. The library will make sure that the config key is replaces by the Spring runtime before the spring bean and property are being processed.
== How to enable dynamic reloading == CCP support dynamic reloading if the "requiresRestart" parameter is set to false in the Parameter xml root. When set to true the CCP will reload the new value in a pre-configured amount of time.
=== Reload Notification ===
Interested parties that wish to get notified on reload event, should use the following API:
"configListener" above should be a class that implements the "CabConfigurationListener" interface.
=== Configuration ===
The following parameters enable this support even when CCP is not enabled:
*configuration.dynamicConfigReload.enabled - this is the global parameter to enable the dynamic configuration. DEFAULT IS SET TO FALSE. *configuration.dynamicConfigReload.refreshDelay- set the refresh delay (milliseconds) to poll the file for changes. DEFAULT IS 30000 millis.
==CCP Compliance==
{{prettywarning|CAB Configuration will work just fine with property files, however in such a case the lib/component will not be a part of the CCP system (i.e. will not be managed by the GUI.)}}
===CCP Standards=== Please follow CAB_Configuration_Platform/Standards_and_Patterns to be fully CCP compliant. Note: although in Runtime everything will work regardless of these conventions, it is very important to follow this so we have a consistent configuration rule-set across components.
===File Structure=== Users of the CAB Configuration 2.x version are familiar with the layered approach of an optional defaultConfiguration.properties file that resides in the jar (/src/main/resources - in your development env.), and another mandatory config.properties files that resides in /etc (/src/test/resources in our development env.).
The problem with the property file approach is that we cannot express metadata on the parameters. For example, we cannot express is this parameter mandatory or not, or what is it's type etc.
From a developer perspective the changes checklist are as follows:
*Convert the property file to an xml file - you do this by running the "proeprty_to_xml" script that comes with CabConfiguration tar file. Note the new file is named configSchema.xml and the old file should be renamed to defaultConfig.properties.old.
*According to the configuration review performed earlier as described #Configuration_Naming_Changes edit the xml and add all relevant metadata.
*Make sure every parameter defined in your config.properties exists in configSchema.xml. If you have a parameter that should not have a default value, just define the metadata and make sure it is marked as required. The system engineers will be prompted to set a value for this parameter when needed.
{{prettynote|note that in the generated xml file you see a reference to the schema located [http://ch-infra.il.nds.com/cabResources/CCP_XML.xsd here]. Use this in an IDE to get auto-completion of all your options.}}
*Minimum required attributes per Parameter are: "name" (provided by the conversion script), "description" and "type".
{{prettywarning|The type value is an enumeration. Use the schema to see supported types}}.
*Take special note for #Structures_Tutorial.
*Components that produce rpm or tar files (e.g. represent processes) should add a #CAB_Dependency_Configuration_Plugin.
{{prettywarning|The examples of xml metadata usage are not exhaustive. Refer to the schema for full xml documentation}}
===Configuration Naming Changes=== CCP tries to create new standards for parameter names. Refer to CAB_Configuration_Review_Project for more info.
===CCP XML Schema===
The full ccp xsd file can be found [http://ch-infra.il.nds.com/cabResources/CCP_XML.xsd here].
====Main Parameter Attributes====
{{prettynote|for the "Parameter" Element, 'name', 'description' and 'type' are the minimum required attributes as all the other attributes have default values. Where the default values for all the other attributes are appropriate, there would be no need to explicitly state the values.}}
Following are the main Parameter attributes and their meaning:
{| border="1" cellpadding="2" !width="50"|Attribute !width="50"|Default !width="225"|Description |- | requiresRestart|| true || Means that the component needs to be restarted for it to consume the new parameter value, as opposed to being able to consume the new value without a restart |- | advanced|| false|| A parameter is advanced if it is to be changed by an expert engineer, usually in unusual/exceptional circumstances |- | readOnly|| false|| Immutable configuration parameter. Highly unusual. |- | required|| true || A parameter is ‘required’ if the component needs that parameter to be assigned a value for it to work. There could be situations where a component does not require a parameter to have a value. |- | hidden|| false|| A hidden parameter will not be exposed by the CCP UI at all. |- | description|| N/A || You must provide a meaningful (up to 512 characters) description of any parameter. |- |}
{{prettynote|There is no need to set the 'instantiationLevel'. It is set by default to 'GROUP', so if you don’t explicitly insert this attribute the parameter will be considered by CCP to be at GROUP level. For non-infrastructure software you would almost always use the GROUP level; only if a parameter could be specific to an instance of the component-type would you use COMPONENT level. Infrastructure parameters are very often GLOBAL as they apply to all components.}}
====Requires Support==== By default each parameter is required. If you want to mark a Parameter as optional please add the 'required="false"' attribute in the Parameter Element. E.g.
====Structures Tutorial==== In some cases (especially in arrays) you would like to express that the Parameter has a structure. For example:
[Old Property Way]
[New Xml Metadata]
====Arrays Tutorial==== Put aside structure arrays defined above you might have a regulat primitive array. Following is an example of a definition:
[Old Property Way]
[New Xml Metadata]
====Relationships between dependencies (Controller)==== In the excel template described above - we introduced the concept of controller. It is quite often that one or more parameters are dependant on a value of a another parameter. Usually if the other parameter has a certain value (or range) then these parameters are relevant. Following is an example of such a relationship (here we see that "service.rmi.retryOnTimeout" is only relevant if "service.rmi.numberOfRetries" is greater than 0):
<Parameter name="service.rmi.retryOnTimeout" type="BOOLEAN"
description="if set to true, will cause the client infrastructure to re-send the current request to the server."
instantiationLevel="GLOBAL">
<DefaultValue>
<PrimitiveValue value="false"/>
</DefaultValue>
<EnabledBy parameterName="service.rmi.numberOfRetries" operator="GT">
<Value>
<PrimitiveValue value="0"/>
</Value>
</EnabledBy>
</Parameter>
<Parameter name="service.rmi.numberOfRetries" type="INTEGER"
description="the number of times the client will retry connectiong to a server when facing communication errors"
instantiationLevel="GLOBAL">
<DefaultValue>
<PrimitiveValue value="3"/>
</DefaultValue>
</Parameter>
====Structure Templates==== In many cases you can find a need to define a template structure. This means you want to express the structure once and reuse many times.
The way to this is by using "ParameterType" instead of simple "Parameter". The ParameterType is similar to a java class (written once) which you can later on instantiate many times.
{{prettynote|We do '''not''' expect users to create such template on their own. Please see the usage guide below.}}
Example of a ParameterType:
<StructureMemberDefinition name="server" type="STRUCTURE" isArray="true" ignoreName="true">
<StructureDefinition>
<StructureMemberDefinition name="host" type="STRING" />
<StructureMemberDefinition name="port" type="INTEGER" />
</StructureDefinition>
</StructureMemberDefinition>
</StructureDefinition>
</ParameterType>
Usage example (note the new "base" attribute):
CAB_Infrastructure/CAB_Communication#Configuration_Using_Template
====Grouping library Namespaces in parent rpm==== In many cases you have a component that has internal libraries all owned by the same logical component. An example would be PPS. You have an rpm called pps which contains internal libs like: pps-db, pps-engine, pps-xml etc. Some of the libraies may contain configSchema.xml files of their own.
By default running the ccp maven plugin "as is" will cause multiple namespaces to appear in the ccpConfig.xml generated file. This is redundant. What we would like to happen is that all internal configurations are grouped with the rpm in a single namespace.
To achieve the above you need to add the "groupWith" attribute in the configSchema.xml file of each internal library (no need to change the configSchema in the rpm project if one exists). The value of the attribute will point to the rpm-software name as defined in the pom file of the rpm maven project.
Example in internal lib (e.g. pps-db):
The above will actually generate a single namespace for pps instead of having multiple names (that mean nothing to operators).
===CCP Maven Plugin=== Components that produce rpm or tar files (e.g. represent processes) should add in their pom file the following maven plugin. This plugin will scan all configSchema.xml files in all the component Maven dependencies and build an xml that holds all the needed data for introducing a new Component into the CCP DB using the CCP GUI.
'''The plugin also reads the relevant (configured) config.properties file - so it introduces parameter overrides "out of the box".'''
The section to be added in the pom file (under build-->plugins) is (note the needed update to the version field):
{{prettywarning|After adding this new plugin a file called ccpConfig.xml will be created under "target" folder. You will need to update the rpm mapping so this file will be added into the rpm in a new docs/ccp folder.
In order to make sure the ccpConfig.xml is copied properly to the docs/ccp folder, please make sure you have this in your pom where you create the rpm:
/opt/nds/installed/${rpmSoftwareName}-${project.version}/docs/ccp
====Web UI Modules==== Web UI Modules that run under NDS Console need special treatment. As the GIS script is generic for all the modules we need to read the generated xml file during runtime. For this each module as part of the mapping must copy the file to the "classes" folder ''' in addition'' to the docs/ccp folder.
====CCP Validation==== The plugin will validate the generated file via a special python script.
Currently this validation will only run when the plugin is executed on a Linux host.
After the build has finished, look in the console of your job and search for "CCP Validation Results" and inspect if you have any warning or errors that need addressing.
=====Configuration Options=====
*To disable this check add in your configuration section: 'false'. *If your component does not use database configuration, please add the following to your configuration section: 'false'. *If your component is not an RMI server, please add the following to your configuration section: 'false'. *If your component is not using cab monitoring, please add the following to your configuration section: 'false'.
Pom snippet example:
See CAB_Infrastructure/CCP_Validator for more info on CCP Validator.
==Troubleshooting== ===Disabling Apache commons configuration character delimiter=== Use the system property: "commonsConfig.delimiterParsingDisabled".
===My tests fail even when I update the properties=== It is common practice that tests use api like the following to update the configuration set (in-memory) for different tests.
The above seems to start working? The reason is probably the additional cache layer we added in front of the configuration object. The reason the cache was added is because we've found out that although commons-configuration library doesn't do I/O for each property read, it does do many locks. This is harmful in multi-threaded environment.
To over come this issue, '''in tests code only''' you can call this code AFTER you update the properties:
===Change Apache commons configuration character delimiter=== Use the system property: "commonsConfig.defaultListDelimiter". Note: default is ','.
==How to use configuration in testing==
===Goal===
#Manage multiple configuration files for different test classes. #Leave the config.properties file to be free of any test configuration parameters and allow it to be clean with values ready for integrator or system engineer (i.e. empty values for db username, password, etc.).
===Usage===
The following API's mandate that the file name used must contain the word "test" (regardless of the letter case) anywhere in the file name.
{{prettynote|The following API can be called with a different file name in different test classes. This will enable you to create multiple configuration sets for different tests.}}
{{prettywarning|The methods detailed below are a prerequisite to loading the configuration either by Spring injection or by Factory API detailed above. If one will not load a new configuration object the new config test file will be loaded.}}
====Maven Dependency==== Please use the following maven dependency to get it:
====Using API====
Call the method
From you test code. For example: